PRB-06

Binary entropy

Max at p=½.

Reading speed

Probabilityentropy

Governing equation

H=-p\\log_2 p-(1-p)\\log_2(1-p)

where

p: p (—)
H: H (bit)

Lecture brief

Historical brief

Bernoulli trials, Poisson, Gauss, Bayes and Shannon entropy turned chance into a calculus of belief and noise. The lab computes likelihood, interval and information in SI-free counts. This sheet (PRB-06 — Binary entropy) is the form associated with entropy. Working symbols: $p$ $\rightarrow$ $H$ . Max at p=½.

Purpose

Purpose: compute

H

from

p

in Probability & statistics via

H=-p\\log_2 p-(1-p)\\log_2(1-p)

Max at p=½. Use it when a real probability & statistics question must be answered in SI before a code check.

Live realistic example

In symbols

Live case. Given

p = 0.300\,\mathrm{—}

, the governing relation

H=-p\\log_2 p-(1-p)\\log_2(1-p)

yields

H = 0.8813\,\mathrm{bit}

. Max at p=½. Move a slider: the numbers are this situation, not a canned story.

Calculator

Inputs

p p0.3 —

Outputs

H H0.8813 bit

Reading speed

Watch on YouTube

Free library

Full library

Free PDF / open book

YouTube channels

00:0 / 00:08

Narration of this film

Max at p=½.

Reading speed

Watch on YouTube