PRB-30

Bernoulli KL

KL between two coins.

Reading speed

GoverningBernoulli KL

Governing equation

D=p\\ln(p/q)+(1-p)\\ln((1-p)/(1-q))

$p \in \left[0.02, 0.98\right]$

where

p: p (—)
q: q (—)
D: Bernoulli KL (nat)

The figure, part by part

Each numbered marker wears the color of its card. Hover a card to light the marker. Tap a marker to jump to that card. The slider walks a red point along the curve; Animate drawing traces it like a pen.

00:0 / 00:08

①

Governing law — red

D=p\\ln(p/q)+(1-p)\\ln((1-p)/(1-q))

$p \in \left[0.02, 0.98\right]$

$p = 0.02$ $D = 0.421$

Along the curve

②

Domain of the variable — blue

p_{\min} = 0.02,\quad p_{\max} = 0.98

$p \in \left[0.02, 0.98\right]$

$p = 0.02$ $p = 0.02$

Along the curve

③

Response curve — green

D=p\\ln(p/q)+(1-p)\\ln((1-p)/(1-q))

$p \in \left[0.02, 0.98\right]$

$p = 0.02$ $D = 0.421$

Along the curve

④

Rate of change — amber

\frac{\mathrm{d}D}{\mathrm{d}p}

$p \in \left[0.02, 0.98\right]$

$p = 0.026$ $D^{\prime} = -3.225$

Along the curve

Lecture brief

Historical brief

Bernoulli trials, Poisson, Gauss, Bayes and Shannon entropy turned chance into a calculus of belief and noise. The lab computes likelihood, interval and information in SI-free counts. This sheet (PRB-30 — Bernoulli KL) is the form associated with Bernoulli KL. Working symbols: $p$ , $q$ $\rightarrow$ $D$ . KL between two coins. Pedagogical SI sheet with a live model and a swept parameter.

Purpose

Purpose: compute

D

from

p

q

in Probability & statistics via

D=p\\ln(p/q)+(1-p)\\ln((1-p)/(1-q))

KL between two coins. Use it when a real probability & statistics question must be answered in SI before a code check.

Live realistic example

In symbols

Live case. Given

p = 0.600\,\mathrm{—}

q = 0.400\,\mathrm{—}

, the governing relation

D=p\\ln(p/q)+(1-p)\\ln((1-p)/(1-q))

yields

D = 0.081\,\mathrm{nat}

. One governing identity, SI units, a single sweep on the sheet. Move a slider: the numbers are this situation, not a canned story.

Calculator

Inputs

p p0.6 —q q0.4 —

Outputs

Bernoulli KL D0.081 nat

Reading speed

Watch on YouTube

Free library

Full library

Free PDF / open book

YouTube channels

00:0 / 00:08

Narration of this film

One governing identity, SI units, a single sweep on the sheet.

KL between two coins. Pedagogical SI sheet with a live model and a swept parameter.

Reading speed

Watch on YouTube