QuantumInfo.ClassicalInfo.Entropy

The one-event entropy function $H_1: [0, 1] \to \mathbb{R}$ maps a probability $p$ to its Shannon entropy, defined as $H_1(p) = -p \ln p$. Here, $p$ is an element of the unit interval $[0, 1]$, and by convention (inherited from the underlying `negMulLog` function), $0 \ln 0 = 0$.

For the Shannon entropy function $H_1: [0, 1] \to \mathbb{R}$ defined by $H_1(p) = -p \ln(p)$, the value of the function at $p = 0$ is equal to $0$.

The Shannon entropy function $H_1$ evaluated at the probability $p = 1$ is equal to $0$.

Let $p \in [0, 1]$ be a probability. The Shannon entropy of $p$, denoted by $H_1(p)$, is non-negative, i.e., $0 \le H_1(p)$.

For any probability $p$ in the unit interval $[0, 1]$, the Shannon entropy $H_1(p)$ is strictly less than 1.

For any probability $p \in [0, 1]$, the Shannon entropy $H_1(p)$ satisfies the inequality $H_1(p) \le e^{-1}$, where $e$ is Euler's number.

For any probabilities $x, y \in [0, 1]$ and any weight $p \in [0, 1]$, the Shannon entropy function $H_1$ is concave, satisfying the inequality: $$p \cdot H_1(x) + (1 - p) \cdot H_1(y) \le H_1(p \cdot x + (1 - p) \cdot y)$$ where $p[a \leftrightarrow b]$ denotes the convex combination $p \cdot a + (1 - p) \cdot b$.

Let $\alpha$ be a finite set and $d$ be a probability distribution on $\alpha$. The Shannon entropy $H_s(d)$ of the distribution is defined as the sum over all elements $x \in \alpha$ of the individual entropy contributions $H_1(p_x)$, where $p_x$ is the probability assigned to $x$ by $d$. Mathematically, this is expressed as: $$H_s(d) = \sum_{x \in \alpha} H_1(d(x))$$ where $H_1(p) = -p \ln(p)$ for $p \in [0, 1]$.

Let $\alpha$ be a finite set and let $d$ be a probability distribution on $\alpha$. The Shannon entropy of the distribution, denoted by $H_s(d)$, is non-negative, i.e., $0 \leq H_s(d)$.

Let $\alpha$ be a finite set and $d$ be a probability distribution on $\alpha$. The Shannon entropy of the distribution, denoted $H_s(d)$, is less than or equal to the natural logarithm of the cardinality of $\alpha$, i.e., $H_s(d) \le \ln |\alpha|$.

Let $\alpha$ be a type and $i \in \alpha$ be an element. Let $d = \delta_i$ be the constant probability distribution centered at $i$ (where $\delta_i(y) = 1$ if $y = i$ and $0$ otherwise). Then the Shannon entropy of this distribution is zero, i.e., $H(d) = 0$.

Let $\alpha$ be a non-empty finite set. For the uniform probability distribution $P$ on $\alpha$, the Shannon entropy $H_s(P)$ is equal to the natural logarithm of the cardinality of $\alpha$, denoted $\ln |\alpha|$.

Let $p \in [0, 1]$ be a probability. The Shannon entropy $H_s$ of a coin distribution (a two-event distribution) with success probability $p$, denoted by $\text{ProbDistribution.coin } p$, is equal to the binary entropy of $p$, denoted by $H_b(p)$ (or `Real.binEntropy p`).

Let $d_1$ and $d_2$ be probability distributions on finite sets $\alpha$ and $\beta$, respectively. If the multiset of probabilities occurring in $d_1$ is equal to the multiset of probabilities occurring in $d_2$, then the Shannon entropy of $d_1$ is equal to the Shannon entropy of $d_2$, denoted $H_s(d_1) = H_s(d_2)$.