← Back to Gallery

Source Coding Theorem

Source Coding Theorem

H(X) ≤ L_avg < H(X) + 1

Shannon's source coding theorem: The average code length L_avg is bounded by the entropy H(X).

Variable-length codes can achieve compression, but cannot go below entropy limit.