Next:Entropy RateUp:Shannon's Entropy
Previous:Information Measures for Discrete Go to:Table of Contents

Entropy Rate and Entropy Series

In this section, we shall discuss two different aspects of Shannon's entropy. One deals with infinite sequence of discrete random variables. The other with infinite sequence of a probability distribution. The entropy measure associated with first aspect is famous as entropy rate, while the entropy measure associated with the second aspect is the entropy series. Discussing the entropy series, we shall study also the important aspect of maximum entropy principle for the discrete case. While, the continuous case is discussed in section 1.8.


Inder Jeet Taneja
Departamento de Matemática - UFSC
88.040-900 Florianópolis, SC - Brazil