In this section, we shall discuss two different aspects of Shannon's
entropy. One deals with infinite sequence of discrete random variables.
The other with infinite sequence of a probability distribution. The entropy
measure associated with first aspect is famous as entropy rate,
while the entropy measure associated with the second aspect is the entropy
series. Discussing the entropy series, we shall study also the important
aspect of maximum entropy principle for the discrete case. While,
the continuous case is discussed in section 1.8.