Next:Characterizations of Generalized EntropiesUp:Entropy-Type Measures

# Unified (r,s)-Entropy for Continuous Probability Distributions

In this section we extend the notion of generalized entropies to the continuous case, and examine some properties of the resulting unified entropy function.

Let X be an absolutely continuous random variable, that is, a random variable having a probability density function p(x). The unified entropy of X is defined as follows:

 (3.14)

provided the integrals exist, where .

The contrast between continuous and discrete distribution is worth emphasising:

(i) The entropy measure of continuous distribution need not exist.
(ii) When it does exist, there is nothing to ensure that is positive because  can exceed unity. We consider the following examples:
Example 3.1. Let  be a random variable with probability density function
Then
with when . (iii) The unified entropy are not limits of the unifiedentropy of the discrete case. This we shall verify in the following example.

Example 3.2. Let  be a discrete random variable taking the values  with equal probabilities . Then

As  increases, the distribution of  converges to a continuous uniform distribution in (0,1). If , we have
however,
(iv) The unified entropy is not invariant with respect to a change of variables. We illustrate this point with the following example:

Example 3.3. We consider a function , where  is a stricly increasing function of . Since the mapping from  to  is one to one, we have

where . Therefore,
which is different from  unless  be the identity function.

These important differences between discrete and continuous cases are a warning that the results for the discrete distributions cannot be translated to continuous case without independent verification. Fortunately, some of the significant concepts rely upon differences between entropies and for these the difficulties disappear.

21-06-2001
Inder Jeet Taneja
Departamento de Matemática - UFSC
88.040-900 Florianópolis, SC - Brazil