CUED Publications database

Estimating the entropy of discrete distributions

Antos, A and Kontoyiannis, I (2001) Estimating the entropy of discrete distributions. IEEE International Symposium on Information Theory - Proceedings. 45-. ISSN 2157-8095

Full text not available from this repository.


Given an i.i.d. sample (X1, Xn) drawn from an unknown discrete distribution P on a countably infinite set, we consider the problem of estimating the entropy of P. We show that the plug-in estimate is universally consistent and that, without further assumptions, no rate-of-convergence results can be obtained for any sequence of entropy estimates. Under additional conditions we get convergence rates for the plug-in estimate and for an estimate based on match-lengths. The behavior of the expected error of the plug-in estimate is shown to be in sharp contrast to the finite-alphabet case.

Item Type: Article
Divisions: Div F > Signal Processing and Communications
Depositing User: Cron Job
Date Deposited: 08 Jan 2018 20:12
Last Modified: 27 Oct 2020 07:12
DOI: 10.1109/ISIT.2001.935908