Antos, A and Kontoyiannis, I (2001) *Estimating the entropy of discrete distributions.* IEEE International Symposium on Information Theory - Proceedings. 45-. ISSN 2157-8095

## Abstract

Given an i.i.d. sample (X1, Xn) drawn from an unknown discrete distribution P on a countably infinite set, we consider the problem of estimating the entropy of P. We show that the plug-in estimate is universally consistent and that, without further assumptions, no rate-of-convergence results can be obtained for any sequence of entropy estimates. Under additional conditions we get convergence rates for the plug-in estimate and for an estimate based on match-lengths. The behavior of the expected error of the plug-in estimate is shown to be in sharp contrast to the finite-alphabet case.

Item Type: | Article |
---|---|

Subjects: | UNSPECIFIED |

Divisions: | Div F > Signal Processing and Communications |

Depositing User: | Cron Job |

Date Deposited: | 08 Jan 2018 20:12 |

Last Modified: | 27 Oct 2020 07:12 |

DOI: | 10.1109/ISIT.2001.935908 |