Nemenman-IEEEIT

From Ilya Nemenman

Back to the full Publications list.

I Nemenman. Inference of entropies of discrete random variables with unknown cardinalities. IEEE Trans. Inf. Thy., submitted 2005. PDF, arXiv.

Abstract

We examine the recently introduced NSB estimator of entropies of severely undersampled discrete variables and devise a procedure for calculating the involved integrals. We discover that the output of the estimator has a well defined limit for large cardinalities of the variables being studied. Thus one can estimate entropies with no a priori assumptions about these cardinalities, and a closed form solution for such estimates is given.