A pragmatic entropy and differential entropy estimator for small datasets

  • Jugurta Montalvão Federal Univ. of Sergipe
  • Romis Attux University of Campinas
  • Daniel Silva University of Brasília
Keywords: Entropy through coincidence, Small datasets, Discrete and/or continuous variables, Uncomplicated algorithms.

Abstract

A pragmatic approach for entropy estimation is presented, first for discrete variables, then in the form of an extension for handling continuous and/or multivariate ones. It is based on coincidence detection, and its application leads to algorithms with three main attractive features: they are easy to use, can be employed without any a priori knowledge concerning source distribution (not even the alphabet cardinality K of discrete sources) and can provide useful estimates even when the number of samples is small (e.g. less than K, for discrete variables).
Published
28-07-2014
How to Cite
Montalvão, J., Attux, R., & Silva, D. (2014). A pragmatic entropy and differential entropy estimator for small datasets. Journal of Communication and Information Systems, 29(1). https://doi.org/10.14209/jcis.2014.8
Section
Letters

Most read articles by the same author(s)