An Introduction to Information Theoretic Learning, Part I: Foundations

  • Daniel Silva University of Brasília
  • Denis Fantinato University of Campinas
  • Janio Canuto Federal University of Sergipe
  • Leonardo Duarte University of Campinas
  • Aline Neves Federal University of ABC
  • Ricardo Suyama Federal University of ABC
  • Jugurta Montalvão Federal University of Sergipe
  • Romis Attux University of Campinas
Keywords: ITL, information theory, entropy, Rényi


With the increasing number of machine learning problems that are out of the linear and Gaussian paradigm, information theoretic learning (ITL) rises as a research field that proposes a modeling method with a wealthier statistical treatment of the adaptation criterion. In the first part of this tutorial, we introduce the main concepts of ITL and a key set of estimators that enable the implementation of algorithms, in the context of a wider view independent of the differentiability property.


How to Cite
Silva, D., Fantinato, D., Canuto, J., Duarte, L., Neves, A., Suyama, R., Montalvão, J., & Attux, R. (2016). An Introduction to Information Theoretic Learning, Part I: Foundations. Journal of Communication and Information Systems, 31(1).
Tutorial Papers