Copula Entropy: Theory and Applications

Abstract: Statistical independence is a core concept in statistics and machine learning. Representing and measuring independence are of fundamental importance in related fields. Copula theory provides the tool for representing statistical independence, while Copula Entropy (CE) presents the tool for measuring statistical independence. This paper first introduces the theory of CE, including its definition, theorem, properties, and estimation method. The theoretical applications of CE to structure learning, association discovery, variable selection, causal discovery, system identification, time lag estimation, domain adaptation, multivariate normality test, two-sample test, and change point detection are reviewed. The relationships between the theoretical applications and their connection to correlation and causality are discussed. The frameworks based on CE, the kernel method, and distance correlation for measuring statistical independence and conditional independence are compared. The advantage of CE based on methods over the other comparable methods is evaluated with simulated and real data. The applications of CE in theoretical physics, astrophysics, geophysics, theoretical chemistry, cheminformatics, materials science, hydrology, climatology, meteorology, environmental science, ecology, animal morphology, agronomy, cognitive neuroscience, motor neuroscience, computational neuroscience, psychology, system biology, bioinformatics, clinical diagnostics, geriatrics, psychiatry, public health, economics, management, sociology, pedagogy, computational linguistics, mass media, law, political science, military science, informatics, energy, food engineering, architecture, civil engineering, transportation, manufacturing, reliability, metallurgy, chemical engineering, aeronautics and astronautics, weapon, automobile, electronics, communication, high performance computing, cybersecurity, remote sensing, ocean, and finance are briefly introduced.