CUED Publications database

Estimating the Directed Information and Testing for Causality

Kontoyiannis, I and Skoularidou, M (2016) Estimating the Directed Information and Testing for Causality. IEEE Transactions on Information Theory, 62. pp. 6053-6067. ISSN 0018-9448

Full text not available from this repository.


The problem of estimating the directed information rate between two discrete processes (Xn) and (Yn) via the plug-in (or maximum-likelihood) estimator is considered. When the joint process ((Xn, Yn)) is a Markov chain of a given memory length, the plug-in estimator is shown to be asymptotically Gaussian and to converge at the optimal rate O(1/√n) under appropriate conditions; this is the first estimator that has been shown to achieve this rate. An important connection is drawn between the problem of estimating the directed information rate and that of performing a hypothesis test for the presence of causal influence between the two processes. Under fairly general conditions, the null hypothesis, which corresponds to the absence of causal influence, is equivalent to the requirement that the directed information rate be equal to zero. In that case, a finer result is established, showing that the plug-in converges at the faster rate O(1/n) and that it is asymptotically χ2-distributed. This is proved by showing that this estimator is equal to (a scalar multiple of) the classical likelihood ratio statistic for the above hypothesis test. Finally, it is noted that these results facilitate the design of an actual likelihood ratio test for the presence or absence of causal influence.

Item Type: Article
Divisions: Div F > Signal Processing and Communications
Depositing User: Cron Job
Date Deposited: 08 Jan 2018 20:11
Last Modified: 27 Oct 2020 07:51
DOI: 10.1109/TIT.2016.2604842