"lucas" <[EMAIL PROTECTED]> wrote:
> a simple question about mutual information:
>
> mutual information is defined as I(X;Y) = H(X) - H(X|Y)
> If X and Y aren't simple discrete random variables, but are defined
> set of random discrete variables (X,Y in {X1,X2, ....,Xn}, with Xi
> and Yi discrete random variables), how can i calculate I(X;Y)?
I(ABC;XYZ) is a well-defined concept in information theory. For
example, if your X is actually a set of variables {X1,X2,X3}, and your
Y is a set of variables {Y1,Y2}, you could define X' to be the
Cartesian product of variables in X: X1xX2xX3, and similarly Y' to be
Y1xY2. Then, you compute I(X';Y'). The trouble with this approach is
that the resulting Cartesian products may be sparse, and then your
probability estimates are bad. Here, you may use the interaction
analysis approach described in http://arxiv.org/abs/cs.AI/0308002, and
approximate I(X';Y') using lower-order interactions.
Best regards,
Aleks
--
mag. Aleks Jakulin
http://ai.fri.uni-lj.si/aleks/
Artificial Intelligence Laboratory,
Faculty of Computer and Information Science, University of Ljubljana.
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
