"lucas" <[EMAIL PROTECTED]> wrote:
> > that the resulting Cartesian products may be sparse, and then your
> > probability estimates are bad.
>
> Why probability estimates are bad?
Say I give you the following three observations of two dichotomous
random variables (x,y), each of which has a range of {0,1}:
0 0
0 1
1 1
Can you estimate the p(x,y)? Maximum likelihood would be p(0,0)=1/3,
p(1,1)=1/3, p(0,1)=1/3 p(1,0)=0, but this is totally unreliable. Now,
imagine you have 1000 observations of 50 dichotomous variables,
x_1,x_2...x_100. Can you estimate p(x_1,x_2,...x_100)?
> > Here, you may use the interaction
> > analysis approach described in http://arxiv.org/abs/cs.AI/0308002,
> > approximate I(X';Y') using lower-order interactions.
>
> How works this approach in a few words?
Say I have four variables, A, B, C, D. I compute the mutual
informations I(A;B), I(A;C)...I(C;D). Then I compute the 3-way
interaction informations I(A;B;C), I(A;B;D)...I(B;C;D). I may neglect
any of these if they are too small.
Now I want to compute the mutual information between A and BCD:
I(A;B,C,D). A quick approximation is:
M ~= I(A;B)+I(A;C)+I(A;D)
But this may overestimate or underestimate the true mutual
information. So I add the 3-interaction effects, and carefully
subtract the overlap between them.
M' ~= M + I(A;B;C)+I(A;B;D)+I(A;C;D)-I(B;C:D)
So that's an approximation to I(A;BCD) that is only based on
trivariate joint PDFs. Still, it's not perfect, because it does not
take 4-way interaction effects into consideration. To do that, just
add I(A;B;C;D)+I(B;C;D).
It's Kikuchi-style mutual information approximation. Probably I was
not clear, but you said "a few words"... :)
Aleks
--
mag. Aleks Jakulin
http://ai.fri.uni-lj.si/aleks/
Artificial Intelligence Laboratory,
Faculty of Computer and Information Science, University of Ljubljana.
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
