I agree with everything you say. Except,
I have found that A'AR gives an awful lot of what you need and may even be better in some ways than a full SVD. The *assumption* that incorporating all n-th degree connections is better in terms of results is just that. Whether it actually is better is a matter of conjecture and I certainly don't have really strong evidence either way. The random indexing community claims to have really good results. The LSA community claims some good results, notably Netflix results. My own experience in document classification is that the results with A'AR (what we used to call one-step learning) and SVD are really, really similar. On Mon, Jan 4, 2010 at 2:55 PM, Jake Mannix <[email protected]> wrote: > When you notice that for text, ngrams like "software engineer" are now > considerably closer to "c++ developer" than to other ngrams, this gives you > information. You don't get that information from a random projection. > You'll get some of that information from A'AR, because you get second-order > correlations, but then you're still losing all correlations beyond > second-order (and a true eigenvector is getting you the full infinite > series of correlations, > properly weighted). > > I mean, I guess you can use SVD purely for dimensional reduction, but like > you say, doing reduction can be done lots of other more efficient ways. > Doing it with reduction which enhances co-occurrence relationships and > distorts > the metric to produce better clusters than when you started is something > that SVD, NMF, and LDA were designed for. > > Maybe I'm missing your point? > -- Ted Dunning, CTO DeepDyve
