This is much denser than I would expect. You are saying that you would have an average of 1000 transactions per user. It is more normal to have 100 or less. If you have these smaller sizes, then in-memory algorithms on a single (large) machine begin to be practical.
On Tue, May 4, 2010 at 2:01 PM, Sean Owen <sro...@gmail.com> wrote: > On Tue, May 4, 2010 at 9:53 PM, First Qaxy <qa...@yahoo.ca> wrote: > > Purely based on estimates, assuming 5 billion transactions, 5 million > users, 100K products normally distributed are expected to create a sparse > item to item matrix of up to 10 Million significant co-occurrences > (significance is not globally defined but in the context of the active item > to recommend from; in other words support can be really tiny, confidence > less so). >
