Exactly! I have always thought that was the main reason why ALS in Giraph was faster.
Doesn't it make strong case for a hybrid environment? Anyway, what i am saying, isn't it more or less truthful to say that in pragmatic ways ALS stuff in Mahout is lagging for the very reason of Mahout being constrained to MR? On Mon, Mar 11, 2013 at 1:16 PM, Ted Dunning <[email protected]> wrote: > Kinda sorta.. > > You can defeat most of the sort if you want to just hash things to buckets. > > On Mon, Mar 11, 2013 at 12:01 PM, Dmitriy Lyubimov <[email protected] > >wrote: > > > Sort component adds log to > > the asymptotic complexity, whereas it is clear that any streaming merge > > algorithm just wouldn't need to do sort and capitalize on the structure > we > > already know . (sure, you can do it map-side with a specific streaming > join > > logic but that would not be pure MR but rather some map task acrobatics). > > >
