Well, with spark, majority of algorithms will have to be rewritten to use RDD/Bagel stuff. But it is only the high level, the math module framework should be pretty much inherited in its entirety. So I think it may exist as a contrib project with dependencies on mahout math, eventually migrating in as a module with optional maven dependencies. We just would need to make a concious effort to pay attention to what's happening there to make sure whatever perturbations happen in math do not break it too much or too often.
-d On Tue, Mar 12, 2013 at 7:20 AM, Ted Dunning <[email protected]> wrote: > What does separate project mean to you? > > Sent from my iPhone > > On Mar 12, 2013, at 3:25 AM, Sean Owen <[email protected]> wrote: > > > YARN + Spark sounds interesting indeed. Could I suggest this sounds like > > definitely a separate project vs yet another experiment in this code > base? > > it's already spread unevenly enough. > > > > > > On Mon, Mar 11, 2013 at 11:59 PM, Ted Dunning <[email protected]> > wrote: > > > >> Yarn by itself won't fix this problem. Yarn + Spark would fix it. But, > >> then again, so would Mesos + Spark or AmigaOS + Spark. > >> > >> Should we open several additional modules which are parallel to core but > >> which depend on alternatives like Giraph or Spark instead? That would > >> allow experimentation to proceed. >
