i guess i was a bit vague. by quasi-agnostic i mean that some code, the smaller part of it, may include specific backend engine dependencies unfortunately. it should be easily portable though.
On Thu, Jul 23, 2015 at 2:50 PM, Dmitriy Lyubimov <[email protected]> wrote: > Mahout is moving to be backend-agnostic. Supports same code on spark or > h20. > > (Disclaimer: some code is quasi-agnostic, such as spark shell, or I think > some co-occurrence drivers also like Spark more than anything else. may be > wrong.) > > > On Thu, Jul 23, 2015 at 2:41 PM, Ankit Goel <[email protected]> > wrote: > >> Thanks a lot guys. >> @Pat is mahout only going to support scala in the near future? and will >> all >> the ml libraries only be from spark? I did read somewhere that mahout was >> heading towards a direction where its more of a framework that supports >> multiple ml libraries. Am I right in my understanding? >> >> On Thu, Jul 23, 2015 at 10:03 PM, Pat Ferrel <[email protected]> >> wrote: >> >> > Just to be clear, mahout runs on AWS just fine. Dmitriy is talking about >> > support and continuance of “MapReduce” which means Hadoop MapReduce. We >> > have been exclusively accepting only more modern engine code for more >> than >> > a year so most of the modern Mahout is in Scala and runs on Spark. The >> > MapReduce paradigm is certainly supported there but it runs on Spark so >> any >> > EMR instances you create should have Spark installed. >> > >> > Amazon now supports Spark on EMR: >> > https://aws.amazon.com/blogs/aws/new-apache-spark-on-amazon-emr/ >> > >> > Make sure you use the correct version of Spark with Mahout. 0.10.0 >> > supports Spark 1.1.1 or less, Mahout 0.10.1 supports Spark 1.2.1 or >> less, >> > the current master snapshot supports Spark 1.3 and runs on Spark 1.4. >> > >> > On Jul 23, 2015, at 7:28 AM, Ankit Goel <[email protected]> >> wrote: >> > >> > Thanks for the heads up Dmitriy..thats exactly the kind of warning I was >> > looking for. I dont have any experience implementing MR yet --i >> understand >> > the algo perfectly-- so this is a great heads up. Any advice oor >> warnings >> > on hadoop installations and versions?? >> > >> > On Thu, Jul 23, 2015 at 6:34 AM, Dmitriy Lyubimov <[email protected]> >> > wrote: >> > >> > > MapReduce things enter de-facto end-of-life. Not that we specifically >> > don't >> > > want to support them, it is de-facto nobody bothers to support them -- >> > > especially risks are high with new versions of hadoop and EMR. >> > > >> > > That said, we'd be grateful for any guide about doing this in EMR. >> > > >> > > On Wed, Jul 22, 2015 at 5:53 PM, Ankit Goel <[email protected]> >> > > wrote: >> > > >> > >> Hi, >> > >> After my runs on my lappy, I'm ready to port my work to the cloud. >> > > Planning >> > >> to use Amazon. One thing I noticed when I started with mahout, that >> > there >> > >> were a lot of things unsaid on the site/wiki and took me a lot of >> time >> > to >> > >> figure out. Pitfalls if I may call them. I will primarily be using >> > >> clustering on the cloud, so the code to accept new data and run it is >> > > what >> > >> I have for now. >> > >> >> > >> So before I port to the cloud, are there any things I should beware >> of >> > or >> > >> lookout for? Like is AWS fine with mahout? Are there any >> configurations >> > I >> > >> should remember? Any advice on implementation to ease my transition >> and >> > > run >> > >> mahout 24hrs? Thanks >> > >> >> > >> -- >> > >> Regards, >> > >> Ankit Goel >> > >> http://about.me/ankitgoel >> > >> >> > > >> > >> > >> > >> > -- >> > Regards, >> > Ankit Goel >> > http://about.me/ankitgoel >> > >> > >> >> >> -- >> Regards, >> Ankit Goel >> http://about.me/ankitgoel >> > >
