Re: Presentation in London: Running Spark on Hive or Hive on Spark
Hi Mich, Would it be on YouTube , post session ? - Harmeet On Jul 7, 2016, at 3:07, Mich Talebzadeh wrote: > Dear forum members > > I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, > your mileage varies" in Future of Data: London > > Details > > Organized by: Hortonworks > > Date: Wednesday, July 20, 2016, 6:00 PM to 8:30 PM > > Place: London > > Location: One Canada Square, Canary Wharf, London E14 5AB. > > Nearest Underground: Canary Warf (map) > > If you are interested please register here > > Looking forward to seeing those who can make it to have an interesting > discussion and leverage your experience. > > Regards, > > Dr Mich Talebzadeh > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > http://talebzadehmich.wordpress.com > > Disclaimer: Use it at your own risk. Any and all responsibility for any loss, > damage or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. >
Re: Book for Machine Learning (MLIB and other libraries on Spark)
Machine learning - I would suggest that you pick up a fine book that explains machine learning. That's the way I went about - pick up each type of machine learning concept - say Linear regression then understand the why/when/how etc and infer results etc. Then apply the learning to a small data set using python or R or scala without Spark. This is to familiarize the learning. Then run the same with MLlib and see it with a big data set on Spark. I would call this consolidation. Few things to remember - all Machine learning algorithms are not available On spark. There is a list of machine learning supported in spark. Kindly look at that. Also look at how to integrate mahout / h20 with spark and see how you can run the machine learning stuff supported by mahout with spark. And then your journey begins :-). Regards, Harmeet On Jun 12, 2016, at 0:31, Mich Talebzadeh wrote: > yes absolutely Ted. > > Thanks for highlighting it > > > > Dr Mich Talebzadeh > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > http://talebzadehmich.wordpress.com > > > On 11 June 2016 at 19:00, Ted Yu wrote: > Another source is the presentation on various ocnferences. > e.g. > http://www.slideshare.net/databricks/apache-spark-mllib-20-preview-data-science-and-production > > FYI > > On Sat, Jun 11, 2016 at 8:47 AM, Mich Talebzadeh > wrote: > Interesting. > > The pace of development in this field is such that practically every single > book in Big Data landscape gets out of data before the ink dries on it :) > > I concur that they serve as good reference for starters but in my opinion the > best way to learn is to start from on-line docs (and these are pretty > respectful when it comes to Spark) and progress from there. > > If you have a certain problem then put to this group and I am sure someone > somewhere in this forum has come across it. Also most of these books' authors > actively contribute to this mailing list. > > > HTH > > > Dr Mich Talebzadeh > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > http://talebzadehmich.wordpress.com > > > On 11 June 2016 at 16:10, Ted Yu wrote: > https://www.amazon.com/Machine-Learning-Spark-Powerful-Algorithms/dp/1783288515/ref=sr_1_1?ie=UTF8&qid=1465657706&sr=8-1&keywords=spark+mllib > > https://www.amazon.com/Spark-Practical-Machine-Learning-Chinese/dp/7302420424/ref=sr_1_3?ie=UTF8&qid=1465657706&sr=8-3&keywords=spark+mllib > > https://www.amazon.com/Advanced-Analytics-Spark-Patterns-Learning/dp/1491912766/ref=sr_1_2?ie=UTF8&qid=1465657706&sr=8-2&keywords=spark+mllib > > > On Sat, Jun 11, 2016 at 8:04 AM, Deepak Goel wrote: > > Hey > > Namaskara~Nalama~Guten Tag~Bonjour > > I am a newbie to Machine Learning (MLIB and other libraries on Spark) > > Which would be the best book to learn up? > > Thanks > Deepak >-- > Keigu > > Deepak > 73500 12833 > www.simtree.net, dee...@simtree.net > deic...@gmail.com > > LinkedIn: www.linkedin.com/in/deicool > Skype: thumsupdeicool > Google talk: deicool > Blog: http://loveandfearless.wordpress.com > Facebook: http://www.facebook.com/deicool > > "Contribute to the world, environment and more : http://www.gridrepublic.org > " > > > >
Re: Running R codes in sparkR
Hi Arunkumar , Yes , R can be integrated with Spark to give you SparkR. There are a couple of blogs on the net. The Spark dev page has it too. https://spark.apache.org/docs/latest/sparkr.html Just remember that all packages of R that you may have worked on in R are not supported in SparkR. There are a good set of R packages in SparkR. As I have understood you cannot run sapply etc for example. The constraint being these packages need to be ported/coded for RDD's. The R community as I understand is not very deeply involved with the Spark community. - this I have understood by seeing you tube videos. On May 31, 2016, at 18:16, Arunkumar Pillai wrote: > Hi > > I have some basic doubt regarding spark R. > > 1. can we run R codes in spark using sparkR or some spark functionalities > that are executed in spark through R. > > > > -- > Thanks and Regards >Arun - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: ANOVA test in Spark
Mayank, Assuming Anova not present in MLIB can you not exploit the Anova from SparkR? I am enquiring not making a factual statement. Thanks On May 13, 2016, at 15:54, mayankshete wrote: > Is ANOVA present in Spark Mllib if not then, when will be this feature be > available in Spark ? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/ANOVA-test-in-Spark-tp26949.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Hello !
Thank you ! On Apr 12, 2016, at 1:41, Ted Yu wrote: > For SparkR, please refer to https://spark.apache.org/docs/latest/sparkr.html > > bq. on Ubuntu or CentOS > > Both platforms are supported. > > On Mon, Apr 11, 2016 at 1:08 PM, wrote: > Dear Experts , > > I am posting this for your information. I am a newbie to spark. > I am interested in understanding Spark at the internal level. > > I need your opinion, which unix flavor should I install spark on Ubuntu or > CentOS. I have had enough trouble with the windows version (1.6.1 with Hadoop > 2.6 pre built binaries , keeps giving me exceptions ). > > I have worked on R on windows till date . Is there an R for unix? I have not > googled this either. Sorry about that.Just want to make sure SparkR has a > smooth run. > > Thanks in advance. > Harry > > > > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Hello !
Dear Experts , I am posting this for your information. I am a newbie to spark. I am interested in understanding Spark at the internal level. I need your opinion, which unix flavor should I install spark on Ubuntu or CentOS. I have had enough trouble with the windows version (1.6.1 with Hadoop 2.6 pre built binaries , keeps giving me exceptions ). I have worked on R on windows till date . Is there an R for unix? I have not googled this either. Sorry about that.Just want to make sure SparkR has a smooth run. Thanks in advance. Harry - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org