Please let us know how you make out. We have NLP requirements on the horizon. I¹ve used NLTK before, but never on Spark. I¹d love to hear if that works out for you.
-brian --- Brian O'Neill Chief Technology Officer Health Market Science The Science of Better Results 2700 Horizon Drive King of Prussia, PA 19406 M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42> healthmarketscience.com This information transmitted in this email message is for the intended recipient only and may contain confidential and/or privileged material. If you received this email in error and are not the intended recipient, or the person responsible to deliver it to the intended recipient, please contact the sender at the email above and delete this email and any attachments and destroy any copies thereof. Any review, retransmission, dissemination, copying or other use of, or taking any action in reliance upon, this information by persons or entities other than the intended recipient is strictly prohibited. From: Mayur Rustagi <mayur.rust...@gmail.com> Reply-To: <user@spark.apache.org> Date: Wednesday, March 12, 2014 at 2:38 PM To: <user@spark.apache.org> Cc: "u...@spark.incubator.apache.org" <u...@spark.incubator.apache.org> Subject: Re: NLP with Spark Would love to know if somebody has tried this, only possible problem I can forsee is non-serializable libraries, else no reason it should not work. Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Wed, Mar 12, 2014 at 11:10 AM, shankark <shankark+...@gmail.com> wrote: > (apologies if this was sent out multiple times before) > > We are about to start a large-scale text-processing research project and are > debating between two alternatives for our cluster -- Spark and Hadoop. I've > researched possibilities of using NLTK with Hadoop and see that there's some > precedent > (http://blog.cloudera.com/blog/2010/03/natural-language-processing-with-hadoop > -and-python/). I wanted to know how easy it might be to use NLTK with pyspark, > or if scalanlp is mature enough to be used with the Scala API for Spark/mllib. > > Thanks! > > > View this message in context: NLP with Spark > <http://apache-spark-user-list.1001560.n3.nabble.com/NLP-with-Spark-tp2612.htm > l> > Sent from the Apache Spark User List mailing list archive > <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
