Re: Scala vs Python for Spark ecosystem
Python can access the JVM - this how it interfaces with Spark. Some of the components do not have a wrapper fro the corresponding Java Api yet and thus are not accessible in Python. Same for elastic search. You need to write a more or less simple wrapper. > On 20 Apr 2016, at 09:53, "kramer2...@126.com" <kramer2...@126.com> wrote: > > I am using python and spark. > > I think one problem might be to communicate spark with third product. For > example, combine spark with elasticsearch. You have to use java or scala. > Python is not supported > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-for-Spark-ecosystem-tp26805p26806.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Scala vs Python for Spark ecosystem
I am using python and spark. I think one problem might be to communicate spark with third product. For example, combine spark with elasticsearch. You have to use java or scala. Python is not supported -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-for-Spark-ecosystem-tp26805p26806.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Scala vs Python for Spark ecosystem
Graphx did not support Python yet. http://spark.apache.org/docs/latest/graphx-programming-guide.html The workaround solution is use graphframes (3rd party API), https://issues.apache.org/jira/browse/SPARK-3789 but some features in Python are not as same as Scala, https://github.com/graphframes/graphframes/issues/57 Jingyu On 20 April 2016 at 16:52, sujeet jog <sujeet@gmail.com> wrote: > It depends on the trade off's you wish to have, > > Python being a interpreted language, speed of execution will be lesser, > but it being a very common language used across, people can jump in hands > on quickly > > Scala programs run in java environment, so it's obvious you will get good > execution speed, although it's not common for people to know this language > readily. > > > Pyspark API's i believe will have everything which Scala Spark API's offer > in long run. > > > > On Wed, Apr 20, 2016 at 12:14 PM, berkerkozan <berkerko...@gmail.com> > wrote: > >> I know scala better than python but my team (2 other my friend) knows only >> python. We want to use graphx or maybe try graphframes. >> What will be the future of these 2 languages for spark ecosystem? Will >> python cover everything scala can in short time periods? what do you >> advice? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-for-Spark-ecosystem-tp26805.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> > -- This message and its attachments may contain legally privileged or confidential information. It is intended solely for the named addressee. If you are not the addressee indicated in this message or responsible for delivery of the message to the addressee, you may not copy or deliver this message or its attachments to anyone. Rather, you should permanently delete this message and its attachments and kindly notify the sender by reply e-mail. Any content of this message and its attachments which does not relate to the official business of the sending company must be taken not to have been sent or endorsed by that company or any of its related entities. No warranty is made that the e-mail or attachments are free from computer virus or other defect.
Re: Scala vs Python for Spark ecosystem
It depends on the trade off's you wish to have, Python being a interpreted language, speed of execution will be lesser, but it being a very common language used across, people can jump in hands on quickly Scala programs run in java environment, so it's obvious you will get good execution speed, although it's not common for people to know this language readily. Pyspark API's i believe will have everything which Scala Spark API's offer in long run. On Wed, Apr 20, 2016 at 12:14 PM, berkerkozan <berkerko...@gmail.com> wrote: > I know scala better than python but my team (2 other my friend) knows only > python. We want to use graphx or maybe try graphframes. > What will be the future of these 2 languages for spark ecosystem? Will > python cover everything scala can in short time periods? what do you > advice? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-for-Spark-ecosystem-tp26805.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Scala vs Python for Spark ecosystem
I know scala better than python but my team (2 other my friend) knows only python. We want to use graphx or maybe try graphframes. What will be the future of these 2 languages for spark ecosystem? Will python cover everything scala can in short time periods? what do you advice? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-for-Spark-ecosystem-tp26805.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
