Hey Phil, Thank you sharing this. The result didn't surprise me a lot, it's normal to do the prototype in Python, once it get stable and you really need the performance, then rewrite part of it in C or whole of it in another language does make sense, it will not cause you much time.
Davies On Fri, Jan 16, 2015 at 7:38 AM, philpearl <p...@tanktop.tv> wrote: > I was interested in this as I had some Spark code in Python that was too slow > and wanted to know whether Scala would fix it for me. So I re-wrote my code > in Scala. > > In my particular case the Scala version was 10 times faster. But I think > that is because I did an awful lot of computation in my own code rather than > in a library like numpy. (I put a bit more detail here > <http://tttv-engineering.tumblr.com/post/108260351966/spark-python-vs-scala> > in case you are interested) > > So there's one data point, if only for the obvious data point comparing > computations in Scala to computations in pure Python. > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Scala-vs-Python-performance-differences-tp4247p21190.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
