This is an interesting question I don't have answer to unfortunately.
Spark (Scala) and PySpark differences are significant enough that I think some might choose one vs another. Spark (Java) or even Spark Goovy vs Spark Scala are pretty close (all JVM), so that is more language preferences. Spark and PySpark interpreters are now in the same interpreter group BTW. From: tog Sent: Thursday, July 23, 10:36 PM Subject: Re: Interpreter newbie (may be stupid) question To: [email protected] I agree with that obviously. Does that mean that we could have also python/scala interpreters - not supporting Spark? (Ok I agree that would not make much sense) The point is that Spark is really polyglot - was supporting scala, Python, Java and now R. how are we going to reflect that - taking into account that from a user perspective what matters is the language + the big data/analytics capabilities of spark. At the end, my interest is through the Groovy language to support the Spark Java api in a REPL so should I call that interpreter Groovy, GroovySpark or JavaSpark ? Are PySpark and Spark seen as different interpreters on the long term ? On Friday, July 24, 2015, <[email protected]> wrote: > Spark is really a superset of Scala > > And likewise > > PySpark is a superset of Python > > > It looks to me like they are exceptions though. Groovy, Hive, Flink, Lens > etc are probably fairly different. > > > > > > On Thu, Jul 23, 2015 at 2:42 PM -0700, "tog" <[email protected] > <javascript:;>> wrote: > Hi > > I am questioning myself regarding the groovy interpreter. As I do progress, > it is becoming quite similar to the spark interpreter except that I am > using the Groovy REPL instead of the Scala REPL > > That gives me 2 options: > - have the Groovy interpreter grouped with the Spark one - but that would > become quite fat and I believe user will choose either scala and/or > groovy/java but most probably not both > - keep it separated > > That brings me to the point of the name of the current spark interpreter, > would it be better if called scala interpreter ? the same could apply to > pyspark versus spark ? > > I have the feeling that the language has precedence over spark itself - but > I am not following zeppelin for a long time so please let me know how it > started and please share your view ? > > Cheers > > -- > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > -- PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net
