Well, uber jar works in YARN, but not with standalone ;)

On Sun, Sep 18, 2016 at 12:44 PM -0700, "Chris Fregly" 
<ch...@fregly.com<mailto:ch...@fregly.com>> wrote:

you'll see errors like this...

"java.lang.RuntimeException: java.io.InvalidClassException: 
org.apache.spark.rpc.netty.RequestMessage; local class incompatible: stream 
classdesc serialVersionUID = -2221986757032131007, local class serialVersionUID 
= -5447855329526097695"

...when mixing versions of spark.

i'm actually seeing this right now while testing across Spark 1.6.1 and Spark 
2.0.1 for my all-in-one, hybrid cloud/on-premise Spark + Zeppelin + Kafka + 
Kubernetes + Docker + One-Click Spark ML Model Production Deployments 
initiative documented here:


and check out my upcoming meetup on this effort either in-person or online:


we're throwing in some GPU/CUDA just to sweeten the offering!  :)

On Sat, Sep 10, 2016 at 2:57 PM, Holden Karau 
<hol...@pigscanfly.ca<mailto:hol...@pigscanfly.ca>> wrote:
I don't think a 2.0 uber jar will play nicely on a 1.5 standalone cluster.

On Saturday, September 10, 2016, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
You should be able to get it to work with 2.0 as uber jar.

What type cluster you are running on? YARN? And what distribution?

On Sun, Sep 4, 2016 at 8:48 PM -0700, "Holden Karau" <hol...@pigscanfly.ca> 

You really shouldn't mix different versions of Spark between the master and 
worker nodes, if your going to upgrade - upgrade all of them. Otherwise you may 
get very confusing failures.

On Monday, September 5, 2016, Rex X <dnsr...@gmail.com> wrote:
Wish to use the Pivot Table feature of data frame which is available since 
Spark 1.6. But the spark of current cluster is version 1.5. Can we install 
Spark 2.0 on the master node to work around this?


Cell : 425-233-8271<tel:425-233-8271>
Twitter: https://twitter.com/holdenkarau

Cell : 425-233-8271<tel:425-233-8271>
Twitter: https://twitter.com/holdenkarau

Chris Fregly
Research Scientist @ PipelineIO<http://pipeline.io>
Advanced Spark and TensorFlow 
San Francisco | Chicago | Washington DC

Reply via email to