Re: Spark shell broken

Andrew Palumbo Tue, 24 Feb 2015 14:44:33 -0800

yeah- i was working off of an old branch and couldn't figure out how toget the mahout/spark-1.2 branch. I got it. just double checkingeverything and will push it shortly.


On 02/24/2015 05:32 PM, Dmitriy Lyubimov wrote:

all i am saying if spark side tests work imo there's no need to go thru PR
process, you can push your squashed patch directly, but if you feel like to
do PR please go ahead.


On Tue, Feb 24, 2015 at 2:30 PM, Andrew Palumbo <[email protected]> wrote:

or rather i have [email protected]:andrewpalumbo/mahout.git as my origin and
there's no spark-1.2 branch there

On 02/24/2015 05:25 PM, Andrew Palumbo wrote:

sorry- trying to get the shell fix out but having trouble pulling the
remote spark 1.2 branch. i think its cause i have git.apache.org as my
remote rather than github.com/apache/mahout


On 02/24/2015 05:15 PM, Andrew Musselman wrote:

Makes sense; I'm still getting those errors after restarting my rebuilt
spark..

On Tue, Feb 24, 2015 at 2:12 PM, Dmitriy Lyubimov <[email protected]>
wrote:

  IIRC MAHOUT_LOCAL doesn't mean a thing with spark mode. It is purely MR

thing.

with shell, important things are MASTER (spark setting) and also other
spark specific settings  that should be added in MAHOUT_OPT (but the
latter
is broken in public branch with spark shell -- one of the bugs that need
fixing).

On Tue, Feb 24, 2015 at 2:03 PM, Andrew Musselman <
[email protected]> wrote:

  Ah, I had an old build of spark server running.

On Tue, Feb 24, 2015 at 1:56 PM, Andrew Musselman <
[email protected]> wrote:

  I only have one spark build that I know of, and it's still reporting

errors in Mahout local mode.

$ echo $SPARK_HOME
/home/akm/spark

$ export MAHOUT_LOCAL="TRUE"
$ bin/mahout spark-shell
MAHOUT_LOCAL is set, so we don't add HADOOP_CONF_DIR to classpath.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in

  [jar:file:/home/akm/mahout/mrlegacy/target/mahout-

mrlegacy-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

  [jar:file:/home/akm/mahout/spark/target/mahout-spark_2.

10-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

  [jar:file:/home/akm/spark/assembly/target/scala-2.10/

spark-assembly-1.1.1-hadoop1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an

explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

                           _                 _
           _ __ ___   __ _| |__   ___  _   _| |_
          | '_ ` _ \ / _` | '_ \ / _ \| | | | __|
          | | | | | | (_| | | | | (_) | |_| | |_
          |_| |_| |_|\__,_|_| |_|\___/ \__,_|\__|  version 1.0


Using Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_03)
Type in expressions to have them evaluated.
Type :help for more information.
15/02/24 13:42:42 WARN Utils: Your hostname, ubuntu resolves to a

loopback

address: 127.0.1.1; using 5.5.8.1 instead (on interface as0t1)
15/02/24 13:42:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind
to
another address
15/02/24 13:42:54 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where

applicable
Created spark context..

Mahout distributed context is available as "implicit val sdc".
mahout> 15/02/24 13:43:32 ERROR Remoting:
org.apache.spark.storage.BlockManagerId; local class incompatible:

stream
classdesc serialVersionUID = 1677335532749418220, local class

serialVersionUID = -7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.
BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:44:05 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:44:38 ERROR TaskSchedulerImpl: Lost an executor 0 (already

removed): remote Akka client disassociated
15/02/24 13:45:15 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:45:48 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:46:21 ERROR TaskSchedulerImpl: Lost an executor 1 (already

removed): remote Akka client disassociated
15/02/24 13:46:57 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:47:30 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:48:04 ERROR TaskSchedulerImpl: Lost an executor 2 (already

removed): remote Akka client disassociated
15/02/24 13:48:40 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:49:14 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:49:47 ERROR TaskSchedulerImpl: Lost an executor 3 (already

removed): remote Akka client disassociated
15/02/24 13:50:23 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:50:56 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:51:30 ERROR TaskSchedulerImpl: Lost an executor 4 (already

removed): remote Akka client disassociated
15/02/24 13:52:07 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

  java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.

java:369)
      at

  akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

      at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
      at

  
akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

      at scala.util.Try$.apply(Try.scala:161)

at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

      at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

      at

  akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

      at
akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)
      at akka.remote.DefaultMessageDispatcher.

dispatch(Endpoint.scala:73)
      at

  akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

      at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)

      at akka.actor.ActorCell.invoke(ActorCell.scala:456)
      at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
      at akka.dispatch.Mailbox.run(Mailbox.scala:219)
      at

  
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

      at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

      at

  scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.

runTask(ForkJoinPool.java:1339)

at

  scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

  
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:52:40 ERROR Remoting:
org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.

BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729
      at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
      at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

      at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

      at

  java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

      at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at

Re: Spark shell broken

Reply via email to