Re: Spark shell broken

Andrew Palumbo Tue, 24 Feb 2015 14:26:55 -0800

sorry- trying to get the shell fix out but having trouble pulling theremote spark 1.2 branch. i think its cause i have git.apache.org as myremote rather than github.com/apache/mahout


On 02/24/2015 05:15 PM, Andrew Musselman wrote:

Makes sense; I'm still getting those errors after restarting my rebuilt
spark..

On Tue, Feb 24, 2015 at 2:12 PM, Dmitriy Lyubimov <[email protected]> wrote:

IIRC MAHOUT_LOCAL doesn't mean a thing with spark mode. It is purely MR
thing.

with shell, important things are MASTER (spark setting) and also other
spark specific settings  that should be added in MAHOUT_OPT (but the latter
is broken in public branch with spark shell -- one of the bugs that need
fixing).

On Tue, Feb 24, 2015 at 2:03 PM, Andrew Musselman <
[email protected]> wrote:

Ah, I had an old build of spark server running.

On Tue, Feb 24, 2015 at 1:56 PM, Andrew Musselman <
[email protected]> wrote:

I only have one spark build that I know of, and it's still reporting
errors in Mahout local mode.

$ echo $SPARK_HOME
/home/akm/spark

$ export MAHOUT_LOCAL="TRUE"
$ bin/mahout spark-shell
MAHOUT_LOCAL is set, so we don't add HADOOP_CONF_DIR to classpath.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in

[jar:file:/home/akm/mahout/mrlegacy/target/mahout-mrlegacy-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

[jar:file:/home/akm/mahout/spark/target/mahout-spark_2.10-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

[jar:file:/home/akm/spark/assembly/target/scala-2.10/spark-assembly-1.1.1-hadoop1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

                          _                 _
          _ __ ___   __ _| |__   ___  _   _| |_
         | '_ ` _ \ / _` | '_ \ / _ \| | | | __|
         | | | | | | (_| | | | | (_) | |_| | |_
         |_| |_| |_|\__,_|_| |_|\___/ \__,_|\__|  version 1.0


Using Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_03)
Type in expressions to have them evaluated.
Type :help for more information.
15/02/24 13:42:42 WARN Utils: Your hostname, ubuntu resolves to a

loopback

address: 127.0.1.1; using 5.5.8.1 instead (on interface as0t1)
15/02/24 13:42:42 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to
another address
15/02/24 13:42:54 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where

applicable

Created spark context..
Mahout distributed context is available as "implicit val sdc".
mahout> 15/02/24 13:43:32 ERROR Remoting:
org.apache.spark.storage.BlockManagerId; local class incompatible:

stream

classdesc serialVersionUID = 1677335532749418220, local class
serialVersionUID = -7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:44:05 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:44:38 ERROR TaskSchedulerImpl: Lost an executor 0 (already
removed): remote Akka client disassociated
15/02/24 13:45:15 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:45:48 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:46:21 ERROR TaskSchedulerImpl: Lost an executor 1 (already
removed): remote Akka client disassociated
15/02/24 13:46:57 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:47:30 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:48:04 ERROR TaskSchedulerImpl: Lost an executor 2 (already
removed): remote Akka client disassociated
15/02/24 13:48:40 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:49:14 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:49:47 ERROR TaskSchedulerImpl: Lost an executor 3 (already
removed): remote Akka client disassociated
15/02/24 13:50:23 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:50:56 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:51:30 ERROR TaskSchedulerImpl: Lost an executor 4 (already
removed): remote Akka client disassociated
15/02/24 13:52:07 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:52:40 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 13:53:13 ERROR TaskSchedulerImpl: Lost an executor 5 (already
removed): remote Akka client disassociated

mahout>
mahout>
mahout> 15/02/24 13:53:49 ERROR Remoting:
org.apache.spark.storage.BlockManagerId; local class incompatible:

stream

classdesc serialVersionUID = 1677335532749418220, local class
serialVersionUID = -7366074099953117729
java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId;
local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

     at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
     at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

     at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
     at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

     at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
     at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
     at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

     at
akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
     at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

     at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)
     at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

On Tue, Feb 24, 2015 at 1:08 PM, Dmitriy Lyubimov <[email protected]>
wrote:

seems like different builds on client and backend.

shell is using your local spark setup (pointed to with SPARK_HOME).

make

sure it points to identical binaries (not just spark version) to what

is

used in the backend.

the reason is spark is not binary-canonical w.r.t. to release version,

it

implies custom builds, perhaps jvm bytecode levels, hadoop

dependencies

etc. etc.

On Tue, Feb 24, 2015 at 12:59 PM, Andrew Musselman <
[email protected]> wrote:

Except after some time it blew up:

$ bin/mahout spark-shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in

[jar:file:/home/akm/mahout/mrlegacy/target/mahout-mrlegacy-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

[jar:file:/home/akm/mahout/spark/target/mahout-spark_2.10-1.0-SNAPSHOT-job.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: Found binding in

[jar:file:/home/akm/spark/assembly/target/scala-2.10/spark-assembly-1.1.1-hadoop1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

                          _                 _
          _ __ ___   __ _| |__   ___  _   _| |_
         | '_ ` _ \ / _` | '_ \ / _ \| | | | __|
         | | | | | | (_| | | | | (_) | |_| | |_
         |_| |_| |_|\__,_|_| |_|\___/ \__,_|\__|  version 1.0


Using Scala version 2.10.0 (Java HotSpot(TM) 64-Bit Server VM, Java
1.7.0_03)
Type in expressions to have them evaluated.
Type :help for more information.
15/02/24 12:23:26 WARN Utils: Your hostname, ubuntu resolves to a

loopback

address: 127.0.1.1; using 5.5.8.1 instead (on interface as0t1)
15/02/24 12:23:26 WARN Utils: Set SPARK_LOCAL_IP if you need to bind

to

another address
15/02/24 12:23:39 WARN NativeCodeLoader: Unable to load

native-hadoop

library for your platform... using builtin-java classes where

applicable

Created spark context..
Mahout distributed context is available as "implicit val sdc".
mahout> 15/02/24 12:24:17 ERROR Remoting:
org.apache.spark.storage.BlockManagerId; local class incompatible:

stream

classdesc serialVersionUID = 1677335532749418220, local class
serialVersionUID = -7366074099953117729
java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:24:50 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:25:23 ERROR TaskSchedulerImpl: Lost an executor 0

(already

removed): remote Akka client disassociated
15/02/24 12:26:00 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:26:33 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:27:06 ERROR TaskSchedulerImpl: Lost an executor 1

(already

removed): remote Akka client disassociated
15/02/24 12:27:43 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:28:16 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:28:49 ERROR TaskSchedulerImpl: Lost an executor 2

(already

removed): remote Akka client disassociated
15/02/24 12:29:26 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:29:59 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:30:32 ERROR TaskSchedulerImpl: Lost an executor 3

(already

removed): remote Akka client disassociated
15/02/24 12:31:09 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:31:42 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:32:15 ERROR TaskSchedulerImpl: Lost an executor 4

(already

removed): remote Akka client disassociated
15/02/24 12:32:51 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:33:24 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:33:58 ERROR TaskSchedulerImpl: Lost an executor 5

(already

removed): remote Akka client disassociated
15/02/24 12:34:34 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:35:07 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:35:41 ERROR TaskSchedulerImpl: Lost an executor 6

(already

removed): remote Akka client disassociated
15/02/24 12:36:17 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:36:50 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:37:23 ERROR TaskSchedulerImpl: Lost an executor 7

(already

removed): remote Akka client disassociated
15/02/24 12:38:00 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:38:33 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:39:06 ERROR TaskSchedulerImpl: Lost an executor 8

(already

removed): remote Akka client disassociated
15/02/24 12:39:43 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:40:16 ERROR Remoting:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

java.io.InvalidClassException:

org.apache.spark.storage.BlockManagerId;

local class incompatible: stream classdesc serialVersionUID =
1677335532749418220, local class serialVersionUID =

-7366074099953117729

at

java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)

at

java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)

at

java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)

at

java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)

at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)

at

java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

at

java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)

at

akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)

at

scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)

at

akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)

at

akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)

     at scala.util.Try$.apply(Try.scala:161)
     at

akka.serialization.Serialization.deserialize(Serialization.scala:98)

at

akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)

at

akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:55)

at

akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:73)

at

akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:764)

     at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
     at akka.actor.ActorCell.invoke(ActorCell.scala:456)
     at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
     at akka.dispatch.Mailbox.run(Mailbox.scala:219)
     at

akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)

at

scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at

scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at

scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at

scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

15/02/24 12:40:49 ERROR TaskSchedulerImpl: Lost an executor 9

(already

removed): remote Akka client disassociated
15/02/24 12:40:49 ERROR SparkDeploySchedulerBackend: Application has

been

killed. Reason: Master removed our application: FAILED
15/02/24 12:40:49 ERROR TaskSchedulerImpl: Exiting due to error from
cluster scheduler: Master removed our application: FAILED


On Tue, Feb 24, 2015 at 12:24 PM, Andrew Musselman <
[email protected]> wrote:

Bingo, works off master now; thanks Dmitriy.

On Tue, Feb 24, 2015 at 11:54 AM, Dmitriy Lyubimov <

[email protected]

wrote:

ok spark 1.2 is mirrored now.
and master should be also ok (back to 1.1)

On Tue, Feb 24, 2015 at 11:53 AM, Andrew Musselman <
[email protected]> wrote:

I reset hard to the previous commit, created a branch, and

pushed

it.

I got email confirmation like so:
Repository: mahout
Updated Branches:
   refs/heads/spark-1.2 [created] 901ef03b4

On Tue, Feb 24, 2015 at 11:47 AM, Dmitriy Lyubimov <

[email protected]

wrote:

yeah ok so you pushed 1.2 branch to asf but it is not yet in

github.

iti

should be there eventually, give it a bit of time.

On Tue, Feb 24, 2015 at 11:35 AM, Dmitriy Lyubimov <

[email protected]

wrote:

what exactly did you try to do?

just resetting HEAD will not work on remote branch -- you

need

force-sync

that (with +) since it is a history-rewriting push, but asf

git

does

not

allow that.

ASF will mirror ALL branches afaik. I think i've done it

before.

so

if

you

create a new one it should (eventually) get there.

On Tue, Feb 24, 2015 at 11:18 AM, Andrew Musselman <
[email protected]> wrote:

Does ASF git get mirrored to GitHub?  I tried pushing a

branch

and

don't

see it there yet.

On Tue, Feb 24, 2015 at 11:16 AM, Dmitriy Lyubimov <

[email protected]

wrote:

On Tue, Feb 24, 2015 at 10:55 AM, Pat Ferrel <

[email protected]

wrote:

to be safe I’d “git reset —hard xyz” to the commit

previous

to

the

1.2.1

As i just explained, that resets are not possible with

ASF

git.

Reverting

is the only option.

-d

Re: Spark shell broken

Reply via email to