[ https://issues.apache.org/jira/browse/SPARK-10221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Max Schmidt updated SPARK-10221: -------------------------------- Description: While using a RowReaderFactory out of the Util API here: com.datastax.spark.connector.japi.CassandraJavaUtil.mapRowToTuple(, Class<ByteBuffer>) against a cassandra table with a column which is described as a ByteBuffer get the following stacktrace: {quote} 8786 [task-result-getter-0] ERROR org.apache.spark.scheduler.TaskSetManager - Task 0.0 in stage 0.0 (TID 0) had a not serializable result: java.nio.HeapByteBuffer Serialization stack: - object not serializable (class: java.nio.HeapByteBuffer, value: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) - object (class scala.Tuple4, (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 11:00:23 CEST 2015,76.808)); not retrying Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0.0 in stage 0.0 (TID 0) had a not serializable result: java.nio.HeapByteBuffer Serialization stack: - object not serializable (class: java.nio.HeapByteBuffer, value: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) - object (class scala.Tuple4, (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 11:00:23 CEST 2015,76.808)) at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1273) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1264) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1263) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1263) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:730) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1457) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1418) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) {quote} Using a kind of wrapper-class following bean conventions, doesn't work either. was: While using a RowReaderFactory out of the Util API here: com.datastax.spark.connector.japi.CassandraJavaUtil.mapRowToTuple(, Class<ByteBuffer>) against a cassandra table with a column which is described as a ByteBuffer get the following stacktrace: 8786 [task-result-getter-0] ERROR org.apache.spark.scheduler.TaskSetManager - Task 0.0 in stage 0.0 (TID 0) had a not serializable result: java.nio.HeapByteBuffer Serialization stack: - object not serializable (class: java.nio.HeapByteBuffer, value: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) - object (class scala.Tuple4, (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 11:00:23 CEST 2015,76.808)); not retrying Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0.0 in stage 0.0 (TID 0) had a not serializable result: java.nio.HeapByteBuffer Serialization stack: - object not serializable (class: java.nio.HeapByteBuffer, value: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) - object (class scala.Tuple4, (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 11:00:23 CEST 2015,76.808)) at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1273) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1264) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1263) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1263) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:730) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1457) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1418) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) Using a kind of wrapper-class following bean conventions, doesn't work either. > RowReaderFactory does not work with blobs > ----------------------------------------- > > Key: SPARK-10221 > URL: https://issues.apache.org/jira/browse/SPARK-10221 > Project: Spark > Issue Type: Bug > Reporter: Max Schmidt > > While using a RowReaderFactory out of the Util API here: > com.datastax.spark.connector.japi.CassandraJavaUtil.mapRowToTuple(, > Class<ByteBuffer>) against a cassandra table with a column which is described > as a ByteBuffer get the following stacktrace: > {quote} > 8786 [task-result-getter-0] ERROR org.apache.spark.scheduler.TaskSetManager > - Task 0.0 in stage 0.0 (TID 0) had a not serializable result: > java.nio.HeapByteBuffer > Serialization stack: > - object not serializable (class: java.nio.HeapByteBuffer, value: > java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) > - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) > - object (class scala.Tuple4, > (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 > 11:00:23 CEST 2015,76.808)); not retrying > Exception in thread "main" org.apache.spark.SparkException: Job aborted due > to stage failure: Task 0.0 in stage 0.0 (TID 0) had a not serializable > result: java.nio.HeapByteBuffer > Serialization stack: > - object not serializable (class: java.nio.HeapByteBuffer, value: > java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]) > - field (class: scala.Tuple4, name: _2, type: class java.lang.Object) > - object (class scala.Tuple4, > (/104.130.160.121,java.nio.HeapByteBuffer[pos=0 lim=2 cap=2],Tue Aug 25 > 11:00:23 CEST 2015,76.808)) > at > org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1273) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1264) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1263) > at > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) > at > org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1263) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730) > at scala.Option.foreach(Option.scala:236) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:730) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1457) > at > org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1418) > at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) > {quote} > Using a kind of wrapper-class following bean conventions, doesn't work either. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org