Sure let me give it a try. Any tips? I've only started looking at Spark code more closely recently. I can compare Spark-1.0.1 code and see what's going on...
Thanks, Ron On Friday, August 8, 2014 10:43 AM, Reynold Xin <r...@databricks.com> wrote: I created a JIRA ticket to track this: https://issues.apache.org/jira/browse/SPARK-2928 Let me know if you need help with it. On Fri, Aug 8, 2014 at 10:40 AM, Reynold Xin <r...@databricks.com> wrote: Yes, I'm pretty sure it doesn't actually use the right serializer in TorrentBroadcast: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L232 > > >And TorrentBroadcast is turned on by default for 1.1 right now. Do you want to >submit a pull request to fix that? This would be a critical fix for 1.1 that's >worth doing. > > > > > >On Fri, Aug 8, 2014 at 10:37 AM, Ron Gonzalez <zlgonza...@yahoo.com> wrote: > >Oops, exception is below. >>For local, it works and that's the case since TorrentBroadcast has if >>!isLocal, then that's the only time the broadcast actually happens. It really >>seems as if the Kryo wrapper didn't kick in for some reason. Do we have a >>unit test that tests the Kryo serialization that I can give a try? >>Thanks, >>Ron >> >> >>Exception in thread "Driver" java.lang.reflect.InvocationTargetException at >>sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>at >>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) at >>org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:180) Caused by: java.io.NotSerializableException: org.apache.avro.generic.GenericData$Record - custom writeObject data (class "scala.collection.mutable.HashMap") >> >> >> >>On Friday, August 8, 2014 10:16 AM, Reynold Xin <r...@databricks.com> wrote: >> >> >> >>Looks like you didn't actually paste the exception message. Do you mind >>doing that? >> >> >> >> >>On Fri, Aug 8, 2014 at 10:14 AM, Reynold Xin <r...@databricks.com> wrote: >> >>> Pasting a better formatted trace: >>> >>> >>> >>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180) >>> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) >>> at >>> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:137) >>> at >>> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:135) >>> at >>> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) >>> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) >>> at >>> scala.collection.mutable.HashTable$class.serializeTo(HashTable.scala:124) >>> at scala.collection.mutable.HashMap.serializeTo(HashMap.scala:39) >>> at scala.collection.mutable.HashMap.writeObject(HashMap.scala:135) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) at >>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) >>> at >>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) >>> at >>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) >>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) >>> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) >>> at org.apache.spark.util.Utils$.serialize(Utils.scala:64) >>> at >>> org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:232) >>> at >>> org.apache.spark.broadcast.TorrentBroadcast.sendBroadcast(TorrentBroadcast.scala:85) >>> at >>> org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:66) >>> at >>> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:36) >>> at >>> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:29) >>> at >>> >>> >>>org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62) >>> at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809) >>> >>> >>> On Fri, Aug 8, 2014 at 10:12 AM, Ron Gonzalez < >>> zlgonza...@yahoo.com.invalid> wrote: >>> >>>> Hi, >>>> I have a running spark app against the released version of 1.0.1. I >>>> recently decided to try and upgrade to the trunk version. Interestingly >>>> enough, after building the 1.1.0-SNAPSHOT assembly, replacing it as my >>>> assembly in my app caused errors. In particular, it seems Kryo >>>> serialization isn't taking. Replacing it with 1.0.1 automatically gets it >>>> working again. >>>> >>>> Any thoughts? Is this a known issue? >>>> >>>> Thanks, >>>> Ron >>>> >>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180) >>>> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at >>>> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:137) >>>> at >>>> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:135) >>>> at >>>> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) >>>> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) at >>>> scala.collection.mutable.HashTable$class.serializeTo(HashTable.scala:124) >>>> at scala.collection.mutable.HashMap.serializeTo(HashMap.scala:39) at >>>> scala.collection.mutable.HashMap.writeObject(HashMap.scala:135) at >>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:606) at >>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) >>>> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) >>>> at >>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) >>>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at >>>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at >>>> org.apache.spark.util.Utils$.serialize(Utils.scala:64) at >>>> org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:232) >>>> at >>>> org.apache.spark.broadcast.TorrentBroadcast.sendBroadcast(TorrentBroadcast.scala:85) >>>> at >>>> org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:66) >>>> at >>>> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:36) >>>> at >>>> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:29) >>>> at >>>> >>>>org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62) >>>> at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809) >>> >>> >>> >> >> >> >
