Actually apparently there is a pull request for it. Thanks for reporting!
https://github.com/apache/spark/pull/1836
On Fri, Aug 8, 2014 at 10:50 AM, Ron Gonzalez wrote:
> Sure let me give it a try. Any tips? I've only started looking at Spark
> code more closely recently.
> I can compare Spar
Sure let me give it a try. Any tips? I've only started looking at Spark code
more closely recently.
I can compare Spark-1.0.1 code and see what's going on...
Thanks,
Ron
On Friday, August 8, 2014 10:43 AM, Reynold Xin wrote:
I created a JIRA ticket to track this:
https://issues.apache.org
I created a JIRA ticket to track this:
https://issues.apache.org/jira/browse/SPARK-2928
Let me know if you need help with it.
On Fri, Aug 8, 2014 at 10:40 AM, Reynold Xin wrote:
> Yes, I'm pretty sure it doesn't actually use the right serializer in
> TorrentBroadcast:
> https://github.com/apa
Yes, I'm pretty sure it doesn't actually use the right serializer in
TorrentBroadcast:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L232
And TorrentBroadcast is turned on by default for 1.1 right now. Do you want
to submit a pull
Oops, exception is below.
For local, it works and that's the case since TorrentBroadcast has if !isLocal,
then that's the only time the broadcast actually happens. It really seems as if
the Kryo wrapper didn't kick in for some reason. Do we have a unit test that
tests the Kryo serialization that
Pasting a better formatted trace:
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at
scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:137)
at
scala.collection.mutable.HashMa
Looks like you didn't actually paste the exception message. Do you mind
doing that?
On Fri, Aug 8, 2014 at 10:14 AM, Reynold Xin wrote:
> Pasting a better formatted trace:
>
>
>
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
> at java.io.ObjectOutputStream.writeObje
Hi,
I have a running spark app against the released version of 1.0.1. I recently
decided to try and upgrade to the trunk version. Interestingly enough, after
building the 1.1.0-SNAPSHOT assembly, replacing it as my assembly in my app
caused errors. In particular, it seems Kryo serialization isn'
