Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3884#issuecomment-71292584
@vanzin Thanks for looking this over. The Python `RDD` objects themselves
are never actually serialized and are used internally in a way that's slightly
different than in Scala/Java Spark. In the existing code, any attempt to
serialize instances of those Python classes throws an exception in the
`__getnewargs__` method, which is why I was able to add new exceptions there.
I'm going to fix the spelling error, take one final look over this, and
commit it so we can get it into the first 1.2.1 RC. I saw a couple of mailing
list questions yesterday that could have been prevented by this patch, which
illustrates why I really want to get this into our next maintenance release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
