Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/15#issuecomment-38772376
By the way - for a while I was really confused how your original patch
actually fixed this problem. I eventually realized that the issue is that the
`DAGScheduler` itself is an actor, so in this case where the `Executor` was
running in the same JVM, also within an actor, it was resetting the class
loader in the thread that the DAGScheduler runs in. So the DAGScheduler got a
`Foo` object back from the task and it tried to put it into an array of `Foo`
objects, but when deserializing the `Foo` from the task it was using the
`replClassLoader` back from the `Executor` which, because delegation was not
properly set-up, had defined `Foo` for itself.
Maybe this is what you meant earlier - it took me some effort to piece it
all together.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
