Github user arahuja commented on the pull request:
https://github.com/apache/spark/pull/1890#issuecomment-52446389
I think this may be the issue I have been wrangling with the last couple
days. I see a variety of odd Kryo related errors, slightly different each
time:
14/08/17 22:52:01 ERROR Executor: Exception in task ID 17061
com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID:
12763
at com.esotericsoftware.kryo.util.DefaultClassResolver.readCl
14/08/17 22:52:01 ERROR Executor: Exception in task ID 17051
java.lang.IndexOutOfBoundsException: Index: 5927, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
Looking through the executor logs I do see
14/08/17 22:52:00 ERROR KryoSerializer: Failed to run spark.kryo.registrator
java.lang.ClassNotFoundException:
org.bdgenomics.guacamole.GuacamoleKryoRegistrator
Also, Sandy we are running on YARN and still seem to see this, is there a
workaround you know of? Or is there any known workaround in general?
Thanks,
Arun
On Sat, Aug 16, 2014 at 6:05 PM, Patrick Wendell <[email protected]>
wrote:
> Hey @GrahamDennis <https://github.com/GrahamDennis> thanks for an
> extremely thorough analysis of this issue here and on the JIRA. I think
> that @rxin <https://github.com/rxin> was able to solve this in a PR that
> improves the way we deal with passing classloaders to our serializers. Do
> you mind trying that fix (which has now been merged) and seeing if it
fixes
> your issue?
>
> #1972 <https://github.com/apache/spark/pull/1972>
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1890#issuecomment-52407167>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
