thanks i will look into that
On Mon, Oct 21, 2013 at 4:49 PM, Doug Cutting <[email protected]> wrote: > On Mon, Oct 21, 2013 at 1:19 PM, Koert Kuipers <[email protected]> wrote: > > doug, could it be a classloader (instead of classpath) issue? looking at > > spark it seems to run the tasks inside the slaves/workers with a custom > > classloader. > > Yes, it could be a classloader issue. Perhaps you need to pass the > Spark task's ClassLoader to SpecificData? Find where your > SpecificDatumReader is constructed, and pass it a SpecificData that > was constructed with that ClassLoader. > > > http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificData.html#SpecificData(java.lang.ClassLoader) > > http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificDatumReader.html#SpecificDatumReader(org.apache.avro.specific.SpecificData) > > Doug >
