Hello,
Newbie alert :-)
I am trying to get a PCollection of the class which is a POJO generated
from my avro schema. Here is what I have:
AvroMode.setSpecificClassLoader(Event.class.getClassLoader());
PCollection<Event> records =
pipeline.read(From.avroFile("/raw/*.avro", Avros.specifics(Event.class)));
pipeline.materialize(records);
The Event class above is generated from the avro schema. I am getting
the following exception:
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.avro.Schema$Parser.parse(Ljava/lang/String;[Ljava/lang/String;)Lorg/apache/avro/Schema;
at com.tfs.interaction.event.avro.Event.<clinit>(Event.java:10)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
at org.apache.crunch.types.avro.Avros.specifics(Avros.java:267)
*at
com.ilabs.dsi.dataQuality.DataQualityChecker.run(DataQualityChecker.java:50)*
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
com.ilabs.dsi.dataQuality.DataQualityChecker.main(DataQualityChecker.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
The line from where the Exception starts is the second line in the
snippet above. I essentially tried reusing the snippet that is mentioned
here: https://issues.apache.org/jira/browse/CRUNCH-442
When I load the data using the following snippet to Record objects,
things work fine:
Schema eventSchema = new Schema.Parser().parse(new
File(schemaPath));
System.out.println("Loaded IDM schema.");
PCollection<Record> events =
pipeline.read(From.avroFile("/raw/*.avro", Avros.generics(eventSchema)));
What am I doing wrong?
Regards.
