Believe the output type for a TextFileOutput has to be a String, not a Table containing a String key with an Avro Map. With the complex object I think you have to write it out as an AvroFile.
On Thu, Aug 6, 2015 at 11:47 AM Hrishikesh P <[email protected]> wrote: > I'm trying to write a PTable<String, Map<String, String>> to a > TextFileSourceTarget but I am running into a ClassCastException. > > {code} > PTable<String, Map<String, String>> table = ...; > > table.write(new > TextFileSourceTarget<>(Path.mergePaths(fs.getHomeDirectory(), path), > Avros.tableOf(Avros.strings(), Avros.maps(Avros.strings())))); > {code} > > The exception is: > > WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : > java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to > org.apache.avro.generic.IndexedRecord > at > org.apache.crunch.types.avro.AvroTableType$IndexedRecordToPair.map(AvroTableType.java:88) > at org.apache.crunch.MapFn.process(MapFn.java:34) > at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) > at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:109) > at org.apache.crunch.impl.mr.run.CrunchMapper.map(CrunchMapper.java:60) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > > > Does someone have an idea why the exception might be occurring? The > version of avro is 1.7.6. I have checked for dependency conflicts but there > doesn't seem to be a problem with it although it looks like it. > > Thanks. >
