Hi Hrishikesh, This should work (or at least, I don't know of a reason why it shouldn't work). I'm having a hard time replicating your problem though -- in all the various setups where I've tried to replicate this, it works fine for me.
Which version of Crunch are you using? Could you put together a small (but complete) test class that demonstrates this problem? - Gabriel On Thu, Aug 6, 2015 at 5:51 PM David Ortiz <[email protected]> wrote: > Believe the output type for a TextFileOutput has to be a String, not a > Table containing a String key with an Avro Map. With the complex object I > think you have to write it out as an AvroFile. > > On Thu, Aug 6, 2015 at 11:47 AM Hrishikesh P <[email protected]> > wrote: > >> I'm trying to write a PTable<String, Map<String, String>> to a >> TextFileSourceTarget but I am running into a ClassCastException. >> >> {code} >> PTable<String, Map<String, String>> table = ...; >> >> table.write(new >> TextFileSourceTarget<>(Path.mergePaths(fs.getHomeDirectory(), path), >> Avros.tableOf(Avros.strings(), Avros.maps(Avros.strings())))); >> {code} >> >> The exception is: >> >> WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : >> java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to >> org.apache.avro.generic.IndexedRecord >> at >> org.apache.crunch.types.avro.AvroTableType$IndexedRecordToPair.map(AvroTableType.java:88) >> at org.apache.crunch.MapFn.process(MapFn.java:34) >> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) >> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:109) >> at org.apache.crunch.impl.mr.run.CrunchMapper.map(CrunchMapper.java:60) >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:422) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) >> >> >> Does someone have an idea why the exception might be occurring? The >> version of avro is 1.7.6. I have checked for dependency conflicts but there >> doesn't seem to be a problem with it although it looks like it. >> >> Thanks. >> >
