Johannes Schwenk created PIG-3079: ------------------------------------- Summary: AvroStorage fails on STORE if "doc" contains hash character Key: PIG-3079 URL: https://issues.apache.org/jira/browse/PIG-3079 Project: Pig Issue Type: Bug Components: piggybank Affects Versions: 0.9.2 Reporter: Johannes Schwenk
I have the following Avro schema: {code} { "name": "test_log", "namespace": "com.example.avro", "type": "record", "doc": "Some long comment containung # character.", "fields": [ {"name": "id", "type": "long"}, {"name": "field1", "type": "int"} ] } {code} The file {code}test_log.avro{code} contains one record (5673565,123) which I can dump: {code} DEFINE AvroStorage org.apache.pig.piggybank.storage.avro.AvroStorage('same', 'test_log.avro'); data = LOAD 'track_log.avro' USING AvroStorage(); dump data; {code} But when I try to store the data... {code} DEFINE AvroStorage org.apache.pig.piggybank.storage.avro.AvroStorage('same', 'test_log.avro'); data = LOAD 'track_log.avro' USING AvroStorage(); STORE data INTO 'out' USING AvroStorage(); {code} I get the following error: {code} Pig Stack Trace --------------- ERROR 6000: <line 3, column 0> Output Location Validation Failed for: 'hdfs://nameservice1/user/schwenk/out More info to follow: Expect 2 fields in character.","fields":[{"name":"id","type":"long"},{"name":"field1","type":"int"}]} org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias data at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1596) at org.apache.pig.PigServer.registerQuery(PigServer.java:584) at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69) at org.apache.pig.Main.run(Main.java:495) at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:208) Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 6000: <line 3, column 0> Output Location Validation Failed for: 'hdfs://nameservice1/user/schwenk/out More info to follow: Expect 2 fields in character.","fields":[{"name":"id","type":"long"},{"name":"field1","type":"int"}]} at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:95) at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64) at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66) at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53) at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:294) at org.apache.pig.PigServer.compilePp(PigServer.java:1360) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297) at org.apache.pig.PigServer.execute(PigServer.java:1289) at org.apache.pig.PigServer.access$400(PigServer.java:125) at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1591) ... 13 more Caused by: java.io.IOException: Expect 2 fields in character.","fields":[{"name":"id","type":"long"},{"name":"field1","type":"int"}]} at org.apache.pig.piggybank.storage.avro.AvroStorage.parseSchemaMap(AvroStorage.java:567) at org.apache.pig.piggybank.storage.avro.AvroStorage.getOutputFormat(AvroStorage.java:581) at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80) ... 25 more {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira