well, here is an attempt. https://issues.apache.org/jira/browse/PIG-3627
Does that help? On Sun, Dec 15, 2013 at 6:24 PM, Cheolsoo Park <[email protected]> wrote: > The schema string only needs to comply with the Pig schema grammar, so it's > not JsonStorage-specific. > > Can you file a jira with a simple example to reproduce your error? That > will allow us to verify whether it's a bug or not. > > > On Sun, Dec 15, 2013 at 6:44 AM, Jay Vyas <[email protected]> wrote: > > > Thanks for the response. > > > > Still no solution, here is where I am at... > > > > - I wasn't explicitly specifying a schema when I got this error... > > > > - I tried changing the generate statements, 1 field ones succeed, but > > others failed to allow Json storage due to the same parse schema error. > > > > - next: How can I know If a schema is valid for json storage or not? > From > > the docs, I think any combo of primitives, maps, bags, and tuples should > be > > ok JsonStorage? After all : each maps to a well defined json structure. > > > > How would i determine if this is a bug in pig? If it is I will file a > nice > > yummy jira :) > > > > On Dec 15, 2013, at 1:24 AM, Cheolsoo Park <[email protected]> wrote: > > > > >>> Syntax error, unexpected symbol at or near ',' > > > > > > From the stack trace, the QueryParser got an invalid token for the > schema > > > string, and I believe JsonStorage got into this error because it > > implements > > > checkSchema()< > > > https://github.com/apache/pig/blob/trunk/src/org/apache/pig/builtin/JsonStorage.java#L108 > > >method > > > unlike PigStorage. PigStorage stores everything in plain text, so it > > > doesn't need to know any schema. But JsonStorage does. > > > > > > Now it seems that your invalid schema string is from this > > > line< > > > https://github.com/apache/pig/blob/trunk/src/org/apache/pig/newplan/logical/rules/InputOutputFileValidator.java#L65 > > >. > > > So my guess is that somehow your store operator is given an invalid > > schema > > > for "uniqcnt". If you set a breakpoint at this line in IDE, you should > be > > > able to see what it's like, and you can go from there. It could be a > bug > > in > > > Pig. :-) > > > > > > > > >> On Thu, Dec 12, 2013 at 8:58 PM, Jay Vyas <[email protected]> > wrote: > > >> > > >> Hi pig ! > > >> > > >> Should JsonStorage support every type of store() that PigStorage > > supports? > > >> It appears that it breaks in a clear case where PigStorage works... > here > > >> are the details. > > >> > > >> I'm able to serialize one of my datasets using PigStorage, but not > > >> JsonStorage. > > >> > > >> <<<< THIS WORKS >>>> > > >> > > >> *1) pigServer.store("uniqcnt", "/tmp/bbb1", "PigStorage");* > > >> > > >> results in this file: > > >> > > >>> cat /tmp/bbb > > >> > > >> storeCode_AK 4 > > >> storeCode_AZ 2 > > >> storeCode_CO 7 > > >> storeCode_CT 4 > > >> storeCode_OK 2 > > >> > > >> <<<< THIS FAILS >>>> > > >> > > >> *2) pigServer.store("uniqcnt", "/tmp/bbb2", "JsonStorage");* > > >> > > >> Results in a stacktrace : > > >> > > >> When I try with JSonStorage: > > >> > > >> Failed to parse: <line 1, column 6> Syntax error, unexpected symbol > at > > or > > >> near ',' > > >> at > > >> > > >> > > > org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:94) > > >> at > > >> > > >> > > > org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:108) > > >> at org.apache.pig.impl.util.Utils.parseSchema(Utils.java:208) > > >> at > org.apache.pig.impl.util.Utils.getSchemaFromString(Utils.java:182) > > >> at > > >> > org.apache.pig.builtin.JsonStorage.prepareToWrite(JsonStorage.java:140) > > >> at > > >> > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125) > > >> at > > >> > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86) > > >> at > > >> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) > > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) > > >> at > > >> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) > > >> > > > -- Jay Vyas http://jayunit100.blogspot.com
