The schema string only needs to comply with the Pig schema grammar, so it's not JsonStorage-specific.
Can you file a jira with a simple example to reproduce your error? That will allow us to verify whether it's a bug or not. On Sun, Dec 15, 2013 at 6:44 AM, Jay Vyas <[email protected]> wrote: > Thanks for the response. > > Still no solution, here is where I am at... > > - I wasn't explicitly specifying a schema when I got this error... > > - I tried changing the generate statements, 1 field ones succeed, but > others failed to allow Json storage due to the same parse schema error. > > - next: How can I know If a schema is valid for json storage or not? From > the docs, I think any combo of primitives, maps, bags, and tuples should be > ok JsonStorage? After all : each maps to a well defined json structure. > > How would i determine if this is a bug in pig? If it is I will file a nice > yummy jira :) > > On Dec 15, 2013, at 1:24 AM, Cheolsoo Park <[email protected]> wrote: > > >>> Syntax error, unexpected symbol at or near ',' > > > > From the stack trace, the QueryParser got an invalid token for the schema > > string, and I believe JsonStorage got into this error because it > implements > > checkSchema()< > https://github.com/apache/pig/blob/trunk/src/org/apache/pig/builtin/JsonStorage.java#L108 > >method > > unlike PigStorage. PigStorage stores everything in plain text, so it > > doesn't need to know any schema. But JsonStorage does. > > > > Now it seems that your invalid schema string is from this > > line< > https://github.com/apache/pig/blob/trunk/src/org/apache/pig/newplan/logical/rules/InputOutputFileValidator.java#L65 > >. > > So my guess is that somehow your store operator is given an invalid > schema > > for "uniqcnt". If you set a breakpoint at this line in IDE, you should be > > able to see what it's like, and you can go from there. It could be a bug > in > > Pig. :-) > > > > > >> On Thu, Dec 12, 2013 at 8:58 PM, Jay Vyas <[email protected]> wrote: > >> > >> Hi pig ! > >> > >> Should JsonStorage support every type of store() that PigStorage > supports? > >> It appears that it breaks in a clear case where PigStorage works... here > >> are the details. > >> > >> I'm able to serialize one of my datasets using PigStorage, but not > >> JsonStorage. > >> > >> <<<< THIS WORKS >>>> > >> > >> *1) pigServer.store("uniqcnt", "/tmp/bbb1", "PigStorage");* > >> > >> results in this file: > >> > >>> cat /tmp/bbb > >> > >> storeCode_AK 4 > >> storeCode_AZ 2 > >> storeCode_CO 7 > >> storeCode_CT 4 > >> storeCode_OK 2 > >> > >> <<<< THIS FAILS >>>> > >> > >> *2) pigServer.store("uniqcnt", "/tmp/bbb2", "JsonStorage");* > >> > >> Results in a stacktrace : > >> > >> When I try with JSonStorage: > >> > >> Failed to parse: <line 1, column 6> Syntax error, unexpected symbol at > or > >> near ',' > >> at > >> > >> > org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:94) > >> at > >> > >> > org.apache.pig.parser.QueryParserDriver.parseSchema(QueryParserDriver.java:108) > >> at org.apache.pig.impl.util.Utils.parseSchema(Utils.java:208) > >> at org.apache.pig.impl.util.Utils.getSchemaFromString(Utils.java:182) > >> at > >> org.apache.pig.builtin.JsonStorage.prepareToWrite(JsonStorage.java:140) > >> at > >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125) > >> at > >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86) > >> at > >> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) > >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) > >> at > >> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) > >> >
