[
https://issues.apache.org/jira/browse/PIG-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478709#comment-13478709
]
Cheolsoo Park commented on PIG-2978:
------------------------------------
Here is the difference between hadoop-1.0.x and 2.0.x:
{code:title=hadoop-1.0.x}
Storer[3].<init>()
Storer[3].setStoreFuncUDFContextSignature(A_1-1)
Storer[3].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
Storer[3].getOutputFormat()
Storer[3].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
{code}
{code:title=hadoop-2.0.x}
Storer[3].<init>()
Storer[3].setStoreFuncUDFContextSignature(A_1-1)
Storer[3].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
Storer[3].getOutputFormat()
Storer[3].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
Storer[4].<init>()
Storer[4].setStoreFuncUDFContextSignature(A_1-1)
Storer[4].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
Storer[4].getOutputFormat()
Storer[4].setStoreLocation(bar, org.apache.hadoop.mapreduce.Job)
{code}
For whatever reason, getStoreFunc is repeated with hadoop-2.0.x. The call stack
of the extra 4th instantiation is below:
{code}
Storer[4].<init> called by
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:577)
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:85)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:67)
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
{code}
> TestLoadStoreFuncLifeCycle fails with hadoop-2.0.x
> --------------------------------------------------
>
> Key: PIG-2978
> URL: https://issues.apache.org/jira/browse/PIG-2978
> Project: Pig
> Issue Type: Sub-task
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.11
>
> Attachments: PIG-2978.patch
>
>
> To reproduce, please run:
> {code}
> ant clean test -Dtestcase=TestLoadStoreFuncLifeCycle -Dhadoopversion=23
> {code}
> This fails with the following error:
> {code}
> Error during parsing. Job in state DEFINE instead of RUNNING
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during
> parsing. Job in state DEFINE instead of RUNNING
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:529)
> at
> org.apache.pig.TestLoadStoreFuncLifeCycle.testLoadStoreFunc(TestLoadStoreFuncLifeCycle.java:332)
> Caused by: Failed to parse: Job in state DEFINE instead of RUNNING
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:193)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
> Caused by: java.lang.IllegalStateException: Job in state DEFINE instead of
> RUNNING
> at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:292)
> at org.apache.hadoop.mapreduce.Job.toString(Job.java:456)
> at java.lang.String.valueOf(String.java:2826)
> at
> org.apache.pig.TestLoadStoreFuncLifeCycle.logCaller(TestLoadStoreFuncLifeCycle.java:270)
> at
> org.apache.pig.TestLoadStoreFuncLifeCycle.access$000(TestLoadStoreFuncLifeCycle.java:41)
> at
> org.apache.pig.TestLoadStoreFuncLifeCycle$InstrumentedStorage.logCaller(TestLoadStoreFuncLifeCycle.java:54)
> at
> org.apache.pig.TestLoadStoreFuncLifeCycle$InstrumentedStorage.getSchema(TestLoadStoreFuncLifeCycle.java:115)
> at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:174)
> at org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:88)
> at
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:839)
> at
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
> at
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
> at
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
> at
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
> at
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
