[
https://issues.apache.org/jira/browse/PIG-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13553352#comment-13553352
]
Joseph Adler commented on PIG-3120:
---
OK, tracked down the issue. It looks like the UDFContextSignature is not
getting propagated if there is a LIMIT statement in the pig code.
Very specifically, in
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.LimitAdjuster.adjust,
it looks like Pig was creating a new POStore object but not copying the
signature. Here is the offending code:
{code}
// this is line 132...
POStore st = new POStore(new
OperatorKey(scope,nig.getNextNodeId(scope)));
st.setSFile(oldSpec);
st.setIsTmpStore(oldIsTmpStore);
st.setSchema(((POStore)mpLeaf).getSchema());
limitAdjustMROp.reducePlan.addAsLeaf(st);
{code}
This is easily fixable by inserting this statement at line 137:
{code}
st.setSignature(((POStore)mpLeaf).getSignature());
{code}
I'll follow up with a path for this issue.
setStoreFuncUDFContextSignature called with null signature
--
Key: PIG-3120
URL: https://issues.apache.org/jira/browse/PIG-3120
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.12
Reporter: Joseph Adler
Priority: Critical
Fix For: 0.12
I'm currently working on PIG-3015 and am having trouble passing the
UDFContextSignature to the store func. It looks like the store func on the
head end is being set to a non-null value, but a null value is being passed
to setStoreFuncUDFContextSignature on the back end. I'm opening this ticket
to track this issue; I'll follow up with a reproducible test case when I have
a clean one.
I suspect this problem occurs when running on a real cluster, but may not
occur in the standard unit tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
