[
https://issues.apache.org/jira/browse/PIG-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247024#comment-15247024
]
Xianda Ke commented on PIG-4876:
--------------------------------
the expected result of D:
{code}
((1,1),{((1,1),{(1,1)},{(1,1)})})
((1,2),{((1,2),{(1,2)},{(1,2)})})
((1,3),{((1,3),{(1,3)},{(1,3)})})
((2,1),{((2,1),{(2,1)},{(2,1)})})
((2,2),{((2,2),{(2,2)},{(2,2)})})
((2,3),{((2,3),{(2,3)},{(2,3)})})
((3,1),{((3,1),{(3,1)},{(3,1)})})
((3,2),{((3,2),{(3,2)},{(3,2)})})
((3,3),{((3,3),{(3,3)},{(3,3)})})
{code}
> OutputConsumeIterator can't handle the last buffered tuples for some Operators
> ------------------------------------------------------------------------------
>
> Key: PIG-4876
> URL: https://issues.apache.org/jira/browse/PIG-4876
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Xianda Ke
> Assignee: Xianda Ke
> Fix For: spark-branch
>
>
> Some Operators, such as MergeCogroup, Stream, CollectedGroup etc buffer some
> input records to constitute the result tuples. The last result tuples are
> buffered in the operator. These Operators need a flag to indicate the end of
> input, so that they can flush and constitute their last tuples.
> Currently, the flag 'parentPlan.endOfAllInput' is targeted for flushing the
> buffered tuples in MR mode. But it does not work with OutputConsumeIterator
> in Spark mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
