[
https://issues.apache.org/jira/browse/PIG-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944263#comment-14944263
]
Rohini Palaniswamy edited comment on PIG-3957 at 10/27/15 9:08 PM:
-------------------------------------------------------------------
Only difference between testFRJoinOut8 and testFRJoinOut9 was 'replicated' vs
'repl'. So repurposed testFRJoinOut9 for adding test.
The code in TezDAGBuilder put in for scalars was not needed any more. So
totally removed it. TestScalarAlias, Scalar and CastScalar e2e tests are fine.
Running the full e2e suite now.
Additionally optimized the Scalar more than one row checking by doing that
while writing the output. MultiQueryOptimizer and UnionOptimizer still do not
handle it. Created PIG-4692 for that. The check in ReadScalarsTez is still
there and cannot be removed as it is required to handle cases where multiple
source tasks write 0 or 1 record each. In that case it will not fail while
writing, but will fail when reading as there will be more than 1 record.
was (Author: rohini):
Only difference between testFRJoinOut8 and testFRJoinOut9 was 'replicated' vs
'repl'. So repurposed testFRJoinOut9 for adding test.
The code in TezDAGBuilder put in for scalars was not needed any more. So
totally removed it. TestScalarAlias, Scalar and CastScalar e2e tests are fine.
Running the full e2e suite now.
Additionally optimized the Scalar more than one row checking by doing that
while writing the output. Did not remove the one in ReadScalarsTez for now as
MultiQueryOptimizer and UnionOptimizer still do not handle it. Will create
separate jira for that.
> Refactor out resetting input key in TezDagBuilder
> -------------------------------------------------
>
> Key: PIG-3957
> URL: https://issues.apache.org/jira/browse/PIG-3957
> Project: Pig
> Issue Type: Improvement
> Components: tez
> Reporter: Daniel Dai
> Assignee: Rohini Palaniswamy
> Fix For: 0.16.0
>
> Attachments: PIG-3957-1.patch
>
>
> In TezDagBuilder, we reset input key of all
> POPackage/POValueInputTez/POIdentityInOutTez, that is very confusing. We
> shall refactor these out.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
