[
https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shravan Matthur Narayanamurthy updated PIG-161:
-----------------------------------------------
Attachment: foreach_gen_comb.patch
A patch for the foreach genearte issue. Includes both the logical side fix from
Santhosh and the physical side fix from me. Also included with this are minor
fixes to get tests working. I have also rolled exprplan.patch into this.
Some Main things to note in the patch apart from exprplan.patch:
New operator: NewForEach which has a list of physical plans and incorporates
the POGenerate functionality
Changes to Translator: visit(LOForeach method to build NewForEach instead of
foreach-generate, the use of seen list disregarding walker to process inner
plans of foreach to handle the implicit split case though no implicit case has
been tested
Changes to MRCompiler: added a new visitNewForEach method for compiling
NewForEach into MR, Modifications to visitSort & visitDistinct methods to work
with NewForEach.
Changes to PhysicalOperator: getNext(Bag) method does accumulation
Changes to POProject: getNext(Bag) changed to handle bag projection
Changes to ConstantExpression to return EOP on second call to the getNext
method and its ripple effects in all test cases.
Status of E2E tests: All previously passing tests pass now. I also included
TestEvalPipeline into the suite. All its test cases except the titleNGrams one
pass which inludes join, sort and distinct. The titleNGrams is a known issue.
> Rework physical plan
> --------------------
>
> Key: PIG-161
> URL: https://issues.apache.org/jira/browse/PIG-161
> Project: Pig
> Issue Type: Sub-task
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: arithmeticOperators.patch, BinCondAndNegative.patch,
> CastAndMapLookUp.patch, exprplan.patch, foreach_gen_comb.patch, incr2.patch,
> incr3.patch, incr4.patch, incr5.patch, logToPhyTranslator.patch,
> missingOps.patch, MRCompilerTests_PlansAndOutputs.txt, Phy_AbsClass.patch,
> physicalOps.patch, physicalOps.patch, physicalOps.patch, physicalOps.patch,
> physicalOps_latest.patch, POCast.patch, POCast.patch, podistinct.patch,
> pogenerate.patch, pogenerate.patch, pogenerate.patch, posort.patch,
> POUserFuncCorrection.patch,
> TEST-org.apache.pig.test.TestLocalJobSubmission.txt,
> TEST-org.apache.pig.test.TestLogToPhyCompiler.txt,
> TEST-org.apache.pig.test.TestLogToPhyCompiler.txt,
> TEST-org.apache.pig.test.TestMapReduce.txt,
> TEST-org.apache.pig.test.TestTypeCheckingValidator.txt,
> TEST-org.apache.pig.test.TestUnion.txt, translator.patch, translator.patch,
> translator.patch, translator.patch
>
>
> This bug tracks work to rework all of the physical operators as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
