[
https://issues.apache.org/jira/browse/PIG-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-2265:
----------------------------
Attachment: PIG-2265-1.patch
There are two issues:
1. SecondaryKeyOptimizer does something wrong when foreach inner plan contains
nested foreach.
2. Order by sampler does not deal with nested tuples.
PIG-2265-1.patch fixed TestSecondarySort.testNestedSortEndToEnd3() by
1. Change SecondaryKeyOptimizer not to optimize when foreach inner plan
contains nested foreach. This alone fix PIG-3049, but not
testNestedSortEndToEnd3
2. Change MRCompiler not to flatten input tuple when doing sorting
Yet to finish all tests.
> Test case TestSecondarySort failure
> -----------------------------------
>
> Key: PIG-2265
> URL: https://issues.apache.org/jira/browse/PIG-2265
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Shengjun Xin
> Attachments: PIG-2265-1.patch
>
>
> Error message:
> Testcase: testNestedSortEndToEnd3 took 53.076 sec
> Caused an ERROR
> Unable to open iterator for alias E. Backend error :
> org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias E. Backend error : org.apache.pig.data.DataByteArray
> cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.PigServer.openIterator(PigServer.java:742)
> at
> org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray
> cannot be cast to org.apache.pig.data.Tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
