[
https://issues.apache.org/jira/browse/PIG-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625007#comment-13625007
]
Cheolsoo Park commented on PIG-2265:
------------------------------------
[~daijy], thank you very much for fixing this. I was curious about how to fix
this.
I have few minor comments on your patch:
* Wouldn't it be better to remove the test cases that you're modifying in
{{TestSecondarySort.java}}? Since you're disabling the optimization, these test
cases don't test anything any more. So why not delete them?
* In addition, can you delete {{testDistinctOptimization1()}}, which is
currently commented out due to PIG-2009? Looking at PIG-2009, this test case
will be no longer valid since you're disabling the optimization anyway.
* Can you remove {{processForEach()}} in {{SecondaryKeyOptimizer.java}} instead
of making it always return true? How about do the following instead?
{code}
@@ -481,10 +481,9 @@ public class SecondaryKeyOptimizer extends MROpPlanVisitor
{
sawInvalidPhysicalOper = processSort((POSort)currentNode);
else if (currentNode instanceof POProject)
sawInvalidPhysicalOper =
processProject((POProject)currentNode);
- else if (currentNode instanceof POForEach)
- sawInvalidPhysicalOper =
processForEach((POForEach)currentNode);
else if (currentNode instanceof POUserFunc ||
- currentNode instanceof POUnion)
+ currentNode instanceof POUnion ||
+ currentNode instanceof POForEach)
break;
{code}
> Test case TestSecondarySort failure
> -----------------------------------
>
> Key: PIG-2265
> URL: https://issues.apache.org/jira/browse/PIG-2265
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Shengjun Xin
> Attachments: PIG-2265-1.patch
>
>
> Error message:
> Testcase: testNestedSortEndToEnd3 took 53.076 sec
> Caused an ERROR
> Unable to open iterator for alias E. Backend error :
> org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias E. Backend error : org.apache.pig.data.DataByteArray
> cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.PigServer.openIterator(PigServer.java:742)
> at
> org.apache.pig.test.TestSecondarySort.testNestedSortEndToEnd3(TestSecondarySort.java:550)
> Caused by: java.lang.ClassCastException: org.apache.pig.data.DataByteArray
> cannot be cast to org.apache.pig.data.Tuple
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:357)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
