liyunzhang_intel created PIG-4456:
-------------------------------------
Summary: Sort the leaves by SparkOperator.operatorKey in
SparkLauncher#sparkOperToRDD
Key: PIG-4456
URL: https://issues.apache.org/jira/browse/PIG-4456
Project: Pig
Issue Type: Bug
Components: spark
Reporter: liyunzhang_intel
Assignee: liyunzhang_intel
Sort the leaves by SparkOperator.operatorKey to guarantee that sparkOperator
with smaller operatorKey is executed before those with larger operatorKey if
both are the leaves of spark plan, for example, sparkOperator(scope-65) and
sparkOperator(scope-67) are all leaves of spark plan. SparkOperator(scope-65)
should be executed before sparkOperator(scope-67) because the output of
scope-65 maybe is used by scope-67.
This patch fixs following unit test failures:
org.apache.pig.test.TestLimitVariable.testLimitVariable1
org.apache.pig.test.TestLimitVariable.testLimitVariable2
org.apache.pig.test.TestLimitVariable.testLimitVariable3
org.apache.pig.test.TestLimitVariable.testLimitVariable4
org.apache.pig.test.TestSample.testSample_VariableNone
org.apache.pig.test.TestSample.testSample_VariableSome
org.apache.pig.test.TestScalarAliases.testScalarAliasesSplitClause
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
