[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19281 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19281 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82054/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #82054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82054/testReport)** for PR 19281 at commit [`78f41e0`](https://github.com/apache/spark/commit/78f41e046ea4e307e43315b23ed3212e1f4c3f1b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/19281 Thank you, @wzhfy, very much for the comments! I have made corresponding changes in my latest commit. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #82054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82054/testReport)** for PR 19281 at commit [`78f41e0`](https://github.com/apache/spark/commit/78f41e046ea4e307e43315b23ed3212e1f4c3f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81980/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #81980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81980/testReport)** for PR 19281 at commit [`a14c9be`](https://github.com/apache/spark/commit/a14c9be030376282253871464287d8e5e6ecce4a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/19281 Thank you very much for the feedback, @tejasapatil, @gatorsmile! All the suggestions/comments have been addressed by my latest check-in. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/19281 @cloud-fan Please refer to https://issues.apache.org/jira/browse/SPARK-18591. The plan is to apply the optimization during physical planning stage, and specifically, when creating Aggregate physical operator. So without the outputOrdering being accurate, such optimizations cannot be done at physical planning stage. And even in post-physical-planning stage, it has to be pushed after EnsureRequirements. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #81980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81980/testReport)** for PR 19281 at commit [`a14c9be`](https://github.com/apache/spark/commit/a14c9be030376282253871464287d8e5e6ecce4a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19281 > So we ended up not getting the correct outputOrdering during physical planning stage before Sort nodes are added to the children. What's the harm of this? I think only `EnsureRequirements` cares about `outputOrdering`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81966/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #81966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81966/testReport)** for PR 19281 at commit [`7893935`](https://github.com/apache/spark/commit/7893935d694663316575a7485ea833fab998d108). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19281 **[Test build #81966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81966/testReport)** for PR 19281 at commit [`7893935`](https://github.com/apache/spark/commit/7893935d694663316575a7485ea833fab998d108). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19281 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user maryannxue commented on the issue: https://github.com/apache/spark/pull/19281 bq. This is not accurate. It depends on the length of required ordering and the length of child ordering. You are right. I did it right in the code but made a mistake in the description here. Thanks for pointing out, @gatorsmile! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19281 > If the childOutputOrdering satisfies (is a superset of) the required child ordering => childOutputOrdering This is not accurate. It depends on the length of required ordering and the length of child ordering. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19281 @gatorsmile @cloud-fan could you trigger tests if it is worth fixing? Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19281: [SPARK-21998][SQL] SortMergeJoinExec did not calculate i...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19281 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
