[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-11-28 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @gatorsmile, I think it would be better to fix codegen than to prevent it from happening with an assertion. If `CodegenFallback` can produce fallback code, why not allow it to when necessary? ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19568 @rdblue Yes, the current implementation implicitly assumes the rule `CollapseCodegenStages ` excludes all the illegal cases. How about adding an `assert` to do the check that the condition of

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @DonnyZone, I don't know of any cases that use codgen after the fix for `CodegenFallback`, but I think this is still a good idea. If Spark is going to generate code, it should generate

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-30 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/19568 @rdblue Could you find any cases that can trigger the problem of wrong INPUT_ROW in SortMergeJoinExec after the fix (https://github.com/apache/spark/pull/18656) for CollapseCodegenStages rule? I

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-30 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/19568 This PR is similar to the initial commit when I try to fix SPARK-21441 in https://github.com/apache/spark/pull/18656

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83090/ Test PASSed. ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83090/testReport)** for PR 19568 at commit

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83090/testReport)** for PR 19568 at commit

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83056/ Test PASSed. ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83056/testReport)** for PR 19568 at commit

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83056/testReport)** for PR 19568 at commit

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19568 Could you please change title from `SPARK-22345` to `[SPARK-22345]`? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @dongjoon-hyun, yes, I'm currently working on it. I just wanted to get the rest up. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83023/ Test FAILed. ---

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83023/testReport)** for PR 19568 at commit

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19568 **[Test build #83023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83023/testReport)** for PR 19568 at commit