[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22141 nvm ;) we still have much time until the next release. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 @maropu Very sorry. I haven't had the time to come back to it. I have some stuff on my plate. I will get to this after i am done. Thanks !! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22141 Any update? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22141 @dilipbiswal Can we resume this for v3.0? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 @maropu I think, this will be considered post 2.4. So we have some time. I am now exploring options to plan better at physical level. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22141 @dilipbiswal Any update? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95091/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #95091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95091/testReport)** for PR 22141 at commit [`844a3ff`](https://github.com/apache/spark/commit/844a3ff82a688e7398bb130a44750aec78420698). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #95091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95091/testReport)** for PR 22141 at commit [`844a3ff`](https://github.com/apache/spark/commit/844a3ff82a688e7398bb130a44750aec78420698). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2431/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95086/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #95086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95086/testReport)** for PR 22141 at commit [`844a3ff`](https://github.com/apache/spark/commit/844a3ff82a688e7398bb130a44750aec78420698). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #95086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95086/testReport)** for PR 22141 at commit [`844a3ff`](https://github.com/apache/spark/commit/844a3ff82a688e7398bb130a44750aec78420698). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2426/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22141 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94932/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94932/testReport)** for PR 22141 at commit [`926c7fc`](https://github.com/apache/spark/commit/926c7fc7a766b1e310798a87ae7a485f731ade99). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2304/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94932/testReport)** for PR 22141 at commit [`926c7fc`](https://github.com/apache/spark/commit/926c7fc7a766b1e310798a87ae7a485f731ade99). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94930/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94930/testReport)** for PR 22141 at commit [`473bfb5`](https://github.com/apache/spark/commit/473bfb500b07626ff42a9e5ddc167970299bde21). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user dmateusp commented on the issue: https://github.com/apache/spark/pull/22141 I reproduced the issue with the following code (was a bit surprised with the behavior) The tables: ```scala scala> spark.sql("SELECT * FROM users").show +---+---+ | id|country| +---+---+ | 0| 10| | 1| 20| +---+---+ scala> spark.sql("SELECT * FROM countries").show +---++ | id|name| +---++ | 10|Portugal| +---++ ``` Without the OR: ```scala scala> spark.sql("SELECT * FROM users u WHERE u.country NOT IN (SELECT id from countries)").show +---+---+ | id|country| +---+---+ | 1| 20| +---+---+ ``` With an OR and IN: scala> spark.sql("SELECT * FROM users u WHERE 1=0 OR u.country IN (SELECT id from countries)").show +---+---+ | id|country| +---+---+ | 0| 10| +---+---+ With an OR and NOT IN: ```scala scala> spark.sql("SELECT * FROM users u WHERE 1=0 OR u.country NOT IN (SELECT id from countries)").show org.apache.spark.sql.AnalysisException: Null-aware predicate sub-queries cannot be used in nested conditions: ((1 = 0) || NOT country#9 IN (list#62 []));; ``` +1 to get that fixed --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22141 **[Test build #94930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94930/testReport)** for PR 22141 at commit [`473bfb5`](https://github.com/apache/spark/commit/473bfb500b07626ff42a9e5ddc167970299bde21). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2303/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22141: [SPARK-25154][SQL] Support NOT IN sub-queries inside nes...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org