[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21747 Thank you for the decision, @mn-mikke and @cloud-fan ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 To backport this fix, we need to backport another improvement PR that allows accessing SQLConf at executor side, which violates the backport rule. I think it's ok to have this fix in 2.4 only. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21747 If nobody has any objections, I'm happy to close this PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21747 Hi, @mn-mikke and @cloud-fan and @maropu . 2.3.2 vote passed today and 2.4.0-rc1 doesn't have this issue. Given that Spark 2.4.0 will come faster than Spark 2.3.3, are we heading to (1) or (2)? 1. Mark this as resolved in 2.4.0 and close this PR? 2. Proceed this PR for 2.3.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92933/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92933/testReport)** for PR 21747 at commit [`4b42c78`](https://github.com/apache/spark/commit/4b42c7870df1447a3d8b8d5c94176e07f043ce06). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21747 IMHO, we could directly use ```DataType.equalsIgnoreCaseAndNullability``` instead of ```sameType``` to make sure that ```mergeTwoDataTypes``` won't blow up if the trait is misused. The proper check is performed by ```checkInputDataTypes``` anyway. If you don't like this approach for any reason, I'm happy to close the PR. :-) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92933/testReport)** for PR 21747 at commit [`4b42c78`](https://github.com/apache/spark/commit/4b42c7870df1447a3d8b8d5c94176e07f043ce06). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21747 How about making a temporary helper function for `sameType` in `ComplexTypeMergingExpression`? You read the config for case sensitivity in a driver side, and hold it as a field value in the closure. Then, the helper function references the (serialized and deserialized) value in executor sides? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 ah, that's hard to fix, maybe just leave it, since Spark 2.4 is coming. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92925/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92925/testReport)** for PR 21747 at commit [`163b880`](https://github.com/apache/spark/commit/163b880a74bb576edad1e9750bcead839a61a7eb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92925/testReport)** for PR 21747 at commit [`163b880`](https://github.com/apache/spark/commit/163b880a74bb576edad1e9750bcead839a61a7eb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92914/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92914/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ComplexTypeMergingExpression extends Expression ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21747 Thanks guys for triggering the builds, but I must implement a fix first. It's failing since the code is accessing ```SQLConf``` from executors (```doCodeGen```-> ```dateType```->```sameType```->```SQLConf```). For master, it shouldn't be a problem thanks to [ #21376](https://github.com/apache/spark/pull/21376). But for this branch, we need to come up with some work around. Maybe, via enforcing serialization of ```dataType```? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92914/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92912/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92912/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ComplexTypeMergingExpression extends Expression ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92912/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92862/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92862/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ComplexTypeMergingExpression extends Expression ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92862/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92859/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92859/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ComplexTypeMergingExpression extends Expression ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92859/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21747 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92856/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92856/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait ComplexTypeMergingExpression extends Expression ` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21747 **[Test build #92856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92856/testReport)** for PR 21747 at commit [`a2fe63e`](https://github.com/apache/spark/commit/a2fe63e1d48b0291feaa9fcd008654da051d1f1b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21747 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21747: [SPARK-24165][SQL][branch-2.3] Fixing conditional expres...
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21747 cc @cloud-fan @ueshin @viirya @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
