[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Thank you, @gatorsmile , @HyukjinKwon , @bersprockets . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20705 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20705 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88245/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88245/testReport)** for PR 20705 at commit [`2975aff`](https://github.com/apache/spark/commit/2975aff63a4a22ed60bded5011ee4ae8169cf987). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88245/testReport)** for PR 20705 at commit [`2975aff`](https://github.com/apache/spark/commit/2975aff63a4a22ed60bded5011ee4ae8169cf987). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1517/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 The failure is irrelevant to this PR. ``` org.apache.spark.sql.execution.streaming.RateSourceV2Suite.basic microbatch execution ``` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88240/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88240/testReport)** for PR 20705 at commit [`2975aff`](https://github.com/apache/spark/commit/2975aff63a4a22ed60bded5011ee4ae8169cf987). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88240/testReport)** for PR 20705 at commit [`2975aff`](https://github.com/apache/spark/commit/2975aff63a4a22ed60bded5011ee4ae8169cf987). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1512/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88167/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88167/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/159489c884cd381584ade0734d6e215a2a67103c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1458/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88167/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/159489c884cd381584ade0734d6e215a2a67103c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88103/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88103/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/159489c884cd381584ade0734d6e215a2a67103c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88103/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/159489c884cd381584ade0734d6e215a2a67103c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1409/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87967/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87967/testReport)** for PR 20705 at commit [`d9d2564`](https://github.com/apache/spark/commit/d9d2564923afad880374d355f2de92c2973e5514). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1284/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87967/testReport)** for PR 20705 at commit [`d9d2564`](https://github.com/apache/spark/commit/d9d2564923afad880374d355f2de92c2973e5514). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87937/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87937/testReport)** for PR 20705 at commit [`d9d2564`](https://github.com/apache/spark/commit/d9d2564923afad880374d355f2de92c2973e5514). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87937/testReport)** for PR 20705 at commit [`d9d2564`](https://github.com/apache/spark/commit/d9d2564923afad880374d355f2de92c2973e5514). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1258/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 For the above two JSON failures, they are `MetastoreDataSourcesSuite` and `PartitionedTablePerfStatsSuite`. They are not designed to `parquet`. They are general test cases. If we explicitly set it to `parquet`, we should change it when we test it to new data source again and again. > How about we explicitly set spark.sql.sources.default to parquet for both test cases in #20705 (comment) too? Instead, I think we had better file two JIRA issues as JSON improvement if that is feasible. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20705 For https://github.com/apache/spark/pull/20705#issuecomment-370119879, yup. JSON uses string for keys in MapType. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87928/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87928/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87928/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1254/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87926/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87926/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87926/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1252/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87925/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87925/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1251/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87925/testReport)** for PR 20705 at commit [`144460d`](https://github.com/apache/spark/commit/144460d791eb92316f609b5934ff10892e3c9be0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Since we verified JSON result, I'll update the PR to address @HyukjinKwon 's comment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/20705
@gatorsmile and @HyukjinKwon .
Two failures are due to limitation of the current JSON data source
implementation. Here, we can see that the test suite correctly tests the target
data source.
1. `resolveRelation for a FileFormat DataSource without userSchema scan
filesystem only once`
For Json source, the statistic count becomes 2.
2. `Pre insert nullability check (MapType)`
Since Json source save as string, it raises ClassCastException when the
given user or table schema is different.
```scala
scala> (Tuple1(Map(1 -> (null: Integer))) ::
Nil).toDF("a").write.mode("overwrite").save("/tmp/json")
scala> spark.read.json("/tmp/json").printSchema
root
|-- a: struct (nullable = true)
||-- 1: string (nullable = true)
scala> (Tuple1(Map(1 -> (null: Integer))) ::
Nil).toDF("a").write.mode("overwrite").saveAsTable("map")
18/03/02 21:13:49 WARN HiveExternalCatalog: Couldn't find corresponding
Hive SerDe for data source provider json. Persisting data source table
`default`.`map` into Hive metastore in Spark SQL specific format, which is NOT
compatible with Hive.
scala> spark.read.json("/tmp/json").printSchema
root
|-- a: struct (nullable = true)
||-- 1: string (nullable = true)
scala> spark.table("map").printSchema
root
|-- a: map (nullable = true)
||-- key: integer
||-- value: integer (valueContainsNull = true)
scala> spark.table("map").show
18/03/02 21:14:12 ERROR Executor: Exception in task 0.0 in stage 10.0 (TID
10)
java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String
cannot be cast to java.lang.Integer
at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
```
For JSON format, could you confirm this, @HyukjinKwon ?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87916/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87916/testReport)** for PR 20705 at commit [`3ec9309`](https://github.com/apache/spark/commit/3ec9309f923405873d73a6e5d9376bba08e050d0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1247/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87916/testReport)** for PR 20705 at commit [`3ec9309`](https://github.com/apache/spark/commit/3ec9309f923405873d73a6e5d9376bba08e050d0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Sure, @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20705 Could you change the default value to json? Can all the tests pass? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 @HyukjinKwon . Yep. Actually, I had a plan for some suitable test suites. So, in general I generalized `parquet` to `load/save`. Your idea also sounds good. I'll try to do for `ParquetPartitionDiscoverySuite.scala` and `python` part according to your advice. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20705 @dongjoon-hyun, I agree with the idea in general but just to be clear do you target to improve the test coverage for both when `orc` and `parquet` are set to `spark.sql.sources.default` here? If not, how about we set `parquet` to `spark.sql.sources.default` explicitly where it's required for now? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87861/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87861/testReport)** for PR 20705 at commit [`5192e6a`](https://github.com/apache/spark/commit/5192e6a152781bdc44a2908cc78e19b0421fb287). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87861/testReport)** for PR 20705 at commit [`5192e6a`](https://github.com/apache/spark/commit/5192e6a152781bdc44a2908cc78e19b0421fb287). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1210/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87851/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87851/testReport)** for PR 20705 at commit [`eb62c2f`](https://github.com/apache/spark/commit/eb62c2f069822eeea684c18296fbcefc4f1ecc2c). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87849/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87849/testReport)** for PR 20705 at commit [`427b6f0`](https://github.com/apache/spark/commit/427b6f01a34938487f162ddf0a38d9217bfb4ece). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87851/testReport)** for PR 20705 at commit [`eb62c2f`](https://github.com/apache/spark/commit/eb62c2f069822eeea684c18296fbcefc4f1ecc2c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1202/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1200/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #87849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87849/testReport)** for PR 20705 at commit [`427b6f0`](https://github.com/apache/spark/commit/427b6f01a34938487f162ddf0a38d9217bfb4ece). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
