[GitHub] [spark] AmplabJenkins removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
AmplabJenkins removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495481943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105745/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
AmplabJenkins removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495481937 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
AmplabJenkins commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495481937 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
AmplabJenkins commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495481943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105745/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495481709 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105744/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495481706 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
SparkQA removed a comment on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495451187 **[Test build #105745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105745/testReport)** for PR 24666 at commit [`aae3653`](https://github.com/apache/spark/commit/aae3653e4dab751ae5bff7e3ef26ac63d4f4cc4f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495481709 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105744/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
AmplabJenkins commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495481706 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
SparkQA removed a comment on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495451189 **[Test build #105744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105744/testReport)** for PR 24668 at commit [`58e9544`](https://github.com/apache/spark/commit/58e95445579a17d5d457dfc7678893d481f0b6f7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page
SparkQA commented on issue #24666: [SPARK-27482][SQL][WEBUI] Show BroadcastHashJoinExec numOutputRows statistics info on SparkSQL UI page URL: https://github.com/apache/spark/pull/24666#issuecomment-495481594 **[Test build #105745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105745/testReport)** for PR 24666 at commit [`aae3653`](https://github.com/apache/spark/commit/aae3653e4dab751ae5bff7e3ef26ac63d4f4cc4f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles
SparkQA commented on issue #24668: [SPARK-27676][SQL][SS] InMemoryFileIndex should respect spark.sql.files.ignoreMissingFiles URL: https://github.com/apache/spark/pull/24668#issuecomment-495481338 **[Test build #105744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105744/testReport)** for PR 24668 at commit [`58e9544`](https://github.com/apache/spark/commit/58e95445579a17d5d457dfc7678893d481f0b6f7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24688: [SPARK-26145][SQL] Support Hive 3.0 metastore
gatorsmile commented on issue #24688: [SPARK-26145][SQL] Support Hive 3.0 metastore URL: https://github.com/apache/spark/pull/24688#issuecomment-495475119 https://issues.apache.org/jira/browse/HIVE-19531?jql=project%20%3D%20HIVE%20AND%20fixVersion%20in%20(3.0.1%2C%203.1.2%2C%203.1.0%2C%203.1.1)%20AND%20component%20%3D%20Metastore I checked the above list. The quality of Hive 3.0 might be bad, but the metastore component [which is what this PR cares] does not have serious bugs. Is my understanding right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
AmplabJenkins removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495474262 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105742/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
AmplabJenkins removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495474257 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
AmplabJenkins commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495474257 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
AmplabJenkins commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495474262 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105742/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
SparkQA removed a comment on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495447419 **[Test build #105742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105742/testReport)** for PR 24647 at commit [`e5e98cb`](https://github.com/apache/spark/commit/e5e98cba7f0f9307459995710016a7f685c08b88). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource.
SparkQA commented on issue #24647: [SPARK-27776][SQL]Avoid duplicate Java reflection in DataSource. URL: https://github.com/apache/spark/pull/24647#issuecomment-495473935 **[Test build #105742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105742/testReport)** for PR 24647 at commit [`e5e98cb`](https://github.com/apache/spark/commit/e5e98cba7f0f9307459995710016a7f685c08b88). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
SparkQA commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495471313 **[Test build #105751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105751/testReport)** for PR 24681 at commit [`828b7f4`](https://github.com/apache/spark/commit/828b7f4c82c32d93d8dc96b399d9b66e8bdcf41b). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495470991 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495470988 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
AmplabJenkins commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495470988 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
AmplabJenkins commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495470991 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
AmplabJenkins removed a comment on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-494987707 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement
gatorsmile commented on issue #24681: [SPARK-27809] [SQL] Make optional clauses order insensitive for CREATE DATABASE/VIEW SQL statement URL: https://github.com/apache/spark/pull/24681#issuecomment-495470711 test this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495470118 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495470122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105747/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454649 **[Test build #105747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105747/testReport)** for PR 24685 at commit [`369c48b`](https://github.com/apache/spark/commit/369c48b6d0b5c70ea144f7f1f7f5680095c11772). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495470122 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105747/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495470118 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495469928 **[Test build #105747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105747/testReport)** for PR 24685 at commit [`369c48b`](https://github.com/apache/spark/commit/369c48b6d0b5c70ea144f7f1f7f5680095c11772). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495469111 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495469120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105746/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495469111 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495469120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105746/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495468932 **[Test build #105746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105746/testReport)** for PR 24685 at commit [`570bbb7`](https://github.com/apache/spark/commit/570bbb7f9313e48cf05be287ee87404b2300a852). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495453504 **[Test build #105746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105746/testReport)** for PR 24685 at commit [`570bbb7`](https://github.com/apache/spark/commit/570bbb7f9313e48cf05be287ee87404b2300a852). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide
dongjoon-hyun commented on issue #24682: [SPARK-27762][SQL] [FOLLOWUP] Add behavior change for Avro writer in migration guide URL: https://github.com/apache/spark/pull/24682#issuecomment-495468265 Shall we create another JIRA issue for this because this becomes beyond `Add behavior change for Avro writer in migration guide`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly.
beliefer commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly. URL: https://github.com/apache/spark/pull/24372#issuecomment-495467953 > Adding @mgaido91 and @maropu since it seems ANSI compliance issue. Thansks for your focus and promote. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on issue #24233: [SPARK-26356][SQL] remove SaveMode
from data source v2
URL: https://github.com/apache/spark/pull/24233#issuecomment-495467864
Thank you for pinging me, @cloud-fan . Mostly, it looks good to me. I have
only one concern about `.mode("default")` behavior,
https://github.com/apache/spark/pull/24233#discussion_r287206462 and the other
comments are minor.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly.
HyukjinKwon commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly. URL: https://github.com/apache/spark/pull/24372#issuecomment-495467369 Adding @mgaido91 and @maropu since it seems ANSI compliance issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly.
beliefer commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly. URL: https://github.com/apache/spark/pull/24372#issuecomment-495464825 > @beliefer can you actually check if this is ANSI standard, rather than comparing to Hive? Yes, it is also a ANSI standard. [https://en.wikipedia.org/wiki/Insert_(SQL)](https://en.wikipedia.org/wiki/Insert_(SQL)) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
AmplabJenkins removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495463320 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105741/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
AmplabJenkins removed a comment on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693#issuecomment-495463533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zhengruifeng commented on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
zhengruifeng commented on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495463566 @srowen I made a detailed review on `ML.XXXEvaluator` & `MLLIB.XXXMetrics` recently and find another several places seems needing to be improved. For example: 1, all metrics in `MultilabelMetrics` & `MulticlassMetrics` can be computed on only one pass, however, in current impl each metric needs one pass. 2, `ML.XXXEvaluator` only supports only one metric at once, which means at least one pass is needed for one metric. I think we can cache the `MLLIB.XXXMetrics` in the impl, and in the following calls, if the input dataset donot change, we can direct get the metric from cached `MLLIB.XXXMetrics` without accumlation on the input dataset. 3, `MultiLabelClassificationEvalutaor` is missing now. 4, in `BinaryClassificationMetrics`, to control the #Bins, direct setting the #Partition in the sort stage seems more reasonable than current impl Would you mind if I open a umbrella ticket "Evaluator & Metrics improvements" to track above points and opened tickets on `sliding job` and `SSreg`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
AmplabJenkins removed a comment on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693#issuecomment-495463527 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
AmplabJenkins commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693#issuecomment-495463527 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
AmplabJenkins commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693#issuecomment-495463533 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
AmplabJenkins removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495463316 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
AmplabJenkins commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495463320 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105741/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
AmplabJenkins commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495463316 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
SparkQA removed a comment on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495424903 **[Test build #105741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105741/testReport)** for PR 24690 at commit [`48de77b`](https://github.com/apache/spark/commit/48de77b56bbeaf91f595dec7e1621b7f105076c1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks
SparkQA commented on issue #24690: [SPARK-27711][CORE] Unset InputFileBlockHolder at the end of tasks URL: https://github.com/apache/spark/pull/24690#issuecomment-495463100 **[Test build #105741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105741/testReport)** for PR 24690 at commit [`48de77b`](https://github.com/apache/spark/commit/48de77b56bbeaf91f595dec7e1621b7f105076c1). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on a change in pull request #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
JoshRosen commented on a change in pull request #24693: [SPARK-27829][SQL] In
Dataset.joinWith inner joins, don't nest data before shuffling
URL: https://github.com/apache/spark/pull/24693#discussion_r287207443
##
File path: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
##
@@ -485,6 +507,28 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
assert(e4.contains("Invalid join type in joinWith: " + LeftAnti.sql))
}
+ test("joinWith left_outer") {
Review comment:
I realized there's an existing left join test, so I'll remove this new test
case and fold in the schema checks into the existing case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
SparkQA commented on issue #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693#issuecomment-495462723 **[Test build #105750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105750/testReport)** for PR 24693 at commit [`33bb4af`](https://github.com/apache/spark/commit/33bb4af4600b772b8eb204f72817d51d9e984559). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen opened a new pull request #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling
JoshRosen opened a new pull request #24693: [SPARK-27829][SQL] In Dataset.joinWith inner joins, don't nest data before shuffling URL: https://github.com/apache/spark/pull/24693 ## What changes were proposed in this pull request? In order to support outer joins with null top-level objects, SPARK-15441 modified Dataset.joinWith to project both inputs into single-column structs prior to the join. For inner joins, however, this step is unnecessary and actually harms performance: performing the nesting before the join increases the shuffled data size. As an optimization for inner joins only, we can move this nesting to occur after the join (effectively switching back to the pre- SPARK-15441 behavior; see #13425). ## How was this patch tested? Existing tests, which I strengthened to also make assertions about the join result's nullability (since this guards against a bug I almost introduced during prototyping). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL]
remove SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r287206904
##
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##
@@ -78,15 +80,14 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
* @since 1.4.0
*/
def mode(saveMode: String): DataFrameWriter[T] = {
-this.mode = saveMode.toLowerCase(Locale.ROOT) match {
+mode(saveMode.toLowerCase(Locale.ROOT) match {
case "overwrite" => SaveMode.Overwrite
case "append" => SaveMode.Append
case "ignore" => SaveMode.Ignore
case "error" | "errorifexists" | "default" => SaveMode.ErrorIfExists
Review comment:
Could you add a test coverage for this? It seems that we don't have yet.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL]
remove SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r287206462
##
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##
@@ -78,15 +80,14 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
* @since 1.4.0
*/
def mode(saveMode: String): DataFrameWriter[T] = {
-this.mode = saveMode.toLowerCase(Locale.ROOT) match {
+mode(saveMode.toLowerCase(Locale.ROOT) match {
case "overwrite" => SaveMode.Overwrite
case "append" => SaveMode.Append
case "ignore" => SaveMode.Ignore
case "error" | "errorifexists" | "default" => SaveMode.ErrorIfExists
Review comment:
This should be changed because "default" is not `ErrorIfExists` for DSv2.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly.
HyukjinKwon commented on issue #24372: [SPARK-27462][SQL] Enhance insert into hive table that could choose some columns in target table flexibly. URL: https://github.com/apache/spark/pull/24372#issuecomment-495461541 @beliefer can you actually check if this is ANSI standard, rather than comparing to Hive? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL]
remove SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r287206026
##
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##
@@ -56,13 +55,16 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
* `SaveMode.Overwrite`: overwrite the existing data.
* `SaveMode.Append`: append the data.
* `SaveMode.Ignore`: ignore the operation (i.e. no-op).
- * `SaveMode.ErrorIfExists`: default option, throw an exception at
runtime.
+ * `SaveMode.ErrorIfExists`: throw an exception at runtime.
*
+ *
+ * When writing to data source v1, the default option is `ErrorIfExist`.
When writing to data
Review comment:
nit `ErrorIfExist` -> `ErrorIfExists`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
AmplabJenkins commented on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495459183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105743/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
AmplabJenkins removed a comment on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495459182 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
AmplabJenkins commented on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495459182 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
AmplabJenkins removed a comment on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495459183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105743/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24348: [SPARK-27444][SQL] multi-select can be used in subquery
HyukjinKwon commented on a change in pull request #24348: [SPARK-27444][SQL]
multi-select can be used in subquery
URL: https://github.com/apache/spark/pull/24348#discussion_r287204320
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala
##
@@ -132,15 +132,19 @@ class PlanParserSuite extends AnalysisTest {
table("a").select(star()).union(table("a").where('s <
10).select(star(
intercept(
"from a select * select * from x where a.s < 10",
- "Multi-select queries cannot have a FROM clause in their individual
SELECT statements")
+ "This select statement can not have FROM cause as its already specified
upfront")
Review comment:
it's fine. I didn't mean we should fix now. just wanted to say it :-).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
SparkQA removed a comment on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495448695 **[Test build #105743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105743/testReport)** for PR 24648 at commit [`79e5eea`](https://github.com/apache/spark/commit/79e5eea4855ebb305a4dd29568915086f9cbf82f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24648: [SPARK-27777][ML] Eliminate uncessary sliding job in AreaUnderCurve
SparkQA commented on issue #24648: [SPARK-2][ML] Eliminate uncessary sliding job in AreaUnderCurve URL: https://github.com/apache/spark/pull/24648#issuecomment-495459003 **[Test build #105743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105743/testReport)** for PR 24648 at commit [`79e5eea`](https://github.com/apache/spark/commit/79e5eea4855ebb305a4dd29568915086f9cbf82f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL] Add count_if functions
HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL]
Add count_if functions
URL: https://github.com/apache/spark/pull/24335#discussion_r287203884
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -373,6 +373,25 @@ object functions {
def countDistinct(columnName: String, columnNames: String*): Column =
countDistinct(Column(columnName), columnNames.map(Column.apply) : _*)
+ /**
+ * Aggregate function: returns the number of true values in a group.
+ *
+ * @group agg_funcs
+ * @since 3.0.0
+ */
+ def count_if(e: Column): Column = withAggregateFunction {
Review comment:
I wouldn't add those functions here for now (see
https://github.com/apache/spark/blob/e4f04654618b1644678c18c4331418fddb2982e8/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L42-L56)
If we're going to add, we should add it in Python and R side as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions
HyukjinKwon commented on issue #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#issuecomment-495458702 cc @cloud-fan too This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL]
remove SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r287203760
##
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##
@@ -268,9 +269,26 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
import
org.apache.spark.sql.execution.datasources.v2.DataSourceV2Implicits._
provider.getTable(dsOptions) match {
+// TODO: for backward compatibility reasons, the builtin file source
needs to support all
+// the save modes, which violates the semantic of `TableProvider`.
Here we special-case
Review comment:
nit. `Here we special-case`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL] Add count_if functions
HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL] Add count_if functions URL: https://github.com/apache/spark/pull/24335#discussion_r287203802 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CountIf.scala ## @@ -0,0 +1,73 @@ +/* Review comment: Can we do `RuntimeReplaceable` like `Every` or `Any`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on a change in pull request #24233: [SPARK-26356][SQL]
remove SaveMode from data source v2
URL: https://github.com/apache/spark/pull/24233#discussion_r287203658
##
File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
##
@@ -268,9 +269,26 @@ final class DataFrameWriter[T] private[sql](ds:
Dataset[T]) {
import
org.apache.spark.sql.execution.datasources.v2.DataSourceV2Implicits._
provider.getTable(dsOptions) match {
+// TODO: for backward compatibility reasons, the builtin file source
needs to support all
Review comment:
Can we have an IDed TODO? `TODO:` -> `TODO(SPARK-XXX):`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
SparkQA commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495458235 **[Test build #105749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105749/testReport)** for PR 24233 at commit [`22ba355`](https://github.com/apache/spark/commit/22ba355c7dd08229343851be0e38746acd73047e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL] Add count_if functions
HyukjinKwon commented on a change in pull request #24335: [SPARK-27425][SQL]
Add count_if functions
URL: https://github.com/apache/spark/pull/24335#discussion_r287203613
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
##
@@ -894,4 +894,29 @@ class DataFrameAggregateSuite extends QueryTest with
SharedSQLContext {
error.message.contains("function min_by does not support ordering on
type map"))
}
}
+
+ test("SPARK-27425: count_if function") {
+def checkError(df: => DataFrame): Unit = {
+ val thrownException = the [AnalysisException] thrownBy
df.queryExecution.analyzed
+ assert(thrownException.message.contains("function count_if requires
boolean type"))
+}
+
+checkAnswer(
+ testData.agg(count_if('key % 2 === 0), count_if('key > 50),
count_if('key < 50)),
+ Row(50L, 50L, 49L))
+checkAnswer(
+ sql("SELECT COUNT_IF(key % 2 = 0), COUNT_IF(key > 50), COUNT_IF(key <
50) FROM testData"),
+ Row(50L, 50L, 49L))
+
+checkAnswer(
+ testData2.groupBy('a).agg(count_if('b % 2 === 0)),
+ Seq(Row(1, 1L), Row(2, 1L), Row(3, 1L)))
+checkAnswer(
+ sql("SELECT a, COUNT_IF(b % 2 = 0) FROM testData2 GROUP BY a"),
+ Seq(Row(1, 1L), Row(2, 1L), Row(3, 1L)))
Review comment:
Can we add a test with `null` as well?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
AmplabJenkins removed a comment on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495457918 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
AmplabJenkins removed a comment on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495457913 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
AmplabJenkins commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495457913 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #24348: [SPARK-27444][SQL] multi-select can be used in subquery
cloud-fan commented on a change in pull request #24348: [SPARK-27444][SQL]
multi-select can be used in subquery
URL: https://github.com/apache/spark/pull/24348#discussion_r287203319
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala
##
@@ -132,15 +132,19 @@ class PlanParserSuite extends AnalysisTest {
table("a").select(star()).union(table("a").where('s <
10).select(star(
intercept(
"from a select * select * from x where a.s < 10",
- "Multi-select queries cannot have a FROM clause in their individual
SELECT statements")
+ "This select statement can not have FROM cause as its already specified
upfront")
Review comment:
this is copied from the old error message:
https://github.com/apache/spark/pull/24348/files#diff-9847f5cef7cf7fbc5830fbc6b779ee10L133
We can update this
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
AmplabJenkins commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495457918 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.
cloud-fan closed pull request #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation.
cloud-fan commented on issue #24617: [SPARK-27732][SQL] Add v2 CreateTable implementation. URL: https://github.com/apache/spark/pull/24617#issuecomment-495457222 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2
dongjoon-hyun commented on issue #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#issuecomment-495457099 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
AmplabJenkins removed a comment on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495455589 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11003/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
AmplabJenkins removed a comment on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495455584 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
AmplabJenkins commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495455584 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
AmplabJenkins commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495455589 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11003/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454649 **[Test build #105747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105747/testReport)** for PR 24685 at commit [`369c48b`](https://github.com/apache/spark/commit/369c48b6d0b5c70ea144f7f1f7f5680095c11772). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
SparkQA commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495454652 **[Test build #105748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105748/testReport)** for PR 24442 at commit [`e7bfcc8`](https://github.com/apache/spark/commit/e7bfcc8eb1cef025abd36913ec50b5221da4ad10). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems
dongjoon-hyun commented on issue #24442: [SPARK-27547][SQL] Fix DataFrame self-join problems URL: https://github.com/apache/spark/pull/24442#issuecomment-495454458 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #24348: [SPARK-27444][SQL] multi-select can be used in subquery
HyukjinKwon commented on a change in pull request #24348: [SPARK-27444][SQL]
multi-select can be used in subquery
URL: https://github.com/apache/spark/pull/24348#discussion_r287200421
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala
##
@@ -132,15 +132,19 @@ class PlanParserSuite extends AnalysisTest {
table("a").select(star()).union(table("a").where('s <
10).select(star(
intercept(
"from a select * select * from x where a.s < 10",
- "Multi-select queries cannot have a FROM clause in their individual
SELECT statements")
+ "This select statement can not have FROM cause as its already specified
upfront")
Review comment:
you know what? cannot is correct too and more usual :-).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454363 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454363 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454367 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11002/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
AmplabJenkins removed a comment on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495454367 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11002/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver
dongjoon-hyun edited a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495453546 @wangyum . Just to be clear, did you regenerate Thrift related code always for Spark? > (Note that we only migrate code without adding features, such as HIVE-4924 and HIVE-15473). I assumed that this generated code came from Hive 3.5.3, but the generated code is not the same with Hive 3.5.3. Please describe how you get this code in the PR description. cc @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal.
SparkQA commented on issue #24685: [SPARK-27814] The cast operation for partition key may push down uncorrect filter, which is fatal. URL: https://github.com/apache/spark/pull/24685#issuecomment-495453504 **[Test build #105746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105746/testReport)** for PR 24685 at commit [`570bbb7`](https://github.com/apache/spark/commit/570bbb7f9313e48cf05be287ee87404b2300a852). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver
dongjoon-hyun edited a comment on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495453546 @wangyum . Just to be clear, did you regenerate Thrift related code always for Spark? > (Note that we only migrate code without adding features, such as HIVE-4924 and HIVE-15473). I assumed that this generated code came from Hive 3.5.3, but the generated code is not the same with Hive 3.5.3. cc @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver
dongjoon-hyun commented on issue #24628: [SPARK-27749][SQL][test-hadoop3.2] hadoop-3.2 support hive-thriftserver URL: https://github.com/apache/spark/pull/24628#issuecomment-495453546 @wangyum . Just to be clear, did you regenerate Thrift related code always for Spark? > (Note that we only migrate code without adding features, such as HIVE-4924 and HIVE-15473). I assumed that this generate code comes from Hive 3.5.3, but the generated code is not the same with Hive 3.5.3. cc @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24278: [SPARK-27350][SQL] Support create table on data source V2
HyukjinKwon commented on issue #24278: [SPARK-27350][SQL] Support create table on data source V2 URL: https://github.com/apache/spark/pull/24278#issuecomment-495453563 I think it's being taken over by https://github.com/apache/spark/pull/24617. Shall we close if I am correct? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
