[GitHub] [spark] karuppayya commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec
karuppayya commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644546701 @maropu Updated the description with benchmarks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ulysses-you commented on a change in pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
ulysses-you commented on a change in pull request #28833:
URL: https://github.com/apache/spark/pull/28833#discussion_r440601141
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/DataTypeParserSuite.scala
##
@@ -61,6 +61,7 @@ class DataTypeParserSuite extends SparkFunSuite {
checkDataType("varchAr(20)", StringType)
checkDataType("cHaR(27)", StringType)
checkDataType("BINARY", BinaryType)
+ checkDataType("void", NullType)
Review comment:
I don't know what is needed, in other word I don't know what is needed
to be compatible with Hive. It may be the reason why this thing was left.
There are 2 different things between Spark and Hive.
1. Hive support both table and view with `Void` type, like `create table t
as select null as c`, `create view v as select null as c`. Spark doesn't
support both.
2. For null value type, Hive use `Void` and Spark use `NullType`
As @cloud-fan said, we shouldn't support `NullType` for tables. But may
compatible with Hive in some DDLs like `desc table`, `show create talbe` ?
I think we should discuss what things to be compatible with Hive first and
then move.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector
AmplabJenkins removed a comment on pull request #28635: URL: https://github.com/apache/spark/pull/28635#issuecomment-644544929 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector
AmplabJenkins commented on pull request #28635: URL: https://github.com/apache/spark/pull/28635#issuecomment-644544929 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector
SparkQA removed a comment on pull request #28635: URL: https://github.com/apache/spark/pull/28635#issuecomment-644488405 **[Test build #124083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124083/testReport)** for PR 28635 at commit [`0a71849`](https://github.com/apache/spark/commit/0a718490ac5e8394674a5b8f64d9f8be382805fe). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector
SparkQA commented on pull request #28635: URL: https://github.com/apache/spark/pull/28635#issuecomment-644544080 **[Test build #124083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124083/testReport)** for PR 28635 at commit [`0a71849`](https://github.com/apache/spark/commit/0a718490ac5e8394674a5b8f64d9f8be382805fe). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644543101 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124081/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644543095 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
AmplabJenkins commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644543095 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
SparkQA removed a comment on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644476504 **[Test build #124081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124081/testReport)** for PR 28810 at commit [`1251b68`](https://github.com/apache/spark/commit/1251b68044611148dcb840d3efa3c39ed9aee18e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins
SparkQA commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644542692 **[Test build #124081 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124081/testReport)** for PR 28810 at commit [`1251b68`](https://github.com/apache/spark/commit/1251b68044611148dcb840d3efa3c39ed9aee18e). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644541643 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124086/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644541638 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644541638 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644508863 **[Test build #124086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124086/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644541195 **[Test build #124086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124086/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array
HeartSaVioR edited a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644538409 cc. @kiszk @viirya as initial reviewers as I can see they're contributed the code part. cc. @bersprockets who helps to review initial patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644536779 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124080/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
SparkQA commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644538725 **[Test build #124098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124098/testReport)** for PR 28833 at commit [`479901d`](https://github.com/apache/spark/commit/479901db17d2e79d4a3001ae86f81faa545a5f4a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644538711 **[Test build #124097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124097/testReport)** for PR 28836 at commit [`2555c49`](https://github.com/apache/spark/commit/2555c497b501f371d2531deb5d8e8f2d0311d5cf). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types
HeartSaVioR commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644538409 cc. @kiszk @viirya as initial reviews as I can see they're contributed the code part. cc. @bersprockets who helps to review initial patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644537540 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124095/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644536706 **[Test build #124095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124095/testReport)** for PR 28836 at commit [`31e6573`](https://github.com/apache/spark/commit/31e65738902c3b8007bce0cb6dd000782fde6ad9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644537530 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644537521 **[Test build #124095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124095/testReport)** for PR 28836 at commit [`31e6573`](https://github.com/apache/spark/commit/31e65738902c3b8007bce0cb6dd000782fde6ad9). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644537530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/
HeartSaVioR commented on a change in pull request #28831:
URL: https://github.com/apache/spark/pull/28831#discussion_r440592664
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
##
@@ -123,26 +123,41 @@ case class ConcatWs(children: Seq[Expression])
val idxVararg = ctx.freshName("idxInVararg")
val evals = children.map(_.genCode(ctx))
+ val supportSplitExpressions =
ctx.supportSplitExpressionsWithCurrentInputs
Review comment:
We may be able to employ some threshold of the number of children or
fallback logic if we want to also exclude the case where the number of blocks
for all three parts are all 1, but that's going to be complicated a bit, so
would like to hear others' voices before applying.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644536999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644537057 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/ar
AmplabJenkins removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644537071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644536999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins removed a comment on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644536555 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array type
AmplabJenkins commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644537071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644536775 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644537057 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644536866 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
AmplabJenkins commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644536775 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types of c
SparkQA commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644536708 **[Test build #124096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124096/testReport)** for PR 28831 at commit [`d75ad5a`](https://github.com/apache/spark/commit/d75ad5aee7d24f4037b950137d529ed2ed3fb1cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644536706 **[Test build #124095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124095/testReport)** for PR 28836 at commit [`31e6573`](https://github.com/apache/spark/commit/31e65738902c3b8007bce0cb6dd000782fde6ad9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
maropu commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644536578 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
AmplabJenkins commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644536555 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
SparkQA removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644460954 **[Test build #124080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124080/testReport)** for PR 28833 at commit [`479901d`](https://github.com/apache/spark/commit/479901db17d2e79d4a3001ae86f81faa545a5f4a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LuciferYang opened a new pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed
LuciferYang opened a new pull request #28838: URL: https://github.com/apache/spark/pull/28838 ### What changes were proposed in this pull request? `SingleSessionSuite` not do `DROP TABLE IF EXISTS test_udtf` when test completed, then if we do mvn test `HiveThriftBinaryServerSuite`, the test case named `SPARK-11595 ADD JAR with input path having URL scheme` will FAILED because it want to re-create an exists table test_udtf. ### Why are the changes needed? test suite shouldn't rely on their execution order ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? Manual test，mvn test SingleSessionSuite and HiveThriftBinaryServerSuite in order This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28833: [SPARK-20680][SQL] Make null type in Spark sql to be compatible with Hive void datatype
SparkQA commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644536074 **[Test build #124080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124080/testReport)** for PR 28833 at commit [`479901d`](https://github.com/apache/spark/commit/479901db17d2e79d4a3001ae86f81faa545a5f4a). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on a change in pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode
jiangxb1987 commented on a change in pull request #28746:
URL: https://github.com/apache/spark/pull/28746#discussion_r440589823
##
File path: core/src/main/scala/org/apache/spark/deploy/LocalSparkCluster.scala
##
@@ -63,23 +65,34 @@ class LocalSparkCluster(
/* Start the Workers */
for (workerNum <- 1 to numWorkers) {
- val workerEnv = Worker.startRpcEnvAndEndpoint(localHostname, 0, 0,
coresPerWorker,
-memoryPerWorker, masters, null, Some(workerNum), _conf,
-conf.get(config.Worker.SPARK_WORKER_RESOURCE_FILE))
+ val (workerEnv, workerRef) =
Worker.startRpcEnvAndEndpoint(localHostname, 0, 0,
+coresPerWorker, memoryPerWorker, masters, null, Some(workerNum), _conf,
+conf.get(config.Worker.SPARK_WORKER_RESOURCE_FILE), isLocalCluster =
true)
workerRpcEnvs += workerEnv
+ workerRefs += workerRef
}
masters
}
def stop(): Unit = {
logInfo("Shutting down local Spark cluster.")
+// SPARK-31922: make sure all the workers have handled the
messages(`KillExecutor`,
+// `ApplicationFinished`) from the Master before we shutdown the workers'
rpcEnvs.
+// Otherwise, we could hit "RpcEnv already stopped" error.
+var busyWorkers = workerRefs
+while (busyWorkers.nonEmpty) {
Review comment:
This is wrong, what if worker didn't reply and throw
RPCTimeoutException? Actually I doubt whether it worth to make so many changes
to avoid an error message that doesn't hurt anything.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak edited a comment on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
sarutak edited a comment on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644531999 Note for reviewers that this fix affects tests with both `sbt` and `Maven` so we need to test with `sbt` and `Maven`. Tests with `sbt` runs first and I'll run tests with `Maven` after the `sbt` test passes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins removed a comment on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644531403 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
SparkQA commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644532968 **[Test build #124094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124094/testReport)** for PR 28837 at commit [`6a1e8c0`](https://github.com/apache/spark/commit/6a1e8c0c4f861d52a7408d0f72373099a59455c0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
sarutak commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644531999 Note for reviewers that this PR modifies both `SparkBuild.scala` and `pom.xml` so we need to test with `sbt` and `Maven`. Tests with `sbt` runs first and I'll run tests with `Maven` after the `sbt` test passes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #28809: [SPARK-31959][SQL][3.0] Fix Gregorian-Julian micros rebasing while switching standard time zone offset
cloud-fan closed pull request #28809: URL: https://github.com/apache/spark/pull/28809 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak opened a new pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
sarutak opened a new pull request #28837: URL: https://github.com/apache/spark/pull/28837 ### What changes were proposed in this pull request? This PR proposes to specify the version of ChromeDriver and RemoteWebDriver which work with guava 14.0.1 explicitly. ### Why are the changes needed? #28585 (SPARK-31765) upgraded HtmlUnit due to a security reason and Selenium was also needed to be upgraded to work with the upgraded HtmlUnit. After upgrading Selenium, ChromeDriver and RemoteWebDriver are implicitly upgraded because of dependency and the implicitly upgraded modules can't work with guava 14.0.1 due to an API compatibility so we need to run ChromeUISeleniumSuite with a guava version specified like `-Dguava.version=25.0-jre`. ``` $ build/sbt -Dguava.version=25.0-jre -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver -Dtest.default.exclude.tags= "testOnly org.apache.spark.ui.UISeleniumSuite" ``` It's a little bit inconvenience so let's use older version which can work with guava 14.0.1. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Confirmed that the following test can finish successfully. ``` build/sbt -Dspark.test.webdriver.chrome.driver=/Users/sarutak/work/oss/chrome-web-driver/chromedriver83 -Dtest.default.exclude.tags= clean "testOnly org.apache.spark.ui.ChromeUISeleniumSuite" ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1
AmplabJenkins commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644531403 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28809: [SPARK-31959][SQL][3.0] Fix Gregorian-Julian micros rebasing while switching standard time zone offset
cloud-fan commented on pull request #28809: URL: https://github.com/apache/spark/pull/28809#issuecomment-644530812 thanks, merging to 3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644530173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124090/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644530166 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644522490 **[Test build #124090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124090/testReport)** for PR 28836 at commit [`77730c0`](https://github.com/apache/spark/commit/77730c049a1a4ec1d0833cac90c67c74c08453ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644530166 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644530103 **[Test build #124090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124090/testReport)** for PR 28836 at commit [`77730c0`](https://github.com/apache/spark/commit/77730c049a1a4ec1d0833cac90c67c74c08453ee). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644527823 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644527787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644527823 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644527787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644527550 **[Test build #124092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124092/testReport)** for PR 28836 at commit [`154edef`](https://github.com/apache/spark/commit/154edefa188f33aa5d465180a155c9bf34f381dd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644527552 **[Test build #124093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
huaxingao commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644526286 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644525663 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124082/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644525654 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644525654 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644478474 **[Test build #124082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124082/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644524371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait
SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644524536 **[Test build #124082 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124082/testReport)** for PR 28710 at commit [`61e93d0`](https://github.com/apache/spark/commit/61e93d0cab590975b5d4f4551edd43cac6b55ce2). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644524371 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644524138 **[Test build #124091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124091/testReport)** for PR 28836 at commit [`be81da3`](https://github.com/apache/spark/commit/be81da3462ed139938b70921ea25a876fcfe38aa). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644522712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644522712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644522490 **[Test build #124090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124090/testReport)** for PR 28836 at commit [`77730c0`](https://github.com/apache/spark/commit/77730c049a1a4ec1d0833cac90c67c74c08453ee). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28831: [WIP][SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed stri
AmplabJenkins removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644519334 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28831: [WIP][SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array
AmplabJenkins commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644519334 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28831: [WIP][SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types
SparkQA commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644519042 **[Test build #124089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124089/testReport)** for PR 28831 at commit [`623aed0`](https://github.com/apache/spark/commit/623aed05f7614f880b5c7acbc9a6a18bf5d1e67d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644515694 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124085/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644516069 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644508803 **[Test build #124085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124085/testReport)** for PR 28836 at commit [`087f8a5`](https://github.com/apache/spark/commit/087f8a5aa6152a3d30887f144694917ec446354f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore
AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644516069 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644515692 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644515665 **[Test build #124085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124085/testReport)** for PR 28836 at commit [`087f8a5`](https://github.com/apache/spark/commit/087f8a5aa6152a3d30887f144694917ec446354f). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644515692 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore
SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644515779 **[Test build #124088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124088/testReport)** for PR 28835 at commit [`9c6ac83`](https://github.com/apache/spark/commit/9c6ac83afbe26f1277a72c04ac3b1ec40a8c7141). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore
yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644514845 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644514283 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644514283 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore
SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644514044 **[Test build #124087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124087/testReport)** for PR 28835 at commit [`9c6ac83`](https://github.com/apache/spark/commit/9c6ac83afbe26f1277a72c04ac3b1ec40a8c7141). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins removed a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644509192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644509162 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AmplabJenkins commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644509162 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
AmplabJenkins commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644509192 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
SparkQA commented on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644508863 **[Test build #124086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124086/testReport)** for PR 28799 at commit [`7969e44`](https://github.com/apache/spark/commit/7969e44a0c1de6155d5183717cb2c740094b41e7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644508803 **[Test build #124085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124085/testReport)** for PR 28836 at commit [`087f8a5`](https://github.com/apache/spark/commit/087f8a5aa6152a3d30887f144694917ec446354f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu opened a new pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause
AngersZh opened a new pull request #28836: URL: https://github.com/apache/spark/pull/28836 ### What changes were proposed in this pull request? In a SELECT statement, the QUALIFY clause filters the results of window functions. QUALIFY does with window functions what HAVING does with aggregate functions and GROUP BY clauses. In the execution order of a query, QUALIFY is therefore evaluated after window functions are computed. Examples: https://docs.snowflake.com/en/sql-reference/constructs/qualify.html#examples ### Why are the changes needed? Support QUALIFY sentence. make SQL more flexiable. ### Does this PR introduce _any_ user-facing change? People can use `QUALIFY` like below: Create and load a table: ``` create table qt (i integer, p char(1), o integer); insert into qt (i, p, o) values (1, 'A', 1), (2, 'A', 2), (3, 'B', 1), (4, 'B', 2); ``` This query uses nesting rather than QUALIFY: ``` select * from ( select i, p, o, row_number() over (partition by p order by o) as row_num from qt ) where row_num = 1 ; +---+---+---+-+ | I | P | O | ROW_NUM | |---+---+---+-| | 1 | A | 1 | 1 | | 3 | B | 1 | 1 | +---+---+---+-+ ``` This query uses QUALIFY: ``` select i, p, o from qt qualify row_number() over (partition by p order by o) = 1 ; +---+---+---+ | I | P | O | |---+---+---| | 1 | A | 1 | | 3 | B | 1 | +---+---+---+ ``` You can also use QUALIFY to reference window functions that are in the SELECT column list: ``` select i, p, o, row_number() over (partition by p order by o) as row_num from qt qualify row_num = 1 ; +---+---+---+-+ | I | P | O | ROW_NUM | |---+---+---+-| | 1 | A | 1 | 1 | | 3 | B | 1 | 1 | +---+---+---+-+ ``` You can see how the QUALIFY acts as a filter by removing the QUALIFY from the previous query and comparing the output: ``` select i, p, o, row_number() over (partition by p order by o) as row_num from qt ; +---+---+---+-+ | I | P | O | ROW_NUM | |---+---+---+-| | 1 | A | 1 | 1 | | 2 | A | 2 | 2 | | 3 | B | 1 | 1 | | 4 | B | 2 | 2 | +---+---+---+-+ ``` The QUALIFY clause can also be combined with aggregates and can have subqueries in the predicate, e.g.: ``` select c2, sum(c3) over (partition by c2) as r from t1 where c3 < 4 group by c2, c3 having sum(c1) > 3 qualify r in ( select min(c1) from test group by c2 having min(c1) > 3); ``` ### How was this patch tested? Added UT This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sarutak commented on a change in pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
sarutak commented on a change in pull request #28799:
URL: https://github.com/apache/spark/pull/28799#discussion_r440561755
##
File path:
core/src/main/resources/org/apache/spark/ui/static/webui-dataTables.css
##
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+table.dataTable thead .sorting { background: url('images/sort_both.png')
no-repeat bottom right; }
+
+table.dataTable thead .sorting_asc { background: url('images/sort_asc.png')
no-repeat bottom right; }
+
+table.dataTable thead .sorting_desc { background: url('images/sort_desc.png')
no-repeat bottom right; }
Review comment:
Yes, as the warning suggests.
https://user-images.githubusercontent.com/4736016/84727938-1db75f00-afcb-11ea-83df-78aaf2242848.png;>
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] liucht-inspur commented on a change in pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column
liucht-inspur commented on a change in pull request #28799:
URL: https://github.com/apache/spark/pull/28799#discussion_r440560179
##
File path:
core/src/main/resources/org/apache/spark/ui/static/webui-dataTables.css
##
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+table.dataTable thead .sorting { background: url('images/sort_both.png')
no-repeat bottom right; }
+
+table.dataTable thead .sorting_asc { background: url('images/sort_asc.png')
no-repeat bottom right; }
+
+table.dataTable thead .sorting_desc { background: url('images/sort_desc.png')
no-repeat bottom right; }
Review comment:
Add just one blank line at the end?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
