[GitHub] [spark] AmplabJenkins removed a comment on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657404261 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657404261 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657404294 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657404294 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #29057: [SPARK-32245][INFRA] Run Spark tests in Github Actions

2020-07-13 Thread GitBox
HyukjinKwon commented on pull request #29057: URL: https://github.com/apache/spark/pull/29057#issuecomment-657455681 @dbtsai that sounds great. I will leave the link at SPARK-32253 This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657455734 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
SparkQA commented on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657455001 **[Test build #125763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125763/testReport)** for PR 29064 at commit

[GitHub] [spark] SaurabhChawla100 commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
SaurabhChawla100 commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657455284 > @dongjoon-hyun / @maropu - Its seems like some issue happened after I rebase this current branch with the master branch. > > Its now showing number of files

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493848 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493848 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

2020-07-13 Thread GitBox
HyukjinKwon commented on a change in pull request #29086: URL: https://github.com/apache/spark/pull/29086#discussion_r453572801 ## File path: dev/run-tests.py ## @@ -589,43 +627,74 @@ def main(): # /home/jenkins/anaconda2/envs/py36/bin os.environ["PATH"] =

[GitHub] [spark] weixiuli commented on pull request #28994: [SPARK-32170][CORE] Improve the speculation for the inefficient tasks by the task metrics.

2020-07-13 Thread GitBox
weixiuli commented on pull request #28994: URL: https://github.com/apache/spark/pull/28994#issuecomment-657384538 @maropu @cloud-fan @gatorsmile @mridulm @dongjoon-hyun Could you help check this PR? Thanks. This is an

[GitHub] [spark] AmplabJenkins commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-657384480 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #28996: [SPARK-29358][SQL] Make unionByName optionally fill missing columns with nulls

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #28996: URL: https://github.com/apache/spark/pull/28996#discussion_r453460284 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -2048,19 +2088,34 @@ class Dataset[T] private[sql]( // Builds a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657400982 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657400982 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

2020-07-13 Thread GitBox
SparkQA commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657423819 **[Test build #125757 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125757/testReport)** for PR 29084 at commit

[GitHub] [spark] SparkQA commented on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

2020-07-13 Thread GitBox
SparkQA commented on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657423860 **[Test build #125758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125758/testReport)** for PR 29075 at commit

[GitHub] [spark] SparkQA commented on pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
SparkQA commented on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657423952 **[Test build #125759 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125759/testReport)** for PR 29064 at commit

[GitHub] [spark] SparkQA commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
SparkQA commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657423953 **[Test build #125760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125760/testReport)** for PR 29045 at commit

[GitHub] [spark] yaooqinn commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
yaooqinn commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453525655 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] [spark] yaooqinn commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
yaooqinn commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453525655 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657453846 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657446183 **[Test build #125762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125762/testReport)** for PR 27735 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657453846 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
SparkQA commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657453623 **[Test build #125762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125762/testReport)** for PR 27735 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657453017 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
SparkQA commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657460682 **[Test build #125756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125756/testReport)** for PR 27428 at commit

[GitHub] [spark] beliefer commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
beliefer commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453537350 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] SparkQA commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
SparkQA commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657461188 **[Test build #125761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125761/testReport)** for PR 27428 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657460424 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657461072 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657460424 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-13 Thread GitBox
SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480292 **[Test build #125768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125768/testReport)** for PR 29085 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657471298 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

2020-07-13 Thread GitBox
SparkQA commented on pull request #29086: URL: https://github.com/apache/spark/pull/29086#issuecomment-657493419 **[Test build #125769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125769/testReport)** for PR 29086 at commit

[GitHub] [spark] dbtsai commented on pull request #29057: [SPARK-32245][INFRA] Run Spark tests in Github Actions

2020-07-13 Thread GitBox
dbtsai commented on pull request #29057: URL: https://github.com/apache/spark/pull/29057#issuecomment-657401623 @HyukjinKwon and @dongjoon-hyun seems there are plugs for Github actions to show the failing tests, for example https://github.com/check-run-reporter/action This might help for

[GitHub] [spark] dbtsai edited a comment on pull request #29057: [SPARK-32245][INFRA] Run Spark tests in Github Actions

2020-07-13 Thread GitBox
dbtsai edited a comment on pull request #29057: URL: https://github.com/apache/spark/pull/29057#issuecomment-657401623 Great job! @HyukjinKwon and @dongjoon-hyun seems there are plugs for Github actions to show the failing tests, for example https://github.com/check-run-reporter/action

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657455734 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657455720 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657455720 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
SparkQA commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657467768 **[Test build #125766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125766/testReport)** for PR 27735 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657459685 **[Test build #125766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125766/testReport)** for PR 27735 at commit

[GitHub] [spark] beliefer commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
beliefer commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453544249 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657468027 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] beliefer commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
beliefer commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453544249 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657468027 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-657473159 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657480218 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

2020-07-13 Thread GitBox
SparkQA commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-657384095 **[Test build #125748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125748/testReport)** for PR 27694 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-13 Thread GitBox
dongjoon-hyun commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-657383898 Hi, @ulysses-you . We already choose the plan. This is a step to forbid that gracefully. For `create view v1 as select null as col`, we can add an `AnalysisException`

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for creating table with void column datatype

2020-07-13 Thread GitBox
dongjoon-hyun edited a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-657383898 Hi, @ulysses-you . We already choose the plan. This is a step to forbid that gracefully. For `create view v1 as select null as col`, we can add an

[GitHub] [spark] cloud-fan commented on a change in pull request #29061: [SPARK-32258][SQL] NormalizeFloatingNumbers directly normalizes IF/CaseWhen/Coalesce child expressions

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #29061: URL: https://github.com/apache/spark/pull/29061#discussion_r453458611 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NormalizeFloatingNumbers.scala ## @@ -116,6 +116,15 @@ object

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
dongjoon-hyun commented on a change in pull request #28957: URL: https://github.com/apache/spark/pull/28957#discussion_r453462469 ## File path: dev/lint-python ## @@ -168,7 +168,15 @@ function sphinx_test { # Check that the documentation builds acceptably, skip check if

[GitHub] [spark] SparkQA commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
SparkQA commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657393233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657393340 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657393278 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657367971 **[Test build #125751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125751/testReport)** for PR 28363 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657393265 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657361898 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657393312 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-657393281 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657393342 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657337043 **[Test build #125746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125746/testReport)** for PR 29075 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29075: [SPARK-32284][SQL] Avoid expanding too many CNF predicates in partition pruning

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29075: URL: https://github.com/apache/spark/pull/29075#issuecomment-657393340 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

2020-07-13 Thread GitBox
cloud-fan commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-657422187 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453513546 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657442634 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453520007 ## File path: sql/core/src/test/resources/sql-tests/results/timezone.sql.out ## @@ -0,0 +1,121 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] SparkQA commented on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
SparkQA commented on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657452051 **[Test build #125753 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125753/testReport)** for PR 29018 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657394110 **[Test build #125753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125753/testReport)** for PR 29018 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657453017 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29086: [SPARK-32292][SPARK-32252][INFRA] Run the relevant tests only in GitHub Actions

2020-07-13 Thread GitBox
HyukjinKwon commented on a change in pull request #29086: URL: https://github.com/apache/spark/pull/29086#discussion_r453573370 ## File path: dev/run-tests.py ## @@ -589,43 +627,74 @@ def main(): # /home/jenkins/anaconda2/envs/py36/bin os.environ["PATH"] =

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29064: URL: https://github.com/apache/spark/pull/29064#issuecomment-657391382 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
HyukjinKwon commented on a change in pull request #28957: URL: https://github.com/apache/spark/pull/28957#discussion_r453467521 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -180,7 +173,7 @@ def create_array(s, t): if len(s) == 0 and

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

2020-07-13 Thread GitBox
AngersZh commented on a change in pull request #29084: URL: https://github.com/apache/spark/pull/29084#discussion_r453496821 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -191,15 +191,19 @@ abstract class SparkStrategies

[GitHub] [spark] cloud-fan commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453509648 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] Ngone51 commented on a change in pull request #29015: [SPARK-32215] Expose a (protected) /workers/kill endpoint on the MasterWebUI

2020-07-13 Thread GitBox
Ngone51 commented on a change in pull request #29015: URL: https://github.com/apache/spark/pull/29015#discussion_r453521642 ## File path: core/src/main/scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala ## @@ -49,6 +56,23 @@ class MasterWebUI( "/app/kill", "/",

[GitHub] [spark] beliefer commented on a change in pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
beliefer commented on a change in pull request #27428: URL: https://github.com/apache/spark/pull/27428#discussion_r453539953 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -102,23 +102,126 @@ import

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657461572 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453546490 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf:

[GitHub] [spark] SparkQA removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

2020-07-13 Thread GitBox
SparkQA removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-657340452 **[Test build #125748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125748/testReport)** for PR 27694 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28996: [SPARK-29358][SQL] Make unionByName optionally fill missing columns with nulls

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #28996: URL: https://github.com/apache/spark/pull/28996#discussion_r453459502 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -2030,7 +2030,25 @@ class Dataset[T] private[sql]( * @group typedrel

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-657384480 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
SparkQA commented on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657394110 **[Test build #125753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125753/testReport)** for PR 29018 at commit

[GitHub] [spark] SparkQA commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657403743 **[Test build #125755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125755/testReport)** for PR 28957 at commit

[GitHub] [spark] SparkQA commented on pull request #27428: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

2020-07-13 Thread GitBox
SparkQA commented on pull request #27428: URL: https://github.com/apache/spark/pull/27428#issuecomment-657403782 **[Test build #125756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125756/testReport)** for PR 27428 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453518043 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala ## @@ -90,6 +93,51 @@ class SparkSqlAstBuilder(conf:

[GitHub] [spark] SparkQA commented on pull request #29045: [SPARK-32234][SQL] Spark sql commands are failing on selecting the orc tables

2020-07-13 Thread GitBox
SparkQA commented on pull request #29045: URL: https://github.com/apache/spark/pull/29045#issuecomment-657459572 **[Test build #125764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125764/testReport)** for PR 29045 at commit

[GitHub] [spark] SparkQA commented on pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-657459581 **[Test build #125765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125765/testReport)** for PR 28957 at commit

[GitHub] [spark] SparkQA commented on pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-07-13 Thread GitBox
SparkQA commented on pull request #27735: URL: https://github.com/apache/spark/pull/27735#issuecomment-657459685 **[Test build #125766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125766/testReport)** for PR 27735 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480868 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-657480868 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
dongjoon-hyun commented on a change in pull request #28957: URL: https://github.com/apache/spark/pull/28957#discussion_r453462469 ## File path: dev/lint-python ## @@ -168,7 +168,15 @@ function sphinx_test { # Check that the documentation builds acceptably, skip check if

[GitHub] [spark] cloud-fan commented on a change in pull request #29064: [SPARK-32272][SQL] Add and extend SQL standard command SET TIME ZONE

2020-07-13 Thread GitBox
cloud-fan commented on a change in pull request #29064: URL: https://github.com/apache/spark/pull/29064#discussion_r453462372 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -240,6 +240,10 @@ statement | MSCK REPAIR TABLE

[GitHub] [spark] SparkQA commented on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

2020-07-13 Thread GitBox
SparkQA commented on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-657388961 **[Test build #125741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125741/testReport)** for PR 28953 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28957: [SPARK-32138] Drop Python 2.7, 3.4 and 3.5

2020-07-13 Thread GitBox
HyukjinKwon commented on a change in pull request #28957: URL: https://github.com/apache/spark/pull/28957#discussion_r453468016 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -180,7 +173,7 @@ def create_array(s, t): if len(s) == 0 and

[GitHub] [spark] AmplabJenkins commented on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
AmplabJenkins commented on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657394198 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29018: [SPARK-32202][ML][WIP] tree models auto infer compact integer type

2020-07-13 Thread GitBox
AmplabJenkins removed a comment on pull request #29018: URL: https://github.com/apache/spark/pull/29018#issuecomment-657394198 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29084: [SPARK-32220][SQL][FOLLOW-UP]SHUFFLE_REPLICATE_NL Hint should not change Non-Cartesian Product join result

2020-07-13 Thread GitBox
SparkQA commented on pull request #29084: URL: https://github.com/apache/spark/pull/29084#issuecomment-657400399 **[Test build #125754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125754/testReport)** for PR 29084 at commit

  1   2   3   4   5   6   7   >