[GitHub] [spark] AmplabJenkins removed a comment on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722936128 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

2020-11-05 Thread GitBox
SparkQA commented on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722936114 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35306/

[GitHub] [spark] HeartSaVioR commented on pull request #30186: [SPARK-23432][UI] Add executor peak jvm memory metrics in executors page

2020-11-05 Thread GitBox
HeartSaVioR commented on pull request #30186: URL: https://github.com/apache/spark/pull/30186#issuecomment-722936210 Never mind. I tried your Github account and found it. Please let me know if I misassigned. This is an

[GitHub] [spark] SparkQA commented on pull request #30251: [SPARK-33347][CORE]Cleanup useless variables of MutableApplicationInfo

2020-11-05 Thread GitBox
SparkQA commented on pull request #30251: URL: https://github.com/apache/spark/pull/30251#issuecomment-722936109 **[Test build #130701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130701/testReport)** for PR 30251 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722936128 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #30186: [SPARK-23432][UI] Add executor peak jvm memory metrics in executors page

2020-11-05 Thread GitBox
HeartSaVioR commented on pull request #30186: URL: https://github.com/apache/spark/pull/30186#issuecomment-722935367 Thanks! Merged into master. @warrenzhu25 Could you please let me know your JIRA ID? I'm not 100% sure so I left the assignee as unassigned. Once I get the information

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
dongjoon-hyun commented on a change in pull request #30276: URL: https://github.com/apache/spark/pull/30276#discussion_r518577972 ## File path: mllib-local/src/test/scala/org/apache/spark/ml/linalg/MatricesSuite.scala ## @@ -862,10 +862,10 @@ class MatricesSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722934583 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722934583 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR closed pull request #30186: [SPARK-23432][UI] Add executor peak jvm memory metrics in executors page

2020-11-05 Thread GitBox
HeartSaVioR closed pull request #30186: URL: https://github.com/apache/spark/pull/30186 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30276: URL: https://github.com/apache/spark/pull/30276#issuecomment-722933731 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA removed a comment on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722817107 **[Test build #130688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130688/testReport)** for PR 30255 at commit

[GitHub] [spark] SparkQA commented on pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
SparkQA commented on pull request #30276: URL: https://github.com/apache/spark/pull/30276#issuecomment-722933702 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35308/

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Support CUBE(...) and ROLLUP(...), GROUPING SETS(...) as group by expr in parser level

2020-11-05 Thread GitBox
maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r518576347 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,26 @@ fromClause ;

[GitHub] [spark] HeartSaVioR commented on pull request #30251: [SPARK-33347][CORE]Cleanup useless variables of MutableApplicationInfo

2020-11-05 Thread GitBox
HeartSaVioR commented on pull request #30251: URL: https://github.com/apache/spark/pull/30251#issuecomment-722933580 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30276: URL: https://github.com/apache/spark/pull/30276#issuecomment-722933731 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722933391 **[Test build #130688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130688/testReport)** for PR 30255 at commit

[GitHub] [spark] maropu commented on a change in pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
maropu commented on a change in pull request #30276: URL: https://github.com/apache/spark/pull/30276#discussion_r518573436 ## File path: mllib-local/src/test/scala/org/apache/spark/ml/linalg/MatricesSuite.scala ## @@ -862,10 +862,10 @@ class MatricesSuite extends

[GitHub] [spark] maropu commented on a change in pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
maropu commented on a change in pull request #30276: URL: https://github.com/apache/spark/pull/30276#discussion_r518573436 ## File path: mllib-local/src/test/scala/org/apache/spark/ml/linalg/MatricesSuite.scala ## @@ -862,10 +862,10 @@ class MatricesSuite extends

[GitHub] [spark] viirya commented on a change in pull request #30245: [SPARK-33337][SQL] Support subexpression elimination in branches of conditional expressions

2020-11-05 Thread GitBox
viirya commented on a change in pull request #30245: URL: https://github.com/apache/spark/pull/30245#discussion_r518571332 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -65,11 +65,46 @@ class

[GitHub] [spark] SparkQA commented on pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
SparkQA commented on pull request #30276: URL: https://github.com/apache/spark/pull/30276#issuecomment-722929230 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35308/

[GitHub] [spark] SparkQA commented on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

2020-11-05 Thread GitBox
SparkQA commented on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722927854 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35306/

[GitHub] [spark] maropu commented on a change in pull request #29170: [SPARK-30876][SQL] Optimizer fails to infer constraints within join

2020-11-05 Thread GitBox
maropu commented on a change in pull request #29170: URL: https://github.com/apache/spark/pull/29170#discussion_r518567508 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -116,7 +116,8 @@ abstract class

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722925083 **[Test build #130700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130700/testReport)** for PR 30255 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722919383 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30274: [SPARK-32860][DOCS][SQL] Updating documentation about map support in Encoders

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722920533 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722917895 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30275: [SPARK-32691][BUILD] Update commons-crypto to v1.1.0

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30275: URL: https://github.com/apache/spark/pull/30275#issuecomment-722917000 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] SparkQA removed a comment on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA removed a comment on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722917156 **[Test build #130697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130697/testReport)** for PR 30255 at commit

[GitHub] [spark] maropu commented on pull request #29170: [SPARK-30876][SQL] Optimizer fails to infer constraints within join

2020-11-05 Thread GitBox
maropu commented on pull request #29170: URL: https://github.com/apache/spark/pull/29170#issuecomment-722922863 > @navinvishy, @maropu, @cloud-fan, @gatorsmile do you think you can revisit this PR? Revisiting this looks fine to me. Are you still here? @navinvishy

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722922482 **[Test build #130699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130699/testReport)** for PR 30255 at commit

[GitHub] [spark] SparkQA commented on pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
SparkQA commented on pull request #30276: URL: https://github.com/apache/spark/pull/30276#issuecomment-722922445 **[Test build #130698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130698/testReport)** for PR 30276 at commit

[GitHub] [spark] dongjoon-hyun opened a new pull request #30276: [SPARK-33333][BUILD][2.4] Upgrade Jetty to 9.4.28.v20200408

2020-11-05 Thread GitBox
dongjoon-hyun opened a new pull request #30276: URL: https://github.com/apache/spark/pull/30276 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] AmplabJenkins commented on pull request #30274: [SPARK-32860][DOCS][SQL] Updating documentation about map support in Encoders

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722920533 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30274: [SPARK-32860][DOCS][SQL] Updating documentation about map support in Encoders

2020-11-05 Thread GitBox
SparkQA commented on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722920519 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35305/

[GitHub] [spark] AmplabJenkins commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722919383 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722919375 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35304/

[GitHub] [spark] AmplabJenkins commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722918725 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722918708 **[Test build #130697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130697/testReport)** for PR 30255 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722917895 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30275: [SPARK-32691][BUILD] Update commons-crypto to v1.1.0

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30275: URL: https://github.com/apache/spark/pull/30275#issuecomment-722917362 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722917156 **[Test build #130697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130697/testReport)** for PR 30255 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30275: [SPARK-32691][BUILD] Update commons-crypto to v1.1.0

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30275: URL: https://github.com/apache/spark/pull/30275#issuecomment-722917000 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30275: [SPARK-32691][BUILD] Update commons-crypto to v1.1.0

2020-11-05 Thread GitBox
dongjoon-hyun edited a comment on pull request #30275: URL: https://github.com/apache/spark/pull/30275#issuecomment-722915891 Thank you for your contribution, @huangtianhua . Could you recover the original PR description style please? Please see a PR like

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30270: [SPARK-33366][SQL] Migrate LOAD DATA command to use UnresolvedTable to resolve the identifier

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30270: URL: https://github.com/apache/spark/pull/30270#issuecomment-722915893 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30270: [SPARK-33366][SQL] Migrate LOAD DATA command to use UnresolvedTable to resolve the identifier

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30270: URL: https://github.com/apache/spark/pull/30270#issuecomment-722915893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #30275: [SPARK-32691][BUILD] Update commons-crypto to v1.1.0

2020-11-05 Thread GitBox
dongjoon-hyun commented on pull request #30275: URL: https://github.com/apache/spark/pull/30275#issuecomment-722915891 Thank you for your contribution, @huangtianhua . Could you recover the original PR description style please?

[GitHub] [spark] SparkQA commented on pull request #30270: [SPARK-33366][SQL] Migrate LOAD DATA command to use UnresolvedTable to resolve the identifier

2020-11-05 Thread GitBox
SparkQA commented on pull request #30270: URL: https://github.com/apache/spark/pull/30270#issuecomment-722915879 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35303/

[GitHub] [spark] huangtianhua opened a new pull request #30275: SPARK-32691 Bump commons-crypto to v1.1.0

2020-11-05 Thread GitBox
huangtianhua opened a new pull request #30275: URL: https://github.com/apache/spark/pull/30275 The package commons-crypto-1.0.0 doesn't support aarch64 platform, it lead the tests of replication failed due to crypto timeout, bump up to v1.1.0 which support aarch64. See:

[GitHub] [spark] dongjoon-hyun commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
dongjoon-hyun commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722914471 This seems to fail still. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722914033 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722914033 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722914004 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35302/

[GitHub] [spark] SparkQA commented on pull request #30274: [SPARK-32860][DOCS][SQL] Updating documentation about map support in Encoders

2020-11-05 Thread GitBox
SparkQA commented on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722911327 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35305/

[GitHub] [spark] SparkQA commented on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

2020-11-05 Thread GitBox
SparkQA commented on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722909952 **[Test build #130696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130696/testReport)** for PR 30154 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30212: [SPARK-33308][SQL] Support CUBE(...) and ROLLUP(...), GROUPING SETS(...) as group by expr in parser level

2020-11-05 Thread GitBox
AngersZh commented on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-722909968 gentle ping @maropu @cloud-fan Do you have any suggestion for this one. A series of PR changes need to be made to accommodate this support

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722909905 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35304/

[GitHub] [spark] AngersZhuuuu removed a comment on pull request #30212: [SPARK-33308][SQL] Support CUBE(...) and ROLLUP(...), GROUPING SETS(...) as group by expr in parser level

2020-11-05 Thread GitBox
AngersZh removed a comment on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-721586653 Any suggestion? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AngersZhuuuu commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

2020-11-05 Thread GitBox
AngersZh commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-722909286 > > Sure, Postgres SQL support this > > Tera DB support too. > > Could you add this info in the PR description? Yea, later for. a more detail. desc

[GitHub] [spark] HyukjinKwon commented on pull request #30242: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

2020-11-05 Thread GitBox
HyukjinKwon commented on pull request #30242: URL: https://github.com/apache/spark/pull/30242#issuecomment-722908940 cc @zsxwing too This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722907809 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722907801 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
SparkQA removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722863678 **[Test build #130694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130694/testReport)** for PR 30222 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722907801 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722907690 **[Test build #130694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130694/testReport)** for PR 30222 at commit

[GitHub] [spark] maropu commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

2020-11-05 Thread GitBox
maropu commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-722907461 > Sure, Postgres SQL support this > Tera DB support too. Could you add this info in the PR description?

[GitHub] [spark] maropu commented on pull request #30260: [SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode

2020-11-05 Thread GitBox
maropu commented on pull request #30260: URL: https://github.com/apache/spark/pull/30260#issuecomment-722906769 Thanks for the request, @gengliangwang ! I'll review this later. This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30273: [SPARK-33369][SQL] Skip schema inference in DataframeWriter.save() if table provider supports external metadata

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30273: URL: https://github.com/apache/spark/pull/30273#issuecomment-722906405 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30273: [SPARK-33369][SQL] Skip schema inference in DataframeWriter.save() if table provider supports external metadata

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30273: URL: https://github.com/apache/spark/pull/30273#issuecomment-722906405 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30273: [SPARK-33369][SQL] Skip schema inference in DataframeWriter.save() if table provider supports external metadata

2020-11-05 Thread GitBox
SparkQA commented on pull request #30273: URL: https://github.com/apache/spark/pull/30273#issuecomment-722906390 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35301/

[GitHub] [spark] maropu commented on a change in pull request #26319: [SPARK-29594][SQL] Provide better error message when creating a Dataset from a Sequence of Case class where a field name started w

2020-11-05 Thread GitBox
maropu commented on a change in pull request #26319: URL: https://github.com/apache/spark/pull/26319#discussion_r518551221 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala ## @@ -543,6 +543,13 @@ object ScalaReflection extends

[GitHub] [spark] SparkQA commented on pull request #30270: [SPARK-33366][SQL] Migrate LOAD DATA command to use UnresolvedTable to resolve the identifier

2020-11-05 Thread GitBox
SparkQA commented on pull request #30270: URL: https://github.com/apache/spark/pull/30270#issuecomment-722901987 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35303/

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

2020-11-05 Thread GitBox
otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r518548353 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -0,0 +1,959 @@ +/* + * Licensed to

[GitHub] [spark] maropu commented on pull request #30251: [SPARK-33347][CORE]Cleanup useless variables of MutableApplicationInfo

2020-11-05 Thread GitBox
maropu commented on pull request #30251: URL: https://github.com/apache/spark/pull/30251#issuecomment-722896647 > What does "binary compatibility check" do? Can you explain it for me @dongjoon-hyun ? Thx ~ NOTE: Since the MiMa plugin checks binary compatibility, you need to update

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722894610 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35302/

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
AngersZh edited a comment on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722892310 > Probably, `arrays_contains(array(1, 2, 3), 2) AND arrays_contains(array(1, 2, 3), 3) `? First, if `arrays_contains_array(array(1, 2, 3, ), array(1, 2,

[GitHub] [spark] AngersZhuuuu commented on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
AngersZh commented on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722892310 > Probably, `arrays_contains(array(1, 2, 3), 2) AND arrays_contains(array(1, 2, 3), 3) `? First, if `arrays_contains_array(array(1, 2, 3, ), array(1, 2, 3, 4,

[GitHub] [spark] maropu commented on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
maropu commented on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722884428 > ArrayOverlap checks if the two arrays contain at least one common element. Ah, I see. You need to describe the clear behaivour in the section, too. Probably,

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
AngersZh edited a comment on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722875681 > arrays_overlap(array(1, 2, 3), array(2, 3))? ArrayOverlap checks if the two arrays contain at least one common element. > Anyway, I think you need to

[GitHub] [spark] SparkQA commented on pull request #30273: [SPARK-33369][SQL] Skip schema inference in DataframeWriter.save() if table provider supports external metadata

2020-11-05 Thread GitBox
SparkQA commented on pull request #30273: URL: https://github.com/apache/spark/pull/30273#issuecomment-722880326 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35301/

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30274: [SPARK-32860][Docs][SQL] Updating documentation

2020-11-05 Thread GitBox
HyukjinKwon commented on a change in pull request #30274: URL: https://github.com/apache/spark/pull/30274#discussion_r518543627 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/Encoders.scala ## @@ -146,7 +146,8 @@ object Encoders { * - String * -

[GitHub] [spark] SparkQA commented on pull request #30274: [SPARK-32860][Docs][SQL] Updating documentation

2020-11-05 Thread GitBox
SparkQA commented on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-72281 **[Test build #130695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130695/testReport)** for PR 30274 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
AngersZh commented on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722875681 > arrays_overlap(array(1, 2, 3), array(2, 3))? ArrayOverlap checks if the two arrays contain at least one common element. > Anyway, I think you need to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722875102 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722875102 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722875077 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35300/

[GitHub] [spark] imback82 commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
imback82 commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722874348 Thanks @dongjoon-hyun / @cloud-fan for the review! This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30274: [SPARK-32860][Docs][SQL] Updating documentation

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722841479 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on pull request #30274: [SPARK-32860][Docs][SQL] Updating documentation

2020-11-05 Thread GitBox
HyukjinKwon commented on pull request #30274: URL: https://github.com/apache/spark/pull/30274#issuecomment-722874048 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] maropu commented on pull request #30243: [SPARK-33335][SQL] Support `array_contains_array` func

2020-11-05 Thread GitBox
maropu commented on pull request #30243: URL: https://github.com/apache/spark/pull/30243#issuecomment-722873034 > If we need to judge SELECT _FUNC_(array(1, 2, 3), array(2, 3)) ? `arrays_overlap(array(1, 2, 3), array(2, 3))`? Anyway, I think you need to describe more in the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins removed a comment on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722869182 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
AmplabJenkins commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722869182 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30255: [WIP][SPARK-33352][CORE][SQL][MLLIB][AVRO][K8S] Fix procedure-like declaration compilation warnings in Scala 2.13

2020-11-05 Thread GitBox
SparkQA commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722869169 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35299/

[GitHub] [spark] dongjoon-hyun commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
dongjoon-hyun commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722865361 Merged to master. The last two commits are only for comment changes. This is an automated message from

[GitHub] [spark] dongjoon-hyun closed pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

2020-11-05 Thread GitBox
dongjoon-hyun closed pull request #30267: URL: https://github.com/apache/spark/pull/30267 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on pull request #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all the outputs are semantic equivalence

2020-11-05 Thread GitBox
viirya commented on pull request #21852: URL: https://github.com/apache/spark/pull/21852#issuecomment-722864515 I agree with @cloud-fan. The `rd` in `CaseWhen` is deterministic already. If it is embeded in `CaseWhen`, this PR should not remove it.

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722863678 **[Test build #130694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130694/testReport)** for PR 30222 at commit

[GitHub] [spark] SparkQA commented on pull request #30270: [SPARK-33366][SQL] Migrate LOAD DATA command to use UnresolvedTable to resolve the identifier

2020-11-05 Thread GitBox
SparkQA commented on pull request #30270: URL: https://github.com/apache/spark/pull/30270#issuecomment-722863596 **[Test build #130693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130693/testReport)** for PR 30270 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
cloud-fan commented on a change in pull request #30222: URL: https://github.com/apache/spark/pull/30222#discussion_r518539170 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -510,6 +510,15 @@ object

[GitHub] [spark] wangyum commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

2020-11-05 Thread GitBox
wangyum commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722858604 We can reproduce it by: ```scala spark.sql("CREATE TABLE t(a int, b int, c int) using parquet") spark.sql( """ |SELECT * | FROM (SELECT CASE

  1   2   3   4   5   6   7   8   >