[GitHub] [spark] HyukjinKwon commented on pull request #32517: [SPARK-35381][R] Fix lambda variable name issues in nested higher order functions at R APIs

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32517: URL: https://github.com/apache/spark/pull/32517#issuecomment-839427517 cc @maropu @zero323 @ueshin FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon opened a new pull request #32517: [SPARK-35381][R] Fix lambda variable name issues in nested higher order functions at R APIs

2021-05-11 Thread GitBox
HyukjinKwon opened a new pull request #32517: URL: https://github.com/apache/spark/pull/32517 ### What changes were proposed in this pull request? This PR fixes the same issue as https://github.com/apache/spark/pull/32424 ```r df <- sql("SELECT array(1, 2, 3) as numbers, arr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138424/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425081 **[Test build #138424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit [`a287ee1`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138424/ -- This

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425510 **[Test build #138424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit [`a287ee1`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839422615 **[Test build #138420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138420/testReport)** for PR 32515 at commit [`baa61a3`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839425100 **[Test build #138425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138425/testReport)** for PR 32512 at commit [`946dd27`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424925 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138414/ -

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425081 **[Test build #138424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit [`a287ee1`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839382118 **[Test build #138414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138414/testReport)** for PR 32511 at commit [`416dd90`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424925 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138414/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] SparkQA commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
SparkQA commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424810 **[Test build #138414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138414/testReport)** for PR 32511 at commit [`416dd90`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839423915 **[Test build #138423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138423/testReport)** for PR 32516 at commit [`99f55c6`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839424336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138423/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839424336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138423/ -- This

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839424322 **[Test build #138423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138423/testReport)** for PR 32516 at commit [`99f55c6`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still co

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839423903 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138413/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue r

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839423903 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138413/ -- This

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839423915 **[Test build #138423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138423/testReport)** for PR 32516 at commit [`99f55c6`](https://github.com

[GitHub] [spark] sarutak commented on a change in pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
sarutak commented on a change in pull request #32512: URL: https://github.com/apache/spark/pull/32512#discussion_r630715420 ## File path: .github/workflows/build_and_test.yml ## @@ -356,6 +356,10 @@ jobs: Rscript -e "install.packages(c('devtools'), repos='https://clou

[GitHub] [spark] SparkQA removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839362651 **[Test build #138413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138413/testReport)** for PR 32399 at commit [`dd431b9`](https://gi

[GitHub] [spark] itholic opened a new pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
itholic opened a new pull request #32516: URL: https://github.com/apache/spark/pull/32516 ### What changes were proposed in this pull request? There are still naming related to Koalas in test and function name. This PR addressed them to fit pandas-on-spark. - kdf -> psdf - kser

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

2021-05-11 Thread GitBox
SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839423094 **[Test build #138413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138413/testReport)** for PR 32399 at commit [`dd431b9`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
SparkQA commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839422755 **[Test build #138422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138422/testReport)** for PR 32452 at commit [`8c53f5a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

2021-05-11 Thread GitBox
SparkQA commented on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-839422709 **[Test build #138421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138421/testReport)** for PR 32457 at commit [`cdbbffe`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839422410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42938/

[GitHub] [spark] SparkQA commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839422615 **[Test build #138420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138420/testReport)** for PR 32515 at commit [`baa61a3`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-839422531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42941/

[GitHub] [spark] AmplabJenkins commented on pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-839422531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42941/ -- T

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-839422508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839422390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839422410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42938/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839421689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42939/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839421688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
HyukjinKwon commented on a change in pull request #32512: URL: https://github.com/apache/spark/pull/32512#discussion_r630713623 ## File path: .github/workflows/build_and_test.yml ## @@ -356,6 +356,10 @@ jobs: Rscript -e "install.packages(c('devtools'), repos='https://

[GitHub] [spark] AmplabJenkins commented on pull request #32514: [SPARK-35379][SQL]Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32514: URL: https://github.com/apache/spark/pull/32514#issuecomment-839421759 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839421689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42939/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839421688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] SparkQA commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
SparkQA commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839420084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] HyukjinKwon edited a comment on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon edited a comment on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414684 Since R and Python ones are merged into 3.1, I will create separate JIRAs: https://issues.apache.org/jira/browse/SPARK-35381 https://issues.apache.org/jira/br

[GitHub] [spark] yaooqinn opened a new pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
yaooqinn opened a new pull request #32515: URL: https://github.com/apache/spark/pull/32515 ### What changes were proposed in this pull request? In https://github.com/yaooqinn/itachi/issues/8, we had a discussion about the current extension injection for the spark

[GitHub] [spark] SparkQA commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839417163 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42940/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414684 Since R and Python ones are merged into 3.1, I will create separate JIRAs -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] HyukjinKwon commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414276 BTW, it has the same problem in Python and R too. I and @ueshin are working on them as followups. -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] HyukjinKwon removed a comment on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon removed a comment on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414140 Yeah, it has in Python and R too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] SparkQA removed a comment on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839275149 **[Test build #138408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138408/testReport)** for PR 32439 at commit [`0df10d1`](https://gi

[GitHub] [spark] HyukjinKwon commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414140 Yeah, it has in Python and R too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839414101 **[Test build #138408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138408/testReport)** for PR 32439 at commit [`0df10d1`](https://github.co

[GitHub] [spark] linhongliu-db commented on a change in pull request #32497: [SPARK-35366][SQL] Avoid using deprecated `buildForBatch` and `buildForStreaming`

2021-05-11 Thread GitBox
linhongliu-db commented on a change in pull request #32497: URL: https://github.com/apache/spark/pull/32497#discussion_r630706338 ## File path: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWrite.scala ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apa

[GitHub] [spark] cloud-fan commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
cloud-fan commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839410836 Why it's a problem only in scala API? how about SQL API? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] wankunde opened a new pull request #32514: Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread GitBox
wankunde opened a new pull request #32514: URL: https://github.com/apache/spark/pull/32514 ### What changes were proposed in this pull request? This PR try to improve `InferFiltersFromConstraints` performance via avoid generating too many constraints. For example: ```j

[GitHub] [spark] SparkQA commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839403755 **[Test build #138419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138419/testReport)** for PR 32439 at commit [`2408a18`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839403654 **[Test build #138417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138417/testReport)** for PR 32512 at commit [`2df99a4`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
SparkQA commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839403661 **[Test build #138418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138418/testReport)** for PR 32506 at commit [`4cf72e8`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-839403607 **[Test build #138416 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138416/testReport)** for PR 32513 at commit [`40cea6b`](https://github.com

[GitHub] [spark] beliefer opened a new pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
beliefer opened a new pull request #32513: URL: https://github.com/apache/spark/pull/32513 ### What changes were proposed in this pull request? Currently, Spark doesn't support `LeafRunnableCommand` as sub query. Because the `LeafRunnableCommand` always output `GenericInternalRow` and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839402590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42937/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839402591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138407/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839402593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42936/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839402592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138406/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839402589 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138409/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839402592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138406/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839402591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138407/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839402590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42937/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839402593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42936/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839402589 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138409/ -- This

[GitHub] [spark] SparkQA commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
SparkQA commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839401304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA removed a comment on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839231915 **[Test build #138407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138407/testReport)** for PR 32476 at commit [`617f89c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
SparkQA commented on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839398453 **[Test build #138407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138407/testReport)** for PR 32476 at commit [`617f89c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
SparkQA commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839398303 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42936/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839322016 **[Test build #138409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138409/testReport)** for PR 32509 at commit [`bfa899d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839396565 **[Test build #138409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138409/testReport)** for PR 32509 at commit [`bfa899d`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839231723 **[Test build #138406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138406/testReport)** for PR 32506 at commit [`fabab22`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
SparkQA commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839395754 **[Test build #138406 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138406/testReport)** for PR 32506 at commit [`fabab22`](https://github.co

[GitHub] [spark] viirya commented on a change in pull request #32474: [SPARK-35347][SQL] Use MethodUtils for looking up methods in Invoke and StaticInvoke

2021-05-11 Thread GitBox
viirya commented on a change in pull request #32474: URL: https://github.com/apache/spark/pull/32474#discussion_r630690560 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -147,30 +149,11 @@ trait InvokeLike extend

[GitHub] [spark] kiszk commented on a change in pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

2021-05-11 Thread GitBox
kiszk commented on a change in pull request #32457: URL: https://github.com/apache/spark/pull/32457#discussion_r630690037 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala ## @@ -180,40 +180,55 @@ case class ExpandExec( } }.u

[GitHub] [spark] sarutak opened a new pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
sarutak opened a new pull request #32512: URL: https://github.com/apache/spark/pull/32512 ### What changes were proposed in this pull request? SPARK-35175 (#32274) added a linter for JS so let's add it to GA. ### Why are the changes needed? To JS code keep clean.

[GitHub] [spark] cloud-fan commented on a change in pull request #32474: [SPARK-35347][SQL] Use MethodUtils for looking up methods in Invoke and StaticInvoke

2021-05-11 Thread GitBox
cloud-fan commented on a change in pull request #32474: URL: https://github.com/apache/spark/pull/32474#discussion_r630684354 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -147,30 +149,11 @@ trait InvokeLike ext

[GitHub] [spark] cloud-fan commented on a change in pull request #32507: [SPARK-35361][SQL] Improve performance for ApplyFunctionExpression

2021-05-11 Thread GitBox
cloud-fan commented on a change in pull request #32507: URL: https://github.com/apache/spark/pull/32507#discussion_r630683577 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ApplyFunctionExpression.scala ## @@ -30,10 +30,11 @@ case class App

[GitHub] [spark] fhygh commented on a change in pull request #32501: [SPARK-35359][SQL] Insert data with char/varchar datatype will fail when data length exceed length limitation

2021-05-11 Thread GitBox
fhygh commented on a change in pull request #32501: URL: https://github.com/apache/spark/pull/32501#discussion_r630681959 ## File path: sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala ## @@ -805,6 +805,18 @@ class FileSourceCharVarcharTestSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42934/

[GitHub] [spark] SparkQA commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
SparkQA commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385174 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42934/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42934/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839381998 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42933/

[GitHub] [spark] SparkQA commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
SparkQA commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839382118 **[Test build #138414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138414/testReport)** for PR 32511 at commit [`416dd90`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
SparkQA commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839382143 **[Test build #138415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138415/testReport)** for PR 32510 at commit [`7f819ea`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839381998 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42933/ -- T

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839381984 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42933/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still co

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839381277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42935/

[GitHub] [spark] AmplabJenkins commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue r

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839381277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42935/ -- T

[GitHub] [spark] SparkQA commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
SparkQA commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839380829 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42934/ -- This is an automated message from the Apache

[GitHub] [spark] shahidki31 edited a comment on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 edited a comment on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377762 > @shahidki31 can you retrigger the test please? https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon Let me check -- This is an auto

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839377859 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42933/ -- This is an automated message from the Apache

[GitHub] [spark] shahidki31 removed a comment on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 removed a comment on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377516 > https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon . Let me check -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] shahidki31 commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377762 > @shahidki31 can you retrigger the test please? https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon Let me check -- This is an automated messa

[GitHub] [spark] shahidki31 commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377516 > https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon . Let me check -- This is an automated message from the Apache Git Service. To respond

<    1   2   3   4   5   6   7   8   >