[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839362554 **[Test build #138411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138411/testReport)** for PR 32473 at commit

[GitHub] [spark] SparkQA commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
SparkQA commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839362618 **[Test build #138412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138412/testReport)** for PR 32452 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #32474: [SPARK-35347][SQL] Use MethodUtils for looking up methods in Invoke and StaticInvoke

2021-05-11 Thread GitBox
cloud-fan commented on a change in pull request #32474: URL: https://github.com/apache/spark/pull/32474#discussion_r630684354 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -147,30 +149,11 @@ trait InvokeLike

[GitHub] [spark] AmplabJenkins commented on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839402589 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138409/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839402593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42936/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839402591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138407/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839402590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42937/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839402592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138406/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839421688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839421689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42939/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32514: [SPARK-35379][SQL]Improve InferFiltersFromConstraints rule performance when parsing spark sql

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32514: URL: https://github.com/apache/spark/pull/32514#issuecomment-839421759 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] maropu commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
maropu commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r630726896 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -103,10 +112,11 @@

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839355590 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42932/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-839361260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138400/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839361259 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42931/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839361258 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42932/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839361258 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42932/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-839361260 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138400/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839361259 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42931/

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

2021-05-11 Thread GitBox
SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839375651 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42935/ --

[GitHub] [spark] cloud-fan commented on a change in pull request #32507: [SPARK-35361][SQL] Improve performance for ApplyFunctionExpression

2021-05-11 Thread GitBox
cloud-fan commented on a change in pull request #32507: URL: https://github.com/apache/spark/pull/32507#discussion_r630683577 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ApplyFunctionExpression.scala ## @@ -30,10 +30,11 @@ case class

[GitHub] [spark] sarutak opened a new pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
sarutak opened a new pull request #32512: URL: https://github.com/apache/spark/pull/32512 ### What changes were proposed in this pull request? SPARK-35175 (#32274) added a linter for JS so let's add it to GA. ### Why are the changes needed? To JS code keep clean.

[GitHub] [spark] HyukjinKwon commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414140 Yeah, it has in Python and R too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA removed a comment on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839275149 **[Test build #138408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138408/testReport)** for PR 32439 at commit

[GitHub] [spark] SparkQA commented on pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

2021-05-11 Thread GitBox
SparkQA commented on pull request #32439: URL: https://github.com/apache/spark/pull/32439#issuecomment-839414101 **[Test build #138408 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138408/testReport)** for PR 32439 at commit

[GitHub] [spark] SparkQA commented on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
SparkQA commented on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839420084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] maropu edited a comment on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
maropu edited a comment on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839430081 > Why it's a problem only in scala API? how about SQL API? In SQL, since user-specified param names are used as they are, the same issue cannot happen; ```

[GitHub] [spark] SparkQA commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839451134 **[Test build #138417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138417/testReport)** for PR 32512 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839403654 **[Test build #138417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138417/testReport)** for PR 32512 at commit

[GitHub] [spark] dongjoon-hyun opened a new pull request #32518: [SPARK-35383][CORE] Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread GitBox
dongjoon-hyun opened a new pull request #32518: URL: https://github.com/apache/spark/pull/32518 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was

[GitHub] [spark] zhengruifeng commented on pull request #32199: [SPARK-35100][ML] Refactor AFT - support virtual centering

2021-05-11 Thread GitBox
zhengruifeng commented on pull request #32199: URL: https://github.com/apache/spark/pull/32199#issuecomment-839467796 @srowen It should be safe to wait for BLAS related changes. I will keep this PR open. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA removed a comment on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-839056599 **[Test build #138400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138400/testReport)** for PR 32494 at commit

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-839356893 **[Test build #138400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138400/testReport)** for PR 32494 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839369117 cc @Ngone51 @mridulm @tgravescs FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon removed a comment on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
HyukjinKwon removed a comment on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839368779 The tests seem failed. Would you please fixing them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839368998 @shahidki31 can you retrigger the test please? https://github.com/shahidki31/spark/actions/runs/832876763 -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839368779 The tests seem failed. Would you please fixing them? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
SparkQA commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839401304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138424/ -- This

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425510 **[Test build #138424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425081 **[Test build #138424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138424/

[GitHub] [spark] SparkQA removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839322042 **[Test build #138410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138410/testReport)** for PR 32473 at commit

[GitHub] [spark] Ngone51 commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-11 Thread GitBox
Ngone51 commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-839437814 Kindly ping @dongjoon-hyun Do you have any more comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839452238 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138417/ -- This

[GitHub] [spark] SparkQA commented on pull request #32517: [SPARK-35381][R] Fix lambda variable name issues in nested higher order functions at R APIs

2021-05-11 Thread GitBox
SparkQA commented on pull request #32517: URL: https://github.com/apache/spark/pull/32517#issuecomment-839451396 **[Test build #138426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138426/testReport)** for PR 32517 at commit

[GitHub] [spark] SparkQA commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839451531 **[Test build #138428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138428/testReport)** for PR 32515 at commit

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839451472 **[Test build #138427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138427/testReport)** for PR 32516 at commit

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r630736286 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -103,10 +112,11 @@

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

2021-05-11 Thread GitBox
shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r630736159 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -86,11 +86,20 @@

[GitHub] [spark] dongjoon-hyun commented on pull request #32518: [SPARK-35383][CORE] Improve s3a magic committer support by inferring missing configs

2021-05-11 Thread GitBox
dongjoon-hyun commented on pull request #32518: URL: https://github.com/apache/spark/pull/32518#issuecomment-839470621 Hi, @steveloughran . Could you review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] srowen commented on pull request #32508: [MINOR][DOCS] Avoid some python docs where first sentence has "e.g." or similar

2021-05-11 Thread GitBox
srowen commented on pull request #32508: URL: https://github.com/apache/spark/pull/32508#issuecomment-839373124 BTW I realize the link I posted doesn't actually show the problem. It's here:

[GitHub] [spark] maropu opened a new pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
maropu opened a new pull request #32511: URL: https://github.com/apache/spark/pull/32511 ### What changes were proposed in this pull request? This PR proposes to print formatted strings for logical plans in the `formatted` mode, e.g., ``` scala> sql("""SELECT (SELECT

[GitHub] [spark] itholic commented on a change in pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

2021-05-11 Thread GitBox
itholic commented on a change in pull request #32161: URL: https://github.com/apache/spark/pull/32161#discussion_r630663662 ## File path: python/pyspark/sql/readwriter.py ## @@ -416,53 +416,10 @@ def parquet(self, *paths, **options): Other Parameters

[GitHub] [spark] shahidki31 commented on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 commented on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377516 > https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon . Let me check -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42934/

[GitHub] [spark] AmplabJenkins commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42934/ --

[GitHub] [spark] SparkQA commented on pull request #32452: [SPARK-35243][SQL] Support columnar execution on ANSI interval types

2021-05-11 Thread GitBox
SparkQA commented on pull request #32452: URL: https://github.com/apache/spark/pull/32452#issuecomment-839385174 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42934/ -- This is an automated message from the

[GitHub] [spark] beliefer opened a new pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
beliefer opened a new pull request #32513: URL: https://github.com/apache/spark/pull/32513 ### What changes were proposed in this pull request? Currently, Spark doesn't support `LeafRunnableCommand` as sub query. Because the `LeafRunnableCommand` always output `GenericInternalRow` and

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Convert LeafRunnableCommand to LocalRelation when query with CTE

2021-05-11 Thread GitBox
SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-839403607 **[Test build #138416 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138416/testReport)** for PR 32513 at commit

[GitHub] [spark] HyukjinKwon edited a comment on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon edited a comment on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414684 Since R and Python ones are merged into 3.1, I will create separate JIRAs: https://issues.apache.org/jira/browse/SPARK-35381

[GitHub] [spark] HyukjinKwon opened a new pull request #32517: [SPARK-35381][R] Fix lambda variable name issues in nested higher order functions at R APIs

2021-05-11 Thread GitBox
HyukjinKwon opened a new pull request #32517: URL: https://github.com/apache/spark/pull/32517 ### What changes were proposed in this pull request? This PR fixes the same issue as https://github.com/apache/spark/pull/32424 ```r df <- sql("SELECT array(1, 2, 3) as numbers,

[GitHub] [spark] HyukjinKwon commented on pull request #32517: [SPARK-35381][R] Fix lambda variable name issues in nested higher order functions at R APIs

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32517: URL: https://github.com/apache/spark/pull/32517#issuecomment-839427517 cc @maropu @zero323 @ueshin FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] shahidki31 edited a comment on pull request #32504: [SPARK-35013][CORE] Don't allow to set spark.driver.cores=0

2021-05-11 Thread GitBox
shahidki31 edited a comment on pull request #32504: URL: https://github.com/apache/spark/pull/32504#issuecomment-839377762 > @shahidki31 can you retrigger the test please? https://github.com/shahidki31/spark/actions/runs/832876763 Sure @HyukjinKwon Let me check -- This is an

[GitHub] [spark] AmplabJenkins commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue r

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839381277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42935/ --

[GitHub] [spark] fhygh commented on a change in pull request #32501: [SPARK-35359][SQL] Insert data with char/varchar datatype will fail when data length exceed length limitation

2021-05-11 Thread GitBox
fhygh commented on a change in pull request #32501: URL: https://github.com/apache/spark/pull/32501#discussion_r630681959 ## File path: sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala ## @@ -805,6 +805,18 @@ class FileSourceCharVarcharTestSuite extends

[GitHub] [spark] viirya commented on a change in pull request #32474: [SPARK-35347][SQL] Use MethodUtils for looking up methods in Invoke and StaticInvoke

2021-05-11 Thread GitBox
viirya commented on a change in pull request #32474: URL: https://github.com/apache/spark/pull/32474#discussion_r630690560 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -147,30 +149,11 @@ trait InvokeLike

[GitHub] [spark] SparkQA removed a comment on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839231915 **[Test build #138407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138407/testReport)** for PR 32476 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32509: [SPARK-35375][INFRA] Use Jinja2 < 3.0.0 for Python linter dependency in GA

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32509: URL: https://github.com/apache/spark/pull/32509#issuecomment-839402589 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138409/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32506: [SPARK-35374][SQL] Add string-to-number conversion support to JacksonParser

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32506: URL: https://github.com/apache/spark/pull/32506#issuecomment-839402592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138406/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839402593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42936/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32476: URL: https://github.com/apache/spark/pull/32476#issuecomment-839402591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138407/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839402590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42937/

[GitHub] [spark] HyukjinKwon removed a comment on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon removed a comment on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414140 Yeah, it has in Python and R too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon commented on pull request #32424: [SPARK-34794][SQL] Fix lambda variable name issues in nested DataFrame functions

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32424: URL: https://github.com/apache/spark/pull/32424#issuecomment-839414276 BTW, it has the same problem in Python and R too. I and @ueshin are working on them as followups. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] yaooqinn opened a new pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
yaooqinn opened a new pull request #32515: URL: https://github.com/apache/spark/pull/32515 ### What changes were proposed in this pull request? In https://github.com/yaooqinn/itachi/issues/8, we had a discussion about the current extension injection for the spark

[GitHub] [spark] SparkQA removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839362651 **[Test build #138413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138413/testReport)** for PR 32399 at commit

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

2021-05-11 Thread GitBox
SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-839423094 **[Test build #138413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138413/testReport)** for PR 32399 at commit

[GitHub] [spark] itholic opened a new pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
itholic opened a new pull request #32516: URL: https://github.com/apache/spark/pull/32516 ### What changes were proposed in this pull request? There are still naming related to Koalas in test and function name. This PR addressed them to fit pandas-on-spark. - kdf -> psdf -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424925 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138414/

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839425081 **[Test build #138424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138424/testReport)** for PR 32516 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839382118 **[Test build #138414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138414/testReport)** for PR 32511 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839422615 **[Test build #138420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138420/testReport)** for PR 32515 at commit

[GitHub] [spark] SparkQA commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
SparkQA commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839425100 **[Test build #138425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138425/testReport)** for PR 32512 at commit

[GitHub] [spark] SparkQA commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
SparkQA commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424810 **[Test build #138414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138414/testReport)** for PR 32511 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32511: [WIP][SPARK-XXXXX][SQL] Show explain formatted strings for logical plans

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32511: URL: https://github.com/apache/spark/pull/32511#issuecomment-839424925 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138414/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32515: [WIP][SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-839424800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-05-11 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-839428571 **[Test build #138410 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138410/testReport)** for PR 32473 at commit

[GitHub] [spark] SparkQA commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
SparkQA commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839443187 **[Test build #138415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138415/testReport)** for PR 32510 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
SparkQA removed a comment on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839382143 **[Test build #138415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138415/testReport)** for PR 32510 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-839449400 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42943/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32512: [SPARK-35377][INFRA] Add JS linter to GA

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32512: URL: https://github.com/apache/spark/pull/32512#issuecomment-839449457 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42946/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32510: [SPARK-35376][CORE] Fallback config should override defaultValue

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32510: URL: https://github.com/apache/spark/pull/32510#issuecomment-839449452 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138415/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
AmplabJenkins commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839449413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42945/ --

[GitHub] [spark] SparkQA commented on pull request #32516: [SPARK-35364][PYTHON] Renaming the existing Koalas related codes

2021-05-11 Thread GitBox
SparkQA commented on pull request #32516: URL: https://github.com/apache/spark/pull/32516#issuecomment-839449369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

2021-05-11 Thread GitBox
SparkQA commented on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-839449229 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] HyukjinKwon closed pull request #32507: [SPARK-35361][SQL] Improve performance for ApplyFunctionExpression

2021-05-11 Thread GitBox
HyukjinKwon closed pull request #32507: URL: https://github.com/apache/spark/pull/32507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #32507: [SPARK-35361][SQL] Improve performance for ApplyFunctionExpression

2021-05-11 Thread GitBox
HyukjinKwon commented on pull request #32507: URL: https://github.com/apache/spark/pull/32507#issuecomment-839359850 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   3   4   5   6   7   8   >