[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698523837 **[Test build #129084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129084/testReport)** for PR 29533 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698532208 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698532031 **[Test build #129084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129084/testReport)** for PR 29533 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

AmplabJenkins removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698532208 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA removed a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698523837 **[Test build #129084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129084/testReport)** for PR 29533 at commit

[GitHub] [spark] nssalian commented on a change in pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

nssalian commented on a change in pull request #29844: URL: https://github.com/apache/spark/pull/29844#discussion_r494580391 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala ## @@ -60,4 +63,15 @@ private[spark]

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698561799 **[Test build #129085 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129085/testReport)** for PR 29844 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612046 **[Test build #129091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129091/testReport)** for PR 29844 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612614 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612626 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698627613 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33712/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250][test-maven][test-hadoop2.7] Upgrade to Hadoop 3.2.1 and move to shaded client

AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698626897 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29843: [WIP][SPARK-29250][test-maven][test-hadoop2.7] Upgrade to Hadoop 3.2.1 and move to shaded client

SparkQA commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698626880 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33709/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29843: [WIP][SPARK-29250][test-maven][test-hadoop2.7] Upgrade to Hadoop 3.2.1 and move to shaded client

AmplabJenkins removed a comment on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698626889 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29843: [WIP][SPARK-29250][test-maven][test-hadoop2.7] Upgrade to Hadoop 3.2.1 and move to shaded client

AmplabJenkins commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698626889 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698651103 **[Test build #129096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129096/testReport)** for PR 29844 at commit

[GitHub] [spark] SparkQA commented on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

SparkQA commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698650576 **[Test build #129095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129095/testReport)** for PR 29863 at commit

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698650598 **[Test build #129096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129096/testReport)** for PR 29844 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698660054 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] github-actions[bot] closed pull request #27604: [SPARK-30849][CORE][SHUFFLE]Fix application failed due to failed to get MapStatuses broadcast block

github-actions[bot] closed pull request #27604: URL: https://github.com/apache/spark/pull/27604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698660054 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698659901 **[Test build #129097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129097/testReport)** for PR 29844 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698652663 **[Test build #129097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129097/testReport)** for PR 29844 at commit

[GitHub] [spark] fhoering commented on a change in pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

fhoering commented on a change in pull request #29806: URL: https://github.com/apache/spark/pull/29806#discussion_r494488079 ## File path: python/docs/source/user_guide/python_packaging.rst ## @@ -0,0 +1,201 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [spark] holdenk commented on pull request #29471: [SPARK-32381][CORE][SQL] Move and refactor parallel listing & non-location sensitive listing to core

holdenk commented on pull request #29471: URL: https://github.com/apache/spark/pull/29471#issuecomment-698499385 Merged to the current development branch :) This is an automated message from the Apache Git Service. To

[GitHub] [spark] asfgit closed pull request #29471: [SPARK-32381][CORE][SQL] Move and refactor parallel listing & non-location sensitive listing to core

asfgit closed pull request #29471: URL: https://github.com/apache/spark/pull/29471 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins removed a comment on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698519635 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698519613 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33703/

[GitHub] [spark] AmplabJenkins commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

AmplabJenkins commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698519635 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] viirya commented on a change in pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

viirya commented on a change in pull request #29806: URL: https://github.com/apache/spark/pull/29806#discussion_r494553533 ## File path: python/docs/source/user_guide/python_packaging.rst ## @@ -0,0 +1,201 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [spark] SparkQA commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

SparkQA commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-698545210 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33705/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins removed a comment on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698581766 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698581766 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

dongjoon-hyun commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698606099 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

SparkQA commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698620823 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33710/

[GitHub] [spark] SparkQA commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698625408 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33711/

[GitHub] [spark] SparkQA commented on pull request #29843: [WIP][SPARK-29250][test-maven][test-hadoop2.7] Upgrade to Hadoop 3.2.1 and move to shaded client

SparkQA commented on pull request #29843: URL: https://github.com/apache/spark/pull/29843#issuecomment-698624838 **[Test build #129092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129092/testReport)** for PR 29843 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29862: [SPARK-32956][SQL] Ensure that the generated and existing headers are not duplicated in CSV DataSource

HyukjinKwon commented on a change in pull request #29862: URL: https://github.com/apache/spark/pull/29862#discussion_r494654857 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVUtils.scala ## @@ -93,6 +93,12 @@ object CSVUtils {

[GitHub] [spark] AmplabJenkins commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

AmplabJenkins commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698630067 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698629444 **[Test build #129083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129083/testReport)** for PR 29828 at commit

[GitHub] [spark] SparkQA commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698630525 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33711/

[GitHub] [spark] AmplabJenkins commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

AmplabJenkins commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698630541 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

dongjoon-hyun commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698636665 The Jenkins jobs are passing Scala/Java tests already. Merged to master. This is an automated message

[GitHub] [spark] dongjoon-hyun closed pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP] Skip special column names test in Hive 1.2

dongjoon-hyun closed pull request #29867: URL: https://github.com/apache/spark/pull/29867 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP][test-hadoop2.7][test-hive1.2] Skip special column names test in Hive 1.2

dongjoon-hyun commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698636232 Thank you, @HyukjinKwon ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] viirya commented on a change in pull request #29831: [SPARK-32351][SQL] show pushed down partition filters in explain()

viirya commented on a change in pull request #29831: URL: https://github.com/apache/spark/pull/29831#discussion_r494671976 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ## @@ -254,22 +254,28 @@ private[hive] trait HiveStrategies { *

[GitHub] [spark] viirya commented on a change in pull request #29831: [SPARK-32351][SQL] show pushed down partition filters in explain()

viirya commented on a change in pull request #29831: URL: https://github.com/apache/spark/pull/29831#discussion_r494671598 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ## @@ -254,22 +254,28 @@ private[hive] trait HiveStrategies { *

[GitHub] [spark] HyukjinKwon closed pull request #29851: [WIP][SPARK-22674][PYTHON] Removed namedtuple hack and made cloudpickle the default serializer

HyukjinKwon closed pull request #29851: URL: https://github.com/apache/spark/pull/29851 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698661308 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698661303 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33717/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698661308 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698663592 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698663577 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33718/

[GitHub] [spark] AmplabJenkins commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698663592 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

SparkQA commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698667296 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33716/

[GitHub] [spark] AmplabJenkins commented on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

AmplabJenkins commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698667313 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

AmplabJenkins removed a comment on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698667313 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] fqaiser94 commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

fqaiser94 commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494698769 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -541,57 +541,105 @@ case class

[GitHub] [spark] fqaiser94 commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

fqaiser94 commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494698809 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -541,57 +541,105 @@ case class

[GitHub] [spark] fqaiser94 commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

fqaiser94 commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494698625 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -541,57 +541,105 @@ case class

[GitHub] [spark] sarutak commented on a change in pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome

sarutak commented on a change in pull request #29827: URL: https://github.com/apache/spark/pull/29827#discussion_r494703217 ## File path: .github/workflows/build_and_test.yml ## @@ -273,6 +273,44 @@ jobs: cd docs jekyll build + webui-tests-with-chrome: +

[GitHub] [spark] sarutak commented on pull request #29827: [SPARK-32957][INFRA] Add a GitHub Actions job to run WebUI tests with Chrome

sarutak commented on pull request #29827: URL: https://github.com/apache/spark/pull/29827#issuecomment-698677232 @dongjoon-hyun I agree. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] zhengruifeng commented on a change in pull request #29852: [SPARK-21481][ML][FOLLOWUP][Trivial] HashingTF use util.collection.OpenHashMap instead of mutable.HashMap

zhengruifeng commented on a change in pull request #29852: URL: https://github.com/apache/spark/pull/29852#discussion_r494713601 ## File path: mllib/src/main/scala/org/apache/spark/ml/feature/HashingTF.scala ## @@ -91,20 +90,13 @@ class HashingTF @Since("3.0.0") private[ml] (

[GitHub] [spark] SparkQA commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

SparkQA commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698689552 **[Test build #129099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129099/testReport)** for PR 29860 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29854: [SPARK-32937][SPARK-32980][K8S] Fix decom & launcher tests and add some comments to reduce chance of breakage

HyukjinKwon commented on pull request #29854: URL: https://github.com/apache/spark/pull/29854#issuecomment-698689697 @holdenk, BTW which PR caused this failure? It's a bit odd that we fix a test as a followup here whereas you ask to revert at

[GitHub] [spark] huaxingao commented on pull request #29868: [SPARK-32973][ML][DOC] FeatureHasher does not check categoricalCols in inputCols

huaxingao commented on pull request #29868: URL: https://github.com/apache/spark/pull/29868#issuecomment-698714611 The change looks good to me, but do you need to change the implementation in https://github.com/apache/spark/pull/29850 ? Seems in that PR you are assuming ` categoricalCols`

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698718868 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33722/

[GitHub] [spark] cloud-fan commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

cloud-fan commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494745836 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

dongjoon-hyun commented on a change in pull request #29828: URL: https://github.com/apache/spark/pull/29828#discussion_r494754892 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeJsonExprs.scala ## @@ -0,0 +1,38 @@ +/* + * Licensed to

[GitHub] [spark] nssalian commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

nssalian commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698662928 @erikerlandson, tests passed. Thanks for your review. This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

SparkQA commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698662720 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33716/

[GitHub] [spark] fqaiser94 commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

fqaiser94 commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494700403 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -901,39 +901,125 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] AmplabJenkins commented on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

AmplabJenkins commented on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698679229 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29863: [SPARK-32877][SQL][TEST][test-hive1.2][test-hadoop2.7] Add test for Hive UDF complex decimal type

AmplabJenkins removed a comment on pull request #29863: URL: https://github.com/apache/spark/pull/29863#issuecomment-698679229 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] LuciferYang edited a comment on pull request #29857: [SPARK-32972][ML] Fix UTs of `mllib` module in Scala 2.13 except RandomForestRegressorSuite

LuciferYang edited a comment on pull request #29857: URL: https://github.com/apache/spark/pull/29857#issuecomment-698679002 > f you like you can try making the ordering of the Map in this code deterministic to see if that does it: > val topNodesForGroup: Map[Int, LearningNode] =

[GitHub] [spark] SparkQA commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

SparkQA commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698704828 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33720/

[GitHub] [spark] AmplabJenkins commented on pull request #29860: [SPARK-32984][TESTS][SQL] Improve showing the differences between approved and actual plans of PlanStabilitySuite

AmplabJenkins commented on pull request #29860: URL: https://github.com/apache/spark/pull/29860#issuecomment-698704837 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

cloud-fan commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698713409 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

cloud-fan closed pull request #29866: URL: https://github.com/apache/spark/pull/29866 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698724864 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33722/

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698724874 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun closed pull request #29863: [SPARK-32877][SQL][TEST] Add test for Hive UDF complex decimal type

dongjoon-hyun closed pull request #29863: URL: https://github.com/apache/spark/pull/29863 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-698724874 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

cloud-fan commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494763126 ## File path: sql/core/src/test/scala/org/apache/spark/sql/UpdateFieldsBenchmark.scala ## @@ -0,0 +1,310 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] cloud-fan commented on a change in pull request #29795: [SPARK-32511][SQL] Add dropFields method to Column class

cloud-fan commented on a change in pull request #29795: URL: https://github.com/apache/spark/pull/29795#discussion_r494762975 ## File path: sql/core/src/test/scala/org/apache/spark/sql/UpdateFieldsBenchmark.scala ## @@ -0,0 +1,310 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] MLnick commented on pull request #29850: [SPARK-32974][ML] FeatureHasher transform optimization

MLnick commented on pull request #29850: URL: https://github.com/apache/spark/pull/29850#issuecomment-698736348 > > @MLnick I just do a simple test, and it shows that we can obtain about 11% speedup. This seems a worthwhile optimization then, thanks!

[GitHub] [spark] SparkQA commented on pull request #29828: [SPARK-32948][SQL] Optimize to_json and from_json expression chain

SparkQA commented on pull request #29828: URL: https://github.com/apache/spark/pull/29828#issuecomment-698511825 **[Test build #129083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129083/testReport)** for PR 29828 at commit

[GitHub] [spark] SparkQA commented on pull request #29585: [WIP][SPARK-32741][SQL] Check if the same ExprId refers to the unique attribute in logical plans

SparkQA commented on pull request #29585: URL: https://github.com/apache/spark/pull/29585#issuecomment-698510784 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33703/

[GitHub] [spark] viirya commented on a change in pull request #29806: [SPARK-32187][PYTHON][DOCS] Doc on Python packaging

viirya commented on a change in pull request #29806: URL: https://github.com/apache/spark/pull/29806#discussion_r494553905 ## File path: python/docs/source/user_guide/python_packaging.rst ## @@ -0,0 +1,201 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [spark] erikerlandson commented on a change in pull request #29844: [SPARK-27872][K8s] Fix executor service account inconsistency for branch-2.4

erikerlandson commented on a change in pull request #29844: URL: https://github.com/apache/spark/pull/29844#discussion_r494609081 ## File path: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/BasicTestsSuite.scala ##

[GitHub] [spark] AmplabJenkins commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612614 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612597 **[Test build #129091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129091/testReport)** for PR 29844 at commit

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698612046 **[Test build #129091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129091/testReport)** for PR 29844 at commit

[GitHub] [spark] SparkQA commented on pull request #29866: [SPARK-32990][SQL] Migrate REFRESH TABLE to use UnresolvedTableOrView to resolve the identifier

SparkQA commented on pull request #29866: URL: https://github.com/apache/spark/pull/29866#issuecomment-698612010 **[Test build #129090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129090/testReport)** for PR 29866 at commit

[GitHub] [spark] SparkQA commented on pull request #29867: [SPARK-32889][SQL][TESTS][FOLLOWUP] Skip special column names test in Hive 1.2

SparkQA commented on pull request #29867: URL: https://github.com/apache/spark/pull/29867#issuecomment-698637895 **[Test build #129087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129087/testReport)** for PR 29867 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698651116 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698651112 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698650598 **[Test build #129096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129096/testReport)** for PR 29844 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

AmplabJenkins removed a comment on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698651112 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29844: [SPARK-27872][K8s][2.4] Fix executor service account inconsistency

SparkQA commented on pull request #29844: URL: https://github.com/apache/spark/pull/29844#issuecomment-698656746 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33717/

  1   2   3   4   5   6   >