[GitHub] [spark] rwpenney commented on a change in pull request #30745: [SPARK-33678][SQL] Product aggregation function

2021-02-20 Thread GitBox
rwpenney commented on a change in pull request #30745: URL: https://github.com/apache/spark/pull/30745#discussion_r579760686 ## File path: python/pyspark/sql/functions.py ## @@ -222,6 +222,45 @@ def sum_distinct(col): return _invoke_function_over_column("sum_distinct", col

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782806787 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39899/

[GitHub] [spark] AmplabJenkins commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782806787 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39899/ -

[GitHub] [spark] SparkQA commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
SparkQA commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782804676 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39899/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782798482 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135318/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782798482 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135318/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782771155 **[Test build #135318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135318/testReport)** for PR 31595 at commit [`7e8c04c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
SparkQA commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782796876 **[Test build #135318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135318/testReport)** for PR 31595 at commit [`7e8c04c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
SparkQA commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782793673 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39899/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782793307 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135319/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782793307 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135319/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782789474 **[Test build #135319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135319/testReport)** for PR 31604 at commit [`a8ecb70`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
SparkQA commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782790327 **[Test build #135319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135319/testReport)** for PR 31604 at commit [`a8ecb70`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
SparkQA commented on pull request #31604: URL: https://github.com/apache/spark/pull/31604#issuecomment-782789474 **[Test build #135319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135319/testReport)** for PR 31604 at commit [`a8ecb70`](https://github.com

[GitHub] [spark] srowen commented on a change in pull request #30745: [SPARK-33678][SQL] Product aggregation function

2021-02-20 Thread GitBox
srowen commented on a change in pull request #30745: URL: https://github.com/apache/spark/pull/30745#discussion_r579742487 ## File path: python/pyspark/sql/functions.py ## @@ -222,6 +222,45 @@ def sum_distinct(col): return _invoke_function_over_column("sum_distinct", col)

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782787880 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782787880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] srowen commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
srowen commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782786865 Merged to master/3.1 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] srowen closed pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
srowen closed pull request #31540: URL: https://github.com/apache/spark/pull/31540 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] SparkQA commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
SparkQA commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782785934 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39897/ ---

[GitHub] [spark] dongjoon-hyun opened a new pull request #31604: [SPARK-34487][K8S][TESTS] Use the runtime Hadoop version in K8s IT

2021-02-20 Thread GitBox
dongjoon-hyun opened a new pull request #31604: URL: https://github.com/apache/spark/pull/31604 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### H

[GitHub] [spark] SparkQA removed a comment on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782769602 **[Test build #135317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135317/testReport)** for PR 31603 at commit [`203c212`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
SparkQA commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782783843 **[Test build #135317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135317/testReport)** for PR 31603 at commit [`203c212`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782782187 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39898/

[GitHub] [spark] AmplabJenkins commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782782187 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39898/ -

[GitHub] [spark] dongjoon-hyun commented on pull request #31599: [SPARK-34481][SQL] Refactor dataframe reader/writer optionsWithPath logic

2021-02-20 Thread GitBox
dongjoon-hyun commented on pull request #31599: URL: https://github.com/apache/spark/pull/31599#issuecomment-782781019 Merged to master. Thanks, @yuchenhuo . This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] dongjoon-hyun closed pull request #31599: [SPARK-34481][SQL] Refactor dataframe reader/writer optionsWithPath logic

2021-02-20 Thread GitBox
dongjoon-hyun closed pull request #31599: URL: https://github.com/apache/spark/pull/31599 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31600: [SPARK-34482][SS] Correct the active SparkSession for streaming query

2021-02-20 Thread GitBox
dongjoon-hyun commented on a change in pull request #31600: URL: https://github.com/apache/spark/pull/31600#discussion_r579735375 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala ## @@ -323,7 +323,7 @@ abstract class StreamEx

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
dongjoon-hyun commented on a change in pull request #31595: URL: https://github.com/apache/spark/pull/31595#discussion_r579734736 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala ## @@ -387,100 +388,104 @@ class DataFrameSetOperation

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
dongjoon-hyun commented on a change in pull request #31595: URL: https://github.com/apache/spark/pull/31595#discussion_r579733849 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -492,12 +492,43 @@ object RemoveRedundantAl

[GitHub] [spark] dongjoon-hyun commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
dongjoon-hyun commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-78292 Thank you for pinging me, @viirya . This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] dongjoon-hyun commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
dongjoon-hyun commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782776559 This is tested locally. ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE

[GitHub] [spark] SparkQA commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
SparkQA commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782776046 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39897/ -

[GitHub] [spark] github-actions[bot] commented on pull request #30329: [SPARK-33420][SQL] Fix BroadCastJoin failure when keys on join side has cast from DateTyte to String

2021-02-20 Thread GitBox
github-actions[bot] commented on pull request #30329: URL: https://github.com/apache/spark/pull/30329#issuecomment-782772974 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] commented on pull request #30348: [SPARK-33413][WEBUI] SparkUI doesn't work when use an external reverse proxy

2021-02-20 Thread GitBox
github-actions[bot] commented on pull request #30348: URL: https://github.com/apache/spark/pull/30348#issuecomment-782772972 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] SparkQA commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
SparkQA commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782771155 **[Test build #135318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135318/testReport)** for PR 31595 at commit [`7e8c04c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
SparkQA commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782769602 **[Test build #135317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135317/testReport)** for PR 31603 at commit [`203c212`](https://github.com

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
dongjoon-hyun edited a comment on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782767150 In these days, the Amplab K8s IT fails due to out of disk. ``` Failed to copy files: failed to copy directory: Error processing tar file(exit status 1): write

[GitHub] [spark] dongjoon-hyun commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
dongjoon-hyun commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782767150 In these days, the Amblab K8s IT fails due to out of disk. ``` Failed to copy files: failed to copy directory: Error processing tar file(exit status 1): write /hadoop

[GitHub] [spark] dongjoon-hyun commented on pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
dongjoon-hyun commented on pull request #31603: URL: https://github.com/apache/spark/pull/31603#issuecomment-782766824 cc @maropu and @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [spark] dongjoon-hyun opened a new pull request #31603: [SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider

2021-02-20 Thread GitBox
dongjoon-hyun opened a new pull request #31603: URL: https://github.com/apache/spark/pull/31603 ### What changes were proposed in this pull request? This is a retry of #31065 . This PR aims to add a basis for columnar encryption test framework by add `OrcEncryptionSuite` and `

[GitHub] [spark] viirya commented on a change in pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
viirya commented on a change in pull request #31595: URL: https://github.com/apache/spark/pull/31595#discussion_r579720302 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -492,12 +492,43 @@ object RemoveRedundantAliases e

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782756848 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135316/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782756848 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135316/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782734564 **[Test build #135316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135316/testReport)** for PR 31602 at commit [`854b2fd`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
SparkQA commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782751880 **[Test build #135316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135316/testReport)** for PR 31602 at commit [`854b2fd`](https://github.co

[GitHub] [spark] tanelk commented on a change in pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
tanelk commented on a change in pull request #31595: URL: https://github.com/apache/spark/pull/31595#discussion_r579705619 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -492,12 +492,43 @@ object RemoveRedundantAliases e

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782742197 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39896/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782742196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135315/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782742196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135315/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782742197 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39896/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782699020 **[Test build #135315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135315/testReport)** for PR 31601 at commit [`5b9c205`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
SparkQA commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782741104 **[Test build #135315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135315/testReport)** for PR 31601 at commit [`5b9c205`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
SparkQA commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782740653 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39896/ ---

[GitHub] [spark] SparkQA commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
SparkQA commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782740093 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39896/ -

[GitHub] [spark] SparkQA commented on pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
SparkQA commented on pull request #31602: URL: https://github.com/apache/spark/pull/31602#issuecomment-782734564 **[Test build #135316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135316/testReport)** for PR 31602 at commit [`854b2fd`](https://github.com

[GitHub] [spark] dongjoon-hyun opened a new pull request #31602: [SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2

2021-02-20 Thread GitBox
dongjoon-hyun opened a new pull request #31602: URL: https://github.com/apache/spark/pull/31602 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### H

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782725442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135314/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782725442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135314/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782695713 **[Test build #135314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135314/testReport)** for PR 31540 at commit [`c9918ab`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
SparkQA commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782718561 **[Test build #135314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135314/testReport)** for PR 31540 at commit [`c9918ab`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782716352 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39894/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782716350 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135313/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782716351 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39895/

[GitHub] [spark] AmplabJenkins commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782716350 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135313/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782716352 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39894/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782716351 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39895/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782632267 **[Test build #135313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135313/testReport)** for PR 31560 at commit [`3fc7288`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
SparkQA commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782715287 **[Test build #135313 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135313/testReport)** for PR 31560 at commit [`3fc7288`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
SparkQA commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782710761 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39895/ ---

[GitHub] [spark] SparkQA commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
SparkQA commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782710527 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39894/ ---

[GitHub] [spark] SparkQA commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
SparkQA commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782707299 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39895/ -

[GitHub] [spark] SparkQA commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
SparkQA commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782705679 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39894/ -

[GitHub] [spark] SparkQA commented on pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
SparkQA commented on pull request #31601: URL: https://github.com/apache/spark/pull/31601#issuecomment-782699020 **[Test build #135315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135315/testReport)** for PR 31601 at commit [`5b9c205`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-782696395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135312/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-782696395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135312/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-782610088 **[Test build #135312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135312/testReport)** for PR 31596 at commit [`6da1d08`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31596: [SPARK-34475][SQL] Rename logical nodes of v2 `ALTER` commands

2021-02-20 Thread GitBox
SparkQA commented on pull request #31596: URL: https://github.com/apache/spark/pull/31596#issuecomment-782696121 **[Test build #135312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135312/testReport)** for PR 31596 at commit [`6da1d08`](https://github.co

[GitHub] [spark] sarutak opened a new pull request #31601: [SPARK-34484][SQL] Introduce a new syntax to represent attributes with the Catalyst DSL

2021-02-20 Thread GitBox
sarutak opened a new pull request #31601: URL: https://github.com/apache/spark/pull/31601 ### What changes were proposed in this pull request? This PR proposes to introduce a new syntax `attr()` to represent attributes with the Catalyst DSL. ### Why are the changes needed?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782609612 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-782695615 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135311/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782695613 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135310/ -

[GitHub] [spark] SparkQA commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
SparkQA commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782695713 **[Test build #135314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135314/testReport)** for PR 31540 at commit [`c9918ab`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-782695615 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135311/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782695613 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135310/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782695614 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135309/ -

[GitHub] [spark] sarutak commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-20 Thread GitBox
sarutak commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-782692634 Thanks all. Merged to `master`. This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [spark] asfgit closed pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-20 Thread GitBox
asfgit closed pull request #31491: URL: https://github.com/apache/spark/pull/31491 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] SparkQA removed a comment on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782601477 **[Test build #135310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135310/testReport)** for PR 31560 at commit [`9178240`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31560: [SPARK-34432][SQL][TESTS] Add JavaSimpleWritableDataSource

2021-02-20 Thread GitBox
SparkQA commented on pull request #31560: URL: https://github.com/apache/spark/pull/31560#issuecomment-782690158 **[Test build #135310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135310/testReport)** for PR 31560 at commit [`9178240`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782599760 **[Test build #135309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135309/testReport)** for PR 31595 at commit [`94ca5e4`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value

2021-02-20 Thread GitBox
SparkQA removed a comment on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-782599904 **[Test build #135311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135311/testReport)** for PR 30421 at commit [`d3b1960`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value

2021-02-20 Thread GitBox
SparkQA commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-782683809 **[Test build #135311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135311/testReport)** for PR 30421 at commit [`d3b1960`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31595: [SPARK-34474][SQL] Remove unnecessary Union under Distinct like operators

2021-02-20 Thread GitBox
SparkQA commented on pull request #31595: URL: https://github.com/apache/spark/pull/31595#issuecomment-782682872 **[Test build #135309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135309/testReport)** for PR 31595 at commit [`94ca5e4`](https://github.co

[GitHub] [spark] srowen commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-20 Thread GitBox
srowen commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-782681399 @sarutak feel free to merge when ready This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-776392734 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] srowen commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
srowen commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-782681238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] srowen commented on a change in pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

2021-02-20 Thread GitBox
srowen commented on a change in pull request #31540: URL: https://github.com/apache/spark/pull/31540#discussion_r579662690 ## File path: core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala ## @@ -449,39 +449,46 @@ class DoubleAccumulator extends AccumulatorV2[jl.Doubl

[GitHub] [spark] AmplabJenkins commented on pull request #31600: [SPARK-34482][SS] Correct the active SparkSession for streaming query

2021-02-20 Thread GitBox
AmplabJenkins commented on pull request #31600: URL: https://github.com/apache/spark/pull/31600#issuecomment-782677205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135306/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31600: [SPARK-34482][SS] Correct the active SparkSession for streaming query

2021-02-20 Thread GitBox
AmplabJenkins removed a comment on pull request #31600: URL: https://github.com/apache/spark/pull/31600#issuecomment-782677205 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135306/ -

  1   2   3   >