[GitHub] [spark] mdianjun commented on pull request #23066: [SPARK-26043][CORE] Make SparkHadoopUtil private to Spark

2020-10-24 Thread GitBox
mdianjun commented on pull request #23066: URL: https://github.com/apache/spark/pull/23066#issuecomment-716095050 @srowen Our app is using the SparkHadoopUtils.addCredentials API to do this before using a custom datasource of ours and are now facing breaking changes. I don't see this

[GitHub] [spark] dongjoon-hyun commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

2020-10-24 Thread GitBox
dongjoon-hyun commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-716088268 Could you add an empty commit whose authorship is the original author, @AngersZh . If then, Apache Spark merge script can give both of you the authorship properly.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716087513 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716087513 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716066052 **[Test build #130240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130240/testReport)** for PR 29695 at commit

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716087302 **[Test build #130240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130240/testReport)** for PR 29695 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
HeartSaVioR commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716084716 Thanks all for reviewing and merging! This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] huskysun commented on pull request #29924: [SPARK-30821][K8S]Handle executor failure with multiple containers

2020-10-24 Thread GitBox
huskysun commented on pull request #29924: URL: https://github.com/apache/spark/pull/29924#issuecomment-716084244 Thanks for merging it in! I just created a JIRA account, and my username is "huskysun" and full name is "Shiqi Sun". Let me know if these work, thanks. On Sat, Oct 24,

[GitHub] [spark] mridulm commented on pull request #30131: [SPARK-33220][CORE]Use `scheduleWithFixedDelay` to avoid repeated unnecessary scheduling for a short time

2020-10-24 Thread GitBox
mridulm commented on pull request #30131: URL: https://github.com/apache/spark/pull/30131#issuecomment-716077517 If we want to take a look at the jdk implementation in case the documentation is unclear, please note that both `scheduleAtFixedRate` and `scheduleWithFixedDelay` rely on

[GitHub] [spark] github-actions[bot] closed pull request #29028: [SPARK-32212][CORE]RDD.takeOrdered can choose to merge intermediate r…

2020-10-24 Thread GitBox
github-actions[bot] closed pull request #29028: URL: https://github.com/apache/spark/pull/29028 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716071649 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34840/

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716071653 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716071653 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716069666 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34840/

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716066052 **[Test build #130240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130240/testReport)** for PR 29695 at commit

[GitHub] [spark] huaxingao commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
huaxingao commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716065891 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] viirya closed pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
viirya closed pull request #27333: URL: https://github.com/apache/spark/pull/27333 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] viirya commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
viirya commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716064216 Thanks. Merging to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716063897 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716063896 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716041521 **[Test build #130239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130239/testReport)** for PR 29695 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716063896 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716063784 **[Test build #130239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130239/testReport)** for PR 29695 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716063066 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716063066 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716062887 **[Test build #130237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130237/testReport)** for PR 27333 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716032552 **[Test build #130237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130237/testReport)** for PR 27333 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716058835 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716037487 **[Test build #130238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130238/testReport)** for PR 29695 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716058835 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716058607 **[Test build #130238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130238/testReport)** for PR 29695 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716050214 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716050211 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34839/

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716050214 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716046668 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34839/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716045815 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716045803 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34838/

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716045812 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716045812 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716043249 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34838/

[GitHub] [spark] AmplabJenkins commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716041751 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716041751 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716041745 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34837/

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716041521 **[Test build #130239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130239/testReport)** for PR 29695 at commit

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716038768 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34837/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-716037905 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-716037905 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715922894 **[Test build #130235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130235/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-716037685 **[Test build #130235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130235/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-10-24 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-716037487 **[Test build #130238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130238/testReport)** for PR 29695 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716035579 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
SparkQA commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716035571 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34836/

[GitHub] [spark] AmplabJenkins commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716035579 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
SparkQA commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716032591 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34836/

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716032552 **[Test build #130237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130237/testReport)** for PR 27333 at commit

[GitHub] [spark] viirya commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
viirya commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716032349 Will merge this once jenkins pass. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] viirya commented on a change in pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
viirya commented on a change in pull request #27333: URL: https://github.com/apache/spark/pull/27333#discussion_r511492575 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/FlatMapGroupsWithStateSuite.scala ## @@ -1019,6 +1019,56 @@ class

[GitHub] [spark] viirya commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

2020-10-24 Thread GitBox
viirya commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-716032205 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716025248 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716024299 **[Test build #130236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130236/testReport)** for PR 28618 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716025211 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716025211 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
SparkQA commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716025161 **[Test build #130236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130236/testReport)** for PR 28618 at commit

[GitHub] [spark] SparkQA commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
SparkQA commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716024299 **[Test build #130236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130236/testReport)** for PR 28618 at commit

[GitHub] [spark] holdenk commented on pull request #28618: [SPARK-31801][API][SHUFFLE] Register map output metadata

2020-10-24 Thread GitBox
holdenk commented on pull request #28618: URL: https://github.com/apache/spark/pull/28618#issuecomment-716016713 Jenkins retest this please. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] holdenk commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

2020-10-24 Thread GitBox
holdenk commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r511493992 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientUtils.scala ## @@ -0,0 +1,124 @@ +/* +

[GitHub] [spark] holdenk commented on a change in pull request #28027: [SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2

2020-10-24 Thread GitBox
holdenk commented on a change in pull request #28027: URL: https://github.com/apache/spark/pull/28027#discussion_r511493573 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala ## @@ -48,6 +48,15 @@ case class

[GitHub] [spark] holdenk commented on pull request #29924: [SPARK-30821][K8S]Handle executor failure with multiple containers

2020-10-24 Thread GitBox
holdenk commented on pull request #29924: URL: https://github.com/apache/spark/pull/29924#issuecomment-715997264 @huskysun what is your JIRA account handle? This is an automated message from the Apache Git Service. To

[GitHub] [spark] asfgit closed pull request #29924: [SPARK-30821][K8S]Handle executor failure with multiple containers

2020-10-24 Thread GitBox
asfgit closed pull request #29924: URL: https://github.com/apache/spark/pull/29924 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] holdenk commented on pull request #29924: [SPARK-30821][K8S]Handle executor failure with multiple containers

2020-10-24 Thread GitBox
holdenk commented on pull request #29924: URL: https://github.com/apache/spark/pull/29924#issuecomment-715994601 Meregd and backported as a fix to branch-3. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715940731 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715940709 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715940709 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715940687 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34835/

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715928128 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34835/

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715922894 **[Test build #130235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130235/testReport)** for PR 30097 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715918066 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715918066 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715887837 **[Test build #130234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130234/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715917808 **[Test build #130234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130234/testReport)** for PR 30097 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715895020 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715895016 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34834/

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715895020 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715892694 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34834/

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30131: [SPARK-33220][CORE]Use `scheduleWithFixedDelay` to avoid repeated unnecessary scheduling for a short time

2020-10-24 Thread GitBox
AngersZh edited a comment on pull request #30131: URL: https://github.com/apache/spark/pull/30131#issuecomment-715889934 > if there is heavy gc after sending rpc request, but before task completes - fixed delay will be gc time + wait time vs just wait time. Double check the code

[GitHub] [spark] AngersZhuuuu commented on pull request #30131: [SPARK-33220][CORE]Use `scheduleWithFixedDelay` to avoid repeated unnecessary scheduling for a short time

2020-10-24 Thread GitBox
AngersZh commented on pull request #30131: URL: https://github.com/apache/spark/pull/30131#issuecomment-715889934 > if there is heavy gc after sending rpc request, but before task completes - fixed delay will be gc time + wait time vs just wait time. Double check the code of

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715889564 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715889562 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715889557 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34833/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715889562 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715887837 **[Test build #130234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130234/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715886332 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34833/

[GitHub] [spark] ScrapCodes commented on pull request #30089: [SPARK-33137][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: update type and nullability of columns (Postgres dialect)

2020-10-24 Thread GitBox
ScrapCodes commented on pull request #30089: URL: https://github.com/apache/spark/pull/30089#issuecomment-715884109 Hi @huaxingao, I wasn't aware of this parallel work here. And created another PR #30142. I am going to copy your rename test and may be you could take a look at the other

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715882828 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715882824 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715880935 **[Test build #130233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130233/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715882814 **[Test build #130233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130233/testReport)** for PR 30097 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

2020-10-24 Thread GitBox
AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-715882824 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] mridulm commented on pull request #30126: [SPARK-33215][WEBUI] Speed up event log download by skipping UI rebuild

2020-10-24 Thread GitBox
mridulm commented on pull request #30126: URL: https://github.com/apache/spark/pull/30126#issuecomment-715882396 Also, CC @zhouyejoe This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] mridulm commented on pull request #30131: [SPARK-33220][CORE]Use `scheduleWithFixedDelay` to avoid repeated unnecessary scheduling for a short time

2020-10-24 Thread GitBox
mridulm commented on pull request #30131: URL: https://github.com/apache/spark/pull/30131#issuecomment-715882017 Are you seeing issues with use of `scheduleAtFixedRate` which are fixed by `scheduleWithFixedDelay` ? There are nuances in the difference, which can have subtle impact. In

  1   2   >