[GitHub] [spark] AmplabJenkins removed a comment on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670827006 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670827006 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670801837 **[Test build #127219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127219/testReport)** for PR 29339 at commit

[GitHub] [spark] SparkQA commented on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
SparkQA commented on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670826865 **[Test build #127219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127219/testReport)** for PR 29339 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29031: [SPARK-32216][SQL] Remove redundant ProjectExec

2020-08-07 Thread GitBox
cloud-fan commented on a change in pull request #29031: URL: https://github.com/apache/spark/pull/29031#discussion_r467349934 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/RemoveRedundantProjects.scala ## @@ -78,9 +86,11 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670807750 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670807750 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670725493 **[Test build #127218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127218/testReport)** for PR 28804 at commit

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670807546 **[Test build #127218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127218/testReport)** for PR 28804 at commit

[GitHub] [spark] HyukjinKwon closed pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
HyukjinKwon closed pull request #29386: URL: https://github.com/apache/spark/pull/29386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
HyukjinKwon commented on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670804169 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on pull request #29331: [SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3

2020-08-07 Thread GitBox
HyukjinKwon commented on pull request #29331: URL: https://github.com/apache/spark/pull/29331#issuecomment-670803903 cc @tgravescs too This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670802009 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670802009 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
SparkQA commented on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670801837 **[Test build #127219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127219/testReport)** for PR 29339 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29385: [SPARK-32191][PySpark][DOC] Migration Guide for PySpark docs

2020-08-07 Thread GitBox
HyukjinKwon commented on a change in pull request #29385: URL: https://github.com/apache/spark/pull/29385#discussion_r467342735 ## File path: python/docs/source/migration_guide/index.rst ## @@ -20,3 +20,14 @@ Migration Guide === +Migration Guide: PySpark

[GitHub] [spark] HyukjinKwon commented on pull request #29385: [SPARK-32191][PySpark][DOC] Migration Guide for PySpark docs

2020-08-07 Thread GitBox
HyukjinKwon commented on pull request #29385: URL: https://github.com/apache/spark/pull/29385#issuecomment-670801380 Nice, thank you @viirya! I will take a look next Monday! This is an automated message from the Apache Git

[GitHub] [spark] stczwd commented on pull request #29339: [Spark-32512][SQL] add alter table add/drop partition command for datasourcev2

2020-08-07 Thread GitBox
stczwd commented on pull request #29339: URL: https://github.com/apache/spark/pull/29339#issuecomment-670801145 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670799920 Got it. I updated my previous comment by marking my misunderstanding. This is an automated message from

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun edited a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670794507 Hi, @Fokko . ~It seems that @srowen clearly gave -1 on this approach. Although I approved this, we cannot merge your PR if there is a -1.~ ~I must admit that

[GitHub] [spark] srowen commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
srowen commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670797950 No -1 here. I don't object to the current change. This is an automated message from the Apache Git Service. To

[GitHub] [spark] github-actions[bot] closed pull request #27026: [SPARK-29596][Web UI] Task duration not updating for running tasks

2020-08-07 Thread GitBox
github-actions[bot] closed pull request #27026: URL: https://github.com/apache/spark/pull/27026 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670794888 While I wrote the above my comment, I didn't notice the last comment from @srowen . However, I believe it's better to adjust now.

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun edited a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670794888 Oh, while I wrote the above my comment, I didn't notice the last comment from @srowen . However, I believe it's better to adjust now.

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670794507 Hi, @Fokko . It seems that @srowen clearly gave -1 on this approach. Although I approved this, we cannot merge your PR if there is a -1. I must admit that I didn't

[GitHub] [spark] srowen commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
srowen commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670794000 No, I don't feel that way. If others would like to merge, go ahead. I would merge the narrower change myself, but do not object to the broader one.

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670791274 Got it. So, you decided to give -1 for the enforcing for that reason. This is an automated message from

[GitHub] [spark] srowen commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
srowen commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670786844 I did. If most of the changes are exceptions, that makes me wonder how much this rule will just trigger false positives. I am not sure how worth it is to enforce this if mostly

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun edited a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670783658 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670783658 Hey, @srowen . Did you see my first comment, "Without the rule enforcement, unused import always happens again"? -

[GitHub] [spark] maropu edited a comment on pull request #29384: [SPARK-32564][SQL][TEST] Inject data statistics to simulate plan generation on actual TPCDS data

2020-08-07 Thread GitBox
maropu edited a comment on pull request #29384: URL: https://github.com/apache/spark/pull/29384#issuecomment-670779886 Thanks, all! @dongjoon-hyun Sure! @cloud-fan okay, I'll do follow-up. This is an automated

[GitHub] [spark] maropu commented on pull request #29384: [SPARK-32564][SQL][TEST] Inject data statistics to simulate plan generation on actual TPCDS data

2020-08-07 Thread GitBox
maropu commented on pull request #29384: URL: https://github.com/apache/spark/pull/29384#issuecomment-670779886 @dongjoon-hyun Sure! @cloud-fan okay, I'll do follow-up. This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670778245 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670778245 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670707796 **[Test build #127217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127217/testReport)** for PR 29121 at commit

[GitHub] [spark] SparkQA commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
SparkQA commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-67074 **[Test build #127217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127217/testReport)** for PR 29121 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29031: [SPARK-32216][SQL] Remove redundant ProjectExec

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29031: URL: https://github.com/apache/spark/pull/29031#issuecomment-670772420 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29031: [SPARK-32216][SQL] Remove redundant ProjectExec

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29031: URL: https://github.com/apache/spark/pull/29031#issuecomment-670772420 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29031: [SPARK-32216][SQL] Remove redundant ProjectExec

2020-08-07 Thread GitBox
SparkQA commented on pull request #29031: URL: https://github.com/apache/spark/pull/29031#issuecomment-670771944 **[Test build #127212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127212/testReport)** for PR 29031 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29031: [SPARK-32216][SQL] Remove redundant ProjectExec

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29031: URL: https://github.com/apache/spark/pull/29031#issuecomment-670642619 **[Test build #127212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127212/testReport)** for PR 29031 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670766308 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670766308 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670645783 **[Test build #127213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127213/testReport)** for PR 29074 at commit

[GitHub] [spark] SparkQA commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
SparkQA commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670763604 **[Test build #127213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127213/testReport)** for PR 29074 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-670751473 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-670751473 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-670632896 **[Test build #127211 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127211/testReport)** for PR 29342 at commit

[GitHub] [spark] SparkQA commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
SparkQA commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-670748063 **[Test build #127211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127211/testReport)** for PR 29342 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670725860 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670725860 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670725493 **[Test build #127218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127218/testReport)** for PR 28804 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670715442 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670715442 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670654894 **[Test build #127215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127215/testReport)** for PR 29386 at commit

[GitHub] [spark] SparkQA commented on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
SparkQA commented on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670714635 **[Test build #127215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127215/testReport)** for PR 29386 at commit

[GitHub] [spark] srowen commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
srowen commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670711092 My last comment was, why do we need to add the rule and then a ton of exclusions? just remove the unused imports. That's a much narrower change

[GitHub] [spark] mridulm commented on pull request #24554: [SPARK-27622][Core] Avoiding the network when block manager fetches disk persisted RDD blocks from the same host

2020-08-07 Thread GitBox
mridulm commented on pull request #24554: URL: https://github.com/apache/spark/pull/24554#issuecomment-670710590 Catching up on PR's ... this essentially means all executors on same host have effectively same preferred locality (modulo concurrent block removal) - did we update the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670707738 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670707728 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670694343 **[Test build #127216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127216/testReport)** for PR 28804 at commit

[GitHub] [spark] SparkQA commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
SparkQA commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670707796 **[Test build #127217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127217/testReport)** for PR 29121 at commit

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670707640 **[Test build #127216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127216/testReport)** for PR 28804 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670707728 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670705663 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670705663 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670705302 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] allisonwang-db commented on pull request #29137: [SPARK-32337][SQL] Show initial plan in AQE plan tree string

2020-08-07 Thread GitBox
allisonwang-db commented on pull request #29137: URL: https://github.com/apache/spark/pull/29137#issuecomment-670699341 I've updated the PR description. Please let me know if it makes sense. This is an automated message from

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467252553 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467251404 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467250468 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467250293 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670694952 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670694952 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670694343 **[Test build #127216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127216/testReport)** for PR 28804 at commit

[GitHub] [spark] karuppayya commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
karuppayya commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467173370 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala ## @@ -353,4 +353,8 @@ object AggUtils {

[GitHub] [spark] Fokko commented on pull request #29121: [SPARK-32319][PYSPARK] Disallow the use of unused imports

2020-08-07 Thread GitBox
Fokko commented on pull request #29121: URL: https://github.com/apache/spark/pull/29121#issuecomment-670690427 Would it be possible to move this forward? :) This is an automated message from the Apache Git Service. To

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467244022 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -314,7 +343,13 @@ private[joins] object

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467244022 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -314,7 +343,13 @@ private[joins] object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670659771 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29331: [SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29331: URL: https://github.com/apache/spark/pull/29331#issuecomment-670659548 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670645866 **[Test build #127214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127214/testReport)** for PR 28804 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670659760 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670659760 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670659564 **[Test build #127214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127214/testReport)** for PR 28804 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29331: [SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29331: URL: https://github.com/apache/spark/pull/29331#issuecomment-670659548 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29331: [SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3

2020-08-07 Thread GitBox
SparkQA removed a comment on pull request #29331: URL: https://github.com/apache/spark/pull/29331#issuecomment-670596918 **[Test build #127208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127208/testReport)** for PR 29331 at commit

[GitHub] [spark] SparkQA commented on pull request #29331: [SPARK-32517][CORE] Add StorageLevel.DISK_ONLY_3

2020-08-07 Thread GitBox
SparkQA commented on pull request #29331: URL: https://github.com/apache/spark/pull/29331#issuecomment-670658671 **[Test build #127208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127208/testReport)** for PR 29331 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre and Hadoop to 3.2.1

2020-08-07 Thread GitBox
dongjoon-hyun commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-670656860 Thank you so much. Yes. I'm looking forward to seeing that~ This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
dongjoon-hyun commented on a change in pull request #29386: URL: https://github.com/apache/spark/pull/29386#discussion_r467209211 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ## @@ -395,7 +395,7 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670655440 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670655440 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
SparkQA commented on pull request #29386: URL: https://github.com/apache/spark/pull/29386#issuecomment-670654894 **[Test build #127215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127215/testReport)** for PR 29386 at commit

[GitHub] [spark] dongjoon-hyun opened a new pull request #29386: [SPARK-32568][BUILD][SS] Upgrade Kafka to 2.6.0

2020-08-07 Thread GitBox
dongjoon-hyun opened a new pull request #29386: URL: https://github.com/apache/spark/pull/29386 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] viirya commented on a change in pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

2020-08-07 Thread GitBox
viirya commented on a change in pull request #29328: URL: https://github.com/apache/spark/pull/29328#discussion_r467199754 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -229,6 +229,7 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670646425 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
AmplabJenkins removed a comment on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670646303 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670646303 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

2020-08-07 Thread GitBox
AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-670646425 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] viirya commented on pull request #29385: [SPARK-32191][PySpark][DOC] Migration Guide for PySpark docs

2020-08-07 Thread GitBox
viirya commented on pull request #29385: URL: https://github.com/apache/spark/pull/29385#issuecomment-670646380 cc @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] SparkQA commented on pull request #29074: [SPARK-32282][SQL] Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-08-07 Thread GitBox
SparkQA commented on pull request #29074: URL: https://github.com/apache/spark/pull/29074#issuecomment-670645783 **[Test build #127213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127213/testReport)** for PR 29074 at commit

  1   2   3   4   5   >