[GitHub] [spark] SparkQA commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

SparkQA commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-672302622 **[Test build #127348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127348/testReport)** for PR 29396 at commit

[GitHub] [spark] huaxingao commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

huaxingao commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r46891 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala ## @@ -0,0 +1,61 @@ +/* + * Licensed to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

AmplabJenkins removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-672297178 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

AmplabJenkins commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-672297178 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

SparkQA commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-672296315 **[Test build #127340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127340/testReport)** for PR 28804 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

SparkQA removed a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-672091607 **[Test build #127340 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127340/testReport)** for PR 28804 at commit

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468882865 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +450,88 @@ class

[GitHub] [spark] Udbhav30 commented on pull request #29387: [SPARK-32481] Support truncate table to move data to trash

Udbhav30 commented on pull request #29387: URL: https://github.com/apache/spark/pull/29387#issuecomment-67229 Gentle ping @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672288827 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

AmplabJenkins removed a comment on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672288736 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672288827 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

AmplabJenkins commented on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672288736 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] c21 commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672288194 @agrawaldevesh - thanks for notes. I totally agree. Just to point out for existing current approach, I already use unsafe row boolean type to store the matched bit in

[GitHub] [spark] SparkQA removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672181632 **[Test build #127343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127343/testReport)** for PR 29367 at commit

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672288032 **[Test build #127343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127343/testReport)** for PR 29367 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

SparkQA removed a comment on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672075384 **[Test build #127338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127338/testReport)** for PR 28617 at commit

[GitHub] [spark] SparkQA commented on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

SparkQA commented on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672287681 **[Test build #127338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127338/testReport)** for PR 28617 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [WIP][SPARK-32180][PYTHON][DOCS] Getting started-Installation guide for pyspark doc

AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672277585 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [WIP][SPARK-32180][PYTHON][DOCS] Getting started-Installation guide for pyspark doc

AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672278185 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [WIP][SPARK-32180][PYTHON][DOCS] Getting started-Installation guide for pyspark doc

AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672277585 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

AmplabJenkins commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-672276640 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

AmplabJenkins removed a comment on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-672276640 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

SparkQA removed a comment on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-672070547 **[Test build #127336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127336/testReport)** for PR 29383 at commit

[GitHub] [spark] SparkQA commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

SparkQA commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-672275557 **[Test build #127336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127336/testReport)** for PR 29383 at commit

[GitHub] [spark] agrawaldevesh commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

agrawaldevesh commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672275450 Hi Cheng, I am wondering if you might have a perf test handy to test this new implementation vs your old approach ? After going through the code and following

[GitHub] [spark] rohitmishr1484 opened a new pull request #29410: [WIP][SPARK-32180][PYTHON][DOCS] Getting started-Installation guide for pyspark doc

rohitmishr1484 opened a new pull request #29410: URL: https://github.com/apache/spark/pull/29410 # What changes were proposed in this pull request? This PR proposes to add getting started- installation to new PySpark docs. ### Why are the changes needed? Better documentation.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672270731 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672270731 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672269109 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672269751 **[Test build #127342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127342/testReport)** for PR 29367 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672137339 **[Test build #127342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127342/testReport)** for PR 29367 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672269096 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672107180 **[Test build #127341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127341/testReport)** for PR 28841 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672269096 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672268829 **[Test build #127341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127341/testReport)** for PR 28841 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

AmplabJenkins commented on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672268078 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

AmplabJenkins removed a comment on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672268078 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

SparkQA commented on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672267613 **[Test build #127347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127347/testReport)** for PR 29409 at commit

[GitHub] [spark] MaxGekk opened a new pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

MaxGekk opened a new pull request #29409: URL: https://github.com/apache/spark/pull/29409 ### What changes were proposed in this pull request? Fix `DaysWritable` by overriding parent's method `def get(doesTimeMatter: Boolean): Date` from `DateWritable` instead of `Date get()` because

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672264729 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672264729 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672264107 **[Test build #127345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127345/testReport)** for PR 29367 at commit

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672264110 **[Test build #127346 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127346/testReport)** for PR 29328 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins removed a comment on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672261135 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672261183 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

AmplabJenkins commented on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672260894 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672261183 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

AmplabJenkins commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672261135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

AmplabJenkins removed a comment on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672260894 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

SparkQA commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672260512 **[Test build #127344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127344/testReport)** for PR 29328 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

SparkQA removed a comment on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672019338 **[Test build #127334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127334/testReport)** for PR 28617 at commit

[GitHub] [spark] SparkQA commented on pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions APIs on DataSourceV2

SparkQA commented on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-672259677 **[Test build #127334 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127334/testReport)** for PR 28617 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672248376 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672248376 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672003736 **[Test build #127332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127332/testReport)** for PR 29360 at commit

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672247240 **[Test build #127332 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127332/testReport)** for PR 29360 at commit

[GitHub] [spark] jkleckner commented on pull request #28423: [SPARK-24266][k8s] Restart the watcher when we receive a version changed from k8s

jkleckner commented on pull request #28423: URL: https://github.com/apache/spark/pull/28423#issuecomment-672244924 It looks a bit different from what I see. For me, it appears to get stuck at the very end of writing data to Bigtable in the very last task of a job. Our partner is working

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672233491 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672233431 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672233431 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672057796 **[Test build #127335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127335/testReport)** for PR 29360 at commit

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-672231939 **[Test build #127335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127335/testReport)** for PR 29360 at commit

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468821063 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468819586 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class

[GitHub] [spark] imback82 commented on pull request #23340: [SPARK-23431][CORE] Expose the new executor memory metrics at the stage level

imback82 commented on pull request #23340: URL: https://github.com/apache/spark/pull/23340#issuecomment-672217163 This can be closed now. We can use changes in #29020 if backporting to 2.4 is needed. (@dongjoon-hyun / @gengliangwang can confirm, but I don't think this will be backported

[GitHub] [spark] stackedsax commented on pull request #23340: [SPARK-23431][CORE] Expose the new executor memory metrics at the stage level

stackedsax commented on pull request #23340: URL: https://github.com/apache/spark/pull/23340#issuecomment-672196580 @imback82 Looks like #29020 is merged into 3.1.0. Should this issue get closed now, or is it being left open for someone to do the same in 2.x versions of Spark?

[GitHub] [spark] viirya commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672194544 sgtm This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] c21 commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672185658 > Any plan to support the case? If we do so, the first key index should be long. @maropu - I think the key index for `LongHashedRelation` can be the index into key array -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672184474 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672184474 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672181632 **[Test build #127343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127343/testReport)** for PR 29367 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672168985 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

AmplabJenkins removed a comment on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-672169721 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

AmplabJenkins commented on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-672169721 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] tgravescs commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

tgravescs commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-672169384 filed https://issues.apache.org/jira/browse/SPARK-32591 as a followup This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672168977 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672168977 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

SparkQA removed a comment on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-671962512 **[Test build #127326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127326/testReport)** for PR 29401 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

SparkQA removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-671980769 **[Test build #127329 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127329/testReport)** for PR 29270 at commit

[GitHub] [spark] SparkQA commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

SparkQA commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672167390 **[Test build #127329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127329/testReport)** for PR 29270 at commit

[GitHub] [spark] SparkQA commented on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

SparkQA commented on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-672167282 **[Test build #127326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127326/testReport)** for PR 29401 at commit

[GitHub] [spark] cloud-fan commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672165234 yea sounds good! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29334: [SPARK-32495][2.4] Update jackson versions to a maintained release, to fix various security vulnerabilities.

AmplabJenkins removed a comment on pull request #29334: URL: https://github.com/apache/spark/pull/29334#issuecomment-672159195 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29334: [SPARK-32495][2.4] Update jackson versions to a maintained release, to fix various security vulnerabilities.

AmplabJenkins removed a comment on pull request #29334: URL: https://github.com/apache/spark/pull/29334#issuecomment-672159126 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29334: [SPARK-32495][2.4] Update jackson versions to a maintained release, to fix various security vulnerabilities.

AmplabJenkins commented on pull request #29334: URL: https://github.com/apache/spark/pull/29334#issuecomment-672159126 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29334: [SPARK-32495][2.4] Update jackson versions to a maintained release, to fix various security vulnerabilities.

SparkQA removed a comment on pull request #29334: URL: https://github.com/apache/spark/pull/29334#issuecomment-672070572 **[Test build #127337 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127337/testReport)** for PR 29334 at commit

[GitHub] [spark] SparkQA commented on pull request #29334: [SPARK-32495][2.4] Update jackson versions to a maintained release, to fix various security vulnerabilities.

SparkQA commented on pull request #29334: URL: https://github.com/apache/spark/pull/29334#issuecomment-672157795 **[Test build #127337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127337/testReport)** for PR 29334 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672156481 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672156463 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672156463 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-671989831 **[Test build #127331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127331/testReport)** for PR 28841 at commit

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672155623 **[Test build #127331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127331/testReport)** for PR 28841 at commit

[GitHub] [spark] c21 commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672142777 Thanks @cloud-fan and @maropu for feedback and discussion. So here is the new proposal of change: * `BytesToBytesMap.java`: Add a new iterator implementation to iterate

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r468763396 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672139882 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672139882 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #29367: URL: https://github.com/apache/spark/pull/29367#issuecomment-672137339 **[Test build #127342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127342/testReport)** for PR 29367 at commit

[GitHub] [spark] mridulm commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

mridulm commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-672111685 @zhouyejoe You had looked at/fixed something similar IIRC - can you please take a look ? Thx This is an

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672107974 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672107974 This is an automated message from the Apache Git Service. To respond to the message, please log on to

<    1   2   3   4   5   6   7   >