[GitHub] [spark] SparkQA commented on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

SparkQA commented on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-648391402 **[Test build #124433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124433/testReport)** for PR 28905 at commit

[GitHub] [spark] MaxGekk commented on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

MaxGekk commented on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-648389771 @cloud-fan @HyukjinKwon @dongjoon-hyun Please, review this PR. This is an automated message from the Apache Git

[GitHub] [spark] HeartSaVioR commented on pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink to avoid memory issue

HeartSaVioR commented on pull request #28904: URL: https://github.com/apache/spark/pull/28904#issuecomment-648380748 Just to be sure, the test app with patch now writes the version 1589, which the log file size is 2.9G, with RES 1.025g.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648379434 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

SparkQA removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648364267 **[Test build #124431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124431/testReport)** for PR 28912 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648379434 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648379121 **[Test build #124431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124431/testReport)** for PR 28912 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648375049 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648375049 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648374670 **[Test build #124432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124432/testReport)** for PR 28906 at commit

[GitHub] [spark] MaxGekk commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

MaxGekk commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648373340 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] viirya commented on pull request #28908: [SPARK-32073][R] Drop R < 3.5 support

viirya commented on pull request #28908: URL: https://github.com/apache/spark/pull/28908#issuecomment-648368937 As we accidentally dropped R < 3.5 in Spark 3.0.0, can we update the doc like https://spark.apache.org/docs/latest/index.html to make it less confusing?

[GitHub] [spark] holdenk commented on a change in pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28708: URL: https://github.com/apache/spark/pull/28708#discussion_r53446 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -0,0 +1,280 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] holdenk commented on a change in pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

holdenk commented on a change in pull request #28370: URL: https://github.com/apache/spark/pull/28370#discussion_r52013 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -1551,30 +1555,36 @@ private[spark] class BlockManager( } /**

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648364897 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648364897 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648364267 **[Test build #124431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124431/testReport)** for PR 28912 at commit

[GitHub] [spark] alismess-db opened a new pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

alismess-db opened a new pull request #28912: URL: https://github.com/apache/spark/pull/28912 ### What changes were proposed in this pull request? #28671 introduced a change where the order in which CANCELED state for SparkExecuteStatementOperation is set was changed. Before setting the

[GitHub] [spark] alismess-db commented on pull request #28912: [SPARK-32057] ExecuteStatement: cancel should not transiently ERROR

alismess-db commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-648361969 cc @juliuszsompolski This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

SparkQA commented on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-648359008 **[Test build #124430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124430/testReport)** for PR 28905 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648356311 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648356005 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648355145 **[Test build #124429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124429/testReport)** for PR 28906 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

AmplabJenkins removed a comment on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-648355907 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648356304 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648356005 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648356295 **[Test build #124429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124429/testReport)** for PR 28906 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28905: [SPARK-32071][SQL] Benchmark `make_interval`

AmplabJenkins commented on pull request #28905: URL: https://github.com/apache/spark/pull/28905#issuecomment-648355907 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648355145 **[Test build #124429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124429/testReport)** for PR 28906 at commit

[GitHub] [spark] MaxGekk commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

MaxGekk commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648353656 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28900: [SPARK-32056][SQL] Coalesce partitions for repartition by expressions when AQE is enabled

AmplabJenkins removed a comment on pull request #28900: URL: https://github.com/apache/spark/pull/28900#issuecomment-648345856 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28900: [SPARK-32056][SQL] Coalesce partitions for repartition by expressions when AQE is enabled

AmplabJenkins commented on pull request #28900: URL: https://github.com/apache/spark/pull/28900#issuecomment-648345856 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28900: [SPARK-32056][SQL] Coalesce partitions for repartition by expressions when AQE is enabled

SparkQA removed a comment on pull request #28900: URL: https://github.com/apache/spark/pull/28900#issuecomment-648167341 **[Test build #124424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124424/testReport)** for PR 28900 at commit

[GitHub] [spark] SparkQA commented on pull request #28900: [SPARK-32056][SQL] Coalesce partitions for repartition by expressions when AQE is enabled

SparkQA commented on pull request #28900: URL: https://github.com/apache/spark/pull/28900#issuecomment-648344696 **[Test build #124424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124424/testReport)** for PR 28900 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648344037 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648344459 @tgravescs a suggestion to improve the title of the PR is also welcome. It is hard to do justice in one simple sentence. I see how you would fail to grasp what the change is for

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648344029 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648342850 **[Test build #124428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124428/testReport)** for PR 28906 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648344029 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648344016 **[Test build #124428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124428/testReport)** for PR 28906 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648343576 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648343576 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648342850 **[Test build #124428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124428/testReport)** for PR 28906 at commit

[GitHub] [spark] MaxGekk commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

MaxGekk commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648342834 @srowen @dongjoon-hyun Please, review this PR. This is an automated message from the Apache Git Service. To

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648339593 > sorry if I wasn't clear. I think this approach of having the fileLostEpoch is better so we avoid the locking in MapOutputTracker. Personally I wouldn't mind fileLost being

[GitHub] [spark] vinooganesh commented on pull request #28899: [SPARK-32062][SQL] Reset listenerRegistered in SparkSession

vinooganesh commented on pull request #28899: URL: https://github.com/apache/spark/pull/28899#issuecomment-648332194 @ulysses-you - quick question for you (mostly for my own knowledge), do people usually destroy and re-create a spark context in the lifetime of a JVM? I was actually

[GitHub] [spark] tgravescs commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

tgravescs commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648329200 sorry if I wasn't clear. I think this approach of having the fileLostEpoch is better so we avoid the locking in MapOutputTracker. Personally I wouldn't mind fileLost being

[GitHub] [spark] dongjoon-hyun commented on pull request #28908: [SPARK-32073][R] Drop R < 3.5 support

dongjoon-hyun commented on pull request #28908: URL: https://github.com/apache/spark/pull/28908#issuecomment-648322331 Thanks, @HyukjinKwon . BTW, if we accidentally dropped it, how can we mention it in this case? This PR should target `Apache Spark 3.0.1` with `releasenotes` label? (For

[GitHub] [spark] tgravescs commented on a change in pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting

tgravescs commented on a change in pull request #28874: URL: https://github.com/apache/spark/pull/28874#discussion_r444392875 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -181,23 +181,23 @@ private[history] class

[GitHub] [spark] dongjoon-hyun commented on pull request #28907: [SPARK-31918][R] Ignore S4 generic methods under SparkR namespace in closure cleaning to support R 4.0.0+

dongjoon-hyun commented on pull request #28907: URL: https://github.com/apache/spark/pull/28907#issuecomment-648307414 Thank you for pinging me, @HyukjinKwon . This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

AmplabJenkins commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-648304842 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting

AmplabJenkins removed a comment on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-648304842 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

SparkQA commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-648304346 **[Test build #124427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124427/testReport)** for PR 28874 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting

AmplabJenkins removed a comment on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-646901034 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] tgravescs commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

tgravescs commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-648303691 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28899: [SPARK-32062][SQL] Reset listenerRegistered in SparkSession

AmplabJenkins removed a comment on pull request #28899: URL: https://github.com/apache/spark/pull/28899#issuecomment-648298062 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28899: [SPARK-32062][SQL] Reset listenerRegistered in SparkSession

AmplabJenkins commented on pull request #28899: URL: https://github.com/apache/spark/pull/28899#issuecomment-648298062 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28899: [SPARK-32062][SQL] Reset listenerRegistered in SparkSession

SparkQA removed a comment on pull request #28899: URL: https://github.com/apache/spark/pull/28899#issuecomment-648089324 **[Test build #124416 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124416/testReport)** for PR 28899 at commit

[GitHub] [spark] tgravescs commented on pull request #28864: [SPARK-32004][ALL] Drop references to slave

tgravescs commented on pull request #28864: URL: https://github.com/apache/spark/pull/28864#issuecomment-648296936 are we doing the docs under a separate jira? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #28899: [SPARK-32062][SQL] Reset listenerRegistered in SparkSession

SparkQA commented on pull request #28899: URL: https://github.com/apache/spark/pull/28899#issuecomment-648296968 **[Test build #124416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124416/testReport)** for PR 28899 at commit

[GitHub] [spark] tgravescs commented on a change in pull request #28864: [SPARK-32004][ALL] Drop references to slave

tgravescs commented on a change in pull request #28864: URL: https://github.com/apache/spark/pull/28864#discussion_r444379610 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -459,9 +459,10 @@ package object config {

[GitHub] [spark] wypoon edited a comment on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon edited a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648293945 @tgravescs as you point out, it is ok to call `mapOutputTracker.removeOutputsOnHost` or `mapOutputTracker.removeOutputsOnExecutor` multiple times with the same

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648293945 @tgravescs as you point out, it is ok to call `mapOutputTracker.removeOutputsOnHost` or `mapOutputTracker.removeOutputsOnExecutor` multiple times with the same host/execId.

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r444377024 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -177,6 +177,8 @@ private[spark] class DAGScheduler( // TODO:

[GitHub] [spark] tgravescs commented on a change in pull request #28864: [SPARK-32004][ALL] Drop references to slave

tgravescs commented on a change in pull request #28864: URL: https://github.com/apache/spark/pull/28864#discussion_r444376608 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -226,9 +226,9 @@ private[spark] class BlockManager( private val

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-648290364 @tgravescs thanks for reviewing. Our customer was not using spark.files.fetchFailure.unRegisterOutputOnHost. In case of `FetchFailure`, in

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28891: [minor] update dev/create-release/known_translations

AmplabJenkins removed a comment on pull request #28891: URL: https://github.com/apache/spark/pull/28891#issuecomment-648280903 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28891: [minor] update dev/create-release/known_translations

AmplabJenkins commented on pull request #28891: URL: https://github.com/apache/spark/pull/28891#issuecomment-648280903 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28891: [minor] update dev/create-release/known_translations

SparkQA removed a comment on pull request #28891: URL: https://github.com/apache/spark/pull/28891#issuecomment-648138887 **[Test build #124423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124423/testReport)** for PR 28891 at commit

[GitHub] [spark] SparkQA commented on pull request #28891: [minor] update dev/create-release/known_translations

SparkQA commented on pull request #28891: URL: https://github.com/apache/spark/pull/28891#issuecomment-648279507 **[Test build #124423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124423/testReport)** for PR 28891 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28909: [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2

AmplabJenkins commented on pull request #28909: URL: https://github.com/apache/spark/pull/28909#issuecomment-648276549 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28909: [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2

AmplabJenkins removed a comment on pull request #28909: URL: https://github.com/apache/spark/pull/28909#issuecomment-648276549 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28909: [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2

SparkQA removed a comment on pull request #28909: URL: https://github.com/apache/spark/pull/28909#issuecomment-648138793 **[Test build #124422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124422/testReport)** for PR 28909 at commit

[GitHub] [spark] SparkQA commented on pull request #28909: [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2

SparkQA commented on pull request #28909: URL: https://github.com/apache/spark/pull/28909#issuecomment-648275212 **[Test build #124422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124422/testReport)** for PR 28909 at commit

[GitHub] [spark] siknezevic commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-648260763 > > Spilling SpillReader with 16000 rows: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative > > UnsafeSorterSpillReader_ratio1 568 727 200 0.5 2219.4

[GitHub] [spark] siknezevic closed pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic closed pull request #27246: URL: https://github.com/apache/spark/pull/27246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] tgravescs commented on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

tgravescs commented on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648257419 thanks for working on this, was interested in this as well. Can you update the description to include details on your overall approach - where do you get the directories

[GitHub] [spark] planga82 commented on pull request #28896: [SPARK-32025][SQL] Csv schema inference problems with different types in the same column

planga82 commented on pull request #28896: URL: https://github.com/apache/spark/pull/28896#issuecomment-648255662 @HyukjinKwon It's exactly what you say, it only happens when the incompatibility is inside one partition. I will change the PR to use compatibleType, and I will make some

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28895: [SPARK-32055][CORE][SQL] Unify getReader and getReaderForRange in ShuffleManager

AmplabJenkins removed a comment on pull request #28895: URL: https://github.com/apache/spark/pull/28895#issuecomment-64825 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28895: [SPARK-32055][CORE][SQL] Unify getReader and getReaderForRange in ShuffleManager

AmplabJenkins commented on pull request #28895: URL: https://github.com/apache/spark/pull/28895#issuecomment-64825 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28895: [SPARK-32055][CORE][SQL] Unify getReader and getReaderForRange in ShuffleManager

SparkQA commented on pull request #28895: URL: https://github.com/apache/spark/pull/28895#issuecomment-648249543 **[Test build #124419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124419/testReport)** for PR 28895 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28895: [SPARK-32055][CORE][SQL] Unify getReader and getReaderForRange in ShuffleManager

SparkQA removed a comment on pull request #28895: URL: https://github.com/apache/spark/pull/28895#issuecomment-648108084 **[Test build #124419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124419/testReport)** for PR 28895 at commit

[GitHub] [spark] tgravescs commented on a change in pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is l

tgravescs commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r444282018 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -177,6 +177,8 @@ private[spark] class DAGScheduler( // TODO:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-648239051 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-648239051 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

SparkQA removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-648101561 **[Test build #124418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124418/testReport)** for PR 28863 at commit

[GitHub] [spark] SparkQA commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

SparkQA commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-648237409 **[Test build #124418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124418/testReport)** for PR 28863 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #28805: [SPARK-28169][SQL] Convert scan predicate condition to CNF

AngersZh commented on a change in pull request #28805: URL: https://github.com/apache/spark/pull/28805#discussion_r444305821 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PruneFileSourcePartitions.scala ## @@ -87,8 +88,17 @@ private[sql]

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648229881 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

AmplabJenkins commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648229881 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

SparkQA removed a comment on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648226563 **[Test build #124426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124426/testReport)** for PR 28911 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

AmplabJenkins removed a comment on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648228067 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

AmplabJenkins removed a comment on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648228080 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA removed a comment on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648089282 **[Test build #124415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124415/testReport)** for PR 28906 at commit

[GitHub] [spark] SparkQA commented on pull request #28906: [SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results

SparkQA commented on pull request #28906: URL: https://github.com/apache/spark/pull/28906#issuecomment-648228200 **[Test build #124415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124415/testReport)** for PR 28906 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

AmplabJenkins commented on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648228067 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

SparkQA commented on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648228052 **[Test build #124426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124426/testReport)** for PR 28911 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

AmplabJenkins removed a comment on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648227256 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

AmplabJenkins commented on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648227256 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

SparkQA commented on pull request #28911: URL: https://github.com/apache/spark/pull/28911#issuecomment-648226563 **[Test build #124426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124426/testReport)** for PR 28911 at commit

<    1   2   3   4   5   6   7   8   9   >