[GitHub] [spark] fqaiser94 edited a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 edited a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645047815 @cloud-fan I've made fairly significant changes to the implementation in order to address the issue where poorly optimized physical plans resulted in massive amounts

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106680 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon closed pull request #28827: [SPARK-31989][SQL] Generate JSON rebasing files w/ 30 minutes step

HyukjinKwon closed pull request #28827: URL: https://github.com/apache/spark/pull/28827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120390 Thanks @WeichenXu123 for review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120346 Let me just merge and go ahead since it's just all about a documentation and warning at the end where we can change without many restrictions. Let me know if you guys

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645151419 **[Test build #124153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r441280737 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -540,6 +540,46 @@ class DAGSchedulerSuite extends

[GitHub] [spark] SparkQA commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

SparkQA commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645104956 **[Test build #124147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124147/testReport)** for PR 28845 at commit

[GitHub] [spark] yaooqinn commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645104925 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146303 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146298 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] jiangxb1987 commented on a change in pull request #28839: [SPARK-32000][CORE][TESTS] Fix the flaky testcase for partially launched task in barrier-mode.

jiangxb1987 commented on a change in pull request #28839: URL: https://github.com/apache/spark/pull/28839#discussion_r441298122 ## File path: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala ## @@ -275,7 +275,8 @@ class BarrierTaskContextSuite

[GitHub] [spark] cchighman closed pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman closed pull request #28841: URL: https://github.com/apache/spark/pull/28841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149534 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149861 @attilapiros This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157060 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157055 **[Test build #124154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645115062 cc @tgravescs This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645114911 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun opened a new pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun opened a new pull request #28847: URL: https://github.com/apache/spark/pull/28847 ### What changes were proposed in this pull request? This aims to replace `executorIdsToBeRemoved` with `executorIdToBeRemoved`. ### Why are the changes needed? Since a wrong

[GitHub] [spark] HyukjinKwon closed pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon closed pull request #28845: URL: https://github.com/apache/spark/pull/28845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120469 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645148461 **[Test build #124143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124143/testReport)** for PR 28761 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645067909 **[Test build #124143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124143/testReport)** for PR 28761 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-64515 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156370 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156028 **[Test build #124154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156370 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645116392 **[Test build #124151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645122517 Thank you, @HyukjinKwon . This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146152 **[Test build #124149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124149/testReport)** for PR 28846 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645102759 **[Test build #124149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124149/testReport)** for PR 28846 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146298 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The

[GitHub] [spark] SparkQA commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149223 **[Test build #124152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124152/testReport)** for PR 28846 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645149128 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149534 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149561 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] wypoon opened a new pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon opened a new pull request #28848: URL: https://github.com/apache/spark/pull/28848 …after executor is lost ### What changes were proposed in this pull request? If an executor is lost, the `DAGScheduler` handles the executor loss by removing the executor but does not

[GitHub] [spark] AmplabJenkins commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645149128 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645116392 **[Test build #124151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28842: [SPARK-32006][SQL] Create date/timestamp formatters once before collect in `hiveResultString()`

cloud-fan commented on a change in pull request #28842: URL: https://github.com/apache/spark/pull/28842#discussion_r441297086 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala ## @@ -72,47 +90,32 @@ object HiveResult { } } - //

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

AmplabJenkins removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167728 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645168383 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645168383 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645114911 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The

[GitHub] [spark] SparkQA removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645151419 **[Test build #124153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152215 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152215 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152202 **[Test build #124153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106680 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106412 **[Test build #124150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124150/testReport)** for PR 28835 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645105255 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

SparkQA removed a comment on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645095283 **[Test build #124147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124147/testReport)** for PR 28845 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins removed a comment on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645105255 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun closed pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun closed pull request #28847: URL: https://github.com/apache/spark/pull/28847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645134832 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cchighman closed pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman closed pull request #28841: URL: https://github.com/apache/spark/pull/28841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149561 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #28827: [SPARK-31989][SQL] Generate JSON rebasing files w/ 30 minutes step

HyukjinKwon commented on pull request #28827: URL: https://github.com/apache/spark/pull/28827#issuecomment-645119748 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157060 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156028 **[Test build #124154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157066 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

AmplabJenkins commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167728 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167153 **[Test build #124146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124146/testReport)** for PR 27066 at commit

[GitHub] [spark] cloud-fan commented on pull request #28778: [SPARK-31949][SQL] Add spark.default.parallelism in SQLConf for isolated across session

cloud-fan commented on pull request #28778: URL: https://github.com/apache/spark/pull/28778#issuecomment-645167188 After more thoughts, I think the file partitions split logic itself is problematic. Its target is to make the number of partitions the same as the total number of cores,

[GitHub] [spark] SparkQA commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645167716 **[Test build #124151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645093212 **[Test build #124146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124146/testReport)** for PR 27066 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec

AmplabJenkins removed a comment on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-64470 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec

AmplabJenkins commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-64470 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] karuppayya commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec

karuppayya commented on a change in pull request #28715: URL: https://github.com/apache/spark/pull/28715#discussion_r440609572 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala ## @@ -226,18 +226,33 @@ case class GenerateExec( "0"

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause

AmplabJenkins removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644554496 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed

AmplabJenkins removed a comment on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644536866 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore

AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644559477 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore

SparkQA removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644515779 **[Test build #124088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124088/testReport)** for PR 28835 at commit

[GitHub] [spark] wangyum commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed

wangyum commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644558909 ok to test. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore

SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644559034 **[Test build #124088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124088/testReport)** for PR 28835 at commit

[GitHub] [spark] yaooqinn commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644564303 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] sarutak edited a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column

sarutak edited a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644553331 @liucht-inspur You can't run Jenkins by yourself for now. If you continue to contribute to Spark, you will be added to the whitelist and be given the right to run

[GitHub] [spark] sarutak edited a comment on pull request #28799: [SPARK-31871][CORE][WEBUI][2.4] Display the canvas element icon for sorting column

sarutak edited a comment on pull request #28799: URL: https://github.com/apache/spark/pull/28799#issuecomment-644553331 @liucht-inspur You can't run Jenkins by yourself for now. If you continue to contribute to Spark, you will be added to the whitelist and given the right to run Jenkins

[GitHub] [spark] SparkQA removed a comment on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause

SparkQA removed a comment on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644527550 **[Test build #124092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124092/testReport)** for PR 28836 at commit

[GitHub] [spark] SparkQA commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

SparkQA commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644574854 **[Test build #124103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124103/testReport)** for PR 28824 at commit

[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644574860 **[Test build #124093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124093/testReport)** for PR 28710 at commit

[GitHub] [spark] SparkQA commented on pull request #28837: [SPARK-31996][BUILD] Specify the version of ChromeDriver and RemoteWebDriver which can work with guava 14.0.1

SparkQA commented on pull request #28837: URL: https://github.com/apache/spark/pull/28837#issuecomment-644574866 **[Test build #124094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124094/testReport)** for PR 28837 at commit

[GitHub] [spark] SparkQA commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause

SparkQA commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644574172 **[Test build #124092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124092/testReport)** for PR 28836 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array type

AmplabJenkins commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644583693 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array types of c

SparkQA commented on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644583204 **[Test build #124111 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124111/testReport)** for PR 28831 at commit

[GitHub] [spark] bart-samwel commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause

bart-samwel commented on pull request #28836: URL: https://github.com/apache/spark/pull/28836#issuecomment-644615299 I'm not a huge fan of this. For one thing, it is not in ANSI SQL 2016. AFAICT it's a Teradata extension that Snowflake adopted as well because they get a lot of Teradata

[GitHub] [spark] xuanyuanking commented on pull request #28838: [SPARK-31997][SQL][TESTS] Drop test_udtf table when SingleSessionSuite test completed

xuanyuanking commented on pull request #28838: URL: https://github.com/apache/spark/pull/28838#issuecomment-644615113 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] sarutak opened a new pull request #28839: [SPARK-32000][CORE][TESTS] Fix the flaky testcase for partially launched task in barrier-mode.

sarutak opened a new pull request #28839: URL: https://github.com/apache/spark/pull/28839 ### What changes were proposed in this pull request? ### Why are the changes needed? This PR fixes the flaky testcase "barrier stage should fail if only partial tasks are launched"

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28839: [SPARK-32000][CORE][TESTS] Fix the flaky testcase for partially launched task in barrier-mode.

AmplabJenkins removed a comment on pull request #28839: URL: https://github.com/apache/spark/pull/28839#issuecomment-644634501 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on pull request #28707: [SPARK-31894][SS] Introduce UnsafeRow format validation for streaming state store

cloud-fan commented on pull request #28707: URL: https://github.com/apache/spark/pull/28707#issuecomment-644635176 I guess we may need to change the state store API to allow Spark to do validation easily. I'm OK to put the validation logic in the state store implementation for now, and

[GitHub] [spark] cloud-fan edited a comment on pull request #28707: [SPARK-31894][SS] Introduce UnsafeRow format validation for streaming state store

cloud-fan edited a comment on pull request #28707: URL: https://github.com/apache/spark/pull/28707#issuecomment-644635176 It seems we need to change the state store API to allow Spark to do validation easily. I'm OK to put the validation logic in the state store implementation for now,

[GitHub] [spark] SparkQA removed a comment on pull request #28839: [SPARK-32000][CORE][TESTS] Fix the flaky testcase for partially launched task in barrier-mode.

SparkQA removed a comment on pull request #28839: URL: https://github.com/apache/spark/pull/28839#issuecomment-644632819 **[Test build #124114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124114/testReport)** for PR 28839 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644559477 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644559483 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] MaxGekk commented on a change in pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

MaxGekk commented on a change in pull request #28824: URL: https://github.com/apache/spark/pull/28824#discussion_r440628505 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/RebaseDateTimeSuite.scala ## @@ -416,38 +418,39 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/ar

AmplabJenkins removed a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644575049 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-644575348 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   7   >