[GitHub] [spark] AmplabJenkins removed a comment on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins removed a comment on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695749303 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695749303 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695749096 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

SparkQA commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695749220 **[Test build #128911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128911/testReport)** for PR 29814 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695749093 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695749093 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695472488 **[Test build #128901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128901/testReport)** for PR 29812 at commit

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695748996 **[Test build #128901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128901/testReport)** for PR 29812 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

AmplabJenkins removed a comment on pull request #29587: URL: https://github.com/apache/spark/pull/29587#issuecomment-695748896 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

AmplabJenkins commented on pull request #29587: URL: https://github.com/apache/spark/pull/29587#issuecomment-695748896 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins removed a comment on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747969 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

SparkQA removed a comment on pull request #29587: URL: https://github.com/apache/spark/pull/29587#issuecomment-695412883 **[Test build #128897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128897/testReport)** for PR 29587 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins removed a comment on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747570 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

SparkQA removed a comment on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747460 **[Test build #128909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128909/testReport)** for PR 29814 at commit

[GitHub] [spark] SparkQA commented on pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

SparkQA commented on pull request #29587: URL: https://github.com/apache/spark/pull/29587#issuecomment-695748762 **[Test build #128897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128897/testReport)** for PR 29587 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

AmplabJenkins removed a comment on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695748031 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29813: [SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName

AmplabJenkins removed a comment on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695748239 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

AmplabJenkins removed a comment on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695746460 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29813: [SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName

SparkQA removed a comment on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695745994 **[Test build #128908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128908/testReport)** for PR 29813 at commit

[GitHub] [spark] SparkQA commented on pull request #29813: [SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName

SparkQA commented on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695748225 **[Test build #128908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128908/testReport)** for PR 29813 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29813: [SPARK-32799][R][SQL] Add allowMissingColumns to SparkR unionByName

AmplabJenkins commented on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695748239 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

AmplabJenkins commented on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695748031 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

SparkQA commented on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695747932 **[Test build #128910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128910/testReport)** for PR 29810 at commit

[GitHub] [spark] SparkQA commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

SparkQA commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747964 **[Test build #128909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128909/testReport)** for PR 29814 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747967 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

AmplabJenkins commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747570 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

SparkQA commented on pull request #29814: URL: https://github.com/apache/spark/pull/29814#issuecomment-695747460 **[Test build #128909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128909/testReport)** for PR 29814 at commit

[GitHub] [spark] zero323 opened a new pull request #29814: [SPARK-32946][R][SQL] Add withColumn to SparkR

zero323 opened a new pull request #29814: URL: https://github.com/apache/spark/pull/29814 ### What changes were proposed in this pull request? This PR adds `withColumn` function SparkR. ### Why are the changes needed? ### Does this PR introduce

[GitHub] [spark] AmplabJenkins commented on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

AmplabJenkins commented on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695746460 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

SparkQA removed a comment on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695502727 **[Test build #128903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128903/testReport)** for PR 29703 at commit

[GitHub] [spark] SparkQA commented on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

SparkQA commented on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695746290 **[Test build #128903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128903/testReport)** for PR 29703 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29813: [SPARK-32799][R] Add allowMissingColumns to SparkR unionByName

AmplabJenkins removed a comment on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695746077 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29813: [SPARK-32799][R] Add allowMissingColumns to SparkR unionByName

AmplabJenkins commented on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695746077 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29813: [SPARK-32799][R] Add allowMissingColumns to SparkR unionByName

SparkQA commented on pull request #29813: URL: https://github.com/apache/spark/pull/29813#issuecomment-695745994 **[Test build #128908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128908/testReport)** for PR 29813 at commit

[GitHub] [spark] zero323 opened a new pull request #29813: [SPARK-32799][R] Add allowMissingColumns to SparkR unionByName

zero323 opened a new pull request #29813: URL: https://github.com/apache/spark/pull/29813 ### What changes were proposed in this pull request? Add optional `allowMissingColumns` argument to SparkR `unionByName`. ### Why are the changes needed?

[GitHub] [spark] dongjoon-hyun commented on pull request #29737: [SPARK-32864][SQL] Support ORC forced positional evolution

dongjoon-hyun commented on pull request #29737: URL: https://github.com/apache/spark/pull/29737#issuecomment-695725164 cc @gatorsmile and @dbtsai This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695702607 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695702580 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695492705 **[Test build #128902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128902/testReport)** for PR 29812 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695702580 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695701924 **[Test build #128902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128902/testReport)** for PR 29812 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

AmplabJenkins removed a comment on pull request #29785: URL: https://github.com/apache/spark/pull/29785#issuecomment-695693141 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

AmplabJenkins commented on pull request #29785: URL: https://github.com/apache/spark/pull/29785#issuecomment-695693141 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

SparkQA removed a comment on pull request #29785: URL: https://github.com/apache/spark/pull/29785#issuecomment-695432655 **[Test build #128899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128899/testReport)** for PR 29785 at commit

[GitHub] [spark] SparkQA commented on pull request #29785: [SPARK-32901][CORE] Do not allocate memory while spilling UnsafeExternalSorter

SparkQA commented on pull request #29785: URL: https://github.com/apache/spark/pull/29785#issuecomment-695689350 **[Test build #128899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128899/testReport)** for PR 29785 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695674717 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695674717 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

SparkQA commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695672817 **[Test build #128907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128907/testReport)** for PR 26935 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29809: [SPARK-16190][CORE] Fix "Duplicate worker" error during failover

AmplabJenkins removed a comment on pull request #29809: URL: https://github.com/apache/spark/pull/29809#issuecomment-69598 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29809: [SPARK-16190][CORE] Fix "Duplicate worker" error during failover

AmplabJenkins commented on pull request #29809: URL: https://github.com/apache/spark/pull/29809#issuecomment-69598 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29809: [SPARK-16190][CORE] Fix "Duplicate worker" error during failover

SparkQA removed a comment on pull request #29809: URL: https://github.com/apache/spark/pull/29809#issuecomment-695402966 **[Test build #128895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128895/testReport)** for PR 29809 at commit

[GitHub] [spark] SparkQA commented on pull request #29809: [SPARK-16190][CORE] Fix "Duplicate worker" error during failover

SparkQA commented on pull request #29809: URL: https://github.com/apache/spark/pull/29809#issuecomment-695663247 **[Test build #128895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128895/testReport)** for PR 29809 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29811: [MINOR][SQL][TEST] Fix an invalid test in DataSourceV2SQLSuite

AmplabJenkins commented on pull request #29811: URL: https://github.com/apache/spark/pull/29811#issuecomment-695658807 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29811: [MINOR][SQL][TEST] Fix an invalid test in DataSourceV2SQLSuite

AmplabJenkins removed a comment on pull request #29811: URL: https://github.com/apache/spark/pull/29811#issuecomment-695658807 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

AmplabJenkins removed a comment on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695656337 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29811: [MINOR][SQL][TEST] Fix an invalid test in DataSourceV2SQLSuite

SparkQA removed a comment on pull request #29811: URL: https://github.com/apache/spark/pull/29811#issuecomment-695366430 **[Test build #128894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128894/testReport)** for PR 29811 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

SparkQA removed a comment on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695422848 **[Test build #128898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128898/testReport)** for PR 29810 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

AmplabJenkins removed a comment on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695656313 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

AmplabJenkins commented on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695656313 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29811: [MINOR][SQL][TEST] Fix an invalid test in DataSourceV2SQLSuite

SparkQA commented on pull request #29811: URL: https://github.com/apache/spark/pull/29811#issuecomment-695654857 **[Test build #128894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128894/testReport)** for PR 29811 at commit

[GitHub] [spark] SparkQA commented on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

SparkQA commented on pull request #29810: URL: https://github.com/apache/spark/pull/29810#issuecomment-695655045 **[Test build #128898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128898/testReport)** for PR 29810 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695624310 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695624310 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

SparkQA commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695622774 **[Test build #128906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128906/testReport)** for PR 26935 at commit

[GitHub] [spark] viirya edited a comment on pull request #29807: [WIP][SPARK-32939][SQL]Avoid re-compute expensive expression

viirya edited a comment on pull request #29807: URL: https://github.com/apache/spark/pull/29807#issuecomment-695526440 @dongjoon-hyun Thanks for pinging me. Hmm, this is actually related to what we are working on SPARK-32943. We should not do it at physical plan level. We plan to tackle

[GitHub] [spark] holdenk edited a comment on pull request #29788: [SPARK-32913][CORE][K8S] Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

holdenk edited a comment on pull request #29788: URL: https://github.com/apache/spark/pull/29788#issuecomment-695530700 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] holdenk commented on pull request #29788: [SPARK-32913][CORE][K8S] Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

holdenk commented on pull request #29788: URL: https://github.com/apache/spark/pull/29788#issuecomment-695556053 (willing to switch back to -0 once the original issue is addressed, I just don't want us in a state with broken tests as normal).

[GitHub] [spark] holdenk commented on pull request #29788: [SPARK-32913][CORE][K8S] Improve ExecutorDecommissionInfo and ExecutorDecommissionState for different use cases

holdenk commented on pull request #29788: URL: https://github.com/apache/spark/pull/29788#issuecomment-695530700 Since there was another PR in the same area committed that broke the existing integration tests in this area I don't feel confident with my soft reservations and switching to a

[GitHub] [spark] viirya commented on pull request #29807: [WIP][SPARK-32939][SQL]Avoid re-compute expensive expression

viirya commented on pull request #29807: URL: https://github.com/apache/spark/pull/29807#issuecomment-695526440 @dongjoon-hyun Thanks for pinging me. Hmm, this is actually similar to what we are working on SPARK-32943. We should not do it at physical plan level. We plan to tackle this

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695522714 **[Test build #128905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128905/testReport)** for PR 29812 at commit

[GitHub] [spark] holdenk commented on pull request #29722: [SPARK-32850][CORE] Simplify the RPC message flow of decommission

holdenk commented on pull request #29722: URL: https://github.com/apache/spark/pull/29722#issuecomment-695517007 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695514525 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695514525 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695512726 **[Test build #128904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128904/testReport)** for PR 29812 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

AmplabJenkins removed a comment on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695504335 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

AmplabJenkins commented on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695504335 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29703: [SPARK-32017][PYTHON][BUILD] Make Pyspark Hadoop 3.2+ Variant available in PyPI

SparkQA commented on pull request #29703: URL: https://github.com/apache/spark/pull/29703#issuecomment-695502727 **[Test build #128903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128903/testReport)** for PR 29703 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695494603 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695494603 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695492705 **[Test build #128902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128902/testReport)** for PR 29812 at commit

[GitHub] [spark] HyukjinKwon closed pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

HyukjinKwon closed pull request #29779: URL: https://github.com/apache/spark/pull/29779 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

HyukjinKwon commented on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695485214 Merged to master. Thank you @srowen, @dongjoon-hyun and @viirya for your time! This is an automated

[GitHub] [spark] HyukjinKwon commented on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

HyukjinKwon commented on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695484206 Let me merge this. It's very unlikely that adding some comments into `setup.py` causes a build failure in JDK 11 ...

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins removed a comment on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695474669 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

AmplabJenkins commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695474669 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

SparkQA commented on pull request #29812: URL: https://github.com/apache/spark/pull/29812#issuecomment-695472488 **[Test build #128901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128901/testReport)** for PR 29812 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

AmplabJenkins removed a comment on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695448253 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

SparkQA removed a comment on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695412880 **[Test build #128896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128896/testReport)** for PR 29779 at commit

[GitHub] [spark] viirya opened a new pull request #29812: [SPARK-32941][SQL] Optimize WithFields expression chain

viirya opened a new pull request #29812: URL: https://github.com/apache/spark/pull/29812 ### What changes were proposed in this pull request? This patch proposes to add more optimization to `WithFields` expression chain. ### Why are the changes needed?

[GitHub] [spark] AngersZhuuuu commented on pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

AngersZh commented on pull request #29692: URL: https://github.com/apache/spark/pull/29692#issuecomment-695464072 > Then we need some estimation work, as the shuffle/scan node may be far away from the join node. We also need to carefully justify if the extra shuffle cost worths the

[GitHub] [spark] AngersZhuuuu commented on pull request #29739: [SPARK-32867][SQL] When explain, HiveTableRelation show limited message

AngersZh commented on pull request #29739: URL: https://github.com/apache/spark/pull/29739#issuecomment-695462208 Any update? ping @cloud-fan @maropu This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon commented on pull request #29771: [SPARK-32635][SQL] Fix foldable propagation

HyukjinKwon commented on pull request #29771: URL: https://github.com/apache/spark/pull/29771#issuecomment-695462706 Nice @peter-toth, LGTM too! cc @vinodkc FYI This is an automated message from the Apache Git Service. To

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29692: [SPARK-32830][SQL] Optimize Skewed BroadcastNestedLoopJoin with AQE

AngersZh commented on a change in pull request #29692: URL: https://github.com/apache/spark/pull/29692#discussion_r491639826 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -262,13 +354,24 @@ case class

[GitHub] [spark] AmplabJenkins commented on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

AmplabJenkins commented on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695448253 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29807: [WIP][SPARK-32939][SQL]Avoid re-compute expensive expression

AngersZh commented on a change in pull request #29807: URL: https://github.com/apache/spark/pull/29807#discussion_r491639643 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanner.scala ## @@ -31,6 +31,12 @@ class SparkPlanner( val

[GitHub] [spark] SparkQA commented on pull request #29779: [SPARK-32180][PYTHON][DOCS][FOLLOW-UP] Rephrase and add some more information in installation guide

SparkQA commented on pull request #29779: URL: https://github.com/apache/spark/pull/29779#issuecomment-695447153 **[Test build #128896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128896/testReport)** for PR 29779 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29767: [SPARK-32896][SS] Add DataStreamWriter.table API

AmplabJenkins removed a comment on pull request #29767: URL: https://github.com/apache/spark/pull/29767#issuecomment-695434433 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HeartSaVioR commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

HeartSaVioR commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-695433636 Same here, I cannot reproduce it locally, and rebase doesn’t seem to help as you see the forced push. This

[GitHub] [spark] AmplabJenkins commented on pull request #29767: [SPARK-32896][SS] Add DataStreamWriter.table API

AmplabJenkins commented on pull request #29767: URL: https://github.com/apache/spark/pull/29767#issuecomment-695434433 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29767: [SPARK-32896][SS] Add DataStreamWriter.table API

SparkQA commented on pull request #29767: URL: https://github.com/apache/spark/pull/29767#issuecomment-695432759 **[Test build #128900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128900/testReport)** for PR 29767 at commit

  1   2   3   >