[GitHub] [spark] AmplabJenkins commented on pull request #34725: [SPARK-37473]BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34725: URL: https://github.com/apache/spark/pull/34725#issuecomment-980516157 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] seayoun opened a new pull request #34725: [SPARK-37473]BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-26 Thread GitBox
seayoun opened a new pull request #34725: URL: https://github.com/apache/spark/pull/34725 ### What changes were proposed in this pull request? We think it has no data when the segment file not exists when all segment files produced by `BypassMergeSortShuffleWriter` is merging;

[GitHub] [spark] AngersZhuuuu commented on pull request #34710: [SPARK-37461][YARN] YARN-CLIENT mode client.appId is always null

2021-11-26 Thread GitBox
AngersZh commented on pull request #34710: URL: https://github.com/apache/spark/pull/34710#issuecomment-980515538 gentle ping @tgravescs @srowen @HyukjinKwon @mridulm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980509763 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145672/

[GitHub] [spark] AmplabJenkins commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980509763 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145672/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980480346 **[Test build #145672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145672/testReport)** for PR 34702 at commit

[GitHub] [spark] SparkQA commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
SparkQA commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980509165 **[Test build #145672 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145672/testReport)** for PR 34702 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980504337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145670/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980504336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145673/

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980504337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145670/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980504336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145673/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980473099 **[Test build #145670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145670/testReport)** for PR 34712 at commit

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980503193 **[Test build #145670 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145670/testReport)** for PR 34712 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980487725 **[Test build #145673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145673/testReport)** for PR 33896 at commit

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980501981 **[Test build #145673 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145673/testReport)** for PR 33896 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980498184 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50143/

[GitHub] [spark] AmplabJenkins commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-980498188 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980498184 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50143/ --

[GitHub] [spark] ulysses-you commented on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

2021-11-26 Thread GitBox
ulysses-you commented on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-980497803 thank you all ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AngersZhuuuu commented on pull request #34679: [SPARK-37437][BUILD] Remove unused hive profile and related CI test

2021-11-26 Thread GitBox
AngersZh commented on pull request #34679: URL: https://github.com/apache/spark/pull/34679#issuecomment-980497001 ping @srowen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980495330 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50143/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980492641 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50142/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980492640 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50141/

[GitHub] [spark] AmplabJenkins commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980492641 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50142/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980492640 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50141/ --

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980491367 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50143/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
SparkQA commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980489244 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50142/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980488878 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50141/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-26 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980487725 **[Test build #145673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145673/testReport)** for PR 33896 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980486829 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50140/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980486828 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145671/

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980486829 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50140/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980486828 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145671/ -- This

[GitHub] [spark] SparkQA commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
SparkQA commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980484762 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50142/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980484750 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50141/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980480329 **[Test build #145671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145671/testReport)** for PR 34717 at commit

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980483121 **[Test build #145671 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145671/testReport)** for PR 34717 at commit

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980482908 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50140/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
SparkQA commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-980480346 **[Test build #145672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145672/testReport)** for PR 34702 at commit

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-980480329 **[Test build #145671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145671/testReport)** for PR 34717 at commit

[GitHub] [spark] c21 commented on a change in pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-26 Thread GitBox
c21 commented on a change in pull request #34702: URL: https://github.com/apache/spark/pull/34702#discussion_r757725944 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala ## @@ -694,7 +694,8 @@ class SQLMetricsSuite extends

[GitHub] [spark] Yikun commented on pull request #34724: [SPARK-37465][Python][WIP] Raise minimum supported Pandas version to 1.0.0

2021-11-26 Thread GitBox
Yikun commented on pull request #34724: URL: https://github.com/apache/spark/pull/34724#issuecomment-980479354 Yep, as @sarutak metioned, @rshkv would you mind help to review #34717? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980477948 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50140/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun edited a comment on pull request #34717: Bump minimum pandas version to 1.0.0

2021-11-26 Thread GitBox
Yikun edited a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-979659317 Just to start the discussion, by using below sql according [1], we can get the all download stat of Pandas in last 3 months. ```SQL SELECT file.version AS

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980473099 **[Test build #145670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145670/testReport)** for PR 34712 at commit

[GitHub] [spark] github-actions[bot] closed pull request #33323: [SPARK-35739][SQL] Add Java-compatible Dataset.join overloads

2021-11-26 Thread GitBox
github-actions[bot] closed pull request #33323: URL: https://github.com/apache/spark/pull/33323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] github-actions[bot] commented on pull request #33747: [SPARK-36453][SQL] Improve consistency processing floating point special literals

2021-11-26 Thread GitBox
github-actions[bot] commented on pull request #33747: URL: https://github.com/apache/spark/pull/33747#issuecomment-980472750 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] github-actions[bot] commented on pull request #33760: [SPARK-36518][Deploy] Spark should support distribute directory to cluster

2021-11-26 Thread GitBox
github-actions[bot] commented on pull request #33760: URL: https://github.com/apache/spark/pull/33760#issuecomment-980472748 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] beliefer commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
beliefer commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980472111 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] bersprockets commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
bersprockets commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980471687 Unfortunately, I don't think you can employ useUTCTimestamp=true to fix TIMESTAMP_NTZ without breaking compatibility for TIMESTAMP between versions. I could be wrong.

[GitHub] [spark] sarutak commented on pull request #34724: [SPARK-37465][Python][WIP] Raise minimum supported Pandas version to 1.0.0

2021-11-26 Thread GitBox
sarutak commented on pull request #34724: URL: https://github.com/apache/spark/pull/34724#issuecomment-980335010 It's already work in progress in #34717 isn't it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980241133 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145664/

[GitHub] [spark] AmplabJenkins commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980241133 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145664/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-979805912 **[Test build #145664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145664/testReport)** for PR 34712 at commit

[GitHub] [spark] SparkQA commented on pull request #34712: [SPARK-37463][SQL] Read/Write Timestamp ntz or ltz to Orc uses UTC timestamp

2021-11-26 Thread GitBox
SparkQA commented on pull request #34712: URL: https://github.com/apache/spark/pull/34712#issuecomment-980191622 **[Test build #145664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145664/testReport)** for PR 34712 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34724: [SPARK-37465][Python] Raise minimum supported Pandas version to 1.0.0

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34724: URL: https://github.com/apache/spark/pull/34724#issuecomment-980176537 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] rshkv opened a new pull request #34724: [SPARK-37465][Python] Raise minimum supported Pandas version to 1.0.0

2021-11-26 Thread GitBox
rshkv opened a new pull request #34724: URL: https://github.com/apache/spark/pull/34724 ### What changes were proposed in this pull request? This raises Spark's minimum supported Pandas version to 1.0.0. If the installed version is below, Spark fails with: _"Pandas >= 1.0.0 must be

[GitHub] [spark] AngersZhuuuu commented on pull request #34715: [SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
AngersZh commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-980160523 ping @dongjoon-hyun @HyukjinKwon @sunchao @cloud-fan @srowen Hope for your review -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-980080356 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145669/

[GitHub] [spark] AmplabJenkins commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-980080356 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145669/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979887726 **[Test build #145669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145669/testReport)** for PR 34723 at commit

[GitHub] [spark] SparkQA commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-980064646 **[Test build #145669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145669/testReport)** for PR 34723 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-980046005 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145668/

[GitHub] [spark] AmplabJenkins commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-980046005 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145668/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-979846280 **[Test build #145668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145668/testReport)** for PR 34367 at commit

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-980038095 **[Test build #145668 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145668/testReport)** for PR 34367 at commit

[GitHub] [spark] nicolasazrak commented on pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

2021-11-26 Thread GitBox
nicolasazrak commented on pull request #34509: URL: https://github.com/apache/spark/pull/34509#issuecomment-980004387 Hey @itholic can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-979967448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145661/

[GitHub] [spark] AmplabJenkins commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-979967448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145661/ -- This

[GitHub] [spark] SparkQA commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

2021-11-26 Thread GitBox
SparkQA commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-979966123 **[Test build #145661 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145661/testReport)** for PR 34060 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-979771136 **[Test build #145661 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145661/testReport)** for PR 34060 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979965454 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145667/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979965454 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145667/

[GitHub] [spark] AmplabJenkins commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979965453 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50139/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979965453 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50139/

[GitHub] [spark] gengliangwang closed pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

2021-11-26 Thread GitBox
gengliangwang closed pull request #31830: URL: https://github.com/apache/spark/pull/31830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] gengliangwang commented on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

2021-11-26 Thread GitBox
gengliangwang commented on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-979962481 Thanks, merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA removed a comment on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979845839 **[Test build #145667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145667/testReport)** for PR 34720 at commit

[GitHub] [spark] SparkQA commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
SparkQA commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979956630 **[Test build #145667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145667/testReport)** for PR 34720 at commit

[GitHub] [spark] SparkQA commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979949080 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50139/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979940240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145663/

[GitHub] [spark] AmplabJenkins commented on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979940240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145663/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979805793 **[Test build #145663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145663/testReport)** for PR 34715 at commit

[GitHub] [spark] SparkQA commented on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
SparkQA commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979939024 **[Test build #145663 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145663/testReport)** for PR 34715 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-979933220 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50138/

[GitHub] [spark] AmplabJenkins commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-979933220 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50138/ --

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-979933187 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50138/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34722: [SPARK-37319][K8S][FOLLOWUP] Set JAVA_HOME for Java 17 installed by apt-get

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34722: URL: https://github.com/apache/spark/pull/34722#issuecomment-979930703 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50136/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979930701 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979930702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #34722: [SPARK-37319][K8S][FOLLOWUP] Set JAVA_HOME for Java 17 installed by apt-get

2021-11-26 Thread GitBox
AmplabJenkins commented on pull request #34722: URL: https://github.com/apache/spark/pull/34722#issuecomment-979930703 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50136/ --

[GitHub] [spark] SparkQA removed a comment on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
SparkQA removed a comment on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979811051 **[Test build #145665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145665/testReport)** for PR 34720 at commit

[GitHub] [spark] SparkQA commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
SparkQA commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979924387 **[Test build #145665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145665/testReport)** for PR 34720 at commit

[GitHub] [spark] SparkQA commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979922902 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50139/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-26 Thread GitBox
SparkQA commented on pull request #34720: URL: https://github.com/apache/spark/pull/34720#issuecomment-979918648 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50137/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34722: [SPARK-37319][K8S][FOLLOWUP] Set JAVA_HOME for Java 17 installed by apt-get

2021-11-26 Thread GitBox
SparkQA commented on pull request #34722: URL: https://github.com/apache/spark/pull/34722#issuecomment-979915303 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50136/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-26 Thread GitBox
SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-979890157 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50138/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-26 Thread GitBox
SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-979887726 **[Test build #145669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145669/testReport)** for PR 34723 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34721: [SPARK-37471][SQL] spark-sql support nested bracketed comment

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34721: URL: https://github.com/apache/spark/pull/34721#issuecomment-979887218 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50132/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34715: [WIP][SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-26 Thread GitBox
AmplabJenkins removed a comment on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-979887220 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50133/

  1   2   >