[GitHub] [spark] AmplabJenkins commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717065671 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu opened a new pull request #30157: [WIP][SPARK-33228][SQL][2.4] Don't uncache data when replacing a view having the same logical plan

maropu opened a new pull request #30157: URL: https://github.com/apache/spark/pull/30157 ### What changes were proposed in this pull request? SPARK-30494's updated the `CreateViewCommand` code to implicitly drop cache when replacing an existing view. But, this change drops

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717065639 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34921/

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717064724 **[Test build #130322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130322/testReport)** for PR 29800 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717063665 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34923/

[GitHub] [spark] beliefer commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

beliefer commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512481955 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala ## @@ -151,10 +169,69 @@ final class

[GitHub] [spark] beliefer commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

beliefer commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512478210 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala ## @@ -171,18 +178,42 @@ trait WindowExecBase extends

[GitHub] [spark] beliefer commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

beliefer commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512477822 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala ## @@ -58,7 +58,7 @@ import

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717055292 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34922/

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717051971 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34921/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717049528 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717049518 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717049483 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34920/

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717049518 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] beliefer commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

beliefer commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512468539 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -355,6 +344,36 @@ abstract class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717048147 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717048147 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717048124 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34919/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins removed a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717045452 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717045434 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34918/

[GitHub] [spark] AmplabJenkins commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717045452 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717038908 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34920/

[GitHub] [spark] stijndehaes edited a comment on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

stijndehaes edited a comment on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-717038227 @redsk @jkleckner The error line you are seeing comes from the class `ExecutorPodsWatchSnapshotSource` this is somewhere else in the code. I thought there was

[GitHub] [spark] stijndehaes commented on pull request #29533: [SPARK-24266][K8S][3.0] Restart the watcher when we receive a version changed from k8s

stijndehaes commented on pull request #29533: URL: https://github.com/apache/spark/pull/29533#issuecomment-717038227 @redsk @jkleckner The error line you are seeing comes from the class `ExecutorPodsWatchSnapshotSource` this is somewhere else. I thought there was another mechanism for the

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717037578 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34919/

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717036263 **[Test build #130321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130321/testReport)** for PR 30097 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less th

AmplabJenkins removed a comment on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035560 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717036225 **[Test build #130320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130320/testReport)** for PR 30156 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins removed a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717034895 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

AmplabJenkins removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717035349 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717036010 > Sorry but your latest change doesn't actually lock properly. Long is immutable, and you always replace the object when you do the calculation and assign to the field,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717034822 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less th

AmplabJenkins removed a comment on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035401 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schem

AmplabJenkins commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035552 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717034727 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins removed a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717034885 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schem

AmplabJenkins commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035401 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717034718 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

AmplabJenkins commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717035334 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] leanken edited a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

leanken edited a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717035399 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less th

AmplabJenkins removed a comment on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035055 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717034813 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then sch

SparkQA removed a comment on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-716970169 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717035252 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34918/

[GitHub] [spark] SparkQA removed a comment on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

SparkQA removed a comment on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-716945800 **[Test build #130311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130311/testReport)** for PR 30097 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema

AngersZh commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035082 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA removed a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717013857 **[Test build #130316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130316/testReport)** for PR 30139 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schem

AmplabJenkins commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717035055 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] leanken commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

leanken commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717035399 retest this, please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA removed a comment on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA removed a comment on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717017189 **[Test build #130318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130318/testReport)** for PR 29800 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717034813 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AmplabJenkins commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717034885 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-717034649 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717034646 **[Test build #130318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130318/testReport)** for PR 29800 at commit

[GitHub] [spark] SparkQA commented on pull request #30097: [SPARK-33140][SQL] remove SQLConf and SparkSession in all sub-class of Rule[QueryPlan]

SparkQA commented on pull request #30097: URL: https://github.com/apache/spark/pull/30097#issuecomment-717034660 **[Test build #130311 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130311/testReport)** for PR 30097 at commit

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717034645 **[Test build #130317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130317/testReport)** for PR 29882 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

AmplabJenkins commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717034718 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717013900 **[Test build #130317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130317/testReport)** for PR 29882 at commit

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717034652 **[Test build #130316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130316/testReport)** for PR 30139 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717033467 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717033455 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717033455 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-716950296 **[Test build #130312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130312/testReport)** for PR 29882 at commit

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717032670 **[Test build #130312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130312/testReport)** for PR 29882 at commit

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717032105 **[Test build #130319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130319/testReport)** for PR 30139 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

cloud-fan commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512451640 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala ## @@ -171,18 +178,42 @@ trait WindowExecBase extends

[GitHub] [spark] cloud-fan commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

cloud-fan commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512451428 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala ## @@ -58,7 +58,7 @@ import

[GitHub] [spark] cloud-fan commented on a change in pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

cloud-fan commented on a change in pull request #29800: URL: https://github.com/apache/spark/pull/29800#discussion_r512450768 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -355,6 +344,36 @@ abstract class

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

HeartSaVioR edited a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717022097 Sorry but your latest change doesn't actually lock properly. Long is immutable, and you always replace the object when you do the calculation and assign to the

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717023200 > Sorry but your latest change doesn't actually lock properly. Long is immutable, and you always replace the object when you do the calculation and assign to the field,

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

HeartSaVioR edited a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717022097 Sorry but your latest change doesn't actually lock properly. Long is immutable, and you always replace the object when you do the calculation and assign to the

[GitHub] [spark] HeartSaVioR commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

HeartSaVioR commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717022097 Sorry but your latest change doesn't actually lock properly. Long is immutable, and you always replace the object when you do the calculation, which is used as a lock. Your

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717019830 > Hmmm. Every update to `chunkSent` and `chunkBeingSent` will compete for the lock on object `totalChunksBeingTransferred` We reduce many race condition on `streams`

[GitHub] [spark] SparkQA commented on pull request #29800: [SPARK-32934][SQL] Improve the performance for NTH_VALUE and reactor the OffsetWindowFunction

SparkQA commented on pull request #29800: URL: https://github.com/apache/spark/pull/29800#issuecomment-717017189 **[Test build #130318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130318/testReport)** for PR 29800 at commit

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh edited a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717016273 > Hmmm. Every update to `chunkSent` and `chunkBeingSent` will compete for the lock on object `totalChunksBeingTransferred` if we add the

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717016273 > Hmmm. Every update to `chunkSent` and `chunkBeingSent` will compete for the lock on object `totalChunksBeingTransferred` if we add the

[GitHub] [spark] SparkQA commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

SparkQA commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717013857 **[Test build #130316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130316/testReport)** for PR 30139 at commit

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717013900 **[Test build #130317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130317/testReport)** for PR 29882 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717013113 > Yes we should ensure the streamState and the totalChunksBeingTransfered are updated synchronically. Other than that the PR looks good! How about current change?

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on a change in pull request #30139: URL: https://github.com/apache/spark/pull/30139#discussion_r512439581 ## File path: common/network-common/src/main/java/org/apache/spark/network/server/OneForOneStreamManager.java ## @@ -43,6 +43,7 @@ private

[GitHub] [spark] otterc commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

otterc commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717008775 > > This should have a considerable impact on the performance when there are multiple open streams because updates of different streams would lock on a single object

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh edited a comment on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717004690 > This should have a considerable impact on the performance when there are multiple open streams because updates of different streams would lock on a single object

[GitHub] [spark] AngersZhuuuu commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

AngersZh commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-717004690 > This should have a considerable impact on the performance when there are multiple open streams because updates of different streams would lock on a single object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717003726 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717003715 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717003715 This is an automated message from the Apache Git Service. To respond to the message, please log on to

<    1   2   3   4   5   6