[GitHub] [spark] AmplabJenkins commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718370669 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718370669 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718295002 **[Test build #130389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130389/testReport)** for PR 30177 at commit

[GitHub] [spark] SparkQA commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718369946 **[Test build #130389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130389/testReport)** for PR 30177 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

AmplabJenkins removed a comment on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718367667 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

AmplabJenkins commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718367667 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718367645 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34997/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

dongjoon-hyun commented on a change in pull request #30162: URL: https://github.com/apache/spark/pull/30162#discussion_r513980401 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCompatibilitySuite.scala ## @@ -0,0 +1,84 @@ +/* +

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718363821 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718363821 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718283783 **[Test build #130388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130388/testReport)** for PR 30177 at commit

[GitHub] [spark] SparkQA commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718363117 **[Test build #130388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130388/testReport)** for PR 30177 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

AmplabJenkins removed a comment on pull request #30178: URL: https://github.com/apache/spark/pull/30178#issuecomment-718362027 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

SparkQA commented on pull request #30178: URL: https://github.com/apache/spark/pull/30178#issuecomment-718362021 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34996/

[GitHub] [spark] AmplabJenkins commented on pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

AmplabJenkins commented on pull request #30178: URL: https://github.com/apache/spark/pull/30178#issuecomment-718362027 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718361145 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34997/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718359146 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718281337 **[Test build #130387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130387/testReport)** for PR 30162 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718359146 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718358492 **[Test build #130387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130387/testReport)** for PR 30162 at commit

[GitHub] [spark] SparkQA commented on pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

SparkQA commented on pull request #30178: URL: https://github.com/apache/spark/pull/30178#issuecomment-718356033 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34996/

[GitHub] [spark] SparkQA commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

SparkQA commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718348533 **[Test build #130394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130394/testReport)** for PR 29247 at commit

[GitHub] [spark] gengliangwang commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

gengliangwang commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718347748 retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718345319 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718345319 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718345308 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34995/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718344106 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718344106 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718343569 **[Test build #130386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130386/testReport)** for PR 30162 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718262767 **[Test build #130386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130386/testReport)** for PR 30162 at commit

[GitHub] [spark] SparkQA commented on pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

SparkQA commented on pull request #30178: URL: https://github.com/apache/spark/pull/30178#issuecomment-718342791 **[Test build #130393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130393/testReport)** for PR 30178 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #24246: [SPARK-24252][SQL] Add TableCatalog API

cloud-fan commented on a change in pull request #24246: URL: https://github.com/apache/spark/pull/24246#discussion_r513932838 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/catalog/v2/TableCatalog.java ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache

[GitHub] [spark] beliefer opened a new pull request #30178: [SPARK-33278][SQL] Improve the performance for FIRST_VALUE

beliefer opened a new pull request #30178: URL: https://github.com/apache/spark/pull/30178 ### What changes were proposed in this pull request? https://github.com/apache/spark/pull/29800 provides a performance improvement for `NTH_VALUE`. `FIRST_VALUE` also could uses the

[GitHub] [spark] HeartSaVioR commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718339453 The current behavior was already a surprise, otherwise I wouldn't insist strongly and try to fix it. Btw, probably I'd just revive the another thread that "disallow

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718337953 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34995/

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HyukjinKwon edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718336795 _In a way_, a user _could_ think it's all internal as long as the tables are created somewhere and they can be read regardless of where the tables are created. And,

[GitHub] [spark] HyukjinKwon commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HyukjinKwon commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718336795 _In a way_, someone could think it's all internal as long as the tables are created somewhere and they can be read regardless of where the tables are created. And, someone

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30176: [SQL][MINOR] Update from_unixtime doc

AmplabJenkins removed a comment on pull request #30176: URL: https://github.com/apache/spark/pull/30176#issuecomment-718335189 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less th

AmplabJenkins removed a comment on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718335063 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30176: [SQL][MINOR] Update from_unixtime doc

AmplabJenkins commented on pull request #30176: URL: https://github.com/apache/spark/pull/30176#issuecomment-718335189 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schem

AmplabJenkins commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718335063 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718335052 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34994/

[GitHub] [spark] SparkQA removed a comment on pull request #30176: [SQL][MINOR] Update from_unixtime doc

SparkQA removed a comment on pull request #30176: URL: https://github.com/apache/spark/pull/30176#issuecomment-718252249 **[Test build #130384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130384/testReport)** for PR 30176 at commit

[GitHub] [spark] SparkQA commented on pull request #30176: [SQL][MINOR] Update from_unixtime doc

SparkQA commented on pull request #30176: URL: https://github.com/apache/spark/pull/30176#issuecomment-718334511 **[Test build #130384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130384/testReport)** for PR 30176 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

HeartSaVioR commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-718334009 @viirya Would you like to have another eyes on reviewing, or review from @gaborgsomogyi could convince you?

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718330774 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718331516 Additionally, if the pattern is normal in Spark codebase I think we should revisit - if users configure something (A) and Spark decides to fail back (B), it must be

[GitHub] [spark] HeartSaVioR edited a comment on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR edited a comment on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718331516 If the pattern is normal in Spark codebase I think we should revisit - if users configure something (A) and Spark decides to fail back (B), it must be only case

[GitHub] [spark] HeartSaVioR commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718331516 If the pattern is normal in Spark codebase I think we should revisit - if users configure something (A) and Spark decides to fail back (B), it must be only case where

[GitHub] [spark] HeartSaVioR commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HeartSaVioR commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-718330774 > For example, for any reason, in a cluster the catalog was set and users used to use it with no problem. User don't want to change their existing workload and code but

[GitHub] [spark] waitinfuture commented on pull request #30168: [SPARK-33208][SQL] Update the document of SparkSession#sql

waitinfuture commented on pull request #30168: URL: https://github.com/apache/spark/pull/30168#issuecomment-718329039 > @waitinfuture can you leave a comment in the JIRA so that I can assign the ticket to you? done

[GitHub] [spark] HeartSaVioR commented on a change in pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

HeartSaVioR commented on a change in pull request #30162: URL: https://github.com/apache/spark/pull/30162#discussion_r513903355 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCompatibilitySuite.scala ## @@ -0,0 +1,84 @@ +/* + *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718327943 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718328603 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34994/

[GitHub] [spark] HeartSaVioR commented on a change in pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

HeartSaVioR commented on a change in pull request #30162: URL: https://github.com/apache/spark/pull/30162#discussion_r513903355 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCompatibilitySuite.scala ## @@ -0,0 +1,84 @@ +/* + *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718327938 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718327928 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34993/

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718327938 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718326137 **[Test build #130392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130392/testReport)** for PR 30162 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

HyukjinKwon commented on a change in pull request #30177: URL: https://github.com/apache/spark/pull/30177#discussion_r513898999 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala ## @@ -137,3 +138,18 @@ trait EvalPythonExec

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

dongjoon-hyun edited a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718324895 Thanks. Could you update the affected version of SPARK-33277 accordingly? This is an automated

[GitHub] [spark] viirya commented on a change in pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

viirya commented on a change in pull request #30162: URL: https://github.com/apache/spark/pull/30162#discussion_r513897382 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCompatibilitySuite.scala ## @@ -0,0 +1,84 @@ +/* + *

[GitHub] [spark] dongjoon-hyun commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

dongjoon-hyun commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718324895 Thanks. Could you update SPARK-33277 accordingly? This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

HyukjinKwon commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718322801 I believe this is a long standing bug even for 2.4 releases .. This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

dongjoon-hyun commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718321831 Is this only for Apache Spark 3.1 and 3.0, @ueshin and @HyukjinKwon ? This is an automated message from

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718321187 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34993/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

dongjoon-hyun commented on a change in pull request #30162: URL: https://github.com/apache/spark/pull/30162#discussion_r513885144 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreCompatibilitySuite.scala ## @@ -0,0 +1,84 @@ +/* +

[GitHub] [spark] SparkQA commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema size

SparkQA commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718314992 **[Test build #130391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130391/testReport)** for PR 30156 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema

AngersZh commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718314367 > Ah, sorry can you update the conflict in migration guide? @AngersZh Done This is an automated

[GitHub] [spark] viirya edited a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

viirya edited a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718305957 > I'm sorry to try back and forth, but I feel it weird that StateStoreCodecsTest belongs to StateStoreCompatibilitySuite.scala file. Probably we can just inline

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718312288 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718312288 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718312272 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34992/

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718308057 **[Test build #130390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130390/testReport)** for PR 30162 at commit

[GitHub] [spark] viirya commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

viirya commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718305957 > I'm sorry to try back and forth, but I feel it weird that StateStoreCodecsTest belongs to StateStoreCompatibilitySuite.scala file. Probably we can just inline

[GitHub] [spark] HyukjinKwon commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

HyukjinKwon commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718305873 cc @BryanCutler for another look if you find some time. This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718305551 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34992/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718305004 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718304998 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #30156: [SPARK-33248][SQL] Add a configuration to control the legacy behavior of whether need to pad null value when value size less then schema

HyukjinKwon commented on pull request #30156: URL: https://github.com/apache/spark/pull/30156#issuecomment-718305127 Ah, sorry can you update the conflict in migration guide? @AngersZh This is an automated message from

[GitHub] [spark] SparkQA removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718260070 **[Test build #130385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130385/testReport)** for PR 30162 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718304998 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718304768 **[Test build #130385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130385/testReport)** for PR 30162 at commit

[GitHub] [spark] HyukjinKwon closed pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon closed pull request #30172: URL: https://github.com/apache/spark/pull/30172 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon closed pull request #30176: [SQL][MINOR] Update from_unixtime doc

HyukjinKwon closed pull request #30176: URL: https://github.com/apache/spark/pull/30176 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718302297 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513859872 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class

[GitHub] [spark] SparkQA commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718301803 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34991/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins removed a comment on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718301816 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30177: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins commented on pull request #30177: URL: https://github.com/apache/spark/pull/30177#issuecomment-718301816 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #30176: [SQL][MINOR] Update from_unixtime doc

HyukjinKwon commented on pull request #30176: URL: https://github.com/apache/spark/pull/30176#issuecomment-718301543 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

HyukjinKwon commented on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718300403 @BigaDev, looks like this is a backport of SPARK-29574. You'll have to pick the commits and PR description as are, and keep the JIRA IDs in the PR title.

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

HyukjinKwon edited a comment on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718300403 @BigaDev, looks like this is a backport of SPARK-29574. You'll have to pick the commits and keep the PR description as are, and keep the JIRA IDs in the PR title.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718299797 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718299797 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718299791 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34990/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718296486 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

AmplabJenkins commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718296486 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA removed a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718191772 **[Test build #130383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130383/testReport)** for PR 30162 at commit

[GitHub] [spark] SparkQA commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

SparkQA commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718295797 **[Test build #130383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130383/testReport)** for PR 30162 at commit

  1   2   3   4   5   6   7   >