[GitHub] [spark] SparkQA commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

SparkQA commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983341397 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50266/ -- This is an automated message from the

[GitHub] [spark] sarutak commented on pull request #34765: [SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is followed by global sort.

sarutak commented on pull request #34765: URL: https://github.com/apache/spark/pull/34765#issuecomment-983352196 > Does this bug also impact the metrics reported by other nodes? For example numOutputRows in FilterExec. Seems so. We need a more comprehensive solution... -- This is

[GitHub] [spark] SparkQA removed a comment on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA removed a comment on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983329673 **[Test build #145797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145797/testReport)** for PR 34657 at commit

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983358111 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50265/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

cloud-fan commented on a change in pull request #34758: URL: https://github.com/apache/spark/pull/34758#discussion_r759918277 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1180,7 +1182,7 @@ class DataSourceV2SQLSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #34753: [SPARK-37494][SQL] Unify v1 and v2 options output of `SHOW CREATE TABLE` command

cloud-fan commented on a change in pull request #34753: URL: https://github.com/apache/spark/pull/34753#discussion_r759921072 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ShowCreateTableSuite.scala ## @@ -200,6 +200,28 @@ abstract class ShowCreateTableSuite

[GitHub] [spark] dongjoon-hyun commented on pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on pull request #34264: URL: https://github.com/apache/spark/pull/34264#issuecomment-983368591 Could you rebase this PR please, @holdenk ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] Peng-Lei commented on a change in pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

Peng-Lei commented on a change in pull request #34758: URL: https://github.com/apache/spark/pull/34758#discussion_r759926219 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala ## @@ -271,6 +271,14 @@ object CatalogUtils

[GitHub] [spark] Peng-Lei commented on a change in pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

Peng-Lei commented on a change in pull request #34758: URL: https://github.com/apache/spark/pull/34758#discussion_r759926219 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala ## @@ -271,6 +271,14 @@ object CatalogUtils

[GitHub] [spark] cloud-fan commented on a change in pull request #34667: [SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

cloud-fan commented on a change in pull request #34667: URL: https://github.com/apache/spark/pull/34667#discussion_r759932599 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -170,16 +170,17 @@ class

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-983270089 **[Test build #145778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145778/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-983120565 **[Test build #145778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145778/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py

SparkQA commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-983270611 **[Test build #145789 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145789/testReport)** for PR 34363 at commit

[GitHub] [spark] SparkQA commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

SparkQA commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983294034 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50257/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py

SparkQA commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-983294418 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50262/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

SparkQA commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983300137 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50256/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983300247 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50260/ -- This is an automated message from the Apache

[GitHub] [spark] xinrong-databricks commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

xinrong-databricks commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983304859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] SparkQA commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

SparkQA commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983320773 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50266/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983321062 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50264/ -- This is an automated message from the Apache

[GitHub] [spark] advancedxy commented on pull request #34640: [SPARK-31585][SQL] Introduce Z-order expression

advancedxy commented on pull request #34640: URL: https://github.com/apache/spark/pull/34640#issuecomment-983338047 > @advancedxy - we haven't rolled out this feature in our production yet, still testing so there should be quite some room for further optimization. For storage efficiency,

[GitHub] [spark] dongjoon-hyun commented on pull request #34706: [SPARK-37458][SS] Remove unnecessary SerializeFromObject from the plan of foreachBatch

dongjoon-hyun commented on pull request #34706: URL: https://github.com/apache/spark/pull/34706#issuecomment-983342994 cc @cloud-fan and @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #34706: [SPARK-37458][SS] Remove unnecessary SerializeFromObject from the plan of foreachBatch

dongjoon-hyun edited a comment on pull request #34706: URL: https://github.com/apache/spark/pull/34706#issuecomment-983342994 cc @cloud-fan, too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983348737 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50264/ -- This is an automated message from the

[GitHub] [spark] MaxGekk commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

MaxGekk commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r759908169 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala ## @@ -1012,6 +1012,196 @@ abstract class CSVSuite

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

dongjoon-hyun commented on a change in pull request #34599: URL: https://github.com/apache/spark/pull/34599#discussion_r759908300 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala ## @@ -381,4 +382,21 @@ object

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

dongjoon-hyun commented on a change in pull request #34599: URL: https://github.com/apache/spark/pull/34599#discussion_r759909040 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala ## @@ -381,4 +382,21 @@ object

[GitHub] [spark] AmplabJenkins commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

AmplabJenkins commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-983362683 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50265/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

AmplabJenkins commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983362682 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50266/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34766: [SPARK-37513][SQL][DOC] Additive expression of date and interval returns different data type between Spark3.2 and Spark3.1

AmplabJenkins commented on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983362686 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145799/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983362685 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145797/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

AmplabJenkins commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983362684 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50264/ --

[GitHub] [spark] cloud-fan commented on a change in pull request #34667: [SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

cloud-fan commented on a change in pull request #34667: URL: https://github.com/apache/spark/pull/34667#discussion_r759931289 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -323,18 +323,25 @@ final class DataFrameWriter[T]

[GitHub] [spark] AmplabJenkins commented on pull request #34363: [SPARK-37083][PYTHON] Inline type hints for python/pyspark/accumulators.py

AmplabJenkins commented on pull request #34363: URL: https://github.com/apache/spark/pull/34363#issuecomment-983275905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145789/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

AmplabJenkins commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-983275908 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145785/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-983275906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145778/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

AmplabJenkins commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983275911 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50252/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

AmplabJenkins commented on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-983275907 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50253/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983275971 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145787/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34439: [SPARK-37095][PYTHON] Inline type hints for files in python/pyspark/broadcast.py

AmplabJenkins commented on pull request #34439: URL: https://github.com/apache/spark/pull/34439#issuecomment-983275972 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50254/ --

[GitHub] [spark] SparkQA commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

SparkQA commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983280331 **[Test build #145793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145793/testReport)** for PR 34212 at commit

[GitHub] [spark] SparkQA commented on pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

SparkQA commented on pull request #34718: URL: https://github.com/apache/spark/pull/34718#issuecomment-983280322 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50259/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983299263 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50261/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

dongjoon-hyun commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-983303850 Merged to master for Apache Spark 3.3. Thank you, @sunchao , @viirya , @huaxingao . -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-98827 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50267/ -- This is an automated message from the Apache

[GitHub] [spark] tanelk commented on pull request #34765: [SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is followed by global sort.

tanelk commented on pull request #34765: URL: https://github.com/apache/spark/pull/34765#issuecomment-983339082 Does this bug also impact the metrics reported by other nodes? For example `numOutputRows` in `FilterExec`. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on a change in pull request #34667: [SPARK-36902][SQL] Migrate CreateTableAsSelectStatement to v2 command

cloud-fan commented on a change in pull request #34667: URL: https://github.com/apache/spark/pull/34667#discussion_r759930581 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -323,18 +323,25 @@ final class DataFrameWriter[T]

[GitHub] [spark] SparkQA commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

SparkQA commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983271213 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50252/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34439: [SPARK-37095][PYTHON] Inline type hints for files in python/pyspark/broadcast.py

SparkQA commented on pull request #34439: URL: https://github.com/apache/spark/pull/34439#issuecomment-983272567 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50254/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

SparkQA commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983279092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50256/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

SparkQA removed a comment on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983223771 **[Test build #145779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145779/testReport)** for PR 34761 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

AmplabJenkins removed a comment on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983278369 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145779/

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983288152 **[Test build #145791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145791/testReport)** for PR 34757 at commit

[GitHub] [spark] SparkQA commented on pull request #34764: [SPARK-37330][SQL] Migrate ReplaceTableStatement to v2 command

SparkQA commented on pull request #34764: URL: https://github.com/apache/spark/pull/34764#issuecomment-983329462 **[Test build #145795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145795/testReport)** for PR 34764 at commit

[GitHub] [spark] SparkQA commented on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

SparkQA commented on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983329503 **[Test build #145796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145796/testReport)** for PR 34763 at commit

[GitHub] [spark] sarutak opened a new pull request #34765: [SPARK-37487][SQL][CORE] Avoid performing CollectMetrics twice if the operation is folloed by global sort.

sarutak opened a new pull request #34765: URL: https://github.com/apache/spark/pull/34765 ### What changes were proposed in this pull request? This PR fixes an issue that `CollectMetrics` performs twice if it's followed by global sort like as follows. ``` val df =

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983329673 **[Test build #145797 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145797/testReport)** for PR 34657 at commit

[GitHub] [spark] cloud-fan commented on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

cloud-fan commented on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983329910 @beliefer @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] advancedxy commented on a change in pull request #34640: [SPARK-31585][SQL] Introduce Z-order expression

advancedxy commented on a change in pull request #34640: URL: https://github.com/apache/spark/pull/34640#discussion_r759890419 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ZOrder.scala ## @@ -0,0 +1,222 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34766: [SPARK-37513][SQL][DOC] Additive expression of date and interval returns different data type between Spark3.2 and Spark3.1

SparkQA removed a comment on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983336928 **[Test build #145799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145799/testReport)** for PR 34766 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

dongjoon-hyun commented on a change in pull request #34599: URL: https://github.com/apache/spark/pull/34599#discussion_r759910290 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala ## @@ -142,15

[GitHub] [spark] MaxGekk closed pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

MaxGekk closed pull request #34596: URL: https://github.com/apache/spark/pull/34596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] mridulm commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

mridulm commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-983356500 It is looking mostly good @venkata91. I would also like @Ngone51 to take a pass through it. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34489: [SPARK-37210][SQL] Write to static partition in dynamic write mode

dongjoon-hyun commented on a change in pull request #34489: URL: https://github.com/apache/spark/pull/34489#discussion_r759917788 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSuite.scala ## @@ -168,4 +172,30 @@ class HiveParquetSuite extends

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34489: [SPARK-37210][SQL] Write to static partition in dynamic write mode

dongjoon-hyun commented on a change in pull request #34489: URL: https://github.com/apache/spark/pull/34489#discussion_r759917283 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHiveSingleton.scala ## @@ -27,9 +27,9 @@ import

[GitHub] [spark] cloud-fan commented on a change in pull request #34719: [SPARK-37381][SQL] Unify v1 and v2 SHOW CREATE TABLE tests

cloud-fan commented on a change in pull request #34719: URL: https://github.com/apache/spark/pull/34719#discussion_r759919791 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowCreateTableSuite.scala ## @@ -0,0 +1,140 @@ +/* + * Licensed to

[GitHub] [spark] wForget commented on a change in pull request #34489: [SPARK-37210][SQL] Write to static partition in dynamic write mode

wForget commented on a change in pull request #34489: URL: https://github.com/apache/spark/pull/34489#discussion_r759919910 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHiveSingleton.scala ## @@ -27,9 +27,9 @@ import

[GitHub] [spark] cloud-fan commented on a change in pull request #34754: [SPARK-37496][SQL] Migrate ReplaceTableAsSelectStatement to v2 command

cloud-fan commented on a change in pull request #34754: URL: https://github.com/apache/spark/pull/34754#discussion_r759935005 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -586,19 +586,13 @@ final class DataFrameWriter[T]

[GitHub] [spark] AmplabJenkins commented on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

AmplabJenkins commented on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983301304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145793/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins removed a comment on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983301303 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50257/

[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-983301547 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50258/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

AmplabJenkins removed a comment on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983301306 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50256/

[GitHub] [spark] AmplabJenkins commented on pull request #34758: [SPARK-37501][SQL] CREATE/REPLACE TABLE should qualify location for v2 command

AmplabJenkins commented on pull request #34758: URL: https://github.com/apache/spark/pull/34758#issuecomment-983301303 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50257/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34761: [SPARK-37508][SQL] Add CONTAINS() string function

AmplabJenkins commented on pull request #34761: URL: https://github.com/apache/spark/pull/34761#issuecomment-983301306 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50256/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

AmplabJenkins commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983301314 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145791/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

AmplabJenkins commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-983301578 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50258/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

AmplabJenkins removed a comment on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983301314 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145791/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

AmplabJenkins removed a comment on pull request #34212: URL: https://github.com/apache/spark/pull/34212#issuecomment-983301304 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145793/

[GitHub] [spark] SparkQA commented on pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-983319286 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50260/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34766: [SPARK-37513][SQL][DOC] Additive expression of date and interval returns different data type between Spark3.2 and Spark3.1

cloud-fan commented on a change in pull request #34766: URL: https://github.com/apache/spark/pull/34766#discussion_r759902213 ## File path: docs/sql-migration-guide.md ## @@ -133,6 +133,8 @@ license: | - In Spark 3.2, create/alter view will fail if the input query output

[GitHub] [spark] mridulm commented on a change in pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

mridulm commented on a change in pull request #33896: URL: https://github.com/apache/spark/pull/33896#discussion_r759892630 ## File path: core/src/main/scala/org/apache/spark/shuffle/ShuffleBlockPusher.scala ## @@ -53,14 +54,18 @@ private[spark] class ShuffleBlockPusher(conf:

[GitHub] [spark] mridulm commented on a change in pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

mridulm commented on a change in pull request #33896: URL: https://github.com/apache/spark/pull/33896#discussion_r759905638 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -3847,6 +3887,76 @@ class DAGSchedulerSuite extends

[GitHub] [spark] MaxGekk commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

MaxGekk commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-983356048 +1, LGTM. Merging to master. Thank you, @sadikovi and @gengliangwang @HyukjinKwon @cloud-fan for review. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

dongjoon-hyun commented on a change in pull request #34599: URL: https://github.com/apache/spark/pull/34599#discussion_r759909487 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KubernetesFeatureConfigStep.scala ## @@ -70,7

[GitHub] [spark] dongjoon-hyun commented on pull request #34489: [SPARK-37210][SQL] Write to static partition in dynamic write mode

dongjoon-hyun commented on pull request #34489: URL: https://github.com/apache/spark/pull/34489#issuecomment-983366425 Could you put your findings into the PR description? - https://github.com/apache/spark/pull/34489#issuecomment-967199860 -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34489: [SPARK-37210][SQL] Write to static partition in dynamic write mode

dongjoon-hyun commented on a change in pull request #34489: URL: https://github.com/apache/spark/pull/34489#discussion_r759919225 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSuite.scala ## @@ -168,4 +170,23 @@ class HiveParquetSuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #34718: [SPARK-37460][DOCS] Add the description of ALTER DATABASE SET LOCATION

cloud-fan commented on a change in pull request #34718: URL: https://github.com/apache/spark/pull/34718#discussion_r759919009 ## File path: docs/sql-ref-syntax-ddl-alter-database.md ## @@ -21,25 +21,47 @@ license: | ### Description -You can alter metadata associated with

[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-983276702 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50258/ -- This is an automated message from the Apache

[GitHub] [spark] gengliangwang commented on pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on pull request #34747: URL: https://github.com/apache/spark/pull/34747#issuecomment-983284395 Merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] gengliangwang closed pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang closed pull request #34747: URL: https://github.com/apache/spark/pull/34747 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-983295626 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50263/ -- This is an automated message from the Apache

[GitHub] [spark] xinrong-databricks commented on a change in pull request #34657: [SPARK-37511][PYTHON] Introduce TimedeltaIndex to pandas API on Spark

xinrong-databricks commented on a change in pull request #34657: URL: https://github.com/apache/spark/pull/34657#discussion_r759874628 ## File path: python/pyspark/pandas/indexes/timedelta.py ## @@ -0,0 +1,100 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-983313959 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50263/ -- This is an automated message from the

[GitHub] [spark] wang-zhun commented on pull request #34072: [SPARK-36680][SQL] Supports Dynamic Table Options for Spark SQL

wang-zhun commented on pull request #34072: URL: https://github.com/apache/spark/pull/34072#issuecomment-983318289 Close this PR first and look forward to a better proposal implementation -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] wang-zhun closed pull request #34072: [SPARK-36680][SQL] Supports Dynamic Table Options for Spark SQL

wang-zhun closed pull request #34072: URL: https://github.com/apache/spark/pull/34072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34766: [SPARK-37513][SQL][DOC] Additive expression of date and interval returns different data type between Spark3.2 and Spark3.1

SparkQA commented on pull request #34766: URL: https://github.com/apache/spark/pull/34766#issuecomment-983344028 **[Test build #145799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145799/testReport)** for PR 34766 at commit

[GitHub] [spark] SparkQA commented on pull request #34763: [SPARK-37389][SQL][FOLLOWUP] SET command shuold not parse comments

SparkQA commented on pull request #34763: URL: https://github.com/apache/spark/pull/34763#issuecomment-983359687 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50271/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34764: [SPARK-37330][SQL] Migrate ReplaceTableStatement to v2 command

SparkQA commented on pull request #34764: URL: https://github.com/apache/spark/pull/34764#issuecomment-983360015 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50270/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34673: [SPARK-37343][SQL] Implement createIndex, IndexExists and dropIndex in JDBC (Postgres dialect)

SparkQA commented on pull request #34673: URL: https://github.com/apache/spark/pull/34673#issuecomment-983363812 **[Test build #145801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145801/testReport)** for PR 34673 at commit

[GitHub] [spark] SparkQA commented on pull request #34757: [SPARK-37504][PYTHON] Pyspark create SparkSession with existed session should not pass static conf

SparkQA commented on pull request #34757: URL: https://github.com/apache/spark/pull/34757#issuecomment-983363743 **[Test build #145800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145800/testReport)** for PR 34757 at commit

  1   2   3   4   5   6   >