Re: [PR] [SPARK-47764][CORE][SQL] Cleanup shuffle dependencies based on ShuffleCleanupMode [spark]

2024-04-24 Thread via GitHub
cloud-fan commented on PR #45930: URL: https://github.com/apache/spark/pull/45930#issuecomment-2074349569 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-47955][SQL] Improve `DeduplicateRelations` performance [spark]

2024-04-24 Thread via GitHub
yaooqinn commented on code in PR #46183: URL: https://github.com/apache/spark/pull/46183#discussion_r1577367019 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DeduplicateRelations.scala: ## @@ -38,28 +38,31 @@ case class RelationWrapper(cls: Class[_],

Re: [PR] [MINOR][DOCS] Add `docs/_generated/` to .gitignore [spark]

2024-04-24 Thread via GitHub
yaooqinn commented on PR #46178: URL: https://github.com/apache/spark/pull/46178#issuecomment-2074202457 Thank you @nchammas -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] [SPARK-47971][PYTHON][CONNECT][TESTS] Reenable `PandasUDFGroupedAggParityTests. test_grouped_with_empty_partition` [spark]

2024-04-24 Thread via GitHub
zhengruifeng opened a new pull request, #46202: URL: https://github.com/apache/spark/pull/46202 ### What changes were proposed in this pull request? Reenable `PandasUDFGroupedAggParityTests. test_grouped_with_empty_partition` ### Why are the changes needed? for test coverage

Re: [PR] [SPARK-47971][PYTHON][CONNECT][TESTS] Reenable `PandasUDFGroupedAggParityTests. test_grouped_with_empty_partition` [spark]

2024-04-24 Thread via GitHub
zhengruifeng commented on code in PR #46202: URL: https://github.com/apache/spark/pull/46202#discussion_r1577376187 ## python/pyspark/sql/tests/pandas/test_pandas_udf_grouped_agg.py: ## @@ -538,11 +538,11 @@ def test_grouped_with_empty_partition(self): data =

Re: [PR] [SPARK-47950] Add Java API Module for Spark Operator [spark-kubernetes-operator]

2024-04-24 Thread via GitHub
jiangzho commented on code in PR #8: URL: https://github.com/apache/spark-kubernetes-operator/pull/8#discussion_r1577383511 ## spark-operator-api/src/main/java/org/apache/spark/kubernetes/operator/BaseResource.java: ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software

Re: [PR] [SPARK-47955][SQL] Improve `DeduplicateRelations` performance [spark]

2024-04-24 Thread via GitHub
peter-toth commented on code in PR #46183: URL: https://github.com/apache/spark/pull/46183#discussion_r1577400706 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DeduplicateRelations.scala: ## @@ -38,28 +38,31 @@ case class RelationWrapper(cls: Class[_],

Re: [PR] [SPARK-47351][SQL] Add collation support for StringToMap & Mask string expressions [spark]

2024-04-24 Thread via GitHub
cloud-fan commented on code in PR #46165: URL: https://github.com/apache/spark/pull/46165#discussion_r1577440230 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala: ## @@ -79,12 +81,14 @@ import

Re: [PR] [SPARK-47414][SQL] Lowercase collation support for regexp expressions [spark]

2024-04-24 Thread via GitHub
cloud-fan commented on code in PR #46077: URL: https://github.com/apache/spark/pull/46077#discussion_r1577446659 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala: ## @@ -158,7 +162,9 @@ case class Like(left: Expression, right:

Re: [PR] [SPARK-47414][SQL] Lowercase collation support for regexp expressions [spark]

2024-04-24 Thread via GitHub
cloud-fan commented on code in PR #46077: URL: https://github.com/apache/spark/pull/46077#discussion_r1577453813 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala: ## @@ -158,7 +162,9 @@ case class Like(left: Expression, right:

Re: [PR] [SPARK-47418][SQL] Add hand-crafted implementations for lowercase unicode-aware contains, startsWith and endsWith and optimize UTF8_BINARY_LCASE [spark]

2024-04-24 Thread via GitHub
cloud-fan commented on PR #46181: URL: https://github.com/apache/spark/pull/46181#issuecomment-2074320175 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-47418][SQL] Add hand-crafted implementations for lowercase unicode-aware contains, startsWith and endsWith and optimize UTF8_BINARY_LCASE [spark]

2024-04-24 Thread via GitHub
cloud-fan closed pull request #46181: [SPARK-47418][SQL] Add hand-crafted implementations for lowercase unicode-aware contains, startsWith and endsWith and optimize UTF8_BINARY_LCASE URL: https://github.com/apache/spark/pull/46181 -- This is an automated message from the Apache Git Service.

Re: [PR] [SPARK-47764][CORE][SQL] Cleanup shuffle dependencies based on ShuffleCleanupMode [spark]

2024-04-24 Thread via GitHub
cloud-fan closed pull request #45930: [SPARK-47764][CORE][SQL] Cleanup shuffle dependencies based on ShuffleCleanupMode URL: https://github.com/apache/spark/pull/45930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

<    1   2   3