[GitHub] [spark] HyukjinKwon closed pull request #39656: [SPARK-42119][SQL] Add built-in table-valued functions inline and inline_outer

HyukjinKwon closed pull request #39656: [SPARK-42119][SQL] Add built-in table-valued functions inline and inline_outer URL: https://github.com/apache/spark/pull/39656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] holdenk opened a new pull request, #39728: [SPARK-42173][CORE] RpcAddress equality can fail

holdenk opened a new pull request, #39728: URL: https://github.com/apache/spark/pull/39728 ### What changes were proposed in this pull request? When constructing an RpcAddress use InetUtils to get a consistently formatted IPv6 address if the env is for an IPv6 address. ### Why

[GitHub] [spark] sadikovi commented on pull request #39729: [SPARK-42175][SQL] Fix cast of a boolean value to timestamp

sadikovi commented on PR #39729: URL: https://github.com/apache/spark/pull/39729#issuecomment-1402899890 @dongjoon-hyun @srowen Can you review this PR? Thanks. It is a small fix for `Cast` expression. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086105668 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086105668 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] rmcyang commented on a diff in pull request #39725: [SPARK-33573][FOLLOW-UP] Increment ignoredBlockBytes when shuffle push blocks are late or colliding

rmcyang commented on code in PR #39725: URL: https://github.com/apache/spark/pull/39725#discussion_r1086005947 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1356,6 +1356,17 @@ private boolean isTooLate(

[GitHub] [spark] otterc commented on a diff in pull request #39725: [SPARK-33573][FOLLOW-UP] Increment ignoredBlockBytes when shuffle push blocks are late or colliding

otterc commented on code in PR #39725: URL: https://github.com/apache/spark/pull/39725#discussion_r1086007667 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1393,6 +1405,7 @@ public void onData(String streamId,

[GitHub] [spark] HyukjinKwon commented on pull request #39721: [SPARK-42171][PYSPARK][TESTS] Fix `pyspark-errors` module and enable it in GitHub Action

HyukjinKwon commented on PR #39721: URL: https://github.com/apache/spark/pull/39721#issuecomment-1402853084 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HyukjinKwon commented on pull request #39724: [SPARK-41775][PYTHON][FOLLOWUP] Fix stdout rerouting

HyukjinKwon commented on PR #39724: URL: https://github.com/apache/spark/pull/39724#issuecomment-1402862371 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] HyukjinKwon closed pull request #39375: [SPARK-36124][SQL] Support subqueries with correlation through UNION

HyukjinKwon closed pull request #39375: [SPARK-36124][SQL] Support subqueries with correlation through UNION URL: https://github.com/apache/spark/pull/39375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon closed pull request #39724: [SPARK-41775][PYTHON][FOLLOWUP] Fix stdout rerouting

HyukjinKwon closed pull request #39724: [SPARK-41775][PYTHON][FOLLOWUP] Fix stdout rerouting URL: https://github.com/apache/spark/pull/39724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #39712: [SPARK-42172][CONNECT] Scala Client Mima Compatibility Tests

HyukjinKwon commented on code in PR #39712: URL: https://github.com/apache/spark/pull/39712#discussion_r1086082530 ## connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/CompatibilitySuite.scala: ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #39717: [SPARK-42168][3.3][SQL][PYTHON] Fix required child distribution of FlatMapCoGroupsInPandas (as in CoGroup)

HyukjinKwon commented on PR #39717: URL: https://github.com/apache/spark/pull/39717#issuecomment-1402875318 cc @sunchao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sadikovi opened a new pull request, #39729: [SPARK-42175] Fix cast of a boolean value to timestamp

sadikovi opened a new pull request, #39729: URL: https://github.com/apache/spark/pull/39729 ### What changes were proposed in this pull request? The PR fixes an issue when casting a boolean to timestamp. While `select cast(true as timestamp)` works and returns

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086105619 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] holdenk commented on a diff in pull request #39728: [SPARK-42173][CORE] RpcAddress equality can fail

holdenk commented on code in PR #39728: URL: https://github.com/apache/spark/pull/39728#discussion_r1086106561 ## core/src/main/scala/org/apache/spark/rpc/RpcAddress.scala: ## @@ -23,30 +23,37 @@ import org.apache.spark.util.Utils /** * Address for an RPC environment, with

[GitHub] [spark] itholic commented on pull request #39721: [SPARK-42171][PYSPARK][TESTS] Fix `pyspark-errors` module and enable it in GitHub Action

itholic commented on PR #39721: URL: https://github.com/apache/spark/pull/39721#issuecomment-1402916066 Late LGTM, thanks for fixing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #39729: [SPARK-42175][SQL] Fix cast of a boolean value to timestamp

dongjoon-hyun commented on code in PR #39729: URL: https://github.com/apache/spark/pull/39729#discussion_r1086138284 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOffSuite.scala: ## @@ -608,6 +608,11 @@ class CastWithAnsiOffSuite extends

[GitHub] [spark] zhenlineo commented on a diff in pull request #39712: [SPARK-42172][CONNECT] Scala Client Mima Compatibility Tests

zhenlineo commented on code in PR #39712: URL: https://github.com/apache/spark/pull/39712#discussion_r1086087241 ## connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/Column.scala: ## @@ -44,7 +45,7 @@ import org.apache.spark.sql.functions.lit * * @since 3.4.0

[GitHub] [spark] gengliangwang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

gengliangwang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086190132 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache

[GitHub] [spark] LuciferYang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

LuciferYang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086234357 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] LuciferYang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

LuciferYang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086234357 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] otterc commented on a diff in pull request #39725: [SPARK-33573][FOLLOW-UP] Increment ignoredBlockBytes when shuffle push blocks are late or colliding

otterc commented on code in PR #39725: URL: https://github.com/apache/spark/pull/39725#discussion_r1086238663 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1382,6 +1395,7 @@ public void onData(String streamId,

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns in Join

zhengruifeng commented on code in PR #39734: URL: https://github.com/apache/spark/pull/39734#discussion_r1086250404 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1174,27 +1174,63 @@ class

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns issue in Join

zhengruifeng commented on code in PR #39734: URL: https://github.com/apache/spark/pull/39734#discussion_r1086250404 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1174,27 +1174,63 @@ class

[GitHub] [spark] itholic commented on a diff in pull request #39702: [SPARK-41487][SQL] Assign name to _LEGACY_ERROR_TEMP_1020

itholic commented on code in PR #39702: URL: https://github.com/apache/spark/pull/39702#discussion_r1086259893 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala: ## @@ -1050,15 +1052,17 @@ class AnalysisErrorSuite extends

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #39726: [SPARK-42123][SQL] Include column default values in DESCRIBE and SHOW CREATE TABLE output #39657

dongjoon-hyun commented on code in PR #39726: URL: https://github.com/apache/spark/pull/39726#discussion_r1086136677 ## sql/core/src/test/resources/sql-tests/inputs/describe.sql: ## @@ -97,3 +97,25 @@ DROP VIEW temp_v; DROP VIEW temp_Data_Source_View; DROP VIEW v; + +--

[GitHub] [spark] HyukjinKwon commented on pull request #39731: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon commented on PR #39731: URL: https://github.com/apache/spark/pull/39731#issuecomment-1403048032 cc @xinrong-meng @dongjoon-hyun @Yikun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon opened a new pull request, #39731: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon opened a new pull request, #39731: URL: https://github.com/apache/spark/pull/39731 ### What changes were proposed in this pull request? Now we cut `branch-3.4`. But the `branch-3.4` points `master` reference (see

[GitHub] [spark] HyukjinKwon closed pull request #39730: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon closed pull request #39730: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions URL: https://github.com/apache/spark/pull/39730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon commented on pull request #39731: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon commented on PR #39731: URL: https://github.com/apache/spark/pull/39731#issuecomment-1403053271 im gonna push this to recover the build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #39731: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon commented on PR #39731: URL: https://github.com/apache/spark/pull/39731#issuecomment-1403053335 Merged to branch-3.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] gengliangwang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

gengliangwang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086194955 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] [spark] gengliangwang opened a new pull request, #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang opened a new pull request, #39732: URL: https://github.com/apache/spark/pull/39732 ### What changes were proposed in this pull request? * Similar to https://github.com/apache/spark/pull/39666, handle remaining null string values in ui protobuf serializer,

[GitHub] [spark] gengliangwang commented on pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang commented on PR #39732: URL: https://github.com/apache/spark/pull/39732#issuecomment-1403122822 cc @LuciferYang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] gengliangwang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086223751 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] otterc commented on a diff in pull request #39725: [SPARK-33573][FOLLOW-UP] Increment ignoredBlockBytes when shuffle push blocks are late or colliding

otterc commented on code in PR #39725: URL: https://github.com/apache/spark/pull/39725#discussion_r1086008666 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1382,6 +1395,7 @@ public void onData(String streamId,

[GitHub] [spark] xinrong-meng opened a new pull request, #39733: Setting version to 3.5.0-SNAPSHOT

xinrong-meng opened a new pull request, #39733: URL: https://github.com/apache/spark/pull/39733 ### What changes were proposed in this pull request? his PR aims to update `master` branch version to 3.5.0-SNAPSHOT. ### Why are the changes needed? Start to prepare Apache Spark

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns in Join

zhengruifeng commented on code in PR #39734: URL: https://github.com/apache/spark/pull/39734#discussion_r1086248363 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1174,27 +1174,63 @@ class

[GitHub] [spark] zhengruifeng opened a new pull request, #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns in Join

zhengruifeng opened a new pull request, #39734: URL: https://github.com/apache/spark/pull/39734 ### What changes were proposed in this pull request? PySpark's `DataFrame.__getattr__` and `DataFrame.__getitem__` invokes `jc = self._jdf.apply(name)` in JVM, which resolve the column

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns issue in Join

zhengruifeng commented on code in PR #39734: URL: https://github.com/apache/spark/pull/39734#discussion_r1086264760 ## connector/connect/server/src/test/scala/org/apache/spark/sql/connect/planner/SparkConnectPlannerSuite.scala: ## @@ -260,21 +260,21 @@ class

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086137202 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] sadikovi commented on a diff in pull request #39729: [SPARK-42176][SQL] Fix cast of a boolean value to timestamp

sadikovi commented on code in PR #39729: URL: https://github.com/apache/spark/pull/39729#discussion_r1086142703 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOffSuite.scala: ## @@ -608,6 +608,11 @@ class CastWithAnsiOffSuite extends

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642: URL: https://github.com/apache/spark/pull/39642#discussion_r1086143970 ## sql/core/src/main/scala/org/apache/spark/status/protobuf/sql/StreamingQueryProgressSerializer.scala: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] [spark] HyukjinKwon opened a new pull request, #39730: [SPARK-42177][INFRA][3.4] Change master to brach-3.4 in GitHub Actions

HyukjinKwon opened a new pull request, #39730: URL: https://github.com/apache/spark/pull/39730 ### What changes were proposed in this pull request? ### Why are the changes needed? To recover the CI, see https://github.com/apache/spark/actions/runs/4002380215/jobs/6869886029.

[GitHub] [spark] HyukjinKwon commented on pull request #39729: [SPARK-42176][SQL] Fix cast of a boolean value to timestamp

HyukjinKwon commented on PR #39729: URL: https://github.com/apache/spark/pull/39729#issuecomment-1403052158 Merged to master, branch-3.4, and branch-3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon closed pull request #39729: [SPARK-42176][SQL] Fix cast of a boolean value to timestamp

HyukjinKwon closed pull request #39729: [SPARK-42176][SQL] Fix cast of a boolean value to timestamp URL: https://github.com/apache/spark/pull/39729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HyukjinKwon closed pull request #39731: [SPARK-42177][INFRA][3.4] Change master to branch-3.4 in GitHub Actions

HyukjinKwon closed pull request #39731: [SPARK-42177][INFRA][3.4] Change master to branch-3.4 in GitHub Actions URL: https://github.com/apache/spark/pull/39731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] sadikovi commented on pull request #39660: [SPARK-42128][SQL] Support TOP (N) for MS SQL Server dialect as an alternative to Limit pushdown

sadikovi commented on PR #39660: URL: https://github.com/apache/spark/pull/39660#issuecomment-1403067310 @beliefer @srowen @dongjoon-hyun Could you please check the following comments: - https://github.com/apache/spark/pull/39660#discussion_r1084578229 -

[GitHub] [spark] LuciferYang commented on pull request #39642: [SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on PR #39642: URL: https://github.com/apache/spark/pull/39642#issuecomment-1403096069 done. Thank you for reviewing the code during the holiday :) @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] LuciferYang commented on a diff in pull request #39712: [SPARK-42172][CONNECT] Scala Client Mima Compatibility Tests

LuciferYang commented on code in PR #39712: URL: https://github.com/apache/spark/pull/39712#discussion_r1086208244 ## connector/connect/client/jvm/pom.xml: ## @@ -75,6 +76,13 @@ mockito-core test + Review Comment: cc @dongjoon-hyun , also cc @pan3793

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39734: [SPARK-41812][SPARK-41823][CONNECT][PYTHON] Fix ambiguous columns in Join

zhengruifeng commented on code in PR #39734: URL: https://github.com/apache/spark/pull/39734#discussion_r1086248363 ## connector/connect/server/src/main/scala/org/apache/spark/sql/connect/planner/SparkConnectPlanner.scala: ## @@ -1174,27 +1174,63 @@ class

[GitHub] [spark] otterc commented on a diff in pull request #39725: [SPARK-33573][FOLLOW-UP] Increment ignoredBlockBytes when shuffle push blocks are late or colliding

otterc commented on code in PR #39725: URL: https://github.com/apache/spark/pull/39725#discussion_r1086246214 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1356,6 +1357,18 @@ private boolean isTooLate(

[GitHub] [spark] itholic commented on a diff in pull request #39701: [SPARK-41489][SQL] Assign name to _LEGACY_ERROR_TEMP_2415

itholic commented on code in PR #39701: URL: https://github.com/apache/spark/pull/39701#discussion_r1086276342 ## core/src/main/resources/error/error-classes.json: ## @@ -933,6 +933,12 @@ ], "sqlState" : "42604" }, + "INVALID_TYPE_FOR_FILTER_EXPR" : { Review

[GitHub] [spark] itholic commented on a diff in pull request #39702: [SPARK-41487][SQL] Assign name to _LEGACY_ERROR_TEMP_1020

itholic commented on code in PR #39702: URL: https://github.com/apache/spark/pull/39702#discussion_r1086279876 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala: ## @@ -1050,15 +1052,17 @@ class AnalysisErrorSuite extends

[GitHub] [spark] gengliangwang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086294168 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -463,7 +492,7 @@ class KVStoreProtobufSerializerSuite extends

[GitHub] [spark] dongjoon-hyun commented on pull request #39735: [SPARK-42179][BUILD][SQL][3.3] Upgrade ORC to 1.7.8

dongjoon-hyun commented on PR #39735: URL: https://github.com/apache/spark/pull/39735#issuecomment-1403211478 Could you review this PR, @gengliangwang ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] peter-toth commented on pull request #39722: [WIP][SPARK-42162] Introduce MultiAdd expression as a memory optimization for canonicalizing large trees of Add expressions

peter-toth commented on PR #39722: URL: https://github.com/apache/spark/pull/39722#issuecomment-1403212470 > - With the https://github.com/apache/spark/pull/37851 in the expression canonicalization, a complex query with a large number of Add operations could end up consuming significantly

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

dongjoon-hyun commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086294697 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -463,7 +492,7 @@ class KVStoreProtobufSerializerSuite extends

[GitHub] [spark] zhengruifeng commented on a diff in pull request #39695: [SPARK-42156] SparkConnectClient supports RetryPolicies now

zhengruifeng commented on code in PR #39695: URL: https://github.com/apache/spark/pull/39695#discussion_r1086269322 ## python/pyspark/sql/connect/client.py: ## @@ -551,37 +592,37 @@ def _execute_and_fetch( logger.info("ExecuteAndFetch") m:

[GitHub] [spark] dongjoon-hyun commented on pull request #39731: [SPARK-42177][INFRA][3.4] Change master to branch-3.4 in GitHub Actions

dongjoon-hyun commented on PR #39731: URL: https://github.com/apache/spark/pull/39731#issuecomment-1403185289 Thank you, @HyukjinKwon ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun opened a new pull request, #39735: [SPARK-42179][BUILD][SQL][3.3] Upgrade ORC to 1.7.8

dongjoon-hyun opened a new pull request, #39735: URL: https://github.com/apache/spark/pull/39735 ### What changes were proposed in this pull request? This PR aims to upgrade ORC to 1.7.8 for Apache Spark 3.3.2. ### Why are the changes needed? Apache ORC 1.7.8 is a

[GitHub] [spark] gengliangwang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086293127 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] gengliangwang commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

gengliangwang commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086292734 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

dongjoon-hyun commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086292024 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -31,6 +34,32 @@ import

[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

dongjoon-hyun commented on code in PR #39732: URL: https://github.com/apache/spark/pull/39732#discussion_r1086292404 ## core/src/test/scala/org/apache/spark/status/protobuf/KVStoreProtobufSerializerSuite.scala: ## @@ -463,7 +492,7 @@ class KVStoreProtobufSerializerSuite extends

[GitHub] [spark] EnricoMi commented on pull request #39717: [SPARK-42168][3.2][SQL][PYTHON] Fix required child distribution of FlatMapCoGroupsInPandas (as in CoGroup)

EnricoMi commented on PR #39717: URL: https://github.com/apache/spark/pull/39717#issuecomment-1403210336 @sunchao good catch! I have renamed the PR and added the Python example as a unit test. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] LuciferYang commented on pull request #39732: [SPARK-42178][UI] Handle remaining null string values in ui protobuf serializer and add tests

LuciferYang commented on PR #39732: URL: https://github.com/apache/spark/pull/39732#issuecomment-1403214341 Another way I can think of is to check each `field` in 'StoreTypes.getDescriptor.toProto.getMessageTypeList', but it should not be better than the current one. -- This is an

[GitHub] [spark] itholic commented on a diff in pull request #39700: [SPARK-41490][SQL] Assign name to _LEGACY_ERROR_TEMP_2441

itholic commented on code in PR #39700: URL: https://github.com/apache/spark/pull/39700#discussion_r1086300683 ## core/src/main/resources/error/error-classes.json: ## @@ -1432,6 +1432,12 @@ }, "sqlState" : "0A000" }, + "UNSUPPORTED_EXPR_FOR_OPERATOR" : { +

[GitHub] [spark] LuciferYang commented on a diff in pull request #39733: Setting version to 3.5.0-SNAPSHOT

LuciferYang commented on code in PR #39733: URL: https://github.com/apache/spark/pull/39733#discussion_r1086307422 ## docs/_config.yml: ## @@ -19,8 +19,8 @@ include: # These allow the documentation to be updated with newer releases # of Spark, Scala, and Mesos.

<    1   2