[PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [Post-Refactor] [spark]

2024-04-13 Thread via GitHub
GideonPotok opened a new pull request, #46040: URL: https://github.com/apache/spark/pull/46040 https://issues.apache.org/jira/browse/SPARK-46830 ### What changes were proposed in this pull request? Add collation support to types of return values for calls to substr,

[PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [Post-Refactor] [spark]

2024-04-13 Thread via GitHub
GideonPotok opened a new pull request, #46039: URL: https://github.com/apache/spark/pull/46039 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

Re: [PR] [SPARK-47591][SQL] Hive-thriftserver: Migrate logInfo with variables to structured logging framework [spark]

2024-04-13 Thread via GitHub
itholic commented on PR #45926: URL: https://github.com/apache/spark/pull/45926#issuecomment-2053576659 Is there any way testing hive-thriftserver on local? I tried couple of commands below: ```shell build/sbt "sql/testOnly *ThriftServerWithSparkContextSuite" build/sbt

Re: [PR] [SPARK-47318][CORE][3.4] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun closed pull request #46015: [SPARK-47318][CORE][3.4] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices URL: https://github.com/apache/spark/pull/46015 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [SPARK-47318][CORE][3.5] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun closed pull request #46014: [SPARK-47318][CORE][3.5] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices URL: https://github.com/apache/spark/pull/46014 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [SPARK-47318][CORE][3.4] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46015: URL: https://github.com/apache/spark/pull/46015#issuecomment-2053805949 Merged to branch-3.4 for Apache Spark 3.4.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2053821293 Gentle ping, @XorSum . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-47841][BUILD] Upgrade `postgresql` to 42.7.3 [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46038: URL: https://github.com/apache/spark/pull/46038#issuecomment-2053821605 Thank you, @panbingkun . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-47841][BUILD] Upgrade `postgresql` to 42.7.3 [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun closed pull request #46038: [SPARK-47841][BUILD] Upgrade `postgresql` to 42.7.3 URL: https://github.com/apache/spark/pull/46038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-47318][CORE][3.5] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46014: URL: https://github.com/apache/spark/pull/46014#issuecomment-2053803012 Yes, correct (https://github.com/apache/spark/pull/46014#discussion_r1561825387) > we will have to forward port that to master as well. -- This is an automated message from

Re: [PR] [SPARK-47318][CORE][3.5] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46014: URL: https://github.com/apache/spark/pull/46014#issuecomment-2053804650 While double-checking the PR, I'd realized that `3.4.3, 3.5.2` is better because the column name is `Since Version`. -- This is an automated message from the Apache Git Service.

Re: [PR] [SPARK-47318][CORE][3.5] Adds HKDF round to AuthEngine key derivation to follow standard KEX practices [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46014: URL: https://github.com/apache/spark/pull/46014#issuecomment-2053805775 Merged to branch-3.5 for Apache Spark 3.5.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [SPARK-47745] Add License to Spark Operator repository [spark-kubernetes-operator]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #3: URL: https://github.com/apache/spark-kubernetes-operator/pull/3#issuecomment-2053812016 For the record, this PR is currently blocked by the following VOTE. - https://lists.apache.org/thread/s9m1pvjogfwpkhjbythpojdvj05yylz7 ``` [VOTE] Add new

Re: [PR] [SPARK-44444][SQL] Enabled ANSI mode by default [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46013: URL: https://github.com/apache/spark/pull/46013#issuecomment-2053817569 Thank you all. For the record, the vote is started. - [[VOTE] SPARK-4: Use ANSI SQL mode by default](https://lists.apache.org/thread/l2qhpfgtfv6gf001vgw677wylhmx9gcb) --

Re: [PR] [SPARK-41954][CORE] Add isDecommissioned in ExecutorDeadException [spark]

2024-04-13 Thread via GitHub
github-actions[bot] commented on PR #43475: URL: https://github.com/apache/spark/pull/43475#issuecomment-2053816422 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [SPARK-47739][SQL] Register logical avro type [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on code in PR #45895: URL: https://github.com/apache/spark/pull/45895#discussion_r1564350531 ## connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroLogicalTypeInitSuite.scala: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-47739][SQL] Register logical avro type [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on code in PR #45895: URL: https://github.com/apache/spark/pull/45895#discussion_r1564350217 ## connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroLogicalTypeInitSuite.scala: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-47745] Add License to Spark Operator repository [spark-kubernetes-operator]

2024-04-13 Thread via GitHub
viirya commented on PR #3: URL: https://github.com/apache/spark-kubernetes-operator/pull/3#issuecomment-2053818216 Yea, let's wait for the vote on the versioning of this Spark operator. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] [SPARK-47739][SQL] Register logical avro type [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on code in PR #45895: URL: https://github.com/apache/spark/pull/45895#discussion_r1564349334 ## connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroLogicalTypeInitSuite.scala: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-13 Thread via GitHub
XorSum commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2053870360 The manual test steps to avoid native zStandard library not available error: 1. a x86_64 linux machine 2. download and extract

Re: [PR] [WIP][SPARK-46549][INFRA] Cache the Python dependencies for SQL tests [spark]

2024-04-13 Thread via GitHub
github-actions[bot] closed pull request #44546: [WIP][SPARK-46549][INFRA] Cache the Python dependencies for SQL tests URL: https://github.com/apache/spark/pull/44546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [SPARK-46574][BUILD] Upgrade maven plugin to latest version [spark]

2024-04-13 Thread via GitHub
github-actions[bot] closed pull request #43908: [SPARK-46574][BUILD] Upgrade maven plugin to latest version URL: https://github.com/apache/spark/pull/43908 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [SPARK-46594][DOCS] The description of `spark.yarn.shuffle.server.recovery.disabled` should be in chapter `Configuring the External Shuffle Service` [spark]

2024-04-13 Thread via GitHub
github-actions[bot] commented on PR #44593: URL: https://github.com/apache/spark/pull/44593#issuecomment-2053816390 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [SPARK-46566][SQL] Session level config was not loaded when isolation is enable. [spark]

2024-04-13 Thread via GitHub
github-actions[bot] closed pull request #44572: [SPARK-46566][SQL] Session level config was not loaded when isolation is enable. URL: https://github.com/apache/spark/pull/44572 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [Don't Merge] upgrade sbt 1.9.8 [spark]

2024-04-13 Thread via GitHub
github-actions[bot] commented on PR #44516: URL: https://github.com/apache/spark/pull/44516#issuecomment-2053816414 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

Re: [PR] [SPARK-46589][CONNECT] Handle a non-existent error class [spark]

2024-04-13 Thread via GitHub
github-actions[bot] commented on PR #44581: URL: https://github.com/apache/spark/pull/44581#issuecomment-2053816398 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.