Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
wForget commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238878202 Hive, Trino, and Clickhouse all seem to be unchunked. Hive: https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFBase64.java Trino: https://github.com/trinodb/trino/blob/4acd387d3e3dd739a8b2ed00fb48f9bbb00c5ab0/core/trino-main/src/main/java/io/trino/operator/scalar/VarbinaryFunctions.java#L61-L64 Clickhouse: https://github.com/ClickHouse/ClickHouse/blob/a07707c56e5c5ad75b19a0e6f75ec02f0ab6c481/src/Common/Base64.cpp#L18 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
yaooqinn commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238844815 Unchunking is the behavior before 3.2(inclusive). Chunking was the behavior introduced accidentally in 3.3. I suggested @wForget in #47303 to make chunking as legacy behavior and unchunking as default, considering that unchunking is more reasonable here and users may not upgrade so eagerly, while chunking is mostly for editors who can not handle long lines. For the unbase64 part, both are valid for decoding. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
cloud-fan commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238780649 It's good to keep the old behavior by default, but I'd like to understand the difference. Which one is more commonly supported in the ecosystem (decoding)? The chunked one or the unchunked one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
yaooqinn commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238593369 Merged to master. Hi @wForget, please help backport this to 3.5. Thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
yaooqinn closed pull request #47410: [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one URL: https://github.com/apache/spark/pull/47410 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
dongjoon-hyun commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238158744 Oh, this one? ``` [info] *** 1 TEST FAILED *** [error] Failed tests: [error] org.apache.spark.sql.connect.ProtoToParsedPlanTestSuite ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
wForget commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238159516 > Oh, this one? yeah -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
wForget commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238157718 The test golden files need to be regenerated, I am doing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
dongjoon-hyun commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238156229 Let's wait for @gatorsmile and @cloud-fan 's review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47307][SQL][FOLLOWUP] Promote spark.sql.legacy.chunkBase64String.enabled from a legacy/internal config to a regular/public one [spark]
wForget commented on PR #47410: URL: https://github.com/apache/spark/pull/47410#issuecomment-2238033219 > BTW, @wForget , if you change the configuration name in this PR, the PR title and description should be updated accordingly. > > Currently, the PR title and description is not matched. Updated, thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
