Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-22 Thread via GitHub
sadikovi commented on PR #46126: URL: https://github.com/apache/spark/pull/46126#issuecomment-2070979626 Yes, I will open a separate PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-22 Thread via GitHub
dongjoon-hyun commented on PR #46126: URL: https://github.com/apache/spark/pull/46126#issuecomment-2070968153 There is a small conflict. Could you make a backporting PR to `branch-3.5`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-22 Thread via GitHub
dongjoon-hyun closed pull request #46126: [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType URL: https://github.com/apache/spark/pull/46126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-21 Thread via GitHub
sadikovi commented on PR #46126: URL: https://github.com/apache/spark/pull/46126#issuecomment-2068451840 Thanks for the review @dongjoon-hyun. I have addressed your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-21 Thread via GitHub
dongjoon-hyun commented on code in PR #46126: URL: https://github.com/apache/spark/pull/46126#discussion_r1574083057 ## connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala: ## @@ -367,6 +367,20 @@ abstract class AvroSuite

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-21 Thread via GitHub
dongjoon-hyun commented on code in PR #46126: URL: https://github.com/apache/spark/pull/46126#discussion_r1574081702 ## connector/avro/src/main/scala/org/apache/spark/sql/avro/SchemaConverters.scala: ## @@ -208,14 +208,13 @@ object SchemaConverters { // could

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-21 Thread via GitHub
dongjoon-hyun commented on PR #46126: URL: https://github.com/apache/spark/pull/46126#issuecomment-2068387729 Sorry for being late. I'll take a look Today, @sadikovi . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-18 Thread via GitHub
sadikovi commented on PR #46126: URL: https://github.com/apache/spark/pull/46126#issuecomment-2065446238 cc @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [SPARK-47904][SQL] Preserve case in Avro schema when using enableStableIdentifiersForUnionType [spark]

2024-04-18 Thread via GitHub
sadikovi opened a new pull request, #46126: URL: https://github.com/apache/spark/pull/46126 ### What changes were proposed in this pull request? When `enableStableIdentifiersForUnionType` is enabled, all of the types are lowercased which creates a problem when field types