MaxGekk opened a new pull request #27200: [SPARK-30509][SQL] Fix deprecation log warning in Avro schema inferring URL: https://github.com/apache/spark/pull/27200 ### What changes were proposed in this pull request? In the PR, I propose to check the `ignoreExtensionKey` option in the case insensitive map of `AvroOption`. ### Why are the changes needed? The map `options` passed to `AvroUtils.inferSchema` contains all keys in the lower cases in fact. Actually, the map is converted from a `CaseInsensitiveStringMap`. Consequently, the check https://github.com/apache/spark/blob/3663dbe541826949cecf5e1ea205fe35c163d147/external/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala#L45 always return `false`, and the deprecation log warning is never printed. ### Does this PR introduce any user-facing change? Yes, after the changes the log warning is printed once. ### How was this patch tested? Added new test to `AvroSuite` which checks existence of log warning.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
