gatorsmile commented on a change in pull request #27621: [SPARK-26071][followup] Improve migration guide of disallowing map type map key URL: https://github.com/apache/spark/pull/27621#discussion_r381740913
########## File path: docs/sql-migration-guide.md ########## @@ -43,7 +43,7 @@ license: | - The `ADD JAR` command previously returned a result set with the single value 0. It now returns an empty result set. - - In Spark version 2.4 and earlier, users can create map values with map type key via built-in function like `CreateMap`, `MapFromArrays`, etc. Since Spark 3.0, it's not allowed to create map values with map type key with these built-in functions. Users can still read map values with map type key from data source or Java/Scala collections, though they are not very useful. + - In Spark version 2.4 and earlier, users can create map values with map type key via built-in function such as `CreateMap`, `MapFromArrays`, etc. Since Spark 3.0, it's not allowed to create map values with map type key with these built-in functions. Users can use array<struct<key, value>> as map keys instead as a workaround. In addition, users can still read map values with map type key from data source or Java/Scala collections, though it is discouraged. Review comment: Could we give a very simple example, how to use array<struct<key, value>> as map keys as a workaround? Also, do we have such a test case? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
