johanl-db opened a new pull request, #40879: URL: https://github.com/apache/spark/pull/40879
### What changes were proposed in this pull request? [StructType.findNestedField](https://github.com/apache/spark/blob/db2625c70a8c3aff64e6a9466981c8dd49a4ca51/sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala#L325) is unable to reach nested fields below two directly nested maps or arrays. Whenever it reaches a map or an array, it'll throw an `invalidFieldName` exception if the child is not a struct. This change updates `findNestedField` to correctly recurse into two or more levels of directly nested maps or arrays. In addition, use `checkError` in tests for `findNestedField` instead of manually checking error messages. ### Why are the changes needed? `findNestedField` is used for example in [`ALTER TABLE` commands](https://github.com/apache/spark/blob/9e17731f46aa26348e69f42d1a96882186022b80/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L3735). It is currently not possible to add or modify a nested field within two or more levels of directly nested maps or arrays. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tests are added to `StructTypeSuite` to cover deeply nested maps and arrays with `findNestedField` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
