voonhous opened a new pull request, #19834: URL: https://github.com/apache/hudi/pull/19834
### Describe the issue this Pull Request addresses Closes #19825 `HoodieSchema.getNonNullType()` returns `this` for a union with no null branch, and a union of the non-null branches when there are two or more of them. Six recursive schema walkers called themselves on that result inside their `UNION` arm, so any schema carrying a field typed `["null","string","int"]` or `["string","int"]` recursed until `StackOverflowError`. The issue names the first two; a sweep of `getNonNullType()` callers found the other four. ### Summary and Changelog | Site | Reachable from | Fix | |---|---|---| | `HoodieSchemaUtils.hasDecimalField` | Streamer JSON sources (`SourceFormatAdapter`) | iterate every branch | | `HoodieSchemaRepair.hasTimestampMillisField` | `HoodieFileGroupReaderBasedFileFormat` | iterate every branch | | `InternalSchemaConverter.collectColNamesFromSchema` | schema-on-read pruning | iterate every branch | | `ValueType.fromSchema` | column stats (`ValueMetadata`) | strip null once, then `IllegalArgumentException` | | `HoodieTableMetadataUtil.coerceToComparable` | column stats (`FileFormatUtils`) | strip null once, then `HoodieNotSupportedException` | | `HoodieSchemaUtils.findNestedField` | file group reader schema handler, write handles | strip null once, then `Option.empty()` | The iterate-every-branch shape is the one `HoodieSchema#containsBlobType` already uses. The two single-answer walkers restore what the strict `getNonNullTypeFromUnion` did before the HoodieSchema migrations (#14311, #18163, #17600) swapped in the lenient call and turned a throw into a loop. The `getNonNullType()` javadoc now states that its result can itself be a union. Tests: multi-branch union cases added to the existing test methods for each site. Each new assertion fails with `StackOverflowError` before the fix. <details> <summary>Related but out of scope</summary> - `InternalSchemaConverter.visitSchemaToBuildType` silently takes the first non-null branch of a multi-branch union when building an InternalSchema type. - `isColumnTypeSupportedV2` is a deny-list that never names `UNION`, which is why the column-stats sites are reachable at all. </details> ### Impact No API change. Six paths that previously threw `StackOverflowError` on multi-branch unions now return a result or throw a descriptive exception. ### Risk Level low. Behavior only changes for schemas with a union of two or more non-null types, which previously could not get past these methods at all. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
