voonhous opened a new pull request, #19835: URL: https://github.com/apache/hudi/pull/19835
### Describe the issue this Pull Request addresses Closes #19826 `HoodieSchemaUtils.generateProjectionSchema` matches field names case-insensitively by lowercasing both the schema's field names and the requested names with the JVM default locale, while `HiveHoodieReaderContext` pre-lowercases the requested names with `Locale.ROOT`. Under a `tr_TR` or `az_AZ` default locale an upper-case `I` in a column name becomes dotless-i (U+0131) on the schema side only, the lookup misses and the Hive read fails with `Field id not found in log schema. Query cannot proceed!`. ### Summary and Changelog Both sides of the lookup now lowercase with `Locale.ROOT`, so the match no longer depends on the JVM's default locale. The legacy realtime reader path lowercased the same way and is aligned in the same commit. <details> <summary>Sites changed, and why the other callers were not affected</summary> - `HoodieSchemaUtils.generateProjectionSchema`: schema-side map key and request-side lookup; javadoc states the `Locale.ROOT` contract. - `HoodieRealtimeRecordReaderUtils.generateProjectionSchema`, `getNameToFieldMap` and `addPartitionFields` (the last one also used the lowercased partition name as the appended null field's name). - `AbstractRealtimeRecordReader.constructHiveOrderedSchema`. `HoodieFileGroupReaderBasedRecordReader`, `HoodieAvroParquetReader`, `MergeIntoHoodieTableCommand` and `BaseBootstrapMetadataHandler` pass names through unchanged, so both sides were already lowercased by the same call. `HiveAvroSerializer` lowercases schema names one-sidedly for Hive's struct type info and is left as is. </details> Tests: `testGenerateProjectionSchema` now projects `PII_COL` (a name with an `I`), and the new `testGenerateProjectionSchemaIgnoresDefaultLocale` sets the default locale to `tr-TR` in a try/finally and covers both directions (upper-cased request, and an upper-cased schema field requested pre-lowercased like `HiveHoodieReaderContext` does). hudi-common runs single-forked and sequentially under surefire, so the toggle cannot leak. The new test fails without the fix. ### Impact Hive reads under a Turkish or Azeri default JVM locale no longer fail on columns whose name contains an upper-case `I`. No behaviour change under any other locale. ### Risk Level low. The change is confined to the locale argument of existing lowercase calls. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
