suryaprasanna opened a new pull request, #18205:
URL: https://github.com/apache/hudi/pull/18205
### Describe the issue this Pull Request addresses
Some runtime systems sanitize or pass-through only `spark.*` prefixed
configs. In these environments, users commonly provide Hadoop and engine
configs using prefixes such as `spark.hadoop.*`.
For Hudi read paths, this creates a usability gap when users need to pass
Hudi datasource configs but can only inject Spark-prefixed keys. This PR
enables using `spark.hoodie.*` as an input form for read configs and normalizes
them to `hoodie.*` with deterministic precedence handling.
### Summary and Changelog
- Extended read option resolution to normalize `spark.hoodie.*` ->
`hoodie.*` inside `parametersWithReadDefaults`.
- Defined and documented precedence during read option merging so behavior
is explicit and predictable.
- Updated `DefaultSource` to pass both `hoodie.*` and `spark.hoodie.*` SQL
confs into common read-default handling.
- Added unit tests to verify:
- `spark.hoodie.*` values are accepted and translated,
- explicit `hoodie.*` values override corresponding `spark.hoodie.*`
values when both are provided.
- Added a relation creation log line with resolved option count for easier
runtime debugging.
### Impact
- Improves compatibility with systems that require user-provided configs to
be prefixed with `spark.*`.
- Lets users pass Hudi configs as `spark.hoodie.*` similarly to how Hadoop
configs are often passed as `spark.hadoop.*`.
- No breaking API changes; this is additive behavior for read config
resolution.
### Risk Level
low
Change is scoped to read-option normalization/merge logic and includes
focused unit coverage for translation and precedence.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
