nada-attia opened a new pull request, #19536:
URL: https://github.com/apache/hudi/pull/19536
### Describe the issue this Pull Request addresses
In HoodieBaseRelation, schema is assumed to always be resolvable from the
table's commit metadata or data files. For datasets that contain no hudi
partitions (only non-hudi partitions), or that have no data written yet, schema
cannot be
resolved that way. Fall back to fetching the schema from HMS in that case
instead of throwing.
Adds a unit test covering select on a freshly created, empty table.
### Summary and Changelog
Reads on a Hudi table would throw when `TableSchemaResolver` could not
resolve a schema from commit metadata or data files on the file system — e.g. a
table with no hudi partitions (only non-hudi partitions), or one with no data
written yet.
`HoodieBaseRelation` now catches that failure and falls back to reading
the table's schema from the Hive Metastore (HMS) catalog entry instead of
throwing.
- `HoodieBaseRelation.scala`: on `TableSchemaResolver.getTableSchema`
failure, fetch the catalog table via
`sparkSession.sessionState.catalog.externalCatalog.getTable(...)` and convert
its schema instead of propagating the exception.
- `TestCreateTable.scala`: added `Test Select On Empty Table Falls Back To
HMS Schema`, which creates a table with no data written and verifies a `select`
on it succeeds (schema resolved from HMS) rather than throwing.
### Impact
No public API changes. Read behavior change: a `select` on a table whose
schema can't be resolved from the file system (e.g. empty table, or
non-hudi-only partitions) now succeeds using the HMS-registered schema instead
of throwing.
### Risk Level
low
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
