cshuo opened a new pull request, #18720:
URL: https://github.com/apache/hudi/pull/18720
### Describe the issue this Pull Request addresses
The Flink base-file-only batch reader for COW/read-optimized queries still
used the legacy parquet iterator path directly inside `CopyOnWriteInputFormat`.
That made base-file-only reads diverge from the file group reader path used
elsewhere, increasing the number of places that need changes when adding reader
features.
This PR keeps `HoodieTableSource` returning `CopyOnWriteInputFormat` and
preserves its existing byte-range split generation, while routing split reads
through `HoodieFileGroupReader` and passing the split `start`/`length` into the
underlying parquet reader path.
### Summary and Changelog
- Refactors `CopyOnWriteInputFormat` to build a `HoodieFileGroupReader` for
each `FileInputSplit`, using base file metadata, partition path, requested/data
schemas, predicates, read props, and split byte range.
- Simplifies `CopyOnWriteInputFormat` constructor inputs to accept Flink
configuration plus table/requested schema strings instead of parquet-specific
field and partition parameters.
- Updates `HoodieTableSource#baseFileOnlyInputFormat` to continue returning
`CopyOnWriteInputFormat` while passing table schema, required schema,
predicates, limit, Flink config, and internal schema manager.
- Extends the Flink row-data parquet reader path so `HoodieFileGroupReader`
propagates `start` and `length` into `RecordIterators`.
- Adds coverage for COW base-file reads with multiple input splits and
updates test utilities to invoke `openInputFormat()` for `RichInputFormat`
instances.
### Impact
- COW/read-optimized base-file-only reads now use the file group reader
internally while preserving existing `CopyOnWriteInputFormat` split behavior
and public input format selection.
- Reduces duplicated reader logic by aligning COW base-file batch reads with
the file group reader abstraction.
### Risk Level
low
### Documentation Update
<!-- Describe any necessary documentation update if there is any new
feature, config, or user-facing change. If not, put "none".
- The config description must be updated if new configs are added or the
default value of the configs are changed.
- Any new feature or user-facing change requires updating the Hudi website.
Please follow the
[instruction](https://hudi.apache.org/contribute/developer-setup#website)
to make changes to the website. -->
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
