yihua opened a new issue, #19846:
URL: https://github.com/apache/hudi/issues/19846
Since #14061, `hoodie.file.group.reader.enabled` is only consulted by the
streaming sources (`HoodieStreamSourceV1`/`V2`) and `HoodieIncrSource`.
`DefaultSource` routes every batch snapshot and read-optimized query to
`HoodieFileGroupReaderBasedFileFormat` regardless of the config, so there is no
runtime way to run the same query against the legacy read path.
That gap shows up when a read regression is suspected on the file group
reader. On 1.1.x this config was the switch used to compare the two read paths
on the same table and cluster; on 1.2.0 and master there is no equivalent, and
the config documentation ("Use engine agnostic file group reader if enabled")
still reads as if it applies to batch reads.
The legacy relations for snapshot queries are all still in the tree
(`BaseFileOnlyRelation`, `MergeOnReadSnapshotRelation`) and still serve
metadata-table reads and streaming reads with the config off, so honoring the
config for batch snapshot queries again is a routing change only.
Proposal: have `DefaultSource` consult `hoodie.file.group.reader.enabled`
for COW snapshot/read-optimized and MOR snapshot queries, leaving the default
(config on) behavior unchanged. CDC, incremental and bootstrap queries stay on
the file group reader unconditionally, since their legacy relations have been
removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
