voonhous opened a new issue, #20637:
URL: https://github.com/apache/hudi/issues/20637

   Part of #20625. Line references are against master `db90a0da`.
   
   ### Problem
   A file slice with log files is always exactly one split 
(`HudiSplitFactory.java:164-185`), where base-file-only slices are cut into 
target-size splits (`:100-160`). `HoodieFileGroupReader` rejects `start != 0` 
when logs are present. With large base files this limits scan parallelism for 
MOR snapshot reads.
   
   ### Direction
   Needs the file group reader to merge a byte range of the base file against 
the log records. Scope and design first; lower priority than the per-split and 
merge-buffer items.
   
   ### Exit criteria
   - Design note on this issue covering how a base file range is merged with 
the slice's log records without re-reading the logs per range.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to