yihua opened a new pull request, #1003:
URL: https://github.com/apache/hudi-rs/pull/1003

   closes #999
   
   ## Description
   
   The parquet base-file reader fetched a row group's column chunks only after 
the previous row group was decoded. `PrefetchingReader`, an `AsyncFileReader` 
wrapper, now starts fetching the next selected row group's chunks in a spawned 
task while the current one decodes. It learns the columns from the reader's own 
requests (under a row filter, only the filter stage's), serves only ranges a 
prefetched chunk fully contains, and falls back to a normal fetch when a 
prefetch fails. It stays at most one row group ahead and aborts in-flight work 
on drop; on a current-thread runtime the overlap is limited to when the 
consumer yields.
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   Tests in `tests::prefetch` check that prefetched reads match unprefetched 
ones, and cover row filters, page-index selections, row-group selection, buffer 
retention and failed prefetches.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to