yihua opened a new pull request, #1004:
URL: https://github.com/apache/hudi-rs/pull/1004

   closes #1000
   
   ## Description
   
   Stacked on #1003; review the last commit.
   
   Large base-file reads were one GET on one connection. `SplittingReader`, 
innermost under the read-volume counters, coalesces ranges the way 
`object_store` does, then splits ranges larger than 8 MiB into concurrent parts 
(at most 8 in flight per call), assembled in place into one buffer per merged 
range; requested ranges are sliced from it zero-copy. Small reads are 
unchanged, and `io_calls` counts once per call. A connection cap shared across 
readers is left for a follow-up.
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   New tests check that split reads return the same bytes as unsplit ones, over 
an in-memory reader and an object store, and cover coalescing, part tiling, 
out-of-order completion, failed and mis-sized parts, and call counting.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to