0z5a opened a new pull request, #906:
URL: https://github.com/apache/arrow-rs-object-store/pull/906

   # Which issue does this PR close?
   
   Closes #905.
   
   # Rationale for this change
   
   The default `ObjectStore::get_ranges` (`coalesce_ranges`) panics with `range 
start must not be greater than end` in one case: a requested range starts after 
the end of the object and is merged with an earlier, in-bounds range. If the 
range starts exactly at the end, it returns an empty buffer instead. Requesting 
the range on its own returns an error, and so do the `InMemory` and 
`LocalFileSystem` implementations of `get_ranges`. The issue has the 
reproduction, including `HttpStore` against an RFC 9110 server.
   
   # What changes are included in this PR?
   
   - `coalesce_ranges` keeps the `fetch` closure usable after the coalesced 
requests finish. If the bytes returned for a merged fetch end before a 
requested range starts, that range is fetched on its own, so the caller gets 
the store's normal error for it.
   - Ranges that start inside the returned bytes are sliced as before. Ranges 
that start inside the object but run past its end are still truncated as today.
   - The signature is unchanged. The extra request only happens in this error 
case.
   - New unit test `test_coalesce_ranges_past_end` uses a fetch closure that 
truncates the way a store does. It covers three cases: a range that runs past 
the end (still truncated), a range that starts past the end, and a range that 
starts exactly at the end. It fails on main (panic) and passes with this change.
   - `put_get_delete_list` now also checks that `get_ranges(&[0..5, 200..300])` 
returns an error. The S3/GCS/Azure/HTTP integration runs therefore cover the 
default implementation. `InMemory` and `LocalFileSystem` already pass this 
check.
   
   # Are there any user-facing changes?
   
   Yes, for `AmazonS3`, `GoogleCloudStorage`, `MicrosoftAzure`, `HttpStore`, 
and any other store that uses the default implementation. For a range that 
starts at or past the end of the object, `get_ranges` now returns an error. 
Before, it panicked or returned an empty buffer. There are no public API 
changes.
   
   This change was drafted with Claude Code (claude-opus-5-5); the test results 
above are from the runs described.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to