0z5a opened a new pull request, #906: URL: https://github.com/apache/arrow-rs-object-store/pull/906
# Which issue does this PR close? Closes #905. # Rationale for this change The default `ObjectStore::get_ranges` (`coalesce_ranges`) panics with `range start must not be greater than end` in one case: a requested range starts after the end of the object and is merged with an earlier, in-bounds range. If the range starts exactly at the end, it returns an empty buffer instead. Requesting the range on its own returns an error, and so do the `InMemory` and `LocalFileSystem` implementations of `get_ranges`. The issue has the reproduction, including `HttpStore` against an RFC 9110 server. # What changes are included in this PR? - `coalesce_ranges` keeps the `fetch` closure usable after the coalesced requests finish. If the bytes returned for a merged fetch end before a requested range starts, that range is fetched on its own, so the caller gets the store's normal error for it. - Ranges that start inside the returned bytes are sliced as before. Ranges that start inside the object but run past its end are still truncated as today. - The signature is unchanged. The extra request only happens in this error case. - New unit test `test_coalesce_ranges_past_end` uses a fetch closure that truncates the way a store does. It covers three cases: a range that runs past the end (still truncated), a range that starts past the end, and a range that starts exactly at the end. It fails on main (panic) and passes with this change. - `put_get_delete_list` now also checks that `get_ranges(&[0..5, 200..300])` returns an error. The S3/GCS/Azure/HTTP integration runs therefore cover the default implementation. `InMemory` and `LocalFileSystem` already pass this check. # Are there any user-facing changes? Yes, for `AmazonS3`, `GoogleCloudStorage`, `MicrosoftAzure`, `HttpStore`, and any other store that uses the default implementation. For a range that starts at or past the end of the object, `get_ranges` now returns an error. Before, it panicked or returned an empty buffer. There are no public API changes. This change was drafted with Claude Code (claude-opus-5-5); the test results above are from the runs described. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
