clbarnes opened a new pull request, #5222:
URL: https://github.com/apache/arrow-rs/pull/5222
- Support suffix and offset ranges in GetOptions and get_opts
- Ensure that, if a range is requested, the response contains exactly that
range
# Which issue does this PR close?
Closes #4611 . Supercedes #5206 .
# Rationale for this change
Motivated by fetching file suffixes without needing 2 requests to find the
length first.
# What changes are included in this PR?
- Add utility struct `GetRange` which represents bounded ranges, offsets,
and suffixes
- `GetOptions` now stores a `GetRange` rather than a `Range`
- Implementations of `ObjectStore` are updated with these changes
# Are there any user-facing changes?
`GetOptions::range` is now a `crate::utils::GetRange` rather than a
`std::ops::Range<usize>`. It implements `From<RangeBounds<usize>>` so this
simply means adding `.into()` to current code.
There are some additional error cases which previously went undetected if a
server didn't return exactly the requested range.
# Open questions
LocalFileSystem is a special-case store whose `get_opts` returns a whole
file which then must be sliced using the `GetResult::range`. Other stores
return a response containing only the requested bytes, with the range returned
only for informational purposes. There is currently little need for users to
deal with this complexity as `get_range(s)` handles it internally, but this PR
may guide more users towards having to do so.
There are a bunch of utilities in the `GetRange` impl and associated free
functions which seemed like they could be useful but are not currently used,
and could be deleted for simplicity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
