clbarnes commented on issue #5272: URL: https://github.com/apache/arrow-rs/issues/5272#issuecomment-1875402851
The `Content-Range` header in the response can contain the length of the full resource (but doesn't have to) as well as the range returned - does anyone familiar with the stores know whether the servers deign to give us that information? In #5222 we just toss it but that can be changed. I think most stores return the range size (S3 way), so some behaviour is changing either way. Maybe if we decide to use the range size we just deprecate the field and have people do `object_meta.range.len()` (hoping nobody's started iterating through it, as `len` here represents the remaining items in ExactSizeIterator); if we want to return the resource size, we could change the name to `total_size` and make it an `Option` in case a range is requested and we get a header like `Content-Range: 0-100/*`. One possible wrinkle is the way that responses in the local file system are handled - in all other stores, `Bytes` representing the requested range are returned, but for the local store a handle to the whole file is returned, which then needs to be sliced using `ObjectMeta.range`. Does that change the mental model of what `.size` means? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
