tustvold commented on PR #7253: URL: https://github.com/apache/arrow-rs/pull/7253#issuecomment-2708275169
> Is the idea that this would be able poll the response stream on a different runtime without all the high level wrapping needed if we did it in ObjectStore? Yes, similar story for metrics, tracing, etc... Caching could also be implemented at this layer quite easily, either by caching anything with an ETag header or by respecting the Cache-Control header, or some other mechanism. > chunks Chunking _can_ be done at this layer, however, it has the exact same restrictions as implementing it at the ObjectStore level: * Request needs to specify an ETag precondition to ensure same object * Requests must specify range(s) * Cannot return chunks out of order It is unclear if these restrictions are too onerous for the feature to be useful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
