alamb commented on PR #382: URL: https://github.com/apache/arrow-rs-object-store/pull/382#issuecomment-2980116275
> > If we want to proceed with removing the `GetResultPayload` I think we should first deprecate the code for a release cycle or two and see how many users show up and worry / offer opinions about it. > > I don't think there's a way to make this non-breaking and TBH I think we have to face one reality: the `object_store` interface is messy at the moment and to make it 1.0 proof, there will be breaking changes; and not all of them can be phased. I am not convinced that this particular API needs to be changed and I think we need a better justification than that the API is complicated. The original idea as I understood it, for this API, is that special casing File operations could make significant performance improvements so the more complicated API was justified. > At the same time I suspect that most users copy the file data into a buffer anyways and don't use any kernel-based IO operations like [sendfile](https://man.archlinux.org/man/sendfile.2). For example, looking at just the DataFusion code, I see quite a few different code paths for `File` vs Stream: https://github.com/search?q=repo%3Aapache%2Fdatafusion%20GetResultPayload%3A%3AFile&type=code I am not claiming that all those cases couldn't be made as performant using a Stream, but I also haven't done the diligence. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
