maartenbreddels commented on pull request #8755:
URL: https://github.com/apache/arrow/pull/8755#issuecomment-733646443
Yes, works beautifully. I can mmap from python, and return a memoryview from
my `read_buffer` method and there are no memory copies at all, while at the
same time returning a bytes object from my `read` method making it compatible
with other libraries (such as pandas).
I did leave in the change of accepting a buffer via `Result<int64_t>
PyReadableFile::Read(int64_t nbytes, void* out)` since it's more consistent
with the `Result<std::shared_ptr<Buffer>> PyReadableFile::Read(int64_t nbytes)`
calling `file_->Read` and accepting a buffer.
Summary:
* In Python land `read` can return a buffer object, but to be compatible
(with e.g. pandas), it should return a bytes object.
* To support zero mem copy and have better compatibility, `read_buffer`
should be implemented and return a buffer object, and `read` should return a
bytes object.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
