kou commented on issue #154: URL: https://github.com/apache/arrow-swift/issues/154#issuecomment-4220049573
Arrow C++ uses IO interface for buffer from memory mapped file. Arrow C++'s IO interface has `supports_zero_copy()` predicate: https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/interfaces.h#L226-L229 If `supports_zero_copy()` returns `true`, the IO can return a buffer that just refers memory mapped data (zero copy) by `Read()` (and its family): https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/file.cc#L664-L680 https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/io/file.cc#L530-L539 Arrow C++ provides `Device` and `MemoryManager` for GPU (and CPU but, in general, they aren't used for CPU expiclity). `MemoryManager` can allocate a buffer from GPU memory and create an IO that read/write from/to GPU memory: https://github.com/apache/arrow/blob/5617e8da18c5888d975fdeb77c98c0d9fb6f0f8b/cpp/src/arrow/gpu/cuda_context.h#L253-L260 All `Buffer`s have a pointer, an offset and a size. A `Buffer` can have parent `Buffer` for a sliced `Buffer`. It's for preventing freeing the parent `Buffer` while children `Buffer`s are alive. Memory mapped IO has a parent `Buffer` and returns a sliced `Buffer` from the parent `Buffer` for `Read()`. A `CudaBuffer` has `CudaMemoryManager`. It's for preventing freeing the `CudaMemoryManager` while allocated `Buffer`s are alive. In Arrow Swift, we may want to add `ArrowBuffer.slice()` to create a sub buffer without copying. (Does this explanation provide enough information...? If there are missing information, please let me know.) > @kou would like to join as well? I can't join it. Sorry! I don't listen/speak English... I only read/write English. If you share summary or something here after the meeting, I may be able to provide additional information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
