totoroyyb commented on issue #6709:
URL: https://github.com/apache/arrow-rs/issues/6709#issuecomment-2569049713
Hi, I also have the need for reading the IPC files via `mmap` for a
low-latency workload. Thanks @tustvold for redirecting the document. The
approach of using `FileDecoder` indeed work for the reading purpose.
However, looks like the `Buffer` is rathor undesired for zero-copy (not
entirely true). The documented approach does incur one extra copy due to the
`Buffer` allocation. I feel like many people who are looking into `mmap`
capability, may want to avoid such copy. Here is one approach (assume `memmap2`
is used):
``` rust
let file = std::fs::File::open("...").unwrap();
let mmap = unsafe { Mmap::map(&file).unwrap() };
let buffer = unsafe {
Buffer::from_custom_allocation(
NonNull::new_unchecked(mmap.as_ptr() as *mut u8),
mmap.len(),
Arc::new(mmap),
)
};
// ... use the buffer as suggested in the document...
```
Nevertheless, may I still request a more structured way (like implemented in
c++ library) of reading IPC file via mmap, which can be more ergnomic to use?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
