tustvold commented on issue #8938: URL: https://github.com/apache/arrow-rs/issues/8938#issuecomment-3738172927
> we can automatically track all created Arrays by claiming inside MutableBuffer Buffers can also be allocated from external allocations, most commonly Vec. > For a second step (if we see step 1 proves feasible) we could have explicit APIs with the context, like what @alamb suggested. My confusion remains how this is practically any different from users registering the memory allocation after it emerges from the kernel. It is the difference between: 1. Memory allocation(s) performed 2. Kernel constructs array(s) 3. Caller registers array with pool And 1. Memory allocation(s) performed 2. Kernel registers buffers with pool 3. Kernel returns arrays Ultimately until Rust stabilises allocator support, our hands are somewhat tied here. That being said, if the desire is just to track global allocations, you can currently define a global allocator - see https://doc.rust-lang.org/std/alloc/index.html. > Also, I think adding a Context struct now (even if it only contains MemoryPool initially) sets up the infra for future features. We prefer to have this configured at the kernel level with options, rather than a single global config object - keeps things decoupled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
