paleolimbot commented on PR #509: URL: https://github.com/apache/arrow-nanoarrow/pull/509#issuecomment-2186589396
> how difficult is it for a user to inject their own implementations of the ArrowDevice object or callbacks to use for the copying It would probably take another PR or two to get to that point. I think this PR is on the right track, though, in that it implements the asynchronous array copy purely in terms of an asynchronous buffer allocation and an asynchronous buffer copy. The two PRs might be: - Better separate the allocation and copying steps in `ArrowBufferInit()` - Let one or more methods accept a custom buffer copier/allocator - Better support splitting up children of a device array (if somebody had more than one stream they had available to use, one might suggest copying children in parallel but right now you might have to do some fudging to get all the pieces in the right place). For something like cudf that has well-abstracted tools for doing these kinds of things, I might suggest building the buffers separately (e.g., how it currently exports a column to an `ArrowArray` using the `ArrowBufferDeallocator`) and using `ArrowDeviceArrayInit()` to do the final export (as opposed to using this generic array copier). That said, I think the generic array copier is still important (if we want people to build device things, they have to test them/print them somehow). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
