clarkzinzow commented on issue #26685: URL: https://github.com/apache/arrow/issues/26685#issuecomment-1766815788
Oh, that makes more sense! So you're saying that an API should be exposed on `Array`/`ArrayData` to recursively truncate buffers, rather than piggyback off of IPC's array visitor + "serialize to a payload" logic? That would probably be cleaner from a frontend language's perspective. There's still type-specific buffer truncation logic that would require a type-based visitor, and it's less clear to me how to marry that array visitor with the IPC serialization code in order to share that buffer truncation logic without make the IPC serialization code less efficient. Right now the IPC serialization truncates buffers and aggregates buffers into a flat list on a single recursive pass over the array tree, while factoring that buffer truncation logic out into an `ArrayData.truncate_buffers()` API would seem to necessitate two passes of the array tree: one to truncate buffers and one to aggregate the flat buffer list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
