anjakefala commented on issue #26685: URL: https://github.com/apache/arrow/issues/26685#issuecomment-1760042012
I had an already started branch for Option (2) (directly using IPC serialization for pickling), and confirmed [that it does not support Pickle Protocol 5](https://github.com/apache/arrow/pull/37683). That made that approach untenable. @pitrou Would you be able to take a look at @clarkzinzow's [branch](https://github.com/apache/arrow/compare/main...clarkzinzow:arrow:arrow-10739-pickle-buffer-truncation-fix). It aims to be an implementation for Option (1): > Refactor the IPC writer's per-type buffer truncation logic into utilities that can be shared by the IPC serialization path and the pickle serialization path. Is it a decent starting point? I would be happy to adopt it, and break it up into smaller PRs if it looks like a promising approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
