Micah Kornfield created ARROW-13690:
---------------------------------------
Summary: [Python] Use IPC writing code for pickling RecordBatches
Key: ARROW-13690
URL: https://issues.apache.org/jira/browse/ARROW-13690
Project: Apache Arrow
Issue Type: Improvement
Components: Python
Reporter: Micah Kornfield
For wide schemas in particular the the recursive nature of the currently
pickling algorithm for record batches makes it less efficient then using the
IPC format (which can be done entirely in C++).
Consider switching the mechanism to use the IPC format. I think this can be a
backwards compatible change if the current leaving: _reconstruct_record_batch
in place if we care about that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
