alamb commented on issue #10392: URL: https://github.com/apache/arrow-rs/issues/10392#issuecomment-5591003058
> I believe a deep-copy of the data into the read column should be mandatory. I see no other way of ensuring a small or even self-contained memory footprint for arrays otherwise. This should be a declared part of the reader imo yes I agree Maybe we can add some configuration option on the IPC reader that forces a copy to occur Note I think @jayzhan211 basically did something like this downstream in datafusion as part of - https://github.com/apache/datafusion/pull/24592 Maybe we could port that to arrow-rs (along with a flag 🤔 ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
