felipecrv commented on PR #35: URL: https://github.com/apache/arrow-experiments/pull/35#issuecomment-2356937438
> > Not so clear cut, because you could use IPC buffer compression and decompress right before returning to save on cache storage space. > > Perhaps... but do any Arrow implements provide efficient, easy-to-use methods to convert an IPC stream with compressed buffers to an IPC stream with uncompressed buffers? This could be done simply enough with a small procedure in most of the Arrow implementations, but I'm not sure how efficient it would be. It would require an intermediate step. But if the cache exists to reduce the need for some expensive computation that produces the RecordBatch stream, even expensive decompression and re-serialization could pay off. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
