tustvold commented on issue #4864: URL: https://github.com/apache/arrow-rs/issues/4864#issuecomment-1737037285
> We can only keep the arrays we need to output and release the others. That is still potentially a non-trivial amount of data, it could theoretically be an entire column chunk worth :sweat_smile: Perhaps we could add a configurable threshold of the maximum number of bytes to keep around in this way, and fallback to decoding columns again if this threshold is exceeded? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
