alamb commented on PR #14766: URL: https://github.com/apache/datafusion/pull/14766#issuecomment-2668448654
> It is a problem of datafusion-cli. If datafusion-cli decides to hold all the result batches in memory, it should create a memory consumer for itself and reserve memory for the result batches. Another thing that I remember `psql` did was to only buffer the first like 1000 lines or so to figure out column spacing -- right now `datafusion-cli` buffers the entire thing to ensure it can format all the rows reasonably which is likely unecessary for the majority of the time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
