pitrou commented on issue #45287: URL: https://github.com/apache/arrow/issues/45287#issuecomment-2616025082
Further diagnosing using #45359 suggests that, on a 1000 cols * 300 chunks table, around 200 MB memory is spent on per-chunk metadata. I suspect it's a combination of the fields on `ChunkedArray`, `NumericArray`, `ArrayData`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
