Dandandan commented on PR #16647: URL: https://github.com/apache/datafusion/pull/16647#issuecomment-3027365462
> > I had a look at this benchmark. It seems it only is testing a single 1M batch per partition/column? > > You mean streaming back RecordBatch (e.g., batch of 8192 rows) instead of 1M all at once, right? > > I'll try to tweak this aspect to make it meet expectations. Yes, otherwise the "optimization" will have no effect, as it can only re-use allocations across batches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
