Dandandan commented on PR #15479: URL: https://github.com/apache/datafusion/pull/15479#issuecomment-2815260149
I think this is a nice experiment. That said, I think we can better try changing the build side of the join to use `Vec<RecordBatch>`. I remember we (I) changed it to concatenate all build batches to one side (to improve performance back then), but it woul d be preferable if we wouldn't concatenate everything in one batch. One downside of doing it is that we can't load > 4GiB of `Utf8` columns in the left side, it will fail with overflowing offsets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
