alamb commented on PR #12969: URL: https://github.com/apache/datafusion/pull/12969#issuecomment-2422462820
> I will go ahead and merge this soon unless someone catches a critical issue. Thank you -- I agree this is better than what was on main > > Here are the things to work on in the near future: > > 1. Add `MemoryReservation` to batch splitting to add rigorous memory accountability. > 2. Exploring if we can pass builders into methods instead of passing arrays around and converting them into builders when possible. > 3. Considering using the`batch_transformer: Box<dyn BatchTransformer>` pattern if it doesn't regress performance I filed https://github.com/apache/datafusion/issues/13003 to track adding memory accounting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
