ariel-miculas commented on PR #22862: URL: https://github.com/apache/datafusion/pull/22862#issuecomment-4667780641
1. I don't like this approach because it's a workaround for the hash aggregate issue, which is going to be reworked; additionally, it's not accurate memory accounting: when the first batch arrives, it reserves the memory for all the subsequent batches; this reservation could fail, meaning that if the operator supports spilling, it would spill on every batch (because each batch carries the entire memory reservation with it) 2. Since GroupedHashAggregateStream could have multiple downstream operators, is the plan to change the memory accouting for those as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
