alamb commented on issue #9562: URL: https://github.com/apache/arrow-datafusion/issues/9562#issuecomment-2015218264
> this issue is a bit tricky for me 😢 , I can only think of the following approaches: change the `GroupedHashAggregateStream::emit()` function return a vector `Result<Vec<RecordBatch>>`, each `RecordBatch` num_rows equal `GroupedHashAggregateStream::batch_size`, and change `ExecutionState::ProducingOutput(RecordBatch)` to `ExecutionState::ProducingOutput(Vec<RecordBatch>)`, then `GroupedHashAggregateStream::poll_next()` function returns one element of a vector at one loop, like `let output = batchs.pop()` [source](https://github.com/apache/arrow-datafusion/blob/main/datafusion/physical-plan/src/aggregates/row_hash.rs#L496) I think this approach sounds good -- nice proposal One thing that could help keep the PRs small and manageable would be to switch the APIs as described above but you could avoid having to change all the `GroupAccumulators` in one PR by returning return a `Vec<>` of size 1 for most of them. Then we can make subsequent PRs to switch over other groups accumulators as needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
