Github user shaoxuan-wang commented on the issue:
https://github.com/apache/flink/pull/3423
@fhueske thanks for the review. I completely agree with your suggestion on
"reworking the batch design". Actually I have proposed the same idea to
"rework the batch" before your review (maybe you have missed my comment on
Feb.27). I was hesitating to make the changes, as I want to keep this PR as
dedicated as possible. But since the performance of the current design is a
concern, let's do the clean up all together within this PR. Regarding to
"pairwise merging", I have different opinion, there are many aggregates that
merging a list at once is much more efficient than merge just two. If we always
use "pairwise merging" in the runtime, we will lose the advantage of
merge(List) API defined in AggregateFunction interface. If the memory is a
concern, we can limit the size of List for each merge iteration, I will provide
the update very soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
