Matthias J. Sax created KAFKA-13842:
---------------------------------------
Summary: Add per-aggregation step before repartitioning
Key: KAFKA-13842
URL: https://issues.apache.org/jira/browse/KAFKA-13842
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Matthias J. Sax
Kafka Streams follows a continuous refinement model for aggregation. For this
reason, we never implement a pre-aggregation step before data repartitioning,
because it won't help much to reduce repartition cost (there is no natural
boundary when a pre-aggregation is finished and when to emit it downstream for
the actual aggregation roll-up).
With https://issues.apache.org/jira/browse/KAFKA-13785 we introduce a
per-aggregation "emit final" feature (different to suppress()) that changes the
continuous refinement model and thus it seems to be a good optimization to add
a pre-aggregation step if this new feature is used.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
