Guozhang Wang created KAFKA-6560:

             Summary: Use single-point queries than range queries for windowed 
aggregation operators
                 Key: KAFKA-6560
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Guozhang Wang
            Assignee: Guozhang Wang

Today for windowed aggregations in Streams DSL, the underlying implementation 
is leveraging the fetch(key, from, to) API to get all the related windows for a 
single record to update. However, this is a very inefficient operation with 
significant amount of CPU time iterating over window stores. On the other hand, 
since the operator implementation itself have full knowledge of the window 
specs it can actually translate this operation into multiple single-point 
queries with the accurate window start timestamp, which would largely reduce 
the overhead.

The proposed approach is to add a single fetch API to the WindowedStore and use 
that in the KStreamWindowedAggregate / KStreamWindowedReduce operators.

This message was sent by Atlassian JIRA

Reply via email to