[
https://issues.apache.org/jira/browse/KAFKA-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-13811:
----------------------------------
Labels: perfomance (was: )
> Investigate sliding windows performance
> ---------------------------------------
>
> Key: KAFKA-13811
> URL: https://issues.apache.org/jira/browse/KAFKA-13811
> Project: Kafka
> Issue Type: Task
> Components: streams
> Reporter: Leah Thomas
> Priority: Major
> Labels: perfomance
>
> We recently fixed a bug in sliding windows so that a grace period of 0ms is
> properly calculated, see https://issues.apache.org/jira/browse/KAFKA-13739.
> Before this patch, sliding windows with a grace period of 0ms would just skip
> all records so nothing would get put into the store.
> When we ran benchmarks for the 3.2 release we saw a significant drop in
> performance for sliding windows on both the 3.2 and trunk branches, see the
> `sliding windows` results
> [here|[http://kstreams-benchmark-results.s3-website-us-west-2.amazonaws.com/summaries/process-cumulative-rate/graph.html].]
> These benchmarks use a sliding window with a 0ms grace period, which means
> until now we weren't sending any values to the state store when running these
> benchmarks.
> I ran benchmarks on the
> [commit|https://github.com/apache/kafka/commit/430f9c99012d1585aa544d4dadf449963296c1fd]
> before KAFKA-13739 and changed the grace period to 2 seconds to see if the
> bug fix changed anything. The performance was still low for [this
> run|[http://kstreams-benchmark-results.s3-website-us-west-2.amazonaws.com/experiments/sliding1min-3-5-3-3-0-430f9c9901-leah-20220408084241-streamsbench/].]
> Given this, it seems like the performance for sliding windows has always been
> low but we didn't realize it because the bug fixed in KAFKA-13739 was present
> in the benchmarks we were running. We should investigate why the current
> algorithm is slow and see if improvements can be made
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
