[
https://issues.apache.org/jira/browse/FLINK-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464544#comment-16464544
]
Shuyi Chen commented on FLINK-7001:
-----------------------------------
Hi [~pgrulich], the paper is a nice read. And the technique applies to Tumble,
Sliding & Session window, which is a good win, and the evaluation result looks
good. Also, it seems you already have an implementation for Scotty using Apache
Flink based on the paper.
Maybe, you and [~jark] can share more, for each approach, about the detail
design, pros and cons, and we can discuss them here?
> Improve performance of Sliding Time Window with pane optimization
> -----------------------------------------------------------------
>
> Key: FLINK-7001
> URL: https://issues.apache.org/jira/browse/FLINK-7001
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Reporter: Jark Wu
> Assignee: Jark Wu
> Priority: Major
>
> Currently, the implementation of time-based sliding windows treats each
> window individually and replicates records to each window. For a window of 10
> minute size that slides by 1 second the data is replicated 600 fold (10
> minutes / 1 second). We can optimize sliding window by divide windows into
> panes (aligned with slide), so that we can avoid record duplication and
> leverage the checkpoint.
> I will attach a more detail design doc to the issue.
> The following issues are similar to this issue: FLINK-5387, FLINK-6990
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
