[
https://issues.apache.org/jira/browse/FLINK-36753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan updated FLINK-36753:
----------------------------
Description:
FLIP-461[1] and FLINK-35549[2] support that rescale could be executed after the
next completed checkpoint. It greatly reduces the amount of data replay after
rescale.
In FLIP-461, Adaptive Scheduler waits for the next periodic checkpoint to be
triggered. In most scenarios, a more efficient solution might be Adaptive
Scheduler actively triggers a Checkpoint after all resources are
ready(Technically desire resources are ready).
The idea comes from an offline discussion between [~mxm] and me.
[1][https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing+for+the+AdaptiveScheduler]
[2] https://issues.apache.org/jira/browse/FLINK-35549
was:
FLIP-461[1] and FLINK-35549[2] support that rescale could be executed after the
next completed checkpoint. It greatly reduces the amount of data replay after
rescale.
In FLIP-461, Adaptive Scheduler waits for the next periodic checkpoint to be
triggered. In most scenarios, a more efficient solution might be Adaptive
Scheduler actively triggers a Checkpoint.
The idea comes from an offline discussion between [~mxm] and me.
[1][https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing+for+the+AdaptiveScheduler]
[2] https://issues.apache.org/jira/browse/FLINK-35549
> Adaptive Scheduler actively triggers a Checkpoint
> -------------------------------------------------
>
> Key: FLINK-36753
> URL: https://issues.apache.org/jira/browse/FLINK-36753
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 2.0-preview
> Reporter: Rui Fan
> Priority: Major
>
> FLIP-461[1] and FLINK-35549[2] support that rescale could be executed after
> the next completed checkpoint. It greatly reduces the amount of data replay
> after rescale.
> In FLIP-461, Adaptive Scheduler waits for the next periodic checkpoint to be
> triggered. In most scenarios, a more efficient solution might be Adaptive
> Scheduler actively triggers a Checkpoint after all resources are
> ready(Technically desire resources are ready).
> The idea comes from an offline discussion between [~mxm] and me.
> [1][https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing+for+the+AdaptiveScheduler]
> [2] https://issues.apache.org/jira/browse/FLINK-35549
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
