[
https://issues.apache.org/jira/browse/FLINK-36753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-36753:
-----------------------------------
Release Note:
The Adaptive Scheduler will now by default actively triggers a checkpoint when
resources change and rescaling is desired, rather than waiting for the next
periodic checkpoint. This reduces rescaling latency, especially when checkpoint
intervals are large. The active trigger respects the configured minimum pause
between checkpoints and will not fire if a checkpoint is already in progress or
being triggered.
You can restore the previous behaviour by setting
{{obmanager.adaptive-scheduler.rescale-trigger.active-checkpoint.enabled}} to
{{false}}.
> Adaptive Scheduler actively triggers a Checkpoint
> -------------------------------------------------
>
> Key: FLINK-36753
> URL: https://issues.apache.org/jira/browse/FLINK-36753
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 2.0-preview
> Reporter: Rui Fan
> Assignee: Samrat Deb
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> FLIP-461[1] and FLINK-35549[2] support that rescale could be executed after
> the next completed checkpoint. It greatly reduces the amount of data replay
> after rescale.
> In FLIP-461, Adaptive Scheduler waits for the next periodic checkpoint to be
> triggered. In most scenarios, a more efficient solution might be Adaptive
> Scheduler actively triggers a Checkpoint after all resources are
> ready(Technically desire resources are ready).
> The idea comes from an offline discussion between [~mxm] and [~fanrui].
> [1][https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing+for+the+AdaptiveScheduler]
> [2] https://issues.apache.org/jira/browse/FLINK-35549
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
