[
https://issues.apache.org/jira/browse/FLINK-26490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502292#comment-17502292
]
刘方奇 commented on FLINK-26490:
-----------------------------
Now, we have so many job without keyed state want to increase their parallelism
when restore from checkpoint/savepoint, but it's hard because of the max
parallelism check.
The above is our background.
As I saw, there are so many check code in the checkpoint restore process. What
I except is that job with keyed state should check their max parallelism but
adjust the max parallelism when they don't contain keyed state. Checkpoint meta
can be rebuild in the restore process. That can make the rest process of the
restoring pass the check.
So what you worry about is that there will be no max parallelism check after
refactor? Am I understand correct?
[~yunta]
> Adjust the MaxParallelism or remove the MaxParallelism check when unnecessary.
> ------------------------------------------------------------------------------
>
> Key: FLINK-26490
> URL: https://issues.apache.org/jira/browse/FLINK-26490
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Reporter: 刘方奇
> Priority: Major
>
> Since Flink introduce key group and MaxParallelism, Flink can rescale with
> less cost.
> But when we want to update the job parallelism bigger than the
> MaxParallelism, it 's impossible cause there are so many MaxParallelism check
> that require new parallelism should not bigger than MaxParallelism.
> Actually, when an operator which don't contain keyed state, there should be
> no problem when update the parallelism bigger than the MaxParallelism,, cause
> only keyed state need MaxParallelism and key group.
> So should we remove this check or auto adjust the MaxParallelism when we
> restore an operator state that don't contain keyed state?
> It can make job restore from checkpoint easier.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
