Lijie Wang created FLINK-30942:
----------------------------------
Summary: Fix the bug that the decided parallelism by adaptive
batch scheduler may be larger than the max parallelism
Key: FLINK-30942
URL: https://issues.apache.org/jira/browse/FLINK-30942
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Reporter: Lijie Wang
Assignee: Lijie Wang
Fix For: 1.17.0
Currently, when using the adaptive batch scheduler, the vertex parallelism
decided by forward group may be larger than the global max parallelism(which
is configured by option {{parallelism.default}} or
{{execution.batch.adaptive.auto-parallelism.max-parallelism}}, see FLINK-30686
for details), which will cause the following exception:
{code:java}
Caused by: java.lang.IllegalArgumentException: Vertex's parallelism should be
smaller than or equal to vertex's max parallelism.
at
org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138)
at
org.apache.flink.runtime.scheduler.DefaultVertexParallelismInfo.setParallelism(DefaultVertexParallelismInfo.java:95)
at
org.apache.flink.runtime.executiongraph.ExecutionJobVertex.setParallelism(ExecutionJobVertex.java:317)
at
org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.changeJobVertexParallelism(AdaptiveBatchScheduler.java:385)
at
org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.initializeVerticesIfPossible(AdaptiveBatchScheduler.java:284)
at
org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler.onTaskFinished(AdaptiveBatchScheduler.java:183)
at
org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:745)
at
org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:725)
at
org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:80)
at
org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:479)
... 30 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
