zentol commented on code in PR #22169:
URL: https://github.com/apache/flink/pull/22169#discussion_r1146013857
##########
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/AdaptiveScheduler.java:
##########
@@ -730,15 +748,49 @@ public CompletableFuture<CoordinationResponse>
deliverCoordinationRequestToCoord
+ " does not exist")));
}
+ @Override
+ public JobResourceRequirements requestJobResourceRequirements() {
+ final JobResourceRequirements.Builder builder =
JobResourceRequirements.newBuilder();
+ for (JobInformation.VertexInformation vertex :
jobInformation.getVertices()) {
+ builder.setParallelismForJobVertex(vertex.getJobVertexID(), 1,
vertex.getParallelism());
+ }
+ return builder.build();
+ }
+
+ @Override
+ public void updateJobResourceRequirements(JobResourceRequirements
jobResourceRequirements) {
+ if (executionMode == SchedulerExecutionMode.REACTIVE) {
+ throw new UnsupportedOperationException(
+ "Cannot change the parallelism of a job running in
reactive mode.");
+ }
+ final Optional<VertexParallelismStore>
maybeUpdateVertexParallelismStore =
+ DefaultVertexParallelismStore.applyJobResourceRequirements(
+ jobInformation.getVertexParallelismStore(),
jobResourceRequirements);
+ if (maybeUpdateVertexParallelismStore.isPresent()) {
+ this.jobInformation =
+ new JobGraphJobInformation(jobGraph,
maybeUpdateVertexParallelismStore.get());
+ declareDesiredResources();
+ state.tryRun(
+ ResourceListener.class,
+ ResourceListener::onNewResourceRequirements,
+ "Current state does not react to desired parallelism
changes.");
Review Comment:
> nothing is (re)declared in declareDesiredResources() - that's fine (?)
Currently, yes. Once we extend the AdaptiveScheduler to support fine-grained
resource management this will change though. But this should happen
automatically if the resource requirements differ.
> but we still want to proceed to onNewResourceRequirements check (?)
yes, so we potentially trigger a restart.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
