Stefan Richter created FLINK-10433:
--------------------------------------
Summary: Stepwise creation of the ExecutionGraph sub-structures
Key: FLINK-10433
URL: https://issues.apache.org/jira/browse/FLINK-10433
Project: Flink
Issue Type: Sub-task
Components: Distributed Coordination
Affects Versions: 1.7.0
Reporter: Stefan Richter
Assignee: Stefan Richter
In this step, we break the construction of ExecutionGraph substructures into
multiple steps. When translating a JobGraph to an ExecutionGraph, we could only
create a “skeleton” ExecutionGraph that is only built up to the
ExecutionJobVertex level. This enables scheduling in two steps. First we can
compute the minimal and desired amount of resources and ask the SlotPool for
them. Only when the available resources match the minimal requirements, the
actual scheduling would start. With this information, we can build the
ExecutionVertex and Execution objects already in the right parallelism and
avoid running out of resources during scheduling. This separation will help us
with the autoscaling use-case, where we figure out the actual parallelism after
we started with scheduling.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
