GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/2447
[FLINK-4490] [distributed coordination] Decouple the JobManage's slots from
the 'Instance'
To allow for a future dynamic slot allocation and release model, the slots
should not depend on 'Instance'.
In this change, the Slots hold most of the necessary information directly
(location, gateway) and
the interact with the Instance only via a 'SlotOwner' interface.
The `ResourceID`, which has been introduces a while back, is a unique
identifier for a TaskManager that is also constant for the lifetime of that
particular TaskManager process. Where previously, the JobManager comparted
`Instance` objects for referential equality to determine if two slots refer to
the same TaskManager, the code now always refers to the `ResourceID` instread.
*Side note: I suggest to promote the `ResourceID` to `TaskManagerID` at
some point.
This also reworks the `InstanceConnectionInfo`to the `TaskManagerLocation`,
which contains in addition to IP/hostname/port also the `ResourceID`.
All logic of the Scheduler and ExecutionGraph remains unchanged.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink slots_works
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2447.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2447
----
commit 198028f4ff1cc01e0e39bd389937a23b9ea45c7f
Author: Stephan Ewen <[email protected]>
Date: 2016-08-29T14:58:31Z
[FLINK-4490] [distributed coordination] (part 1) Change
InstanceConnectionInfo to TaskManagerLocation
This adds the ResourceId to the TaskManagerLocation
commit 02ad27e065e3f32b4503fac6ac5a3138a3b607e1
Author: Stephan Ewen <[email protected]>
Date: 2016-08-30T18:34:20Z
[FLINK-4490] [distributed coordination] (part 2) Make slots independent of
'Instance'.
To allow for a future dynamic slot allocation and release model, the slots
should not depend on 'Instance'.
In this change, the Slots hold most of the necessary information directly
(location, gateway) and
the interact with the Instance only via a 'SlotOwner' interface.
commit e7efae594015e2788f840b0fc3c685880fd35e85
Author: Stephan Ewen <[email protected]>
Date: 2016-08-31T11:52:45Z
[FLINK-4525] [core] (followup) Remove remaining redundant code for
pre-defined strictly local assignments.
commit 7b30d8f6246af60b7fceebb0cdde761aee89a714
Author: Stephan Ewen <[email protected]>
Date: 2016-08-31T11:59:01Z
[FLINK-4490] [distributed coordination] (part 3) Rename methods on
'Instance' to have more intuitive names
getResourceID() --> getTaskManagerID()
getInstanceConnectionInfo() --> getTaskManagerLocation()
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
