GitHub user ifndef-SleePy opened a pull request:
https://github.com/apache/flink/pull/5048
[Flink-7871] [flip6] SlotPool should release unused slots to RM
## What is the purpose of the change
* This pull request makes SlotPool release unused slots back to RM if the
slots is idle for some time
## Brief change log
* Add a TimerService in SlotPool
* Each available slot will register a timer in TimerService, and will
unregister when the slot is removed or becomes allocated
* If timeout happened, the slot will be released by notifying TM
* Add notifySlotUnused method in TaskExecutorGateway
* Make some fixed params configurable in SlotPool
## Verifying this change
This change added tests in AvailableSlotsTest
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no, but add a new `@Public(Evolving)` class SlotOptions)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alibaba/flink FLINK-7871
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5048.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5048
----
commit b16d206b1ff1a459de10dd5d08094172386d3707
Author: biao.liub <[email protected]>
Date: 2017-11-16T10:11:20Z
[FLINK-7871] [flip6] SlotPool will release unused slot if it is idle for a
period.
commit 19e9ccfb69b6e1529ed346a00389b682eecdfcfd
Author: biao.liub <[email protected]>
Date: 2017-11-16T11:15:16Z
[FLINK-7871] [flip6] Stop timer service in SlotPool elegantly.
commit 0484f7457e46180fe49ae4f3483afa92c0b20ac5
Author: biao.liub <[email protected]>
Date: 2017-11-17T02:58:57Z
[FLINK-7871] [flip6] Fix rebasing error.
commit 2a012582b1e169a90c5674ed294d95c12029ee8d
Author: biao.liub <[email protected]>
Date: 2017-11-17T03:19:40Z
[FLINK-7871] [flip6] Fix AvailableSlotsTest.
commit bb8b8f889735ab7a93c837db3c33b3f0900755db
Author: biao.liub <[email protected]>
Date: 2017-11-17T08:15:01Z
[FLINK-7871] [flip6] Add super.postStop() in SlotPool.postStop()
commit 29fe70cec7de136e5959c9f128d32ece97fb68fc
Author: biao.liub <[email protected]>
Date: 2017-11-17T08:36:41Z
[Flink-7871] [flip6] SlotPool should release unused slots to RM
Summary:
## What is the purpose of the change
* This pull request makes SlotPool release unused slots back to RM if the
slots is idle for some time
## Brief change log
* Add a TimerService in SlotPool
* Each available slot will register a timer in TimerService, and will
unregister when the slot is removed or becomes allocated
* If timeout happened, the slot will be released by notifying TM
* Add notifySlotUnused method in TaskExecutorGateway
* Make some fixed params configurable in SlotPool
## Verifying this change
This change added tests in AvailableSlotsTest
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no, but add a new '@Public(Evolving)' class SlotOptions)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
Test Plan: UT done
Reviewers: æµ·æ¶, è¾
æº
Differential Revision: https://aone.alibaba-inc.com/code/D350078
commit 7b144e853df1d9a807464832a3582922c91672d1
Author: biao.liub <[email protected]>
Date: 2017-11-17T09:19:40Z
[Flink-7871] [flip6] SlotPool should release unused slots to RM
Summary:
## What is the purpose of the change
* This pull request makes SlotPool release unused slots back to RM if the
slots is idle for some time
## Brief change log
* Add a TimerService in SlotPool
* Each available slot will register a timer in TimerService, and will
unregister when the slot is removed or becomes allocated
* If timeout happened, the slot will be released by notifying TM
* Add notifySlotUnused method in TaskExecutorGateway
* Make some fixed params configurable in SlotPool
## Verifying this change
This change added tests in AvailableSlotsTest
## Does this pull request potentially affect one of the following parts:
* Dependencies (does it add or upgrade a dependency): (no)
* The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no, but add a new `@Public(Evolving)` class SlotOptions)
* The serializers: (no)
* The runtime per-record code paths (performance sensitive): (no)
* Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
* The S3 file system connector: (no)
## Documentation
* Does this pull request introduce a new feature? (no)
* If yes, how is the feature documented? (not applicable)
Test Plan: UT done
Reviewers: haitao.w, è¾
æº
Differential Revision: https://aone.alibaba-inc.com/code/D350105
commit 4f2cddf3e35b4f9203850cfee4122b12bfd1f046
Author: biao.liub <[email protected]>
Date: 2017-11-22T10:57:41Z
[FLINK-7871] Fix typos and and some tests.
----
---
