georgew5656 opened a new pull request, #16310:
URL: https://github.com/apache/druid/pull/16310
Adding a api on the overlord to order a task group to checkpoint and exit
early before taskDuration is met. This is helpful when trying to update old
middle managers (by ending the existing tasks on them) and also could
potentially let druid run streaming tasks indefinitely (since they can do
intermediate checkpointing and can always be shutdown manually when needed).
### Description
- Add an API that given a supervisor-id, task-group-id, will tell the tasks
in that task group to shutdown the next time the supervisor loop runs.
#### Fixed the bug ...
#### Renamed the class ...
#### Added a forbidden-apis entry ...
I thought integrating this into the checkTaskDuration flow was the best way
to to this since it does everything needed during a regular handoff (when
taskDuration is exceeded), and the logic we want here is exactly the same as
what is done during regular handoff (as opposed to a intermediate handoff or a
supervisor pause)
This is still WIP so I can definitely move some of the logic around
(checkTaskDuration -> shouldTaskHandoff, or something like this)
#### Release note
Add new API for telling streaming task groups to exit early.
##### Key changed/added classes in this PR
* `SupervisorManager`
* `SupervisorResource`
* `SeekableStreamSupervisor`
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency
checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md)
(Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked
related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in
[licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
