kgyrtkirk opened a new pull request, #18510:
URL: https://github.com/apache/druid/pull/18510
Phantom tasks could have appeared which were running indefinetly and may
have caused the system to try to kill them repeatedly without much success
* if `HttpRemoteTaskRunner` is used
* launch a task (must be still running when the worker is stopped)
* the `Task` object will be fully registered into the
`HttpRemoteTaskRunnerWorkItem`
* cycle the overlord
* the running task is picked up from the still alive worker; however it
will be
[registered](https://github.com/apache/druid/blob/45cf67d09e10aaf493556d2fe512f2034eead897/indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunner.java#L1541)
w/o `Task` informations
* cycle the worker
* during connection some
[expectedAnnouncements](https://github.com/apache/druid/blob/45cf67d09e10aaf493556d2fe512f2034eead897/indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/HttpRemoteTaskRunner.java#L610-L625)
are forged...however w/o the `Task` that's not possible
* the task remains in `RUNNING` state
note:
* killing a stuck task like the above will:
* change its status to FAILED
* sends shutdown to the worker
* which will be ignored
* and thus no `Announcement` will be generated
* it will remain stuck in the overlord
* killed again in the next round
The proposed fix fills in the announcement w/o using the `Task` details; the
only purpose of that announcement is to emit the the [fake
Announcement](https://github.com/apache/druid/blob/45cf67d09e10aaf493556d2fe512f2034eead897/indexing-service/src/main/java/org/apache/druid/indexing/overlord/hrtr/WorkerHolder.java#L354-L367)
about that the task have disappeared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
