[jira] [Comment Edited] (MESOS-8573) Container stuck in PULLING when Docker daemon hangs
[
https://issues.apache.org/jira/browse/MESOS-8573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367705#comment-16367705
]
Gilbert Song edited comment on MESOS-8573 at 2/20/18 7:21 AM:
--
https://reviews.apache.org/r/65689/
[https://reviews.apache.org/r/65712/]
was (Author: gilbert):
[https://reviews.apache.org/r/65689/
https://reviews.apache.org/r/65712/
|https://reviews.apache.org/r/65689/]
> Container stuck in PULLING when Docker daemon hangs
> ---
>
> Key: MESOS-8573
> URL: https://issues.apache.org/jira/browse/MESOS-8573
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Greg Mann
>Assignee: Gilbert Song
>Priority: Major
> Labels: mesosphere
>
> When the {{force}} argument is not set to {{true}}, {{Docker::pull}} will
> always perform a {{docker inspect}} call before it does a {{docker pull}}. If
> either of these two Docker CLI calls hangs indefinitely, the Docker container
> will be stuck in the PULLING state. This means that we make no further
> progress in the {{launch()}} call path, so the executor binary is never
> executed, the {{Future}} associated with the {{launch()}} call is never
> failed or satisfied, and {{wait()}} is never called on the container. The
> agent chains the executor cleanup onto that {{wait()}} call which is never
> made. So, when the executor registration timeout elapses,
> {{containerizer->destroy()}} is called on the executor container, but the
> rest of the executor cleanup is never performed, and no terminal task status
> update is sent.
> This leaves the task destined for that Docker executor stuck in TASK_STAGING
> from the framework's perspective, and attempts to kill the task will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-8573) Container stuck in PULLING when Docker daemon hangs
[
https://issues.apache.org/jira/browse/MESOS-8573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367705#comment-16367705
]
Gilbert Song edited comment on MESOS-8573 at 2/20/18 7:21 AM:
--
[https://reviews.apache.org/r/65689/
https://reviews.apache.org/r/65712/
|https://reviews.apache.org/r/65689/]
was (Author: gilbert):
https://reviews.apache.org/r/65689/
> Container stuck in PULLING when Docker daemon hangs
> ---
>
> Key: MESOS-8573
> URL: https://issues.apache.org/jira/browse/MESOS-8573
> Project: Mesos
> Issue Type: Improvement
>Affects Versions: 1.5.0
>Reporter: Greg Mann
>Assignee: Gilbert Song
>Priority: Major
> Labels: mesosphere
>
> When the {{force}} argument is not set to {{true}}, {{Docker::pull}} will
> always perform a {{docker inspect}} call before it does a {{docker pull}}. If
> either of these two Docker CLI calls hangs indefinitely, the Docker container
> will be stuck in the PULLING state. This means that we make no further
> progress in the {{launch()}} call path, so the executor binary is never
> executed, the {{Future}} associated with the {{launch()}} call is never
> failed or satisfied, and {{wait()}} is never called on the container. The
> agent chains the executor cleanup onto that {{wait()}} call which is never
> made. So, when the executor registration timeout elapses,
> {{containerizer->destroy()}} is called on the executor container, but the
> rest of the executor cleanup is never performed, and no terminal task status
> update is sent.
> This leaves the task destined for that Docker executor stuck in TASK_STAGING
> from the framework's perspective, and attempts to kill the task will fail.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
