[ https://issues.apache.org/jira/browse/MESOS-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16257398#comment-16257398 ]
Yan Xu commented on MESOS-8185: ------------------------------- I think so. [~ipronin] with MESOS-7215 no tasks will be killed by the master, even if the framework is not partition-aware. Will this work? > Tasks can be known to the agent but unknown to the master. > ---------------------------------------------------------- > > Key: MESOS-8185 > URL: https://issues.apache.org/jira/browse/MESOS-8185 > Project: Mesos > Issue Type: Bug > Affects Versions: 1.2.0 > Reporter: Ilya Pronin > Assignee: Ilya Pronin > Labels: reliability > > Currently, when a master re-registers an agent that was marked unreachable, > it shutdowns all not partition-aware frameworks on that agent. When a master > re-registers an agent that is already registered, it doesn't check that all > tasks from the slave's re-registration message are known to it. > It is possible that due to a transient loss of connectivity an agent may miss > {{SlaveReregisteredMessage}} along with {{ShutdownFrameworkMessage}} and thus > will not kill not partition-aware tasks. But the master will mark the agent > as registered and will not re-add tasks that it thought will be killed. The > agent may re-register again, this time successfully, before becoming marked > unreachable while never having terminated tasks of not partition-aware > frameworks. The master will simply forget those tasks ever existed, because > it has "removed" them during the previous re-registration. > Example scenario: > # Connection from the master to the agent stops working > # Agent doesn't see pings from the master and attempts to re-register > # Master sends {{SlaveRegisteredMessage}} and {{ShutdownSlaveMessage}}, which > don't get to the agent because of the connection failure. Agent is marked > registered. > # Network issue resolves, connection breaks. Agent retries re-registration. > # Master thinks that the agent was registered since step (3) and just > re-sends {{SlaveRegisteredMessage}}. Tasks remain running on the agent. > One of the possible solutions would be to compare the list of tasks the the > already registered agent reports in {{ReregisterSlaveMessage}} and the list > of tasks the master has. In this case anything that the master doesn't know > about should not exist on the agent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)