[
https://issues.apache.org/jira/browse/MESOS-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045840#comment-15045840
]
Anindya Sinha commented on MESOS-4048:
--------------------------------------
I think the high level thinking is that mesos master be responsible of updating
frameworks with an appropriate task status and let frameworks decide how they
would like to handle tasks, instead of mesos-master doing the cleanup on its
own.
Currently, when a mesos-slave is unreachable from mesos-master, master gets a
disconnect/deactivate slave event and starts the health check timeout (default:
5*15 sec) and on its expiry migrates the task to TASK_LOST, cleans up slave
from its registered list and marks this slave as removed, and notifies
frameworks with a LOST_SLAVE.
Similarly, slave_reregister_timeout controls how long the master waits for a
slave to reregister after master failover before marking that slave as a
LOST_SLAVE (with a notification to the framework).
Instead, mesos-master could just be responsible for notifying frameworks of
events. When a slave disconnects/deactivates, mesos-master could notify the
framework of an unknown state for the affected tasks with a TASK_WANDERING
status. The basic idea is that frameworks can decide what to do with this
TaskStatus, ie. reschedule it immediately or wait for a certain amount of time
(timer maintained on framework) and then reschedule. The wandering task can
certainly get back to TASK_RUNNING at which point the framework can decide to
kill this old instance if that task has already been rescheduled.
We maintain a single timer that denotes the time in which a slave must register
(either on master failover or after a slave disconnect/deactivate event). At
this point, we notify frameworks of a LOST_SLAVE event.
Please also refer to MESOS-4049 for details on how this approach helps in
managing persistent tasks, ie. tasks that should not be terminated by anyone
other than the frameworks.
> Consider unifying slave timeout behavior between steady state and master
> failover
> ---------------------------------------------------------------------------------
>
> Key: MESOS-4048
> URL: https://issues.apache.org/jira/browse/MESOS-4048
> Project: Mesos
> Issue Type: Improvement
> Components: master, slave
> Reporter: Neil Conway
> Assignee: Anindya Sinha
> Priority: Minor
> Labels: mesosphere
>
> Currently, there are two timeouts that control what happens when an agent is
> partitioned from the master:
> 1. {{max_slave_ping_timeouts}} + {{slave_ping_timeout}} controls how long the
> master waits before declaring a slave to be dead in the "steady state"
> 2. {{slave_reregister_timeout}} controls how long the master waits for a
> slave to reregister after master failover.
> It is unclear whether these two cases really merit being treated differently
> -- it might be simpler for operators to configure a single timeout that
> controls how long the master waits before declaring that a slave is dead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
