Nice work! First question: don't you think that operations should differentiate short and long maintenance? I am thinking about frameworks that use persistent storage on disk for example. A short maintenance such as a slave reboot or upgrade could be done without moving the data to another slave. However decommissioning requires to drain the storage too.
If you have an HDFS datanode with 50TB of (replicated) data, you might not want to drain it for a reboot (assuming your replication factor is high enough) since it takes ages. However for decommission it might make sense to drain it. Not sure if this is a good example but I feel the need to know if the maintenance is planned to be short or is forever. I know this does not fit the nice modeling you describe :-/ Actually for HDFS we could define a threshold where "good enough" replication without the slave would be considered enough and thus we could deactivate the slave. This would prevent a rolling restart to go too fast. However could you specify that when you drain a slave with hard:false you don't enter the drained state even when the deadline has passed if tasks are still running? This is not explicit in the document and we want to make sure operators have the information about this and could avoid unfortunate rolling restarts. On Aug 25, 2014 9:25 PM, "Benjamin Mahler" <[email protected]> wrote: > Hi all, > > I wanted to take a moment to thank Alexandra Sava, who completed her OPW > internship this past week. We worked together in the second half of her > internship to create a design document for maintenance primitives in Mesos > (the original ticket is MESOS-1474 > <https://issues.apache.org/jira/browse/MESOS-1474>, but the design > document is the most up-to-date plan). > > Maintenance in this context consists of anything that requires the tasks > running on the slave to be killed (e.g. kernel upgrades, machine > decommissioning, non-recoverable mesos upgrades / configuration changes, > etc). > > The desire is to expose maintenance events to frameworks in a generic > manner, as to allow frameworks to respect their SLAs, perform better task > placement, and migrate tasks if necessary. > > The design document is here: > > https://docs.google.com/document/d/1NjK7MQeJzTRdfZTQ9q1Q5p4dY985bZ7cFqDpX4_fgjM/edit?usp=sharing > > Please take a moment before the end of next week to go over this design. > *Higher > level feedback and questions can be discussed most effectively in this > thread.* > > Let's thank Alexandra for her work! > > Ben >
