Updated timestamps will be generated. On Tue, Sep 30, 2014 at 6:21 AM, Whitney Sorenson <[email protected]> wrote:
> That makes sense and I suppose your first sentence was the real > summary/confirmation of the change in 0.20 that I was looking for. > > A quick question - can we trust that the taskStatus objects sent as a > result of calling reconciliation themselves are the exact same object as > was (possibly) sent before, or would they have updated timestamps? > > > > > On Tue, Sep 30, 2014 at 4:14 AM, Benjamin Mahler < > [email protected]> wrote: > >> We want reconciliation to be a process that eventually terminates. >> >> In <= 0.19.0, the following two cases are conflated through no update >> being sent: >> (1) No state difference. >> (2) Master temporarily cannot reply / dropped message. >> >> As a result, a scheduler cannot determine when it is finished reconciling >> (is my state correct? or was my message not processed?). >> >> >> The way we want to steer frameworks to use reconciliation is as follows. >> >> (1) You should only need to reconcile with the master after a >> re-registration occurs (either master failed over, or framework failed >> over). Some frameworks may want to be more defensive against both >> themselves and against Mesos, and may reconcile on a periodic basis (e.g. >> hourly or daily). >> >> (2) Reconciliation is a process which terminates when an update has been >> received for each task. Here is some pseudo-code that demonstrates how a >> scheduler would implement reliable reconciliation: >> >> >> # Reconciles state against the master. >> # TODO: If you call this twice, it will start two reconciliation >> cycles, instead of starting a new one. >> def reconcile(): >> start_time = now() >> remaining_tasks = [all non terminal tasks] >> driver.reconcileTasks() # Implicit reconciliation, lets you >> discover unknown tasks. >> delay(Seconds(30), _reconcile, start_time, remaining_tasks) >> >> def _reconcile(start_time, remaining_tasks): >> remaining_tasks = [t for t in remaining_tasks if >> t.latest_update_time() < start_time] >> if not remaining_tasks.empty(): >> driver.reconcileTasks(remaining_tasks) >> delay(Seconds(30), _reconcile, start_time, remaining_tasks) # >> TODO: Use backoff instead. >> >> >> The idea is that you reconcile a set of tasks until you receive updates >> for each one, this set will converge to become empty. You would call >> reconcile() when a (re-)registration occurs. >> >> This is the model you should use in 0.20.0, there are some edge cases >> that we'll fix for 0.21.0, but you likely will not notice them: >> https://issues.apache.org/jira/browse/MESOS-1407 >> >> Appreciate you starting this thread. Let me know if anything is not clear. >> >> Ben >> >> On Sun, Sep 28, 2014 at 11:58 AM, Whitney Sorenson <[email protected] >> > wrote: >> >>> I'm trying to understand the changes in >>> https://issues.apache.org/jira/browse/MESOS-1453 and the >>> SchedulerDriver JavaDoc. >>> >>> In the 0.19 behavior, it made sense to me that a framework would hold >>> onto a copy of all the latest task statuses it knew about, and could poll >>> reconcileTasks with these statuses in order to request delivery of any lost >>> messages (covering the case of both the framework being absent for a while >>> or just a general loss of messages.) >>> >>> Is the idea behind the changes in 0.20 that a framework now need only >>> call reconcileTasks once after registering with a master? In that case, >>> what is the use case for having the API still take a list of taskStatus >>> objects - so frameworks can decide that they don't want to know about >>> unknown tasks [1379]? If frameworks should still routinely ask for missing >>> messages - then why bother sending all updates and causing the framework to >>> have to handle the work of routinely ignoring duplicate status updates? >>> >>> Thanks, >>> >>> -Whitney >>> >>> >> >
