> > It falls apart for terminal tasks when executor process is not running > anymore.
This sounds important...can you recall what would not work in that scenario? I figured it would work ~identically because the observer follows the lifecycle of the task sandbox directory. On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <ma...@apache.org> wrote: > I think we've discussed this option before. It falls apart for > terminal tasks when executor process is not running anymore. > > One of the possible ways forward could be extending Mesos UI to > opportunistically consume task data periodically dumped by an executor > into a json file. That could cover the functionality gap created by > killing the observer and let other frameworks customize their task > views in a standard and pluggable way. > > On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sniem...@apache.org> wrote: > > It seems like the easiest path forward would be to have the executor > itself > > host the observer web UI, if the HTTP port for the UI were configured as > > just another port on the task, the aurora UI could just link to /mname > for > > that instance. > > > > I think the overall "what is running on this machine" view the observer > > displays (if you go to it without a task ID) is much less useful and > could > > probably be removed without much sadness. > > > > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wfar...@apache.org> wrote: > > > >> > > >> > why don't we revisit the problem from the other direction and see if > we > >> > can remove checkpoints? > >> > >> > >> Simplicity, again :-) If it turns out we don't need the observer > anyhow, > >> it saves a lot of time. I'm just poking at different parts to make > sure we > >> can still justify their weight. > >> > >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zma...@apache.org> wrote: > >> > >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfar...@apache.org> > wrote: > >> > > >> > > We clearly have different experiences - i've never really benefited > >> from > >> > > viewing the process graph, as most jobs have very simple sequences > that > >> > > could be easily explained by a text file in the sandbox. On the > >> > contrary, > >> > > i've encountered people confused by the process graph, the observer, > >> and > >> > > sandbox browsing...so i must respectfully disagree that it is > >> universally > >> > > appreciated. > >> > > > >> > > What i'm trying to achieve is simplicity. The observer is an extra > >> > moving > >> > > part, and another thing for operators to understand and maintain. > It > >> > also > >> > > couples Aurora to one relatively specific way of running tasks, > which > >> > makes > >> > > it difficult to open new use cases like Docker tasks. Removing the > >> > > observer starts to pull on a thread of complexity that i don't think > >> > Aurora > >> > > benefits much from, for example state checkpointing by the executor. > >> > > > >> > > My goal is not to apply pressure, but to perform a gut check. If > the > >> > > answer is "No", that's fine. > >> > > > >> > > >> > > >> > Bill, > >> > > >> > I think you are pulling on the right thread here but I think > revisiting > >> the > >> > observer is the wrong way of approaching the problem. I also agree > that > >> > Aurora doesn't benefit much from state checkpointing by the executor > and > >> > the observer is an extension of that since it provides a read only > human > >> > friendly view of the data in the checkpoints. However, instead of > >> removing > >> > the observer (and degrading the UX around accessing the data in the > >> > checkpoints), why don't we revisit the problem from the other > direction > >> and > >> > see if we can remove checkpoints? > >> > > >> > > >> > -- > >> > Zameer Manji > >> > > >> >
