> > There is no process to host the observer UI when a task terminates.
Aha, i didn't realize that was in reference to Steve's comment "have the executor itself host the observer web UI". I agree, that approach is encumbered for terminal tasks. On Mon, Apr 4, 2016 at 5:10 PM, Maxim Khutornenko <ma...@apache.org> wrote: > Sorry, I should have tried harder explaining myself. There is no > process to host the observer UI when a task terminates. We still want > (and arguably more so) to look at terminal task details but since > there is no process left to host the http server, there is no way to > access that data in the current way of things. > > On Mon, Apr 4, 2016 at 3:54 PM, Bill Farner <wfar...@apache.org> wrote: > >> > >> It falls apart for terminal tasks when executor process is not running > >> anymore. > > > > > > This sounds important...can you recall what would not work in that > > scenario? I figured it would work ~identically because the observer > > follows the lifecycle of the task sandbox directory. > > > > On Mon, Apr 4, 2016 at 2:43 PM, Maxim Khutornenko <ma...@apache.org> > wrote: > > > >> I think we've discussed this option before. It falls apart for > >> terminal tasks when executor process is not running anymore. > >> > >> One of the possible ways forward could be extending Mesos UI to > >> opportunistically consume task data periodically dumped by an executor > >> into a json file. That could cover the functionality gap created by > >> killing the observer and let other frameworks customize their task > >> views in a standard and pluggable way. > >> > >> On Mon, Apr 4, 2016 at 2:31 PM, Steve Niemitz <sniem...@apache.org> > wrote: > >> > It seems like the easiest path forward would be to have the executor > >> itself > >> > host the observer web UI, if the HTTP port for the UI were configured > as > >> > just another port on the task, the aurora UI could just link to /mname > >> for > >> > that instance. > >> > > >> > I think the overall "what is running on this machine" view the > observer > >> > displays (if you go to it without a task ID) is much less useful and > >> could > >> > probably be removed without much sadness. > >> > > >> > On Mon, Apr 4, 2016 at 5:23 PM, Bill Farner <wfar...@apache.org> > wrote: > >> > > >> >> > > >> >> > why don't we revisit the problem from the other direction and see > if > >> we > >> >> > can remove checkpoints? > >> >> > >> >> > >> >> Simplicity, again :-) If it turns out we don't need the observer > >> anyhow, > >> >> it saves a lot of time. I'm just poking at different parts to make > >> sure we > >> >> can still justify their weight. > >> >> > >> >> On Mon, Apr 4, 2016 at 1:54 PM, Zameer Manji <zma...@apache.org> > wrote: > >> >> > >> >> > On Mon, Apr 4, 2016 at 1:47 PM, Bill Farner <wfar...@apache.org> > >> wrote: > >> >> > > >> >> > > We clearly have different experiences - i've never really > benefited > >> >> from > >> >> > > viewing the process graph, as most jobs have very simple > sequences > >> that > >> >> > > could be easily explained by a text file in the sandbox. On the > >> >> > contrary, > >> >> > > i've encountered people confused by the process graph, the > observer, > >> >> and > >> >> > > sandbox browsing...so i must respectfully disagree that it is > >> >> universally > >> >> > > appreciated. > >> >> > > > >> >> > > What i'm trying to achieve is simplicity. The observer is an > extra > >> >> > moving > >> >> > > part, and another thing for operators to understand and maintain. > >> It > >> >> > also > >> >> > > couples Aurora to one relatively specific way of running tasks, > >> which > >> >> > makes > >> >> > > it difficult to open new use cases like Docker tasks. Removing > the > >> >> > > observer starts to pull on a thread of complexity that i don't > think > >> >> > Aurora > >> >> > > benefits much from, for example state checkpointing by the > executor. > >> >> > > > >> >> > > My goal is not to apply pressure, but to perform a gut check. If > >> the > >> >> > > answer is "No", that's fine. > >> >> > > > >> >> > > >> >> > > >> >> > Bill, > >> >> > > >> >> > I think you are pulling on the right thread here but I think > >> revisiting > >> >> the > >> >> > observer is the wrong way of approaching the problem. I also agree > >> that > >> >> > Aurora doesn't benefit much from state checkpointing by the > executor > >> and > >> >> > the observer is an extension of that since it provides a read only > >> human > >> >> > friendly view of the data in the checkpoints. However, instead of > >> >> removing > >> >> > the observer (and degrading the UX around accessing the data in the > >> >> > checkpoints), why don't we revisit the problem from the other > >> direction > >> >> and > >> >> > see if we can remove checkpoints? > >> >> > > >> >> > > >> >> > -- > >> >> > Zameer Manji > >> >> > > >> >> > >> >
