Michael Hudson-Doyle <[email protected]> writes: > Andy Doan <[email protected]> writes: > >> On 01/14/2013 08:32 PM, Michael Hudson-Doyle wrote: >>> Apologies if this mail is a little incomprehensible. I'm brain dumping >>> in a hurry :-) >>> >>> The lava dispatcher currently communicates where the results it has >>> uploaded to the dashboard have ended up to the scheduler by having an >>> extra file descriptor open to the scheduler monitor process -- both >>> stdout and stderr are pretty messy and I didn't want to take the risk of >>> e.g. a boot message being mis-interpreted as structured data. >>> >>> As well as being silly and over-engineered, this is implicated in my >>> debugging of the "jobs don't always exit properly" bug. So let's stop >>> using it. >>> >>> What I'd like to do instead is add an --output-dir option to the >>> dispatcher, pass it from the scheduler, have the scheduler write the >>> location of the uploaded results to a known location in this output >>> directory. In fact, I've done this: >>> >>> http://bazaar.launchpad.net/~mwhudson/lava-dispatcher/output-dir/revision/530 >>> http://bazaar.launchpad.net/~mwhudson/lava-scheduler/use-dispatcher-output-dir/revision/236 >>> >>> Doesn't seem so bad really. >>> >>> This fixed the immediate problem, but I want to go further: I want to >>> store more structured log output (filtering boot logs away, for example) >>> in this output directory and gradually teach the scheduler web bits how >>> to interpret this structured data. >>> >>> Thoughts? >> >> It simplifies things and looks better, so I'm a big fan. > > Cool. > >> I have a couple of minor nitpicks with the actual patches, but that >> can wait until the MP's are created if you prefer. > > Feel free to elaborate, especially your nitpicks are structural things, > but the patches were pretty much hacks so I'm not super surprised > they're not perfect :)
I've pushed the dispatcher further in what I think is a sort of interesting direction. Branch at https://code.launchpad.net/~mwhudson/lava-dispatcher/moar-output-dir, diff to trunk at http://paste.ubuntu.com/1540064/. The basic idea is to create a few files in the specified output directory and funnel different sorts of output to different files. So in this branch we have a file for the serial output, we have a file for log messages and we have a file for "everything". There is also a way for code to say that output is not very interesting and shouldn't be sprayed to stdout (although the use of this feature in the branch is a bit silly). These choices are perhaps not very exciting, but I think the facility could be useful. I'd like to extend this to some kind of structured output file -- which would do things like point to another file that contained the master boot logs and then the scheduler could interpret this and allow the master boot logs to be hidden by default but expanded on request. Similar for things like lmc output. This is your chance to say that this is all a bit crazy and I should stop (as I'm off until next Tuesday) but I do actually like where this is going (I've been wanting to do something like this for over a year, I think). Cheers, mwh _______________________________________________ linaro-validation mailing list [email protected] http://lists.linaro.org/mailman/listinfo/linaro-validation
