Bryan Stearns wrote: > I'd like to see the native Python logging mechanism used, since we > already use it for other debug output in Chandler; output that needs to > be found by Tinderbox could be logged by its own logger object, which > would make parsing it much less prone to false-positive errors (the > lines are in a well-defined format, so finding the lines produced by one > logger is easy); this would also make diagnosis of failures easier (the > big problem with the current output) because all the output (including > regular debug output) would be combined in correct chronological order > (so we wouldn't have to correlate multiple logs).
There might be something here that we could pursue. But I also realized I did not explain everything we do and one thing I said was confusing... chandler.log (and twisted.log, although we probably don't include it currently on Tbox) are produced by the python logger. One other kind of output in the full log is produce by our Tinderbox client script, and we obviously have control over how that is formatted. Unit test output is currently in a transitional mode, and at least currently not very much under our control. It might be a lot of work, but somebody would need to take a look. Functional test output is totally under our control, and switched to the new cats format pretty recently. Unfortunately it outputs many lines that get recognized by the Tbox log parser as error lines even though they are not errors. Performance tests can fail as well; these are transitioning to new cats at some point as well. Finally there is a lot of output we don't really have much control over, which comes from make, compiler, etc. The parser we use is the generic 'unix error parser' that comes with Tinderbox. I don't know how difficult it would be to change that. In fact, I'd rather not change it unless absolutely necessary, because maintaining a hacked tbox is not fun. The confusing thing I said about concatenated logs... Each tbox client concatenates the logs from a single run for the email, but the Tinderbox server maintains each email (or build report, rather) separately. > This sounds like an easy thing to scan for (a line starting with "exit > code=" immediately after the 'starting functional tests' line). And we could actually make that say: error exit code=1 which would be recognized as failure by the Tbox parser and you'd get a link to that location. > Alternatively, the thing that concatenates everything could notice when > a log that's supposed to be non-empty is empty or nonexistant, and > substitute an easily found error message instead. Yeah, I think we should output an error message in place of a log if we expected the log to be there. > This case also sounds readily identifiable, even though "all tests > passed" followed by "tests failed" looks confusing. If the tinderbox > doesn't panic when this case is identified, I won't either. :-) It doesn't panic, and it correctly reports test failure. It is just confusing to read the log. It would be clearer if we could say 'error tests failed because python exit code=XXX'. I'll see if I can make the changes for the last three things. -- Heikki Toivonen
signature.asc
Description: OpenPGP digital signature
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev
