If it's not too much work, I think the extra bit of noise will not be a problem. But I don't think it is really necessary either.
I assume the actual flip will be a small change that we can back out easily if we notice troubles... After a sufficient grace period we can remove the old formatter altogether and hopefully simplify the code somewhat. pl On 9 December 2015 at 17:08, Todd Fiala <todd.fi...@gmail.com> wrote: > Here's what I can do. > > Put in the change (setting the default to use the new format). > > Separately, put in a trial balloon commit with one failing test, one > exceptional exit test, and one timeout test, and watch the ubuntu 14.04 > buildbot catch it and fail. Then reverse this out. That should show beyond > a reasonable doubt whether the buildbot catches new failures and errors. (I > think this is a noisy way to accomplish this, but it certainly would > validate if its working). > > -Todd > > On Wed, Dec 9, 2015 at 8:06 AM, Todd Fiala <todd.fi...@gmail.com> wrote: >> >> Specifically, the markers for issue details are: >> >> FAIL >> ERROR >> UNEXPECTED SUCCESS >> TIMEOUT >> >> (These are the fourth field in the array entries (lines 275 - 290) of >> packages/Python/lldbsuite/test/basic_results_formatter.py). >> >> -Todd >> >> On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala <todd.fi...@gmail.com> wrote: >>> >>> That's a good point, Tamas. >>> >>> I use (so I claim) the same all upper-case markers for the test result >>> details. Including, not using XPASS but rather UNEXPECTED SUCCESS for >>> unexpected successes. (The former would trigger the lit script IIRC to >>> parse that as a failing-style result). >>> >>> The intent is this is a no-op on the test runner. >>> >>> On Wed, Dec 9, 2015 at 8:02 AM, Tamas Berghammer <tbergham...@google.com> >>> wrote: >>>> >>>> +Ying Chen >>>> >>>> Ying, what do we have to do on the build bot side to support a change in >>>> the default test result summary formatter? >>>> >>>> On Wed, Dec 9, 2015 at 4:00 PM Todd Fiala via lldb-dev >>>> <lldb-dev@lists.llvm.org> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Per a previous thread on this, I've made all the changes I intended to >>>>> make last night to get the intended replacement of test run results meet >>>>> or >>>>> exceed current requirements. >>>>> >>>>> I'd like to switch over to that by default. I'm depending on the test >>>>> event system to be able to handle test method reruns in test results >>>>> accounting. >>>>> >>>>> The primary thing missing before was that timeouts were not routed >>>>> through the test events system, nor were exception process exits (i.e. >>>>> test >>>>> inferiors exiting with a signal on POSIX systems). Those were added last >>>>> night so that test events are generated for those, and the >>>>> BasicResultsFormatter presents that information properly. >>>>> >>>>> I will switch it over to being the default output in a bit here. >>>>> Please let me know if you have any concerns once I flip it on by default. >>>>> >>>>> Thanks! >>>>> -- >>>>> -Todd >>>>> _______________________________________________ >>>>> lldb-dev mailing list >>>>> lldb-dev@lists.llvm.org >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>> >>> >>> >>> >>> -- >>> -Todd >> >> >> >> >> -- >> -Todd > > > > > -- > -Todd _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev