As a first step, I think there's consensus on increasing the test timeout to
~3x the length of the slowest test we know of. That test appears to be
TestDataFormatterObjC, which takes 388 seconds on Davide's machine. So I
propose 20 minutes as the timeout value.
Separately, regarding x-failed pexpect()-backed tests, I propose deleting them
if they've been x-failed for over a year. That seems like a long enough time to
wait for someone to step up and fix them given that they're a real
testing/maintenance burden. For any group of to-be-deleted tests, like the 22
lldb-mi tests x-failed in all configurations, I'd file a PR about potentially
bringing the tests back. Thoughts?
> On Mar 13, 2018, at 11:52 AM, Davide Italiano <dccitali...@gmail.com> wrote:
> On Tue, Mar 13, 2018 at 11:26 AM, Jim Ingham <jing...@apple.com> wrote:
>> It sounds like we timing out based on the whole test class, not the
>> individual tests? If you're worried about test failures not hanging up the
>> test suite the you really want to do the latter.
>> These are all tests that contain 5 or more independent tests. That's
>> probably why they are taking so long to run.
>> I don't object to having fairly long backstop timeouts, though I agree with
>> Pavel that we should choose something reasonable based on the slowest
>> running tests just so some single error doesn't cause test runs to just
>> never complete, making analysis harder.
> Vedant (cc:ed) is going to take a look at this as he's babysitting the
> bots for the week. I'll defer the call to him.
lldb-dev mailing list