On 10/25/07, Dan Kegel <[EMAIL PROTECTED]> wrote: > James wrote: > >Looking at the test data, all of the msi:install tests timeout. I > >just ran the install tests in XP running under vmware on a 3ghz > >machine. The tests took 9m41s. That completely blows away the 2min > >timeout. There's nothing wrong with the tests, they just take a long > >time. I don't think we should extend the timeout, because it's very > >subjective and more tests will be added, meaning we'll have to change > >the timeout eventually. I do think we should have a flag or variable > >that allows the timeout to be ignored for certain tests. Any > >opinions? > > I'd like a way to specify the expected runtime, > for use on tests that we observe take a long time. > It would be used in two ways: > 1) runtest could take an option --skip-long-tests > which would skip all tests that had that option set, and > 2) by default runtest would only abort those tests > if they took one minute over their expected time (say). > > That would give us both a quick interactive make test > and a more reliable slow-but-complete make test. > - Dan >
I don't think that's fair to long tests, say msi:install. There will always be people that don't want to wait for the tests, and thus the long tests get less exposure. In the case of msi:install, there's technically nothing wrong with the tests, so it should get just as much exposure as any other test. Just today I noticed a bug in msi:install that I didn't know was there before because the tests were timing out. Like I said before, we shouldn't specify a limit for certain tests like msi:install because they'll always get longer and we'll always be chasing a moving target. -- James Hawkins
