On Wed, Aug 18, 2021 at 12:01 PM 'Weizhong Xia' via blink-dev < [email protected]> wrote:
> Dear blink-devs, > > As another effort trying to speed up blink_web_tests on CQ, We are looking > at slow tests that also timeouts. From the timing stats(example > <https://test-results.appspot.com/data/layout_results/linux-rel/773524/blink_web_tests%20%28with%20patch%29/layout-test-results/results.html>) > and some research, there are about 0.25% such tests, but running such tests > takes about 4% of the total run time. So we are thinking of > doing something about it. > > We would like to propose as below, and get feedback on this: > 1. We add Skip to test expectations for slow tests that also timeouts. > This will effectively skip all such tests on swarming bots, and save ~50s > real time on linux-rel. > 2. We add a presubmit check to make any unskipped Slow+Timeout an error. > (We can also make this a warning, but such warning will persist, so kind of > annoying) > 3. Devs are encouraged to take actions to either fix the timeout, or break > the slow tests into pieces to make it not slow. > > I recently found a test that was TIMEOUT in TestExpectations, and still timed out in Slow, so I left it in TestExpectations under the assumption that it will timeout faster there. The underlying issue is not a slow test but a test that hangs and we may or may not want to skip such a test. Would identifying these cases and moving them from Slow to TestExpectations have any noticeable impact? > Please let us know your opinions on this. > > Thanks, Weizhong > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSipU3NDSp7tWd%3DqA%3DrZWMrmXjfT4NzTG7U8EoEAgsQ1QXw%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSipU3NDSp7tWd%3DqA%3DrZWMrmXjfT4NzTG7U8EoEAgsQ1QXw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGsbWzTW6WD9bZk0xaW%2B3CynEkARLXjSNiLYNMojoy8N7%2Bto7A%40mail.gmail.com.
