On Tue, Aug 17, 2021 at 10:08 AM 'Weizhong Xia' via blink-dev < [email protected]> wrote:
> tl;dr: We introduced a mechanism that pre-collects test cases to > //third_party/blink/web_tests/AllTestsByDirectories.json, and uses a > presubmit check to ensure this file is up to date. You can stop here if you > don't write web test cases. > > Dear blink-devs, > > You may have noticed that when you try to upload a CL, you are prompted to > update AllTestsByDirectories.json if your CL changes web test cases. Yes, > today we landed a CL > <https://chromium-review.googlesource.com/c/chromium/src/+/3067372> that > pre-collects test cases and saves that to the json file. The reason to do > this is that rwt tries to collect tests on every swarming bots, and that > takes about 30 seconds (the best case when running on SSD). This is not a > small amount of time as we are targeting a 10-min CQ. After this CL lands, > collecting tests now takes less than 2 seconds for all platforms. > > Impact to you: if your CL adds/deletes/renames web test cases, you will be > prompted. Follow the instructions there should be enough. But if you still > have an issue, (even though we have tested many different scenarios we can > think about), feel free to ping/email me. > > There is no change to how you run rwt locally. We added a new switch to > tell the swarming bots to load tests from the json file. This switch is > turned off by default. Developers usually don't need to turn this on, > unless all of your tests are saved to the json file and you are running a > full test so want to save about 30 seconds. > FWIW, I think this is an important and probably worthwhile change (disclaimer: I did review it and was involved in the design). The number of tests we have now is substantial and the startup delay is a significant cost. We need to be looking into ways to reduce this. I think it's possible we should turn this on by default and/or do some hooking to make things aware of when you've added new tests (e.g., via git status) to try and make this a little more seamless, but I also think landing it off by default was a good first step. Some may ask if we can just generate this at compile time. Unfortunately, no, we can't, because there's no way to keep it correctly up-to-date when you do add tests after having generated the file at least once [ we asked this ourselves and actually tried this approach and broke deterministic builds :( ]. I'm happy to hear other thoughts. Perhaps there are other ways we can optimize things to be fast enough ... -- Dirk > > Thanks for your time! > > Cheers, Weizhong > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSiprRWy_UCC0w2oohbSGGTbeP%3DdkyYnBOx7j8Y-abDSe2A%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXrSiprRWy_UCC0w2oohbSGGTbeP%3DdkyYnBOx7j8Y-abDSe2A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEoffTAc389E6wteVXUAuKX8ZBFWBa2UEMwAbd9oCxJ9pxNCKQ%40mail.gmail.com.
