On Mon, Feb 10, 2025 at 11:01 AM Weizhong Xia <weizh...@google.com> wrote:
> Looking into the log I see a lot of tests timed out. Maybe that is the > reason why the total test time increased a lot. > Just to double-check: *Have you also looked at mac-rel <https://ci.chromium.org/ui/p/chromium/builders/try/mac-rel/2079605/overview> step 164 ("headless_shell_wpt_tests (without patch) on Mac-14") which is red *without* my patch?* Is it also red because of test timeouts? FWIW I see that stdout <https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8723633334109810449/+/u/headless_shell_wpt_tests__without_patch__on_Mac-14/stdout> of this step reports only 2 timeouts: $ cat stdout | grep TIMEOUT.*expected ▶ TIMEOUT [expected OK] external/wpt/svg/interact/scripted/focus-events.svg ▶ TIMEOUT [expected OK] external/wpt/compute-pressure/compute_pressure_disconnect_idempotent.https.window.html?globalScope=window > Can you check if your change has slowed down the test? I see on mac-rel > the total test time for headless_shell_wpt_tests is 352 min, while the P50 > now is 190 min. This is similarly true for linux-rel and win-rel. > *There are indeed some timeouts, but much less than the "1450 failures"* reported at the top of the mac-rel results. In stdout <https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8723633334109810449/+/u/headless_shell_wpt_tests__with_patch__on_Mac-14/stdout?format=raw> of step 130 I see 133 timeouts: $ cat stdout\?format\=raw | grep TIMEOUT.*expected | wc -l grep: (standard input): binary file matches *I expect that my CL will have some impact on performance of PNG decoding and encoding.* Looking at the (Google-internal link) Finch/UMA dashboard at https://uma.googleplex.com/p/chrome/variations?sid=d8ca164209cfbd8c7cff6fd2574a03bb, I indeed see some slowdown in PNG decoding speed as reported by Renderer4.ImageDecodeTaskDurationUs.Png.Software and ImageDecoder.Png.UiGfxIntoSkBitmap. The encoding speed seems to be neutral or improved as reported by Blink.Canvas.ToDataURLScaledDuration.PNG and Blink.Canvas.ToBlob.TotalEncodingDelay.PNG. OTOH, so far I have not been able to reproduce the same magnitude of the slowdown in local decoding benchmarks - see https://docs.google.com/document/d/12gj3dnGyc9ZPvsu-53gTN38WneOnF_QddJ5WvsiizXg/edit?usp=sharing *Nevertheless, I do *not* expect my CL to almost double the test time*. The PNG decoding and encoding slowdown I observed in Canary trials has been surprisingly high, but it still shouldn't cause such a dramatic increase in test time, because I expect that PNG decoding/encoding time should be a small percentage of the total test runtime (at least for most tests). *I cannot repro the timeouts locally. *For example, win-rel <https://ci.chromium.org/ui/p/chromium/builders/try/win-rel/907626/overview> reported <https://chromium-layout-test-archives.storage.googleapis.com/results.html?json=chromium/try/win-rel/907626/headless_shell_wpt_tests%20%28with%20patch%29/full_results_jsonp.js> a timeout in external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html; and stdout <https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8723633334109810449/+/u/headless_shell_wpt_tests__with_patch__on_Mac-14/stdout> of step 130 of the massively-failing mac-rel <https://ci.chromium.org/ui/p/chromium/builders/try/mac-rel/2079605/overview> seems to report a timeout of virtual/scalefactor200/external/wpt/css/css-backgrounds/hidpi/simple-bg-color.html. And I can't repro the timeout in the first test on my gWindows CloudTop (I tried the second test on my gLinux CloudTop - not quite matching the bot setup but I don't have a Mac to test with.) I also tried to measure the test duration on gWindows CloudTop with and without enabling my feature (I used `run_web_tests.bat`, because I didn't find an equivalent of `--timing` flag in the output of `run_wpt_tests.bat --help`) - the feature doesn't seem to have a big effect on the test runtime: With my feature the average test runtime is 0.68712s: C:\src\chromium\src>third_party\blink\tools\run_web_tests.bat -t rel external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html --iterations=10 --timing --verbose ... [1/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.7871s [2/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6953s [3/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6670s [4/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6659s [5/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6802s [6/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6816s [7/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6650s [8/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6680s [9/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6823s [10/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6788s Finally stop servers and clean up All 10 tests ran as expected in 50.72s (49.42s in rwt, 1x). Without my feature (misspelling the `enabled_features` entry in `fieldtrial_testing_config.json` and recompiling via `autoninja -C out\rel blink_tests`) the average runtime is 0.6703s: C:\src\chromium\src>third_party\blink\tools\run_web_tests.bat -t rel external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html --iterations=10 --timing --verbose ... [1/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6939s [2/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6660s [3/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6523s [4/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6504s [5/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.7513s [6/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6651s [7/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6514s [8/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6489s [9/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6636s [10/10] external/wpt/css/geometry/DOMMatrix2DInit-validate-fixup.html passed 0.6601s Finally stop servers and clean up All 10 tests ran as expected in 50.43s (49.13s in rwt, 1x). > > Please always feel free to reach out if you need help, :-). > Thanks for taking a look! > > thanks, Weizhong > > > > On Mon, Feb 10, 2025 at 10:06 AM Xianzhu Wang <wangxian...@chromium.org> > wrote: > >> I believe this is an infra issue. I've moved the bug to Blink>Infra. >> >> On Mon, Feb 10, 2025 at 9:14 AM Łukasz Anforowicz <luka...@chromium.org> >> wrote: >> >>> Hello @blink-dev, >>> >>> I would appreciate help with understanding why >>> `headless_shell_wpt_tests` step is red in my tryjobs (presumably because of >>> my CL which enables Rust PNG via `fieldtrial_testing_config.json`?). In >>> https://crbug.com/395446442 I link to an example where `mac-rel` is >>> red, and says things like "result unexpectedly crashed" or "1450 failures", >>> but I am unable to find test logs that would show a callstack of a crash, >>> or a specific test assertion that fails. >>> >>> Did I miss something when looking at the tryjob results? Maybe I should >>> ask the infra team for help with this? >>> >>> Best regards, >>> >>> Lukasz >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAA_NCUFV7MgpXgK9PgaNFBrFtqBNYAGeYWf7F3t3-iAjhCrb9w%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAA_NCUFV7MgpXgK9PgaNFBrFtqBNYAGeYWf7F3t3-iAjhCrb9w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAA_NCUHJNy4aiJ7KMuWibNOy2JRACxiq7r5MMVx8Z48eJXwBsQ%40mail.gmail.com.
