I write a lot of web platform tests as a Web Platform engineer; recently I wrote one in external/wpt/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/external/wpt/> (the external web platform tests directory), and was shocked to find the eradication of `-expected.txt` files. I placed my expectations file next to the source file as we've done for many years, and found that my test was "failing" because the test runner couldn't find my test expectations file.
I dug deeper and found https://crrev.com/c/3603221 which was responsible for moving more than 21,000 *platform-agnostic* test expectations files away from their source files and into web_tests/platform/generic <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/platform/generic/> directory. I found more discussion in this email thread <https://groups.google.com/a/chromium.org/g/blink-dev/c/0WmmgEkqdOo> which I missed because blink-dev emails do not go directly in my inbox. I must say I find this change extraordinarily inconvenient as a Web Platform engineer, and I want to push back against this. A minority of web platform tests have platform-specific failures, which justifies the need for *some* platform-specific test expectations directories, but I believe a huge majority have generic baselines that are wildly convenient to have right next to the actual tests themselves. Putting them in a separate directory means I and others have to open a separate browser tab to view how many expectations there are for a given directory, and requires a lot of unnecessary context switching. It is particularly confusing for clusters of tests whose names are all *very* similar and vary by only a few numbers or suffixes—this increases the cost of the context switching. Furthermore, it renders tons of directories absolutely useless! All ~150 directories in web_tests/virtual <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/virtual/> (for VirtualTestSuites) are just empty directories with README files—these used to house virtualtest-specific expectations. So now for fenced frames (the project I'm working on right now), we have the following test directories: - web_tests/wpt_internal/fenced_frame/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/wpt_internal/fenced_frame/> - web_tests/virtual/fenced-frame-mparch/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/virtual/fenced-frame-mparch/> - web_tests/virtual/fenced-frame-shadow-dom/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/virtual/fenced-frame-shadow-dom/> - web_tests/platform/generic/wpt_internal/fenced_frame/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/platform/generic/wpt_internal/fenced_frame/> - web_tests/platform/generic/virtual/fenced-frame-mparch/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/platform/generic/virtual/fenced-frame-mparch/> - web_tests/platform/generic/virtual/fenced-frame-shadow-dom/ <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/web_tests/platform/generic/virtual/fenced-frame-shadow-dom/> - + platform-specific directories, which are relatively rare for us This is so weird! Regardless of whether or not there are plans to clean this up, I can't see the upsides. The ousting of platform-agnostic expectations is only an inconvenience for WP engineers, while there might be some test-infra conveniences around BUILD.gn dependencies <https://bugs.chromium.org/p/chromium/issues/detail?id=1299834#:~:text=This%20way%20all%20the%20baselines%20will%20be%20in%20platform%20directory%2C%20make%20it%20a%20little%20bit%20easier%20to%20specify%20dependency%20in%20BUILD.gn.> (maybe?). In any case, I am hard pressed to find justification in this move, and would love to see if we can reconsider this. Thoughts? Dom -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAP-uykAN06y5o-WYznnicvm1YREbSsLbs6dM57LtL4vCWB%3Duzw%40mail.gmail.com.
