There is no active plan to export or migrate the WebGPU CTS into WPT. It's
something we'd theoretically like to do in the long run, but there are a
lot of blockers:
- If we just export without moving it, then browser-to-wpt auto-export will
not be usable, so I would want to move it.
- TypeScript meaningfully enables our test development and we would not
want to strip it from the code, so we would want to somehow support that
inside WPT. (We actually had an idea involving running babel in a service
worker that could make this work now, but it's something we wouldn't want
to do just for WebGPU.)
- Each browser's wpt-to-browser auto-import will need to run the WebGPU
tests on all hardware/software configurations used on that browser's CQ
equivalent, and be able to collect the test results and update expectations
files automatically. Without this, auto-imports will frequently be unable
to import without breaking the build.
- Chromium doesn't actually run most of the WebGPU CTS through
WPT/web_tests anymore - only reftests still use web_tests. For numerous
reasons, it was more practical to run under Chromium's GPU integration test
framework. Other browsers are integrating the tests in whatever way is most
practical for them (for example WebKit can't run it under their WPT runner
using the WPT "variants" feature - they need it split up into files or
somehow baked into the WPT manifest).
- The test tree is very large and deep and test runtimes are highly
variable, we needed a heartbeat mechanism for timeouts, which WPT does not
have.
- We needed expectations
<https://source.chromium.org/chromium/chromium/src/+/main:third_party/dawn/webgpu-cts/expectations.txt>
to depend on the GPU hardware and various configurations of Chromium's
graphics and WebGPU stack, which we already had in that framework.
- It uses the real browser instead of content_shell: tests what we
ship, and we also had difficult-to-debug flakiness somehow relating to GPU
initialization in content_shell.
-Kai (he/they)
On Tue, Feb 7, 2023 at 5:28 AM Corentin Wallez <[email protected]> wrote:
> Hey Rego,
>
> The WebGPU CTS is meant to be exportable into a WPT subdirectory, see
> https://github.com/gpuweb/cts/blob/main/docs/build.md. However I don't
> know that there's a specific plan to integrate into WPT proper since
> development uses a lot of combination testing and Typescript, which is not
> something WPT support (the export step creates individual HTML pages for
> each test case and compiles the Typescript to Javascript). +Kai Ninomiya
> <[email protected]>, feel free to correct me if I missed something.
>
> Cheers,
>
> Corentin
>
> On Mon, Feb 6, 2023 at 5:55 PM Manuel Rego Casasnovas <[email protected]>
> wrote:
>
>>
>>
>> On 14/12/2022 18:02, Corentin Wallez wrote:
>> > The WebGPU Conformance Test Suite is being built
>> > at https://github.com/gpuweb/cts <https://github.com/gpuweb/cts> and
>> can
>> > be integrated as a subdirectory of WPT. Coverage is still incomplete due
>> > to the complexity of the API but progressing quickly. We expect to ship
>> > with coverage holes, but with most important and risky aspects of
>> > interoperability well tested.
>>
>> Any plans about integrating the test suite into WPT?
>>
>> Thanks,
>> Rego
>>
>
--
You received this message because you are subscribed to the Google Groups
"blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANxMeyD6G97bLgnCFRpqMCmkXhtkpkV9rP7wk2ByoiGgMznoSw%40mail.gmail.com.
