the web-platform-tests repository includes tooling that enables running those 
tests against a supported browser product. I'd like to propose adding generic 
WebKit support there.

Current changes only assume usage of the WebDriver protocol, and the WebDriver 
binary accepting the --port flag. Selenium executors are used for test harness 
and reftests. Same WebDriver implementation can also be tested against the 
WebDriver tests included in the web-platform-tests directory, presuming the 
tests are enabled or explicitly specified.

Only port-specific bit is the specification of capabilities that are passed to 
the WebDriver binary, idea being that these capabilities are the same as those 
supported by the WebDriver implementation.

GTK is for now the only port that's supported, and it's leveraging the 
WebDriver implementation under Source/WebDriver/ in WebKit. WPE will be doing 
the same. Safari I suppose could use its own WebDriver implementation, or 
perhaps even a separate product.

Here's the current set of changes:

The web-platform-tests suite can then be run like this for the GTK port, 
assuming a tip-of-trunk build:

$ /work/web-platform-tests/wpt run --webkit-port=gtk \
    --webdriver-binary=WebKitBuild/Release/bin/WebKitWebDriver \
    --binary=WebKitBuild/Release/bin/MiniBrowser \
    --binary-arg=--automation \
    --binary-arg=--javascript-can-open-windows-automatically=true \
    --binary-arg=--enable-xss-auditor=false \
    webkit /2dcontext

This can be further wrapped into a python script and run as part of the 
continuous integration system. These changes add a run-web-platform-tests 
script that invokes the web-platform-tests runner tool, also allowing each port 
to specify what tests to enable and what the expected failures are:

Only a small subset of tests is enabled there, for prototype purposes. The 
expected results system could also be improved to avoid each expected failure 
having to be marked as such in separate .ini files.

But foremost, I'd like to have a consensus of sorts about how various WebKit 
ports should be handled in the web-platform-tests repository, so that the 
changes there can proceed -- whether it's fine to implement a generic WebKit 
product, or whether Safari would like to be treated as a separate browser[1], 


[1] There's for instance this from a year ago (though not sure about its 
webkit-dev mailing list

Reply via email to