On Mon, Feb 12, 2018, at 6:43 PM, youenn fablet wrote:
> Hi Zan,
> I like the idea of using WebDriver for WPT conformance testing.
> Such results will probably be more meaningful for conformance than what WTR
> or DRT could produce.
> For WPT regression testing, we would stick to using WTR/DRT and internals
> methods instead of WebDriver, am I right?
Correct, this isn't meant as a replacement for WPT regression testing, but at
some point in the future WebKit developers might agree that it can properly
But as it stands, I'm only selling it as a conformance testing process that
produces information that (in case of current failures or future regressions)
nudges WebKit towards conformance compliance.
> Le lun. 12 févr. 2018 à 07:08, <z...@falconsigh.net> a écrit :
>> the web-platform-tests repository includes tooling that enables running
>> those tests against a supported browser product. I'd like to propose adding
>> generic WebKit support there.
>> Current changes only assume usage of the WebDriver protocol, and the
>> WebDriver binary accepting the --port flag. Selenium executors are used for
>> test harness and reftests. Same WebDriver implementation can also be tested
>> against the WebDriver tests included in the web-platform-tests directory,
>> presuming the tests are enabled or explicitly specified.
>> Only port-specific bit is the specification of capabilities that are passed
>> to the WebDriver binary, idea being that these capabilities are the same as
>> those supported by the WebDriver implementation.
>> GTK is for now the only port that's supported, and it's leveraging the
>> WebDriver implementation under Source/WebDriver/ in WebKit. WPE will be
>> doing the same. Safari I suppose could use its own WebDriver implementation,
>> or perhaps even a separate product.
>> Here's the current set of changes:
>> The web-platform-tests suite can then be run like this for the GTK port,
>> assuming a tip-of-trunk build:
>> $ /work/web-platform-tests/wpt run --webkit-port=gtk \
>> --webdriver-binary=WebKitBuild/Release/bin/WebKitWebDriver \
>> --binary=WebKitBuild/Release/bin/MiniBrowser \
>> --binary-arg=--automation \
>> --binary-arg=--enable-xss-auditor=false \
>> webkit /2dcontext
>> This can be further wrapped into a python script and run as part of the
>> continuous integration system. These changes add a run-web-platform-tests
>> script that invokes the web-platform-tests runner tool, also allowing each
>> port to specify what tests to enable and what the expected failures are:
>> Only a small subset of tests is enabled there, for prototype purposes. The
>> expected results system could also be improved to avoid each expected
>> failure having to be marked as such in separate .ini files.
>> But foremost, I'd like to have a consensus of sorts about how various
>> WebKit ports should be handled in the web-platform-tests repository, so that
>> the changes there can proceed -- whether it's fine to implement a generic
>> WebKit product, or whether Safari would like to be treated as a separate
>> browser, etc.
>>  There's for instance this from a year ago (though not sure about its
>> webkit-dev mailing list
webkit-dev mailing list