Re: [webkit-dev] Exporting WPT tests
21.12.2017, 04:29, "youenn fablet" : > Hi all, > > Just to let you know that a new script landed in WebKit: > Tools/Scripts/export-w3c-test-changes. > Please have a look if you are interested in and ping me if you have any > question or suggestion. > This script can: > - Push your LayoutTests/imported/w3c/web-platform-tests changes made within a > WebKit repository to your own GitHub WPT clone > - Create a PR on W3C WPT repo and add a comment on the related WebKit > bugzilla (-c option) > > There is some information that is needed the first time: your GitHub > username, a GitHub token to allow the script to create a PR on your behalf. It might be useful to add GitHub usernames to contributors.json > Script documentation might help. > > One potential workflow could be something like: > 1. Author a webkit patch > 2. Upload patch to bugzilla > 3. At the time patch is set to "r?" on bugzilla, use the script to create a > W3C WPT PR. > When the patch is "r+", merge the W3C WPT PR and land the WebKit patch. > > Merry Christmas, > y > , > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev -- Regards, Konstantin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Exporting WPT tests
Hi all, Just to let you know that a new script landed in WebKit: Tools/Scripts/export-w3c-test-changes. Please have a look if you are interested in and ping me if you have any question or suggestion. This script can: - Push your LayoutTests/imported/w3c/web-platform-tests changes made within a WebKit repository to your own GitHub WPT clone - Create a PR on W3C WPT repo and add a comment on the related WebKit bugzilla (-c option) There is some information that is needed the first time: your GitHub username, a GitHub token to allow the script to create a PR on your behalf. Script documentation might help. One potential workflow could be something like: 1. Author a webkit patch 2. Upload patch to bugzilla 3. At the time patch is set to "r?" on bugzilla, use the script to create a W3C WPT PR. When the patch is "r+", merge the W3C WPT PR and land the WebKit patch. Merry Christmas, y ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Exporting WPT tests
On Fri, Apr 28, 2017 at 8:24 AM, Mike Pennisi wrote: > Hi Youenn. My name is Mike, and I've been working with Google for the past 4 > months or so to improve various aspects of the Web Platform Tests project > (more > on that here [1]). > >> The only constraint I know of is that the test does not give flaky tests >> from >> WPT Chrome/Firefox bots. > > The full set of validation steps are described in the project's > `.travis.yml` > file [2]. That's a bit tough to read even if you're familiar with TravisCI > (we're > working on it!), but from WebKit's perspective, the only other relevant > check > is for file linting. It's not very opinionated (mostly limited to objective > concerns) but still something to be aware of. We can probably integrate W3C's lint into our style checker. > I'm wondering if we can avoid duplicating effort by making a standalone > tool. > It might even be the kind of thing we could host in the W3C GitHub > organization--whose to say that Edge (for example) wouldn't benefit from > that, > too? I would love to be involved in that implementation. That would be fantastic. I suppose each vendor can configure the scripts for their own needs? - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Exporting WPT tests
Jeff has just created a document to explore what this tool might look like: https://bugs.chromium.org/p/chromium/issues/detail?id=691653#c3 Youenn, this sounds like it's right up your alley! On Fri, Apr 28, 2017 at 11:44 AM, youenn fablet wrote: > Hi Mike, > > Thanks for the information. > It is really great to see Safari be integrated in the bots :) > https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py > seems like a really good potential candidate for WPT upstream. > y > > Le ven. 28 avr. 2017 à 08:25, Mike Pennisi a écrit : >> >> Hi Youenn. My name is Mike, and I've been working with Google for the past >> 4 >> months or so to improve various aspects of the Web Platform Tests >> project (more >> on that here [1]). >> >> > The only constraint I know of is that the test does not give flaky >> tests from >> > WPT Chrome/Firefox bots. >> >> The full set of validation steps are described in the project's >> `.travis.yml` >> file [2]. That's a bit tough to read even if you're familiar with >> TravisCI (we're >> working on it!), but from WebKit's perspective, the only other relevant >> check >> is for file linting. It's not very opinionated (mostly limited to >> objective >> concerns) but still something to be aware of. >> >> Also note that we're very close to including both Edge and Safari in the >> set of >> browsers used to identify flaky tests! [3] >> >> >> > We do not have yet the tooling to automate the creation of a WPT >> GitHub PR >> > from a WebKit patch that lands. >> >> I've recently been migrating tests for Service Workers from the Chromium >> project to WPT. The process in place there is pretty slick. (Context for >> other >> folks on the list: it's able to create commits that exclude >> Chromium-specific >> files [4] and then submit GitHub pull requests from those, merging when CI >> passes [5]. The patch Youenn mentioned is based on those files.) >> >> I'm wondering if we can avoid duplicating effort by making a standalone >> tool. >> It might even be the kind of thing we could host in the W3C GitHub >> organization--whose to say that Edge (for example) wouldn't benefit from >> that, >> too? I would love to be involved in that implementation. >> >> But I'm getting ahead of myself :) I've CC'd Jeff Carp and Quinten >> Yearsley of >> the Chromium team since they are currently working with that tooling. >> >> So what do you folks think? Would it be practical to share code like this? >> >> [1] https://bocoup.com/blog/diving-into-the-web-platform-tests >> [2] https://github.com/w3c/web-platform-tests/blob/master/.travis.yml >> [3] https://github.com/w3c/web-platform-tests/pull/5231 >> [4] >> >> https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit.py >> [5] >> >> https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py >> >> ___ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Exporting WPT tests
Hi Mike, Thanks for the information. It is really great to see Safari be integrated in the bots :) https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py seems like a really good potential candidate for WPT upstream. y Le ven. 28 avr. 2017 à 08:25, Mike Pennisi a écrit : > Hi Youenn. My name is Mike, and I've been working with Google for the past > 4 > months or so to improve various aspects of the Web Platform Tests > project (more > on that here [1]). > > > The only constraint I know of is that the test does not give flaky > tests from > > WPT Chrome/Firefox bots. > > The full set of validation steps are described in the project's > `.travis.yml` > file [2]. That's a bit tough to read even if you're familiar with > TravisCI (we're > working on it!), but from WebKit's perspective, the only other relevant > check > is for file linting. It's not very opinionated (mostly limited to objective > concerns) but still something to be aware of. > > Also note that we're very close to including both Edge and Safari in the > set of > browsers used to identify flaky tests! [3] > > > We do not have yet the tooling to automate the creation of a WPT > GitHub PR > > from a WebKit patch that lands. > > I've recently been migrating tests for Service Workers from the Chromium > project to WPT. The process in place there is pretty slick. (Context for > other > folks on the list: it's able to create commits that exclude > Chromium-specific > files [4] and then submit GitHub pull requests from those, merging when CI > passes [5]. The patch Youenn mentioned is based on those files.) > > I'm wondering if we can avoid duplicating effort by making a standalone > tool. > It might even be the kind of thing we could host in the W3C GitHub > organization--whose to say that Edge (for example) wouldn't benefit from > that, > too? I would love to be involved in that implementation. > > But I'm getting ahead of myself :) I've CC'd Jeff Carp and Quinten > Yearsley of > the Chromium team since they are currently working with that tooling. > > So what do you folks think? Would it be practical to share code like this? > > [1] https://bocoup.com/blog/diving-into-the-web-platform-tests > [2] https://github.com/w3c/web-platform-tests/blob/master/.travis.yml > [3] https://github.com/w3c/web-platform-tests/pull/5231 > [4] > > https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit.py > [5] > > https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Exporting WPT tests
Hi Youenn. My name is Mike, and I've been working with Google for the past 4 months or so to improve various aspects of the Web Platform Tests project (more on that here [1]). > The only constraint I know of is that the test does not give flaky tests from > WPT Chrome/Firefox bots. The full set of validation steps are described in the project's `.travis.yml` file [2]. That's a bit tough to read even if you're familiar with TravisCI (we're working on it!), but from WebKit's perspective, the only other relevant check is for file linting. It's not very opinionated (mostly limited to objective concerns) but still something to be aware of. Also note that we're very close to including both Edge and Safari in the set of browsers used to identify flaky tests! [3] > We do not have yet the tooling to automate the creation of a WPT GitHub PR > from a WebKit patch that lands. I've recently been migrating tests for Service Workers from the Chromium project to WPT. The process in place there is pretty slick. (Context for other folks on the list: it's able to create commits that exclude Chromium-specific files [4] and then submit GitHub pull requests from those, merging when CI passes [5]. The patch Youenn mentioned is based on those files.) I'm wondering if we can avoid duplicating effort by making a standalone tool. It might even be the kind of thing we could host in the W3C GitHub organization--whose to say that Edge (for example) wouldn't benefit from that, too? I would love to be involved in that implementation. But I'm getting ahead of myself :) I've CC'd Jeff Carp and Quinten Yearsley of the Chromium team since they are currently working with that tooling. So what do you folks think? Would it be practical to share code like this? [1] https://bocoup.com/blog/diving-into-the-web-platform-tests [2] https://github.com/w3c/web-platform-tests/blob/master/.travis.yml [3] https://github.com/w3c/web-platform-tests/pull/5231 [4] https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit.py [5] https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Exporting WPT tests
Hi all, After talking with some WebKittens, I'd like to discuss the process of upstreaming WPT tests from WebKit. Here is my take of it. WPT tests need a review before being merged. This review can be done in WPT GitHub at PR time. This review can also be done as part of the usual WebKit review flow. As done by other browser teams, a test that is reviewed in WebKit bugzilla and landed in WebKit can be readily merged into WPT without additional review (although additional review is always great!). The only constraint I know of is that the test does not give flaky tests from WPT Chrome/Firefox bots. We do not have yet the tooling to automate the creation of a WPT GitHub PR from a WebKit patch that lands. In the meantime, for those of us planning to contribute to WPT, it might make sense to start using a consistent flow. How about marking explicitly the PRs as coming from WebKit in the title, with the related bugzilla URL in the comment? Patch at https://bugs.webkit.org/show_bug.cgi?id=169462 might be handy for that. When done so, how about allowing any WebKitten with WPT repository merge access to merge those PRs without further review? It might (and did) happen that a proposed test is conforming to the spec but the spec is expected to change at some point. I don't think we should refrain from landing such test in WPT. A test exercising the currently described behavior is a good thing anyway. It might actually ease the job of those who will later need to provide a test that will cover the spec change. Thanks y ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
