Re: [webkit-dev] Another WPT bite

2017-05-15 Thread Ryosuke Niwa
On Mon, May 15, 2017 at 10:07 PM, Anne van Kesteren  wrote:
> On Tue, May 16, 2017 at 5:45 AM, Ryosuke Niwa  wrote:
>>> I think the main problem with not running a server is that behavior
>>> for file URLs is not defined. And browsers tend to impose different
>>> restrictions there. So you might end up debugging something only to
>>> later found out it didn't work due to file URL restrictions. And you
>>> can't guarantee that something will work from a file URL, because
>>> there's no agreement on what will.
>>
>> That's not an issue with most ref or testharness.js tests which tests
>> WebIDL, CSS OM, etc... because none of them have to do network
>> requests.
>
> Regardless of the feature, there's no defined agreement on how it
> should work when loaded from a file URL. A test author cannot divide
> "this can load from a file URL" from "this needs to be loaded over
> HTTP(S)". Except that loading over HTTP(S) always ought to work as
> that much we've written down in standards.

Given we're talking about how these tests are ran inside WebKit,
whether there is an agreement about this or not is sort of irrelevant.
If a test doesn't run as expected, we can run it inside a HTTP server.

Also, I'm not aware of any browser putting restrictions on how script
or link elements are loaded. For example, if z.js exits at
/Volumes/Data/bin/ and /Volumes/Data/bin/x/y.html loads z.js by
 then Chrome, Firefox, and Safari all
happily execute z.js.

Always using HTTP just because that's the lowest common denominator is
not great given that it significantly affects the common development
workflow people follow in WebKit.

I really don't think there is a viable path for importing and using
testharness.js in WebKit if we required that everyone must use HTTP
sever. Even I wouldn't do it even though I'm otherwise supportive of
writing more tests using testharness.js and upstreaming those tests
back to web-platform-tests.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-15 Thread youenn fablet
It makes sense to run WPT tests as HTTP URLs for conformance/regression
purposes.
It is fine to run WPT tests as file based URLs for development purposes.

Tooling should make it possible to run WPT tests as HTTP URLs for
development purposes with minimum to no cost.
We are not there yet.

Le lun. 15 mai 2017 à 22:08, Anne van Kesteren  a écrit :

> On Tue, May 16, 2017 at 5:45 AM, Ryosuke Niwa  wrote:
> >> I think the main problem with not running a server is that behavior
> >> for file URLs is not defined. And browsers tend to impose different
> >> restrictions there. So you might end up debugging something only to
> >> later found out it didn't work due to file URL restrictions. And you
> >> can't guarantee that something will work from a file URL, because
> >> there's no agreement on what will.
> >
> > That's not an issue with most ref or testharness.js tests which tests
> > WebIDL, CSS OM, etc... because none of them have to do network
> > requests.
>
> Regardless of the feature, there's no defined agreement on how it
> should work when loaded from a file URL. A test author cannot divide
> "this can load from a file URL" from "this needs to be loaded over
> HTTP(S)". Except that loading over HTTP(S) always ought to work as
> that much we've written down in standards.
>
>
> --
> https://annevankesteren.nl/
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-15 Thread Anne van Kesteren
On Tue, May 16, 2017 at 5:45 AM, Ryosuke Niwa  wrote:
>> I think the main problem with not running a server is that behavior
>> for file URLs is not defined. And browsers tend to impose different
>> restrictions there. So you might end up debugging something only to
>> later found out it didn't work due to file URL restrictions. And you
>> can't guarantee that something will work from a file URL, because
>> there's no agreement on what will.
>
> That's not an issue with most ref or testharness.js tests which tests
> WebIDL, CSS OM, etc... because none of them have to do network
> requests.

Regardless of the feature, there's no defined agreement on how it
should work when loaded from a file URL. A test author cannot divide
"this can load from a file URL" from "this needs to be loaded over
HTTP(S)". Except that loading over HTTP(S) always ought to work as
that much we've written down in standards.


-- 
https://annevankesteren.nl/
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Where do we put WPT tests to be exported

2017-05-15 Thread youenn fablet
I see two main cases:
- Writer of the patch is making sure to upstream WPT test changes at WebKit
landing time. It is ok to make the changes directly in
LayoutTests/imported/w3c/web-platform-tests/
- Writer plans to upstream WPT test changes at some point but wants more
time. It is better to develop the tests in LayoutTests/http/wpt and then
migrate them later on.

I would start with an experimental phase with some of us making direct
changes in LayoutTests/imported/w3c/web-platform-tests/.
When we are happy with the tools and think the risk for issues is low
enough (or when the bots can handle most of it for us), hacking
LayoutTests/imported/w3c/web-platform-tests/ could be the default.

Le lun. 15 mai 2017 à 21:02, Ryosuke Niwa  a écrit :

> Hi all,
>
> Youenn is working on a patch to automatically create a GitHub PR to
> export tests from a WebKit patch which includes changes to
> web-platform-tests [1].
>
> That raises a question as to where we should put new tests or modified
> tests intended to be exported to web-platform-tests from WebKit.
>
> I think the most obvious option is to use
> LayoutTests/imported/w3c/web-platform-tests/.  However, in the other
> thread about adopting testharness.js (titled Another WPT bite), Maciej
> briefly expressed the preference for creating a new directory:
> https://lists.webkit.org/pipermail/webkit-dev/2017-May/029022.html
>
> Do other people have strong opinions about this?
>
> - R. Niwa
>
> [1] https://bugs.webkit.org/show_bug.cgi?id=169462
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


[webkit-dev] Where do we put WPT tests to be exported

2017-05-15 Thread Ryosuke Niwa
Hi all,

Youenn is working on a patch to automatically create a GitHub PR to
export tests from a WebKit patch which includes changes to
web-platform-tests [1].

That raises a question as to where we should put new tests or modified
tests intended to be exported to web-platform-tests from WebKit.

I think the most obvious option is to use
LayoutTests/imported/w3c/web-platform-tests/.  However, in the other
thread about adopting testharness.js (titled Another WPT bite), Maciej
briefly expressed the preference for creating a new directory:
https://lists.webkit.org/pipermail/webkit-dev/2017-May/029022.html

Do other people have strong opinions about this?

- R. Niwa

[1] https://bugs.webkit.org/show_bug.cgi?id=169462
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev


Re: [webkit-dev] Another WPT bite

2017-05-15 Thread Ryosuke Niwa
On Sun, May 14, 2017 at 12:35 AM, Anne van Kesteren  wrote:
> On Sun, May 14, 2017 at 8:24 AM, Maciej Stachowiak  wrote:
>> For the engineer use case, we can make a command-line tool to launch the 
>> server and load the test. But it's kind of sad that in ~95% of cases, the 
>> only value provided by the server is resolving the path to testharness.js. 
>> If tests referenced testharness.js via relative path, then most of the time 
>> they could be loaded as local files just fine, which would be more 
>> convenient (as well as, I believe, solving the "external trac link" issue).
>
> I think the main problem with not running a server is that behavior
> for file URLs is not defined. And browsers tend to impose different
> restrictions there. So you might end up debugging something only to
> later found out it didn't work due to file URL restrictions. And you
> can't guarantee that something will work from a file URL, because
> there's no agreement on what will.

That's not an issue with most ref or testharness.js tests which tests
WebIDL, CSS OM, etc... because none of them have to do network
requests.

> I personally just keep web-platform-tests running and don't notice
> much overhead (MacBook Air, Mid 2012).

People at Apple tend to reboot their machines and install new OS all
the time so this is clearly an annoyance for us. I've rarely debugged
or created our own HTTP tests for the extra step that involves.

Even with all the shadow DOM and custom elements I wrote, I used
relative paths when I write them, and only removed relative path right
before I uploaded them to WPT repository. Furthermore, whenever I
debug those tests, the first thing I do is to rewrite the paths to
relative paths although these days, I've worked around that by adding
symlinks from /resources to LayoutTests/resources/ on multiple OS
installations.  Nonetheless, this remains to be one of the biggest
issue I have with WPT tests.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev