[webkit-dev] New EWS queue: Stress Test EWS

[Aakash Jain via webkit-dev]

Hi Everyone,

I am happy to announce that I have added a new EWS queue: Stress Test EWS. From 
now on, you would see a new status bubbles (on Bugzilla bug) named: 
'mac-wk2-stress'. For the patches that add or modify layout tests, this EWS 
will run those added/modified layout-tests repeatedly large number of times 
(currently 100 iterations). It will also run those added/modified layout-tests 
repeatedly in guard-malloc mode. Example: 
https://ews-build.webkit.org/#/builders/62/builds/249. The aim is to find any 
issue with the layout-tests, especially flakiness, or failure when run in a 
certain manner.

Flaky layout-tests are currently a huge problem, as they sometimes cause false 
positives on EWS (and other parts of infrastructure), and slow down our 
infrastructure (as for every test failure, EWS needs to run the test-suite few 
times to ensure the failures are not flaky or pre-existing). Until now, we 
didn't have any good automated way to indicate the test flakiness to the 
engineers. With this EWS, we are adding pre-commit testing for layout-test 
flakiness. We understand that layout tests might be flaky in a variety of ways, 
sometimes in a very specific or unexpected manner. I am looking at covering 
more and more scenarios in this EWS by making incremental improvements to it.

Few ideas to further improve this EWS are:
- Test on more OS/configurations (this EWS is currently using macOS release wk2 
mode)
- Run the test being added/modified repeatedly along-with other layout-tests in 
same directory
- Improve the logic to identify layout-tests, current logic is naive and only 
recognizes html layout-tests

More ideas are welcome.

Thanks
Aakash
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Request for position: ALPS and ACCEPT_CH HTTP/2 and HTTP/3 frames

[David Benjamin via webkit-dev]

On Tue, Apr 6, 2021 at 2:37 PM Alex Christensen 
wrote:

> I’m also wondering why
> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#retry-limits
> says it should only retry GET requests.  Is that just to avoid re-uploading
> large POST requests?
>

POST requests are not safe
 or idempotent
, so the client
shouldn't automatically resend them. Though, yes, large POST bodies would
also be a nuisance.

> On Apr 6, 2021, at 10:02 AM, David Benjamin  wrote:
>
> Hi Alex, thanks for the comments! Responses inline.
>
> On Mon, Apr 5, 2021 at 9:04 PM Alex Christensen 
> wrote:
>
>> I’m glad to see ALPS and bytes sent over the network used instead of
>> additional reliance on state on the client.  We don’t want to introduce a
>> super cookie on the client, and we want to minimize breakage when a user
>> agent decides to remove state to prevent tracking.
>>
>
> Well, with regards to cross-site tracking, I'll note that Accept-CH cache
> is already naturally partitioned because it only applies to top-level
> loads. Subresources follow a delegation model. But, yeah, one of the nice
> outcomes of Client Hint Reliability is it makes the Accept-CH cache
> *actually* a cache, so the UA can scope or clear it with less worry. I
> think reducing the performance and functionality gap between new and
> returning clients is generally valuable for this sort of thing. I hope
> Client Hint Reliability is useful in this regard.
>
> I can’t say I’ve followed this development closely or even thought through
>> it all completely, but here are some initial thoughts:
>>
>> My first thought is that it seems excessive to have a way to specify
>> support of client hints both in the TLS handshake and in HTTP/{2,3}
>> frames.  I guess that’s why you wrote
>> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#why-two-mechanisms
>
>
> I think you may have misunderstood the reference to two mechanisms. The
> TLS ALPS extension and h2/h3 frames are part of the same mechanism. It's a
> layering thing. TLS provides a generic spot to stick protocol-specific
> settings early enough in the handshake, and then HTTP/{2,3} define how to
> use it. (We don't want every new feature like this to require an update to
> the TLS server.)
>
> Rather, the reference is to (1) Critical-CH HTTP response *header* and
> (2) TLS ALPS + h2/h3 frames. I'd love to avoid the redundancy, but I think
> this is the best option given all the design constraints. And yeah the
> explainer discusses why.
>
>
>> I don’t think that requiring a site to be running software that supports
>> client hints is a good prerequisite to using client hints, so I don’t think
>> that’s a good reason to have two mechanisms.
>>
>
> I'm not sure I'm parsing this sentence right. It sounds like you both
> don't think server software changes are a good requisite, but also don't
> think it's good to have a mechanism with lower server software requirements?
>
> Oops!  Remove the first “don’t” in my sentence.
>

Ah. :-)

I think the other constraints are more fundamental. I view this more as a
nice bonus. Especially given how slowly TLS sadly moves in much of the
ecosystem. Features in TLS libraries, software releases, those releases
making their way into Linux distros, and then to LTS versions of those
distros, etc.

In particular, the rest of Client Hints works with just headers, so having
Critical-CH makes it easier for sites to get reliability. That, in turn,
reduces the functionality gap when the UA needs to clear or scope some
state, as we talked about above. (Though the connection-level optimization
is still needed to close the performance gap.)

> Sites can change with open connections, but if a site changes its client
>> hints acceptance, wouldn’t that be a good reason to terminate all the open
>> connections and require renegotiation?
>>
>
> Sites don't really work that way architecturally. You may have, for
> instance, a CDN frontend handling TLS and H2/H3, but it contacts the origin
> server that developers actually upload content to. In such deployments,
> there usually isn't a way to signal an update to all connections like that.
> Moreover, there's a race condition here. The client may request the
> resource at the same time as the server signaling the new preferences.
>
>
>> Wildcard subdomains in the certificate is an interesting problem.
>>
>
> I'll add that cross-name pooling further complicates any hope of signaling
> existing connections above.
>
>
>> If it is decided that multiple mechanisms are necessary, their
>> interaction should be well defined.  What if the server said one thing in
>> ALPS but said something different in an HTTP/{2,3} frame?  What if I have
>> multiple connections open to the same server and get different client hint
>> headers?
>>
>
> Agreed it should be well-defined. I touched on this briefly
> in 

Re: [webkit-dev] Request for position: ALPS and ACCEPT_CH HTTP/2 and HTTP/3 frames

[Alex Christensen via webkit-dev]

I’m also wondering why 
https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#retry-limits
 says it should only retry GET requests.  Is that just to avoid re-uploading 
large POST requests?  

> On Apr 6, 2021, at 10:02 AM, David Benjamin  wrote:
> 
> Hi Alex, thanks for the comments! Responses inline.
> 
> On Mon, Apr 5, 2021 at 9:04 PM Alex Christensen  > wrote:
> I’m glad to see ALPS and bytes sent over the network used instead of 
> additional reliance on state on the client.  We don’t want to introduce a 
> super cookie on the client, and we want to minimize breakage when a user 
> agent decides to remove state to prevent tracking.
> 
> Well, with regards to cross-site tracking, I'll note that Accept-CH cache is 
> already naturally partitioned because it only applies to top-level loads. 
> Subresources follow a delegation model. But, yeah, one of the nice outcomes 
> of Client Hint Reliability is it makes the Accept-CH cache actually a cache, 
> so the UA can scope or clear it with less worry. I think reducing the 
> performance and functionality gap between new and returning clients is 
> generally valuable for this sort of thing. I hope Client Hint Reliability is 
> useful in this regard.
> 
> I can’t say I’ve followed this development closely or even thought through it 
> all completely, but here are some initial thoughts:
> 
> My first thought is that it seems excessive to have a way to specify support 
> of client hints both in the TLS handshake and in HTTP/{2,3} frames.  I guess 
> that’s why you wrote 
> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#why-two-mechanisms
>  
> 
> 
> I think you may have misunderstood the reference to two mechanisms. The TLS 
> ALPS extension and h2/h3 frames are part of the same mechanism. It's a 
> layering thing. TLS provides a generic spot to stick protocol-specific 
> settings early enough in the handshake, and then HTTP/{2,3} define how to use 
> it. (We don't want every new feature like this to require an update to the 
> TLS server.)
> 
> Rather, the reference is to (1) Critical-CH HTTP response header and (2) TLS 
> ALPS + h2/h3 frames. I'd love to avoid the redundancy, but I think this is 
> the best option given all the design constraints. And yeah the explainer 
> discusses why.
>  
> I don’t think that requiring a site to be running software that supports 
> client hints is a good prerequisite to using client hints, so I don’t think 
> that’s a good reason to have two mechanisms.
> 
> I'm not sure I'm parsing this sentence right. It sounds like you both don't 
> think server software changes are a good requisite, but also don't think it's 
> good to have a mechanism with lower server software requirements?
Oops!  Remove the first “don’t” in my sentence.
>  
> Sites can change with open connections, but if a site changes its client 
> hints acceptance, wouldn’t that be a good reason to terminate all the open 
> connections and require renegotiation?
> 
> Sites don't really work that way architecturally. You may have, for instance, 
> a CDN frontend handling TLS and H2/H3, but it contacts the origin server that 
> developers actually upload content to. In such deployments, there usually 
> isn't a way to signal an update to all connections like that. Moreover, 
> there's a race condition here. The client may request the resource at the 
> same time as the server signaling the new preferences.
>  
> Wildcard subdomains in the certificate is an interesting problem.
> 
> I'll add that cross-name pooling further complicates any hope of signaling 
> existing connections above.
>  
> If it is decided that multiple mechanisms are necessary, their interaction 
> should be well defined.  What if the server said one thing in ALPS but said 
> something different in an HTTP/{2,3} frame?  What if I have multiple 
> connections open to the same server and get different client hint headers?
> 
> Agreed it should be well-defined. I touched on this briefly in 
> draft-davidben-http-client-hint-reliability-02, but not in full detail. The 
> IETF and W3C/WHATWG split is a bit fun at the boundaries of the web platform 
> and network protocols. :-) I think we'll probably put the full Fetch 
> integration in https://github.com/WICG/client-hints-infrastructure 
> , alongside the other 
> Client Hints bits.
> 
> The interaction we think works best is that ALPS/frames and 
> Accept-CH/Critical-CH are conceptually two separate sources of hint requests, 
> with the expectation that the former is an optimization for the latter. You 
> end up roughly unioning them. This avoids weird behaviors when they mismatch 
> and meshes well with the constraints that led to two mechanisms in the first 
> place.
>  
> In 
> 

Re: [webkit-dev] Request for position: ALPS and ACCEPT_CH HTTP/2 and HTTP/3 frames

[David Benjamin via webkit-dev]

(Oops, couldn't post to the list without being subscribed. Apologies for
the duplicate mail to anyone who got both!)

On Tue, Apr 6, 2021 at 1:02 PM David Benjamin  wrote:

> Hi Alex, thanks for the comments! Responses inline.
>
> On Mon, Apr 5, 2021 at 9:04 PM Alex Christensen 
> wrote:
>
>> I’m glad to see ALPS and bytes sent over the network used instead of
>> additional reliance on state on the client.  We don’t want to introduce a
>> super cookie on the client, and we want to minimize breakage when a user
>> agent decides to remove state to prevent tracking.
>>
>
> Well, with regards to cross-site tracking, I'll note that Accept-CH cache
> is already naturally partitioned because it only applies to top-level
> loads. Subresources follow a delegation model. But, yeah, one of the nice
> outcomes of Client Hint Reliability is it makes the Accept-CH cache
> *actually* a cache, so the UA can scope or clear it with less worry. I
> think reducing the performance and functionality gap between new and
> returning clients is generally valuable for this sort of thing. I hope
> Client Hint Reliability is useful in this regard.
>
> I can’t say I’ve followed this development closely or even thought through
>> it all completely, but here are some initial thoughts:
>>
>> My first thought is that it seems excessive to have a way to specify
>> support of client hints both in the TLS handshake and in HTTP/{2,3}
>> frames.  I guess that’s why you wrote
>> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#why-two-mechanisms
>
>
> I think you may have misunderstood the reference to two mechanisms. The
> TLS ALPS extension and h2/h3 frames are part of the same mechanism. It's a
> layering thing. TLS provides a generic spot to stick protocol-specific
> settings early enough in the handshake, and then HTTP/{2,3} define how to
> use it. (We don't want every new feature like this to require an update to
> the TLS server.)
>
> Rather, the reference is to (1) Critical-CH HTTP response *header* and
> (2) TLS ALPS + h2/h3 frames. I'd love to avoid the redundancy, but I think
> this is the best option given all the design constraints. And yeah the
> explainer discusses why.
>
>
>> I don’t think that requiring a site to be running software that supports
>> client hints is a good prerequisite to using client hints, so I don’t think
>> that’s a good reason to have two mechanisms.
>>
>
> I'm not sure I'm parsing this sentence right. It sounds like you both
> don't think server software changes are a good requisite, but also don't
> think it's good to have a mechanism with lower server software requirements?
>
>
>> Sites can change with open connections, but if a site changes its client
>> hints acceptance, wouldn’t that be a good reason to terminate all the open
>> connections and require renegotiation?
>>
>
> Sites don't really work that way architecturally. You may have, for
> instance, a CDN frontend handling TLS and H2/H3, but it contacts the origin
> server that developers actually upload content to. In such deployments,
> there usually isn't a way to signal an update to all connections like that.
> Moreover, there's a race condition here. The client may request the
> resource at the same time as the server signaling the new preferences.
>
>
>> Wildcard subdomains in the certificate is an interesting problem.
>>
>
> I'll add that cross-name pooling further complicates any hope of signaling
> existing connections above.
>
>
>> If it is decided that multiple mechanisms are necessary, their
>> interaction should be well defined.  What if the server said one thing in
>> ALPS but said something different in an HTTP/{2,3} frame?  What if I have
>> multiple connections open to the same server and get different client hint
>> headers?
>>
>
> Agreed it should be well-defined. I touched on this briefly
> in draft-davidben-http-client-hint-reliability-02, but not in full detail.
> The IETF and W3C/WHATWG split is a bit fun at the boundaries of the web
> platform and network protocols. :-) I think we'll probably put the full
> Fetch integration in https://github.com/WICG/client-hints-infrastructure,
> alongside the other Client Hints bits.
>
> The interaction we think works best is that ALPS/frames and
> Accept-CH/Critical-CH are conceptually two separate sources of hint
> requests, with the expectation that the former is an optimization for the
> latter. You end up roughly unioning them. This avoids weird behaviors when
> they mismatch and meshes well with the constraints that led to two
> mechanisms in the first place.
>
>
>> In
>> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#retry-limits
>> you specify that a client should not retry more than once per request to
>> avoid infinite loops, but in
>> https://github.com/WICG/client-hints-infrastructure/blob/main/reliability.md#server-triggered-retry
>> you use the possibility of infinite loops as a reason that a
>> server-triggered