Thanks Etienne; questions inline:
On Thursday, October 20, 2022 at 8:04:19 AM UTC-7 Etienne Pierre-doray wrote: > Thanks for taking the time to discuss this. > > I'm wondering if we understand the constituent test in Speedometer (or the >> harness) that is favouring Safari's out-of-spec behaviour? >> > There's some context in crbug.com/1297550 and in speedometer2.1 release > notes <https://webkit.org/blog/13083/speedometer-2-1/>; Speedometer 2.1 > hopefully fixes the benchmark to mitigate the impact of throttling > setTimeout(0) (in local experiment, it does reduce improvements we can get > with this change). > So if I go directly to browserbench.org and click on "speedometer", it takes me to: https://browserbench.org/Speedometer2.1/ Is that a very new change? Is there a reason for us to continue to look at (or cite) 2.0 when 2.1 is live? > Speedometer seems like the key motivator here, rather than public content >> > Correct. I think Speedometer2.0 is the main motivator to shipping this in > a timely manner. +sky@ who has been championing moving forward with this > change. > What's the residual delta now that 2.1 is available to test against? > Ideally we can prove making this change has no negative impact on metrics > we care about. > Another (long-term) benefit is perhaps to move away from the > spec-mendated threshold (which is somewhat arbitrary) and hopefully take it > away from the spec. A hard-to-prove benefit of removing the 4ms clamping is > to match more closely the devs intent when they write setTimeout(0), and > give the browser more leeway in implementing a throttling policy. > > I'd support finching this on for Stable for some releases while we get >> resolution on fixing the benchmark. >> > I'm experimenting on M107 (with nesting threshold = 15) and will ramp up > to 1% Stable soon. > (We also experimented with Stable 1% on M104-105 for a different value > (nesting = 100), which showed no regression on Windows / MacOs, but > regressed startup time by 0.5% at the median on Android). > If finching for one milestone is enough to confirm no regression (from a > metrics perspective, I believe it's enough to get statistically significant > data), I'm hoping we can optimistically ship on M108 through a waterfall > roll-out. Otherwise, maybe we can delay shipping 1+ milestone. > Presumably the downside of this change is in power/battery? Are there other impacts we're looking at? > Another option we discussed is to ship as-is on desktop only (and figure > out Android later), but I feel like this creates a more inconsistent > platform. > +1 to doing this in a more uniform way. Best, Alex > On Wed, Oct 19, 2022 at 11:52 AM Alex Russell <slightly...@chromium.org> > wrote: > >> This intent was the subject of a long discussion at API OWNERS today, and >> I'm wondering if we understand the constituent test in Speedometer (or the >> harness) that is favouring Safari's out-of-spec behaviour? >> >> Speedometer seems like the key motivator here, rather than public >> content, and winning it matters in the interim while Apple is gaming this >> for the purposes of benchmarketing. I'd support finching this on for Stable >> for some releases while we get resolution on fixing the benchmark. >> >> Best, >> >> Alex >> >> >> >> On Thursday, October 13, 2022 at 1:00:12 PM UTC-7 Etienne Pierre-doray >> wrote: >> >>> Contact emailsetien...@chromium.org >>> >>> Specification >>> https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html >>> >>> Design docs >>> >>> https://docs.google.com/document/d/1boT0k8BQjl7mXXzvI9SdN4XJPSza27vE8T0CNxmMhCI >>> >>> Summary >>> >>> Increase the nesting threshold before which setTimeout(..., <4ms) start >>> being clamped, from 5 to 15. setTimeout(..., 0) is commonly used to break >>> down long Javascript tasks and let other internal tasks run, which prevents >>> the browser from hanging. setTimeouts and setIntervals with an interval < >>> 4ms are not clamped as aggressively as they were before. This improves >>> short horizon performance, but websites abusing the API will still >>> eventually have their set setTimeouts clamped >>> >>> >>> Blink componentBlink >>> <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink> >>> >>> TAG review >>> >>> TAG review statusNot applicable >>> >>> Risks >>> >>> >>> Interoperability and Compatibility >>> >>> setTimeout is a well established and mature API. This change poses a >>> risk of breaking websites and tests that rely on the current timing caused >>> by clamping and the subtle task ordering that it entails. As an example, >>> this change breaks assumptions about the ordering between setTimeout(0) and >>> unrelated tasks in at least one case in Chrome tests (crbug.com/1302309). >>> On the flip side, the implementation in Chrome is already non compliant ( >>> crbug.com/1108877). There's also a similar experiment on beta that is >>> ongoing (crbug.com/1263190). Devs can use >>> chrome://flags#unthrottled-nested-timeout to test their sites for >>> compatibility issues. >>> >>> >>> *Gecko*: No signal >>> >>> *WebKit*: Shipped/Shipping ( >>> https://sourcegraph.com/github.com/WebKit/WebKit/-/commit/786e3e0b252e38fb01c8db97a94d52cb0f57891e >>> ) >>> >>> *Web developers*: No signals >>> >>> *Other signals*: >>> >>> WebView application risks >>> >>> Does this intent deprecate or change behavior of existing APIs, such >>> that it has potentially high risk for Android WebView-based applications? >>> >>> >>> >>> Debuggability >>> >>> setTimeout() and setInterval() have an associated trace event in >>> DevTools. >>> https://developer.chrome.com/docs/devtools/evaluate-performance/performance-reference/ >>> >>> >>> Will this feature be supported on all six Blink platforms (Windows, Mac, >>> Linux, Chrome OS, Android, and Android WebView)?No >>> >>> Is this feature fully tested by web-platform-tests >>> <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> >>> ?No >>> >>> Flag nameunthrottled-nested-timeout >>> >>> Requires code in //chrome?False >>> >>> Tracking bughttps://crbug.com/1108877 >>> >>> Launch bughttps://launch.corp.google.com/launch/4201069 >>> >>> Estimated milestones >>> Chrome for desktop: 108 >>> Chrome for Android: 108 >>> Android Webview 108 >>> >>> Anticipated spec changesThe spec dictates a nesting threshold of 5 >>> https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html "If >>> nesting level is greater than 5, and timeout is less than 4, then set >>> timeout to 4." Chrome has never respected the exact behavior ( >>> crbug.com/1108877), and safari recently updated the threshold to 10 ( >>> https://github.com/WebKit/WebKit/commit/786e3e0b252e38fb01c8db97a94d52cb0f57891e). >>> >>> A potential change to the spec is to make the threshold "implementation >>> dependent" to match reality. >>> >>> Link to entry on the Chrome Platform Status >>> https://chromestatus.com/feature/5710690097561600 >>> >>> Links to previous Intent discussionsReady for Trial: >>> https://groups.google.com/a/chromium.org/g/blink-dev/c/-TjeYs7shTQ/m/FhJq0mQyDAAJ >>> Intent to Experiment: >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALoDvsZNwh5uBANxryWHCdgFVFCts6noKSU9FY1BcqYH0%3D55sg%40mail.gmail.com >>> >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALoDvsZNwh5uBANxryWHCdgFVFCts6noKSU9FY1BcqYH0=5...@mail.gmail.com> >>> Intent to Extend Experiment: >>> https://mail.google.com/mail/u/0/#search/in%3Asent+settimeout/KtbxLzGLjTQPrFFjRfPrfQCtcCmwvTksJV >>> >>> <https://mail.google.com/mail/u/0/#search/in:sent+settimeout/KtbxLzGLjTQPrFFjRfPrfQCtcCmwvTksJV> >>> >>> This was previously enabled through field-trial on Beta 50% and Stable >>> 1% on M104-105, with a more aggressive nesting threshold = 100. No breakage >>> was reported, but it showed small guiding metrics (startup) regressions on >>> Android. I'm confident that having a lower threshold will eliminate the >>> adverse effects. Ideally, I would conduct another round of field-trial, but >>> I think we're better off with doing a waterfall roll-out, and experiment >>> later on (1% stable) to confirm no regressions: >>> >>> - Gradually rolling-out to each channel at 100% has more chances of >>> teasing out potential brokerage and won't be perceived as flaky >>> failures. I >>> will loop back after 100% beta before we hit stable. >>> - This will involve fewer back and forth with blink-dev / API >>> owners, and allow us to benefit from performance gains sooner. >>> >>> >>> >>> >>> This intent message was generated by Chrome Platform Status >>> <https://chromestatus.com/>. >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0997686f-eafb-4b47-9666-cef65cb0d470n%40chromium.org.
