> 19 июня 2020 г., в 9:59 AM, Mark Lam <mark....@apple.com> написал(а): > > > >> On Jun 19, 2020, at 9:53 AM, Alexey Proskuryakov <a...@webkit.org >> <mailto:a...@webkit.org>> wrote: >> >> >> >>> 18 июня 2020 г., в 9:30 PM, Saam Barati <sbar...@apple.com >>> <mailto:sbar...@apple.com>> написал(а): >>> >>> Why are we insisting on doing something on the bots that takes ~10x longer >>> to run than necessary? I’d rather have that time spent running more tests. >> >> Replying to this point specifically, I wanted to point out that WebKit tests >> take 2x longer in debug, not 10x longer. JSC tests take 3.6x longer. > > Since I collected real data on this last night on the actual bot that runs > the JSC stress tests, I’ll share the data here: > > Build time > A clean build using buid-jsc for a normal Debug build on bot638 takes about > 4.5 minutes. > A clean build using build-jsc for a --force-opt=O3 Debug build on bot638 > takes about 6 minutes. > > Test run time > Running with a regular Debug build, the test completed in about 4 hours 41 > minutes with 1 timeout. > Running with a --force-opt=O3 Debug build, the test completed in about 39 > minutes with 0 timeouts. > > The difference in test run time is 281 minutes vs 29 minutes. That is a 7.2x > ratio, not 3.6x.
https://build.webkit.org/builders/Apple-Catalina-Debug-JSC-Tests/builds/1080 - 4 hrs, 5 secs https://build.webkit.org/builders/Apple-Catalina-Release-JSC-Tests/builds/2546 - 1 hrs, 6 mins, 27 secs That's 3.6x. - Alexey >>> Overall, how we’re doing things now feels like a bad allocation of bot >>> resources. The differences I see between O3 with no-inlining vs O0 is: >> >> Last time this was discussed, we talked about -Og, which is specifically >> designed for the purpose I believe. Where do we stand on understanding and >> adopting that? > > I tried -Og last time after your suggestion, and I remember thinking that the > perf was not acceptable back then. I’ll collect the data again and report > back with real number later today. > > Mark > >> >> - Alexey >> >>> - Some race conditions will behave differently. Race conditions are already >>> non predictable. I don’t think we’re losing anything here. >>> - O0 vs O3 is a different compiler. We may encounter bugs in O3 we don’t in >>> O0, and vice versa. In general, we probably care more about O3 compiler >>> bugs than O0, since we don’t ship O0, but ship a lot of O3. >>> >>> (And if we’re going to insist on “I want it to run what I build at my >>> desk”: I run debug with O3 at my desk, and I can run debug tests in a >>> reasonable amount of time now.) >>> >>> In evaluating what’s the better setup, I think it’s helpful to think about >>> this from the other side. Let’s imagine we had Debug+O3 as our current >>> setup. And someone proposed to move it to O0 and make our tests take ~10x >>> longer. I think that’d be a non-starter. >>> >>> - Saam >>> >>>> On Jun 17, 2020, at 9:48 PM, Simon Fraser <simon.fra...@apple.com >>>> <mailto:simon.fra...@apple.com>> wrote: >>>> >>>> I also object to losing good stack traces for crashes on Debug bots. >>>> >>>> Also, I don't think Debug bots should build something different from what >>>> I build at my desk. >>>> >>>> Simon >>>> >>>>> On Jun 17, 2020, at 1:36 PM, Mark Lam <mark....@apple.com >>>>> <mailto:mark....@apple.com>> wrote: >>>>> >>>>> Hi folks, >>>>> >>>>> We're planning to switch the JSC EWS bot and build.webkit.org >>>>> <http://build.webkit.org/> Debug build and test bots to building with the >>>>> following set first: >>>>> ./Tools/Scripts/set-webkit-configuration --force-opt=O3 >>>>> >>>>> This means the Debug builds will be built with optimization level forced >>>>> to O3. >>>>> >>>>> Why are we doing this? >>>>> 1. So that the JSC EWS will start catching ASSERT failures. >>>>> 2. JSC stress test Debug bots have been timing out and not running tests >>>>> at all. Hopefully, this change will fix this issue. >>>>> 3. Tests will run to completion faster and we’ll catch regressions sooner. >>>>> >>>>> The downside: crash stack traces will be like Release build stack traces. >>>>> But I don’t think we should let this deter us. It’s not like there’s no >>>>> stack information. And just as we do with debugging Release build test >>>>> failures, we can always do a Debug build locally to do our debugging. >>>>> >>>>> We would like to apply this change to all Debug build and test bots, not >>>>> just the JSC ones. Does anyone strongly object to this change? >>>>> >>>>> Thanks. >>>>> >>>>> Cheers, >>>>> Mark >>>>> >>>>> _______________________________________________ >>>>> webkit-dev mailing list >>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>> >>>> _______________________________________________ >>>> webkit-dev mailing list >>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>> _______________________________________________ >>> webkit-dev mailing list >>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >> >> - Alexey >> >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >> https://lists.webkit.org/mailman/listinfo/webkit-dev >> <https://lists.webkit.org/mailman/listinfo/webkit-dev>
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
