> On Jun 19, 2020, at 1:04 PM, Alexey Proskuryakov <a...@webkit.org> wrote: > > > >> 19 июня 2020 г., в 10:24 AM, Geoffrey Garen <gga...@apple.com >> <mailto:gga...@apple.com>> написал(а): >> >>>> On Jun 19, 2020, at 8:04 AM, Geoffrey Garen <gga...@apple.com >>>> <mailto:gga...@apple.com>> wrote: >>>> >>>> Can you explain more about what "O3 with no-inlining” is? How does >>>> --force-opt=O3 avoid inlining? Would this fully resolve Simon concern >>>> about stack traces, or would something still be different about stack >>>> traces? >>> There doesn't exist a way to do this now, but it'd be trivial to add a way. >>> I won't claim it fixes all stack traces differences, but I'd think >>> compiling using "-fno-inline -fno-optimize-sibling-calls" would get us >>> pretty far in crashing stack traces being similar enough. >> >> Sounds good. >> >> I think we should try to refine the proposal along these lines, to minimize >> the downsides. I won’t speak for Simon, but for me, being able to ensure a >> clear backtrace from a bot would be a big improvement. > > Enabling some level of optimization is reasonable; whether it should be -O3 > with inlining disabled or -Og is a technical question that probably can't be > answered without hard data. Also, building locally in the same way as bots do > could be a show stopper, as people don't like slow builds.
I just sent an email with the hard data a few minutes ago. -Og is 3.5x slower than -O3 on a Debug test run. Building locally is totally acceptable. It adds 6 minutes to the total test run time. This is coming from a current run that takes 4.7 hours and will be reduced to about 45 minutes for a local build + O3 Debug test run. > >>>> And again, I think this discussion would get a lot more focused if the >>>> change could apply only to JSC code, and not also to WTF code. >>> I believe Mark's proposal, initially, is just to make JSC do this. So I >>> don't see the point of compiling WTF differently. JSC can kick off its own >>> build, and run Debug+O3 tests faster than it can run Debug+O0 tests. Given >>> people working on JSC want this, and people working on JSC defend these >>> tests, and that these test results are more stable (see below), we should >>> make this change for JSC. >>> >>> I was trying to convince folks defending non-JSC testing, that they too, >>> should want this. I'm not going to pull teeth here. If folks want their >>> tests to take ~10x longer to run, they're entitled to make that tradeoff. >> >> Got it. > > To do this only for JSC builds, we'd need separate builders and storage, so > it becomes a question of allocating more resources, not just switching over > to a different configuration. While EWS builds for JSC independently, > post-commit testing shares build artifacts across all testers. I don’t think we should go with separate storage unless there are other tests that also depend on a Debug JSC build. Maybe I’m mistaken but I don’t think we’ll have any other reason to keep the build artifacts around. While today’s post-commit test script for Debug JSC tests uses common build artifacts, why can’t we change the steps for this test to work like the EWS bot and just build locally? In terms of total run time, even if we have a separate bot to build the binary, the test bot still has to wait for it because it would be a different build from the common build artifact that other test bots use. So, there is no time saved here. Mark >>>> And again, on the run more tests front, it would be helpful to know >>>> whether this change was enough to get the stress tests running or not. >>> My experience running the tests locally supports this fully. I don't get >>> timeouts when running O3+Debug locally. When running Debug+O0 locally, I'd >>> get timeouts all the time, and the total test run would take ~4-8 hours. We >>> can wait for official confirmation from Mark. >> >> Alexey, do the JSC stress tests run now on bots? If not, how fast would they >> need to run in order to be eligible to run on bots? > > I don't think that there is a simple answer, as certain variations of stress > tests get disabled on certain bots, JSC tests have a lot of variations that > are handpicked. I wouldn't even know how to find the complex answer, but > perhaps you can get the answer from <https://build.webkit.org/dashboard/ > <https://build.webkit.org/dashboard/>> > > - Alexey > > > >> Thanks, >> Geoff >> >>> >>> - Saam >>> >>>> >>>> Thanks, >>>> Geoff >>>> >>>>> On Jun 18, 2020, at 9:30 PM, Saam Barati <sbar...@apple.com >>>>> <mailto:sbar...@apple.com>> wrote: >>>>> >>>>> Why are we insisting on doing something on the bots that takes ~10x >>>>> longer to run than necessary? I’d rather have that time spent running >>>>> more tests. >>>>> >>>>> Overall, how we’re doing things now feels like a bad allocation of bot >>>>> resources. The differences I see between O3 with no-inlining vs O0 is: >>>>> - Some race conditions will behave differently. Race conditions are >>>>> already non predictable. I don’t think we’re losing anything here. >>>>> - O0 vs O3 is a different compiler. We may encounter bugs in O3 we don’t >>>>> in O0, and vice versa. In general, we probably care more about O3 >>>>> compiler bugs than O0, since we don’t ship O0, but ship a lot of O3. >>>>> >>>>> (And if we’re going to insist on “I want it to run what I build at my >>>>> desk”: I run debug with O3 at my desk, and I can run debug tests in a >>>>> reasonable amount of time now.) >>>>> >>>>> In evaluating what’s the better setup, I think it’s helpful to think >>>>> about this from the other side. Let’s imagine we had Debug+O3 as our >>>>> current setup. And someone proposed to move it to O0 and make our tests >>>>> take ~10x longer. I think that’d be a non-starter. >>>>> >>>>> - Saam >>>>> >>>>>> On Jun 17, 2020, at 9:48 PM, Simon Fraser <simon.fra...@apple.com >>>>>> <mailto:simon.fra...@apple.com>> wrote: >>>>>> >>>>>> I also object to losing good stack traces for crashes on Debug bots. >>>>>> >>>>>> Also, I don't think Debug bots should build something different from >>>>>> what I build at my desk. >>>>>> >>>>>> Simon >>>>>> >>>>>>> On Jun 17, 2020, at 1:36 PM, Mark Lam <mark....@apple.com >>>>>>> <mailto:mark....@apple.com>> wrote: >>>>>>> >>>>>>> Hi folks, >>>>>>> >>>>>>> We're planning to switch the JSC EWS bot and build.webkit.org >>>>>>> <http://build.webkit.org/> Debug build and test bots to building with >>>>>>> the following set first: >>>>>>> ./Tools/Scripts/set-webkit-configuration --force-opt=O3 >>>>>>> >>>>>>> This means the Debug builds will be built with optimization level >>>>>>> forced to O3. >>>>>>> >>>>>>> Why are we doing this? >>>>>>> 1. So that the JSC EWS will start catching ASSERT failures. >>>>>>> 2. JSC stress test Debug bots have been timing out and not running >>>>>>> tests at all. Hopefully, this change will fix this issue. >>>>>>> 3. Tests will run to completion faster and we’ll catch regressions >>>>>>> sooner. >>>>>>> >>>>>>> The downside: crash stack traces will be like Release build stack >>>>>>> traces. But I don’t think we should let this deter us. It’s not like >>>>>>> there’s no stack information. And just as we do with debugging Release >>>>>>> build test failures, we can always do a Debug build locally to do our >>>>>>> debugging. >>>>>>> >>>>>>> We would like to apply this change to all Debug build and test bots, >>>>>>> not just the JSC ones. Does anyone strongly object to this change? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Cheers, >>>>>>> Mark >>>>>>> >>>>>>> _______________________________________________ >>>>>>> webkit-dev mailing list >>>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>>>> >>>>>> _______________________________________________ >>>>>> webkit-dev mailing list >>>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>>> _______________________________________________ >>>>> webkit-dev mailing list >>>>> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >>>>> https://lists.webkit.org/mailman/listinfo/webkit-dev >>>>> <https://lists.webkit.org/mailman/listinfo/webkit-dev> >>>> >>> >> >> _______________________________________________ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> >> https://lists.webkit.org/mailman/listinfo/webkit-dev > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org> > https://lists.webkit.org/mailman/listinfo/webkit-dev > <https://lists.webkit.org/mailman/listinfo/webkit-dev>
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
