(Note: I have not looked into the details of CART/TART and their interaction
with OMTC)
It's entirely possible that (b) is true *now* -- the test may have been good
and proper for the previous environment, but now the environment
characteristics were changed such that the test needs tweaks. Empirically, I
have not seen any regressions on any of my Windows machines (which is basically
all of them); things like tab animations and the like have started feeling
smoother even after a long-running browser session with many tabs. I realize
this is not the same as cold hard numbers, but it does suggest to me that we
need to take another look at the tests now.
- Vlad
----- Original Message -----
> From: "Gijs Kruitbosch" <gijskruitbo...@gmail.com>
> To: "Bas Schouten" <bschou...@mozilla.com>, "Gavin Sharp"
> <ga...@gavinsharp.com>
> Cc: dev-tech-...@lists.mozilla.org, "mozilla.dev.platform group"
> <dev-platform@lists.mozilla.org>, "release-drivers"
> <release-driv...@mozilla.org>
> Sent: Thursday, May 22, 2014 4:46:29 AM
> Subject: Re: OMTC on Windows
>
> Looking on from m.d.tree-management, on Fx-Team, the merge from this
> change caused a >40% CART regression, too, which wasn't listed in the
> original email. Was this unforeseeen, and if not, why was this
> considered acceptable?
>
> As gavin noted, considering how hard we fought for 2% improvements (one
> of the Australis folks said yesterday "1% was like Christmas!") despite
> our reasons of why things were really OK because of some of the same
> reasons you gave (e.g. running in ASAP mode isn't realistic, "TART is
> complicated", ...), this hurts - it makes it seem like (a) our
> (sometimes extremely hacky) work was done for no good reason, or (b) the
> test is fundamentally flawed and we're better off without it, or (c)
> when the gfx team decides it's OK to regress it, it's fine, but not when
> it happens to other people, quite irrespective of reasons given.
>
> All/any of those being true would give me the sad feelings. Certainly it
> feels to me like (b) is true if this is really meant to be a net
> perceived improvement despite causing a 40% performance regression in
> our automated tests.
>
> ~ Gijs
>
> On 18/05/2014 19:47, Bas Schouten wrote:
> > Hi Gavin,
> >
> > There have been several e-mails on different lists, and some communication
> > on some bugs. Sadly the story is at this point not anywhere in a condensed
> > form, but I will try to highlight a couple of core points, some of these
> > will be updated further as the investigation continues. The official bug
> > is bug 946567 but the numbers and the discussion there are far outdated
> > (there's no 400% regression ;)):
> >
> > - What OMTC does to tart scores differs wildly per machine, on some
> > machines we saw up to 10% improvements, on others up to 20% regressions.
> > There also seems to be somewhat more of a regression on Win7 than there is
> > on Win8. What the average is for our users is very hard to say, frankly I
> > have no idea.
> > - One core cause of the regression is that we're now dealing with two D3D
> > devices when using Direct2D since we're doing D2D drawing on one thread,
> > and D3D11 composition on the other. This means we have DXGI locking
> > overhead to synchronize the two. This is unavoidable.
> > - Another cause is that we're now having two surfaces in order to do double
> > buffering, this means we need to initialize more resources when new layers
> > come into play. This again, is unavoidable.
> > - Yet another cause is that for some tests we composite 'ASAP' to get
> > interesting numbers, but this causes some contention scenario's which are
> > less likely to occur in real-life usage. Since the double buffer might
> > copy the area validated in the last frame from the front buffer to the
> > backbuffer in order to prevent having to redraw much more. If the
> > compositor is compositing all the time this can block the main thread's
> > rasterization. I have some ideas on how to improve this, but I don't know
> > how much they'll help TART, in any case, some cost here will be
> > unavoidable as a natural additional consequence of double buffering.
> > - The TART number story is complicated, sometimes it's hard to know what
> > exactly they do, and don't measure (which might be different with and
> > without OMTC) and how that affects practical performance. I've been told
> > this by Avi and it matches my practical experience with the numbers. I
> > don't know the exact reasons and Avi is probably a better person to talk
> > about this than I am :-).
> >
> > These are the core reasons that we were able to identify from profiling.
> > Other than that the things I said in my previous e-mail still apply. We
> > believe we're offering significant UX improvements with async video and
> > are enabling more significant improvements in the future. Once we've fixed
> > the obvious problems we will continue to see if there's something that can
> > be done, either through tiling or through other improvements, particularly
> > in the last point I mentioned there might be some, not 'too' complex
> > things we can do to offer some small improvement.
> >
> > If we want to have a more detailed discussion we should probably pick a
> > list to have this on and try not to spam people too much :-).
> >
> > Bas
> >
> > ----- Original Message -----
> > From: "Gavin Sharp" <ga...@gavinsharp.com>
> > To: "Bas Schouten" <bschou...@mozilla.com>
> > Cc: "dev-tree-management" <dev-tree-managem...@lists.mozilla.org>,
> > dev-tech-...@lists.mozilla.org, "release-drivers"
> > <release-driv...@mozilla.org>, "mozilla.dev.platform group"
> > <dev-platform@lists.mozilla.org>
> > Sent: Sunday, May 18, 2014 6:23:58 PM
> > Subject: Re: OMTC on Windows
> >
> >> but tart will regress by ~20%, and several other suites will regress as
> >> well.
> >> We've investigated this extensively and we believe the majority of these
> >> regressions are due to the nature of OMTC and the fact that we have to do
> >> more work.
> >
> > Where can I read more about the TART investigations? I'd like to
> > understand why it is seen as inevitable, and get some of the details
> > of the regression. OMTC is important, and I'm excited to see it land
> > on Windows, but the Firefox and Performance teams have just come off a
> > months-long effort to make significant wins in TART, and the thought
> > of taking a 20% regression (huge compared to some of the improvements
> > we fought for) is pretty disheartening.
> >
> > Gavin
> >
> > On Sun, May 18, 2014 at 12:16 AM, Bas Schouten <bschou...@mozilla.com>
> > wrote:
> >> Hey all,
> >>
> >> After quite a lot of waiting we've switched on OMTC on Windows by default
> >> today (bug 899785). This is a great move towards moving all our platforms
> >> onto OMTC (only linux is left now), and will allow us to remove a lot of
> >> code that we've currently been duplicating. Furthermore it puts us on
> >> track for enabling other features on desktop like APZ, off main thread
> >> animations and other improvements.
> >>
> >> Having said that we realize that what we've currently landed and turned on
> >> is not completely bug free. There's several bugs still open (some more
> >> serious than others) which we will be addressing in the coming weeks,
> >> hopefully before the merge to Aurora. The main reason we've switched it
> >> on now is that we want to get as much data as possible from the nightly
> >> channel and our nightly user base before the aurora merge, as well as
> >> wanting to prevent any new regressions from creeping in while we fix the
> >> remaining problems. This was extensively discussed both internally in the
> >> graphics team and externally with other people and we believe we're at a
> >> point now where things are sufficiently stabilized for our nightly
> >> audience. OMTC is enabled and disabled with a single pref so if
> >> unforeseen, serious consequences occur we can disable it quickly at any
> >> stage. We will inevitably find new bugs in the coming weeks, please link
> >> any bugs you happen to come across to bug 899785, if anything
> se
> >> ems very serious, please let us know, we'll attempt to come up with a
> >> solution on the short-term rather than disabling OMTC and reducing the
> >> amount of feedback we get.
> >>
> >> There's also some important notes to make on performance, which we expect
> >> to be reported by our automated systems:
> >>
> >> - Bug 1000640 is about WebGL. Currently OMTC regresses WebGL performance
> >> considerably, patches to fix this are underway and this should be fixed
> >> on the very short term.
> >>
> >> - Several of the Talos test suite numbers will change considerably
> >> (especially with Direct2D enabled), this means Tscroll for example will
> >> improve by ~25%, but tart will regress by ~20%, and several other suites
> >> will regress as well. We've investigated this extensively and we believe
> >> the majority of these regressions are due to the nature of OMTC and the
> >> fact that we have to do more work. We see no value in holding off OMTC
> >> because of these regressions as we'll have to go there anyway. Once the
> >> last correctness and stability problems are all solved we will go back to
> >> trying to find ways to get back some of the performance regressions.
> >> We're also planning to move to a system more like tiling in desktop,
> >> which will change the performance characteristics significantly again, so
> >> we don't want to sink too much time into optimizing the current
> >> situation.
> >>
> >> - Memory numbers will increase somewhat, this is unavoidable, there's
> >> several steps which have to be taken when doing off main thread
> >> compositing (like double-buffering), which inherently use more memory.
> >>
> >> - On a brighter note: Async video is also enabled by these patches. This
> >> means that when the main thread is busy churning JavaScript, instead of
> >> stuttering your video should now happily continue playing!
> >>
> >> - Also there's some indications that there's a subjective increase in
> >> scrolling performance as well.
> >>
> >>
> >> If you have any questions please feel free to reach out to myself or other
> >> members of the graphics team!
> >>
> >>
> >> Bas
> >> _______________________________________________
> >> dev-platform mailing list
> >> dev-platform@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-platform
>
> _______________________________________________
> dev-tech-gfx mailing list
> dev-tech-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-gfx
>
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
