Re: [whatwg] Worker requestAnimationFrame
We build a major HTML5 game engine, and I feel the deadline approach is unsuitable for gaming. Games often do heavy CPU work to run the game logic before drawing. (Please do not drag me in to arguments over time stepping; stepping less often than rAF does not look smooth, and stepping more often wastes CPU and battery, or the CPU is simply not fast enough to fit more than one step in 16ms.) If a game misses any kind of deadline, it can't just choose to not miss the deadline. We think the best behavior in that case is to drop to half v-sync rate. Note that many top console games specifically target 30 FPS and no higher[1]. Many of our users have asked for this behavior[2]. Running at say 45 FPS looks uneven and is perceived as janky due to the irregular screen update, whereas 30 FPS (i.e. half v-sync rate) is perceived as more steady and even, despite the technical performance being worse. We have tried to implement half-vsync rate ourselves in the past by simply ignoring alternate rAFs, but in practice it looked terrible - I guess we broke implementation-defined heuristics that expect every rAF call to do a similar amount of work. I think it would be more useful if we could indicate to the browser what we want to happen given that we *expect* to miss the deadline, either: 1) display ASAP and callback again ASAP (this appears to be the current behavior and allows uneven framerates like 45 FPS) 2) drop to half rAF rate (producing a steady 30 FPS on a 60 Hz display). I understand this may be left as implementation-defined, but I guess there may be valid use cases for display ASAP as well (VR?), so perhaps the spec should be extended to allow both cases? When it comes to rAF in a worker, for the reasons above I do not see the point in any divergence from what the existing main-thread rAF does, unless both are changed. If both are changed I hope it takes in to account half-vsync mode. If main-thread rAF is not changed, then the rAF in a worker should fire simultaneously with the main-thread rAF (as if main-thread rAF posted a message to the worker, minus the latency of posting a message) such that a game engine running in a worker can align with the main window painting. [1] http://www.gamespot.com/articles/fallout-4-will-run-1080p-30fps-on-ps4-xbox-one-pc-/1100-6428362/ [2] See crbug.com/461719 which has 53 stars On 19 August 2015 at 22:32, Justin Novosad ju...@google.com wrote: On Wed, Aug 19, 2015 at 5:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Thu, Aug 20, 2015 at 4:21 AM, Ian Vollick voll...@chromium.org wrote: On Wed, Aug 19, 2015 at 7:36 AM Justin Novosad ju...@google.com wrote: On Wed, Aug 19, 2015 at 10:13 AM, Rick Byers rby...@chromium.org wrote: Sounds great to me. I agree this is an important scenario. *Ian*, thoughts? I would certainly like to see requestAnimationFrame in a worker, but there is more risk of falling out of step with vsync in a vanilla worker that has access to APIs that are dangerous from a performance point of view. It would also be nice to run a worker with rAF on an elevated priority thread (or maybe even the compositor thread), but that would be a bad idea if it can do stuff like synchronous IO. I fear a proliferation of different kinds of restricted Workers that would make it hard to handle multiple kinds of callbacks in a single Worker, so I'd rather not introduce a new restricted kind unless it's absolutely necessary. I think it would be fine to support rAF in a general DedicatedWorker and then later, if absolutely necessary, provide an elevated-priority Worker with restricted API. After implementing rAF for DedicatedWorker, the first slow-script mitigation I'd like to introduce is the you missed your frame deadline event that we've been talking about for a whlie. In the requestIdleCallback proposal ( https://w3c.github.io/requestidlecallback/ ) a deadline argument is passed to the callback. That's an idea we could apply here as well. Not sure whether it is better or worse than using an event. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] Worker requestAnimationFrame
On Thu, Aug 20, 2015 at 11:02 PM, Ashley Gullen ash...@scirra.com wrote: We have tried to implement half-vsync rate ourselves in the past by simply ignoring alternate rAFs, but in practice it looked terrible - I guess we broke implementation-defined heuristics that expect every rAF call to do a similar amount of work. I think it would be more useful if we could indicate to the browser what we want to happen given that we *expect* to miss the deadline, either: 1) display ASAP and callback again ASAP (this appears to be the current behavior and allows uneven framerates like 45 FPS) 2) drop to half rAF rate (producing a steady 30 FPS on a 60 Hz display). I've heard different application developers give different answers about which behavior is better. If the answer actually depends on the application, we could add an API to control this. When it comes to rAF in a worker, for the reasons above I do not see the point in any divergence from what the existing main-thread rAF does, unless both are changed. If both are changed I hope it takes in to account half-vsync mode. If main-thread rAF is not changed, then the rAF in a worker should fire simultaneously with the main-thread rAF (as if main-thread rAF posted a message to the worker, minus the latency of posting a message) such that a game engine running in a worker can align with the main window painting. I agree that we should not pass a deadline parameter and frame-skipping policy should be the same on a worker as on the main thread. But I wouldn't describe it as fire simultaneously with the main thread. For one thing, if the main thread is blocked by long-running scripts then you could get more rAF callbacks firing on the worker than on the main thread. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] Worker requestAnimationFrame
Sounds great to me. I agree this is an important scenario. *Ian*, thoughts? Is anyone actively working on worker-thread canvas in blink at the moment? Note that Ian's CompositorWorker prototype https://docs.google.com/document/d/18GGuTRGnafai17PDWjCHHAvFRsCfYUDYsi720sVPkws/edit# currently names this method requestCompositorFrame to help make the semantic difference from requestAnimationFrame more clear to developers. But otherwise it's basically the same. Lots of debate could be had on the admission / timing policy, but only data from real apps is likely to resolve this debate so we should probably start with something simple and flexible like you propose. Rick On Tue, Aug 18, 2015 at 10:53 PM, Robert O'Callahan rob...@ocallahan.org wrote: For OffscreenCanvas we need a way for a Worker to draw once per composited frame. I suggest we create DedicatedWorkerGlobalScope.requestAnimationFrame(callback) that works similarly to Window.requestAnimationFrame. To reduce latency for applications such as VR, I'd like to run the callback after vsync and let the compositor wait for some implementation-defined amount of time for the callback to complete before compositing the frame; this will give the callback a chance to finish rendering and get the results composited before the next vsync. If the callback runs too long its updates will be applied to some later frame. I suggest we later extend this for worker-based control of CSS transforms etc (i.e. CompositorWorker). Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] Worker requestAnimationFrame
On Wed, Aug 19, 2015 at 10:13 AM, Rick Byers rby...@chromium.org wrote: Sounds great to me. I agree this is an important scenario. *Ian*, thoughts? Is anyone actively working on worker-thread canvas in blink at the moment? Work not yet started, but it is in my queue. On Tue, Aug 18, 2015 at 10:53 PM, Robert O'Callahan rob...@ocallahan.org wrote: For OffscreenCanvas we need a way for a Worker to draw once per composited frame. I suggest we create DedicatedWorkerGlobalScope.requestAnimationFrame(callback) that works similarly to Window.requestAnimationFrame. To reduce latency for applications such as VR, I'd like to run the callback after vsync and let the compositor wait for some implementation-defined amount of time for the callback to complete before compositing the frame; this will give the callback a chance to finish rendering and get the results composited before the next vsync. If the callback runs too long its updates will be applied to some later frame. Without this API, I think the best one could do is to post a message to the worker from from the main thread rAF callback, which is not very good because it adds unnecessary latency to the signal, and will induce Jank if the main thread is busy. So: +1 to this proposal I suggest we later extend this for worker-based control of CSS transforms etc (i.e. CompositorWorker). Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] Worker requestAnimationFrame
On Thu, Aug 20, 2015 at 4:21 AM, Ian Vollick voll...@chromium.org wrote: On Wed, Aug 19, 2015 at 7:36 AM Justin Novosad ju...@google.com wrote: On Wed, Aug 19, 2015 at 10:13 AM, Rick Byers rby...@chromium.org wrote: Sounds great to me. I agree this is an important scenario. *Ian*, thoughts? I would certainly like to see requestAnimationFrame in a worker, but there is more risk of falling out of step with vsync in a vanilla worker that has access to APIs that are dangerous from a performance point of view. It would also be nice to run a worker with rAF on an elevated priority thread (or maybe even the compositor thread), but that would be a bad idea if it can do stuff like synchronous IO. I fear a proliferation of different kinds of restricted Workers that would make it hard to handle multiple kinds of callbacks in a single Worker, so I'd rather not introduce a new restricted kind unless it's absolutely necessary. I think it would be fine to support rAF in a general DedicatedWorker and then later, if absolutely necessary, provide an elevated-priority Worker with restricted API. After implementing rAF for DedicatedWorker, the first slow-script mitigation I'd like to introduce is the you missed your frame deadline event that we've been talking about for a whlie. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] Worker requestAnimationFrame
On Wed, Aug 19, 2015 at 5:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Thu, Aug 20, 2015 at 4:21 AM, Ian Vollick voll...@chromium.org wrote: On Wed, Aug 19, 2015 at 7:36 AM Justin Novosad ju...@google.com wrote: On Wed, Aug 19, 2015 at 10:13 AM, Rick Byers rby...@chromium.org wrote: Sounds great to me. I agree this is an important scenario. *Ian*, thoughts? I would certainly like to see requestAnimationFrame in a worker, but there is more risk of falling out of step with vsync in a vanilla worker that has access to APIs that are dangerous from a performance point of view. It would also be nice to run a worker with rAF on an elevated priority thread (or maybe even the compositor thread), but that would be a bad idea if it can do stuff like synchronous IO. I fear a proliferation of different kinds of restricted Workers that would make it hard to handle multiple kinds of callbacks in a single Worker, so I'd rather not introduce a new restricted kind unless it's absolutely necessary. I think it would be fine to support rAF in a general DedicatedWorker and then later, if absolutely necessary, provide an elevated-priority Worker with restricted API. After implementing rAF for DedicatedWorker, the first slow-script mitigation I'd like to introduce is the you missed your frame deadline event that we've been talking about for a whlie. In the requestIdleCallback proposal ( https://w3c.github.io/requestidlecallback/ ) a deadline argument is passed to the callback. That's an idea we could apply here as well. Not sure whether it is better or worse than using an event. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
