Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
> > On Tue, Oct 22, 2013 at 2:48 AM, Robert O'Callahan >wrote: > >> >> > This code actually does something potentially useful which can't easily >> be >> > done with attachToCanvas: generating a series of images as fast as >> possible >> > which will be processed on another thread in some way other than just >> > rendering them on the screen. (E.g., be encoded into an animated image >> or >> > video file.) >> > (Err, wait. A few issues come to mind. 1: You can already say createImageBitmap(canvas) to create an ImageBitmap, which handles the "store a snapshot of a frame" use cases. 2: If the reason to have a "transfer" version for these use cases is just an optimization, then it's not obvious that it's a useful optimization. The use cases you mention suggest a GPU readback anyway. 3: If you're doing the encoding yourself in script, you want ImageData anyway, not ImageBitmap. I don't object as such to adding such a method if it's useful, and I don't think I have the energy right now to debate these in much depth, but this feels like taking a proposal and searching for uses for it.) On Tue, Oct 22, 2013 at 12:20 PM, Kenneth Russell wrote: > On Tue, Oct 22, 2013 at 7:37 AM, Glenn Maynard wrote: > > I just noticed that Canvas already has a Canvas.setContext() method > > That's there in support of CanvasProxy, which is a flawed API and > which this entire discussion is aiming to rectify. > I don't see flaws with the setContext() API, which appears to have already solved the problem of being able to make one context render to multiple canvases. Any relation to CanvasProxy isn't relevant to this. > > , which > > seems to do exactly what I'm proposing, even down to clearing the > backbuffer > > on attach. The only difference is that it lives on Canvas instead of the > > context--the only reason I put it there in my proposal was because this > only > > seemed useful for WebGL. Given that, I think this proposal can be > > simplified down to just: "put setContext on WorkerCanvas too". > > Also, adding a present() method to Canvas. > That's mixing up proposals, actually. Adding present() is for the "explicitpresent" proposal, which aims at solving the "synchronizing rendering in a worker to DOM changes in the main thread" use cases. Reusing setContext() replaces my attachToCanvas() proposal, which is for the "one context rendering to multiple canvases)" use cases. They're orthogonal, not mutually exclusive, and solve different problems. (We're mixing up proposals because we're trying to solve too many problems simultaneously, which is one reason I've tried to split this stuff into smaller chunks.) > At a high level I prefer the form of the WorkerCanvas API, including > transferToImageBitmap and the ability to transfer an ImageBitmap into > an HTMLImageElement for viewing, and removing the CanvasProxy concept > and associated APIs. I'd like to focus my own efforts in writing a > full draft for WorkerCanvas under > http://wiki.whatwg.org/wiki/Category:Proposals . > Again, this is a supplement to WorkerCanvas, not a replacement for it. (It may be compatible with CanvasProxy too, but I haven't looked at it closely to see.) We're circling around: you keep saying we should use transferToImageBitmap, I keep pointing out the problems with it that my proposal solves, and you reply by saying "we should use transferToImageBitmap", without addressing those problems. I don't think we have any more information to bring to the discussion right now, so I think we're at a good point to wait for Hixie to get around to these threads rather than going over the same stuff again (and giving him more reading material :). Here's a summary of my proposal: - The WorkerCanvas adjustments to CanvasProxy (minus the transferToImageBitmap stuff), to better address the "rendering to a Canvas from a worker" and "creating off-screen Canvases in a worker"-related use cases. - Include setContext() on WorkerCanvas, to support rendering from one context to multiple canvases when in a worker. - Add "explicitpresent" and present() to Canvas, to support synchronizing rendering in a worker to DOM changes in the main thread without forcing that synchronization on everybody. The second and third are independent and can be implemented separately, after WorkerCanvas itself has time to settle. -- Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Tue, Oct 22, 2013 at 7:37 AM, Glenn Maynard wrote: > I just noticed that Canvas already has a Canvas.setContext() method That's there in support of CanvasProxy, which is a flawed API and which this entire discussion is aiming to rectify. > , which > seems to do exactly what I'm proposing, even down to clearing the backbuffer > on attach. The only difference is that it lives on Canvas instead of the > context--the only reason I put it there in my proposal was because this only > seemed useful for WebGL. Given that, I think this proposal can be > simplified down to just: "put setContext on WorkerCanvas too". Also, adding a present() method to Canvas. > On Mon, Oct 21, 2013 at 9:03 PM, Kenneth Russell wrote: >> >> There are some unexpected consequences of the attachToCanvas API >> style. For example, what if two contexts use attachToCanvas to target >> the same canvas? > > > I left out these details in my initial post in order to see what people > thought at a high level before delving into details. At a high level I prefer the form of the WorkerCanvas API, including transferToImageBitmap and the ability to transfer an ImageBitmap into an HTMLImageElement for viewing, and removing the CanvasProxy concept and associated APIs. I'd like to focus my own efforts in writing a full draft for WorkerCanvas under http://wiki.whatwg.org/wiki/Category:Proposals . -Ken
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Tue, Oct 22, 2013 at 4:37 PM, Glenn Maynard wrote: > On Tue, Oct 22, 2013 at 2:48 AM, Robert O'Callahan >wrote: > > > This code actually does something potentially useful which can't easily > be > > done with attachToCanvas: generating a series of images as fast as > possible > > which will be processed on another thread in some way other than just > > rendering them on the screen. (E.g., be encoded into an animated image or > > video file.) > > > > (This is a proposal for attachToCanvas--now setContext--not against > transferToImageBitmap, if there are use cases that transferToImageBitmap > solves best in its own right. It seems like toBlob already handles this, > though.) > It doesn't, because toBlob encodes to an image format. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
I just noticed that Canvas already has a Canvas.setContext() method, which seems to do exactly what I'm proposing, even down to clearing the backbuffer on attach. The only difference is that it lives on Canvas instead of the context--the only reason I put it there in my proposal was because this only seemed useful for WebGL. Given that, I think this proposal can be simplified down to just: "put setContext on WorkerCanvas too". On Mon, Oct 21, 2013 at 9:03 PM, Kenneth Russell wrote: > There are some unexpected consequences of the attachToCanvas API > style. For example, what if two contexts use attachToCanvas to target > the same canvas? I left out these details in my initial post in order to see what people thought at a high level before delving into details. Attaching when already attached would replace the old attachment. It's not possible for two workers to attach to the same canvas, since only a single WorkerCanvas can exist for any given canvas; and the original Canvas can't be attached to if a WorkerCanvas was created (eg. it's in the "proxied" mode). > What if one of those contexts is 2D and the other is > WebGL? Currently it's illegal to try to fetch two different context > types for a single Canvas. The current CanvasProxy spec contains > several complex rules for these cases, and they're not easy to > understand. > This is handled by setContext: attaching a context detaches any previously-attached context. > Will it be guaranteed that if you have a WebGL context, attachToCanvas > to canvas1, do some rendering, and then attachToCanvas to canvas2, > that the only remaining buffer in canvas1 is its color buffer? No > depth buffers, multisample buffers, etc. will have to remain for some > reason? > If you reattach to canvas1 in the future, the buffers are cleared, which means you can discard or reuse those buffers as soon as you attach to a different canvas. > > How would WebGL's preserveDrawingBuffer attribute, which is a property > of the context, interact with directing its output to multiple > canvases? > Since attaching the canvas clears it, that would override preserveDrawingBuffer. > Fundamentally I think the behavior is easier to spec, and the > implementation is easier to make correct, if the ultimate destination > is an image rather than a canvas, and the color buffer is transferred > out of the WorkerCanvas in an explicit step. > Whether that's true or not, making things easy for the user takes priority over making things easy for spec writers and implementation. On Tue, Oct 22, 2013 at 2:48 AM, Robert O'Callahan wrote: > This code actually does something potentially useful which can't easily be > done with attachToCanvas: generating a series of images as fast as possible > which will be processed on another thread in some way other than just > rendering them on the screen. (E.g., be encoded into an animated image or > video file.) > (This is a proposal for attachToCanvas--now setContext--not against transferToImageBitmap, if there are use cases that transferToImageBitmap solves best in its own right. It seems like toBlob already handles this, though.) -- Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Tue, Oct 22, 2013 at 12:39 AM, Glenn Maynard wrote:
> Using ImageBitmap for this has a lot of issues. It requires synchronizing
> with scripts in the UI thread. It requires manualling resize your canvas
> repeatedly to fit different destinations. It also may potentially create
> lots of backbuffers. Here's an example of code using ImageBitmap
> incorrectly, leading to excess memory allocation:
>
> function render()
> {
> var canvas = myWorkerCanvas;
> renderTo(canvas);
> var buffer = canvas.transferToImageBitmap();
> postMessage(buffer);
> }
> setTimeout(render, 1);
>
This code actually does something potentially useful which can't easily be
done with attachToCanvas: generating a series of images as fast as possible
which will be processed on another thread in some way other than just
rendering them on the screen. (E.g., be encoded into an animated image or
video file.)
Rob
--
Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w *
*
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 4:34 PM, Glenn Maynard wrote: > On Mon, Oct 21, 2013 at 6:08 PM, Kenneth Russell wrote: >> >> > Using ImageBitmap for this has a lot of issues. It requires >> > synchronizing >> > with scripts in the UI thread. >> >> This isn't difficult, and amounts to a few additional lines of code in >> the main thread's onmessage handler. > > > Synchronization with the UI thread isn't bad because it's difficult. > Avoiding synchronization with the main thread has been raised as a desirable > goal: > http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0152.html > "including that it isn't possible to render from a worker without > synchronizing with the main thread." > > (My previous comments on this are here: > http://www.mail-archive.com/whatwg@lists.whatwg.org/msg35959.html) > >> >> The ImageBitmap style proposal has another significant advantage in >> that it allows a single canvas context to present results in multiple >> output regions on the page. > > > You can do that. You just create a WorkerCanvas for each canvas you want to > present to, hand them to the worker, then attachToCanvas in the worker to > switch from canvas to canvas. (That's orthogonal to explicitpresent.) OK, I misunderstood that part of your attachToCanvas proposal. There are some unexpected consequences of the attachToCanvas API style. For example, what if two contexts use attachToCanvas to target the same canvas? What if one of those contexts is 2D and the other is WebGL? Currently it's illegal to try to fetch two different context types for a single Canvas. The current CanvasProxy spec contains several complex rules for these cases, and they're not easy to understand. Will it be guaranteed that if you have a WebGL context, attachToCanvas to canvas1, do some rendering, and then attachToCanvas to canvas2, that the only remaining buffer in canvas1 is its color buffer? No depth buffers, multisample buffers, etc. will have to remain for some reason? How would WebGL's preserveDrawingBuffer attribute, which is a property of the context, interact with directing its output to multiple canvases? Fundamentally I think the behavior is easier to spec, and the implementation is easier to make correct, if the ultimate destination is an image rather than a canvas, and the color buffer is transferred out of the WorkerCanvas in an explicit step. -Ken >> >> This sort of resource exhaustion is certainly possible, but I view >> this downside as smaller than the upside of addressing both of the >> above use cases. > > > I can only find one thing above that you might be referring to as a use case > (the one I replied to immediately above). What was the other? > > -- > Glenn Maynard >
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 6:08 PM, Kenneth Russell wrote: > > Using ImageBitmap for this has a lot of issues. It requires > synchronizing > > with scripts in the UI thread. > > This isn't difficult, and amounts to a few additional lines of code in > the main thread's onmessage handler. > Synchronization with the UI thread isn't bad because it's difficult. Avoiding synchronization with the main thread has been raised as a desirable goal: http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0152.html"including that it isn't possible to render from a worker without synchronizing with the main thread." (My previous comments on this are here: http://www.mail-archive.com/whatwg@lists.whatwg.org/msg35959.html) > The ImageBitmap style proposal has another significant advantage in > that it allows a single canvas context to present results in multiple > output regions on the page. > You can do that. You just create a WorkerCanvas for each canvas you want to present to, hand them to the worker, then attachToCanvas in the worker to switch from canvas to canvas. (That's orthogonal to explicitpresent.) > This sort of resource exhaustion is certainly possible, but I view > this downside as smaller than the upside of addressing both of the > above use cases. > I can only find one thing above that you might be referring to as a use case (the one I replied to immediately above). What was the other? -- Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 3:39 PM, Glenn Maynard wrote:
> On Sun, Oct 20, 2013 at 11:16 PM, Robert O'Callahan
> wrote:
>>
>> With all these proposals I think it's OK to allow the main thread to do
>> (e.g.) a toDataURL and read what the current contents of the canvas is,
>
>
> We can defer this discussion, since it's not something new to this proposal
> (or any other proposal we're discussing).
>
>
> On Sun, Oct 20, 2013 at 11:33 PM, Robert O'Callahan
> wrote:
>>
>> To me, passing the image data explicitly in an ImageBuffer along with the
>> "present" message seems like a better fit to the workers message-passing
>> model than this proposal, where the data is stored as hidden state in the
>> canvas element with (effectively) a setter in the worker and a getter in the
>> main thread, and that setting and getting has to be coordinated with
>> postMessage for synchronization. The relationship between a "commit" and its
>> "present" has to be deduced by reasoning about the timing of messages,
>> rather than by just reasoning about JS data flow through postMessage.
>
>
> Using ImageBitmap for this has a lot of issues. It requires synchronizing
> with scripts in the UI thread.
This isn't difficult, and amounts to a few additional lines of code in
the main thread's onmessage handler.
The ImageBitmap style proposal has another significant advantage in
that it allows a single canvas context to present results in multiple
output regions on the page.
> It requires manualling resize your canvas
> repeatedly to fit different destinations. It also may potentially create
> lots of backbuffers. Here's an example of code using ImageBitmap
> incorrectly, leading to excess memory allocation:
>
> function render()
> {
> var canvas = myWorkerCanvas;
> renderTo(canvas);
> var buffer = canvas.transferToImageBitmap();
> postMessage(buffer);
> }
> setTimeout(render, 1);
>
> We start with one backbuffer available, render to it (renderTo), peel it off
> the canvas to be displayed somewhere, and toss it off to the main thread.
> (For the sake of the example, the main thread is busy and doesn't process it
> immediately.) The worker enters render() again, and when it gets to
> renderTo, a new backbuffer has to be allocated, since the one buffer we have
> is still used by the ImageBuffer and can't be changed. This happens
> repeatedly, creating new backbuffers each time, since none of them can be
> reused.
>
> This is an extreme example, but if this ever happens even once, it means
> potentially allocating an extra backbuffer.
This sort of resource exhaustion is certainly possible, but I view
this downside as smaller than the upside of addressing both of the
above use cases.
-Ken
>>
>> This proposal also requires that whenever a worker is going to return
>> image data to the main thread, the main thread must start things off by
>> creating a canvas element. It's also not possible for a worker to spawn off
>> sub-workers to do drawing (at least, not without some really ugly
>> coordination with the main thread.)
>
>
> Sure it is. If you want an offscreen buffer, you just "new WorkerCanvas()".
> This is unrelated to offscreen drawing.
>
> --
> Glenn Maynard
>
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Sun, Oct 20, 2013 at 11:16 PM, Robert O'Callahan
wrote:
> With all these proposals I think it's OK to allow the main thread to do
> (e.g.) a toDataURL and read what the current contents of the canvas is,
>
We can defer this discussion, since it's not something new to this proposal
(or any other proposal we're discussing).
On Sun, Oct 20, 2013 at 11:33 PM, Robert O'Callahan wrote:
> To me, passing the image data explicitly in an ImageBuffer along with the
> "present" message seems like a better fit to the workers message-passing
> model than this proposal, where the data is stored as hidden state in the
> canvas element with (effectively) a setter in the worker and a getter in
> the main thread, and that setting and getting has to be coordinated with
> postMessage for synchronization. The relationship between a "commit" and
> its "present" has to be deduced by reasoning about the timing of messages,
> rather than by just reasoning about JS data flow through postMessage.
>
Using ImageBitmap for this has a lot of issues. It requires synchronizing
with scripts in the UI thread. It requires manualling resize your canvas
repeatedly to fit different destinations. It also may potentially create
lots of backbuffers. Here's an example of code using ImageBitmap
incorrectly, leading to excess memory allocation:
function render()
{
var canvas = myWorkerCanvas;
renderTo(canvas);
var buffer = canvas.transferToImageBitmap();
postMessage(buffer);
}
setTimeout(render, 1);
We start with one backbuffer available, render to it (renderTo), peel it
off the canvas to be displayed somewhere, and toss it off to the main
thread. (For the sake of the example, the main thread is busy and doesn't
process it immediately.) The worker enters render() again, and when it
gets to renderTo, a new backbuffer has to be allocated, since the one
buffer we have is still used by the ImageBuffer and can't be changed. This
happens repeatedly, creating new backbuffers each time, since none of them
can be reused.
This is an extreme example, but if this ever happens even once, it means
potentially allocating an extra backbuffer.
> This proposal also requires that whenever a worker is going to return
> image data to the main thread, the main thread must start things off by
> creating a canvas element. It's also not possible for a worker to spawn off
> sub-workers to do drawing (at least, not without some really ugly
> coordination with the main thread.)
>
Sure it is. If you want an offscreen buffer, you just "new
WorkerCanvas()". This is unrelated to offscreen drawing.
--
Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Sun, Oct 20, 2013 at 5:33 PM, Glenn Maynard wrote:
> Example:
>
>
>
> var canvas = document.querySelector("#canvas");
> var worker = createWorker();
> worker.postMessage({
> cmd: "init",
> canvas: canvas.getWorkerCanvas(),
> });
>
> worker.onmessage = function(e)
> {
> // The worker told us that a frame has been committed. Present it for
> display.
> canvas.present();
>
> // Tell the worker that it should start rendering the next frame.
> worker.postMessage({cmd: "update"});
>
> // Do any DOM changes here, to synchronize them with displaying the new
> canvas.
> updateUI();
> }
>
>
> Worker:
>
> onmessage = function(e)
> {
> // On initialization only:
> if(e.data.cmd == "init")
> canvas = e.data.canvas;
>
> // Render our scene.
> renderFrame(canvas);
>
> // Commit the scene.
> canvas.commit();
>
> // Tell the main thread that the frame is ready.
> postMessage("present");
> }
> function renderFrame(workerCanvas) { }
>
To me, passing the image data explicitly in an ImageBuffer along with the
"present" message seems like a better fit to the workers message-passing
model than this proposal, where the data is stored as hidden state in the
canvas element with (effectively) a setter in the worker and a getter in
the main thread, and that setting and getting has to be coordinated with
postMessage for synchronization. The relationship between a "commit" and
its "present" has to be deduced by reasoning about the timing of messages,
rather than by just reasoning about JS data flow through postMessage.
This proposal also requires that whenever a worker is going to return image
data to the main thread, the main thread must start things off by creating
a canvas element. It's also not possible for a worker to spawn off
sub-workers to do drawing (at least, not without some really ugly
coordination with the main thread.)
Rob
--
Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w *
*
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 4:46 AM, Glenn Maynard wrote: > On Sun, Oct 20, 2013 at 9:26 PM, Kyle Huey wrote: > > > On Sun, Oct 20, 2013 at 11:33 PM, Glenn Maynard >wrote: > > > >> It must not be possible for the UI thread to detect whether present() > did > >> anything--if there's no frame in the ready buffer, nothing changes and > the > >> UI thread can't detect this. Similarly, it must not be possible for the > >> rendering thread to detect if the ready frame has been presented. These > >> rules are to prevent exposing asynchronous behavior. > >> > > > > Well you can readback from es, so how is that going to work? > > However it works today, since CanvasProxy needs the same thing. If a > CanvasProxy/WorkerCanvas exists for a canvas, you should have to use a > toBlob method on that, and calls to that (and earlier calls in progress) on > the Canvas itself should fail. (If CanvasProxy isn't doing that it seems > like a bug.) > With all these proposals I think it's OK to allow the main thread to do (e.g.) a toDataURL and read what the current contents of the canvas is, and possibly observe that they have changed. This is already possible today with drawing video elements to a canvas; you can observe that the current video frame sometimes changes. (It is of course essential that you don't read a partially updated frame.) If we don't want that, one way to fix it is to lock the frame used by a particular script execution. So the first time you do a drawImage(video), or a toDataURL within a particular script execution, we lock the current contents of the video element or the canvas element for the purposes of that script execution and ensure that any further usage of that element within this script execution returns the same data. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
(Whoops. Why did Gmail send that as my work email? It shouldn't have made it through to the list, since it's not subscribed...) On Sun, Oct 20, 2013 at 9:26 PM, Kyle Huey wrote: > On Sun, Oct 20, 2013 at 11:33 PM, Glenn Maynard wrote: > >> It must not be possible for the UI thread to detect whether present() did >> anything--if there's no frame in the ready buffer, nothing changes and the >> UI thread can't detect this. Similarly, it must not be possible for the >> rendering thread to detect if the ready frame has been presented. These >> rules are to prevent exposing asynchronous behavior. >> > > Well you can readback from es, so how is that going to work? > However it works today, since CanvasProxy needs the same thing. If a CanvasProxy/WorkerCanvas exists for a canvas, you should have to use a toBlob method on that, and calls to that (and earlier calls in progress) on the Canvas itself should fail. (If CanvasProxy isn't doing that it seems like a bug.)
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Sun, Oct 20, 2013 at 11:33 PM, Glenn Maynard wrote: > It must not be possible for the UI thread to detect whether present() did > anything--if there's no frame in the ready buffer, nothing changes and the > UI thread can't detect this. Similarly, it must not be possible for the > rendering thread to detect if the ready frame has been presented. These > rules are to prevent exposing asynchronous behavior. > Well you can readback from es, so how is that going to work? - Kyle
[whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
(This is a spin-off thread from "Canvas in workers". Some of this is
written in terms of the WorkerCanvas proposal, but it works fine with the
current CanvasProxy API. I'm skipping some steps here and going straight
to a proposal, since my main goal at the moment is to detangle this from
the other thread...)
Here's a way to synchronize updates to DOM changes, so scenes rendered in a
worker only appear when the UI thread is ready for them to be.
- Add a flag to the Canvas to enable this. For now, let's call this
explicitpresent, eg. .
- When a script finishes rendering (eg. calls commit()), the buffer is not
automatically displayed. Instead, it's simply made available to be
displayed.
- Add a method, Canvas.present(), to present the most recently-available
frame.
To describe this in terms of triple-buffering, you have three buffers: a
rendering buffer (aka the backbuffer), a display buffer (aka the
front-buffer), and a ready buffer. You render (possibly in a worker) to
the rendering buffer. When you're finished, you call commit(), and the
rendering buffer and the ready buffer are swapped. Now that a new frame is
ready, you can call canvas.present() to swap the ready buffer and the
display buffer. Essentially, that's it.
You don't actually need to allocate a third buffer, as long as the user
doesn't start rendering a new frame before present()ing the previous one.
This could be a behind-the-scenes optimization to avoid the extra memory
cost--only allocate a third buffer if actually needed.
It must not be possible for the UI thread to detect whether present() did
anything--if there's no frame in the ready buffer, nothing changes and the
UI thread can't detect this. Similarly, it must not be possible for the
rendering thread to detect if the ready frame has been presented. These
rules are to prevent exposing asynchronous behavior.
Example:
var canvas = document.querySelector("#canvas");
var worker = createWorker();
worker.postMessage({
cmd: "init",
canvas: canvas.getWorkerCanvas(),
});
worker.onmessage = function(e)
{
// The worker told us that a frame has been committed. Present it for
display.
canvas.present();
// Tell the worker that it should start rendering the next frame.
worker.postMessage({cmd: "update"});
// Do any DOM changes here, to synchronize them with displaying the new
canvas.
updateUI();
}
Worker:
onmessage = function(e)
{
// On initialization only:
if(e.data.cmd == "init")
canvas = e.data.canvas;
// Render our scene.
renderFrame(canvas);
// Commit the scene.
canvas.commit();
// Tell the main thread that the frame is ready.
postMessage("present");
}
function renderFrame(workerCanvas) { }
--
Glenn Maynard
