Re: [whatwg] Canvas in workers
We talked through this proposal with a lot of Mozilla people in a meeting and collectively decided that we don't care about the case of workers that commit multiple frames to a canvas without yielding --- at least for now. So we want to remove commit() and copy the main-thread semantics that a canvas frame is eligible for presentation whenever script is not running in the worker. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
This is not the current WebGL semantics: WebGL presents its drawing buffer to the HTML page compositor immediately before a compositing operation[...] I can't find where this is specified for canvas, though I assume it's all implied. -Jeff - Original Message - From: Robert O'Callahan rob...@ocallahan.org To: Justin Novosad ju...@google.com Cc: WHAT Working Group wha...@whatwg.org, Glenn Maynard gl...@zewt.org, Kenneth Russell k...@google.com, Kyle Huey m...@kylehuey.com Sent: Thursday, October 24, 2013 2:24:27 AM Subject: Re: [whatwg] Canvas in workers We talked through this proposal with a lot of Mozilla people in a meeting and collectively decided that we don't care about the case of workers that commit multiple frames to a canvas without yielding --- at least for now. So we want to remove commit() and copy the main-thread semantics that a canvas frame is eligible for presentation whenever script is not running in the worker. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
- Original Message - From: Robert O'Callahan rob...@ocallahan.org We talked through this proposal with a lot of Mozilla people in a meeting and collectively decided that we don't care about the case of workers that commit multiple frames to a canvas without yielding --- at least for now. So we want to remove commit() and copy the main-thread semantics that a canvas frame is eligible for presentation whenever script is not running in the worker. On Thu, Oct 24, 2013 at 7:25 AM, Jeff Gilbert jgilb...@mozilla.com wrote: This is not the current WebGL semantics: WebGL presents its drawing buffer to the HTML page compositor immediately before a compositing operation[...] (Can you please quote correctly? Having one person top-quoting makes a mess of the whole thread, and it looked like you were saying that the WebGL spec language you were quoting was incorrect.) The assumption WebGL is making here is that compositing is a synchronous task in the event loop, which happens while no script is running. That is, the semantics Robert describes are the same as what the WebGL spec is trying to say. That's not necessarily how compositing actually works, though, and that language also won't make sense with threaded rendering. It might be better for WebGL to define this using the global script clean-up jobs task that HTML now defines. http://www.whatwg.org/specs/web-apps/current-work/#run-the-global-script-clean-up-jobs I'd recommend spinning off a separate thread if we want to go into this further. -- Glenn Maynard
Re: [whatwg] Canvas in workers
On Thu, Oct 24, 2013 at 6:59 AM, Glenn Maynard gl...@zewt.org wrote: - Original Message - From: Robert O'Callahan rob...@ocallahan.org We talked through this proposal with a lot of Mozilla people in a meeting and collectively decided that we don't care about the case of workers that commit multiple frames to a canvas without yielding --- at least for now. So we want to remove commit() and copy the main-thread semantics that a canvas frame is eligible for presentation whenever script is not running in the worker. On Thu, Oct 24, 2013 at 7:25 AM, Jeff Gilbert jgilb...@mozilla.com wrote: This is not the current WebGL semantics: WebGL presents its drawing buffer to the HTML page compositor immediately before a compositing operation[...] (Can you please quote correctly? Having one person top-quoting makes a mess of the whole thread, and it looked like you were saying that the WebGL spec language you were quoting was incorrect.) The assumption WebGL is making here is that compositing is a synchronous task in the event loop, which happens while no script is running. That is, the semantics Robert describes are the same as what the WebGL spec is trying to say. That's not necessarily how compositing actually works, though, and that language also won't make sense with threaded rendering. It might be better for WebGL to define this using the global script clean-up jobs task that HTML now defines. http://www.whatwg.org/specs/web-apps/current-work/#run-the-global-script-clean-up-jobs I'd recommend spinning off a separate thread if we want to go into this further. The time that compositing occurs is already specified by the HTML event loop processing model (7.1.4.2): http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#processing-model-4 An event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop must continually run through the following steps for as long as it exists: 1. Run the oldest taskhttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#concept-task on one of the event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop 's task queueshttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#task-queue, if any, ignoring tasks whose associated Documenthttp://www.whatwg.org/specs/web-apps/current-work/multipage/dom.html#documents are not fully activehttp://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#fully-active. The user agent may pick any task queuehttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#task-queue . 2. If the storage mutexhttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#storage-mutex is now owned by the event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop, release it so that it is once again free. 3. If a task was run in the first step above, remove that task from its task queuehttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#task-queue . 4. If this event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop is not a worker's event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop, run these substeps: 1. Perform a microtask checkpointhttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#perform-a-microtask-checkpoint . 2. Provide a stable statehttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#provide-a-stable-state . 3. If necessary, update the rendering or user interface of any Documenthttp://www.whatwg.org/specs/web-apps/current-work/multipage/dom.html#document or browsing contexthttp://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#browsing-context to reflect the current state. 5. Otherwise, if this event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop is running for a WorkerGlobalScopehttp://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html#workerglobalscope, but there are no events in the event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop 's task queueshttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#task-queue and the WorkerGlobalScopehttp://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html#workerglobalscope object's closinghttp://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html#dom-workerglobalscope-closing flag is true, then destroy the event loophttp://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#event-loop, aborting these steps. 6. Return to the
Re: [whatwg] Canvas in workers
On Tue, Oct 22, 2013 at 5:59 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Oct 22, 2013 at 10:44 PM, Robert O'Callahan rob...@ocallahan.org wrote: No problem at all. Can you do it? I need to get a WHATWG account :-). OK, I added the proposal here: http://wiki.whatwg.org/wiki/WorkerCanvas A couple of changes from the previous version: -- Added ImageBitmap.transferToImage. -- Removed HTMLImageElement.srcObject (though it may come back in other contexts) -- Made ImageBitmap Transferable. This makes it possible to ensure the worker that generated an ImageBitmap does not keep alive a reference to the buffer. -- Removed ImageBitmaps.close() since the use-cases driving this proposal probably don't need it anymore. Interface Transferable is not defined.
Re: [whatwg] Canvas in workers
On Wed, Oct 23, 2013 at 11:46 AM, Justin Novosad ju...@google.com wrote: Interface Transferable is not defined. Never mind. :-)
Re: [whatwg] Canvas in workers
On Mon, Oct 21, 2013 at 8:03 AM, Glenn Maynard gl...@zewt.org wrote: On Sun, Oct 20, 2013 at 11:53 PM, Robert O'Callahan rob...@ocallahan.org wrote: Glenn, taking a step back for a bit, is there anything in https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal that you would actually object to? IOW, is there anything there that you would think is completely superfluous to the platform if all your proposals were to be adopted as well? I have no objection to the overall change from CanvasProxy to WorkerCanvas, eg. the stuff in Kyle's original mail to the thread. (Being able to settle on that is one of the reasons I've tried to detach discussion for the other use cases.) I'd only recommend leaving out transferToImageBitmap, srcObject and ImageBitmap.close() parts. I do think that would be redundant with with present proposal. They can always be added later, and leaving them out keeps the WorkerCanvas proposal itself focused. Robert, please don't remove those APIs from your proposal. They're needed in order to address known use cases, and splitting them off will make it difficult to understand how they interact with WorkerCanvas later. I would like to suggest changing the 'srcObject' property on HTMLImageElement into some sort of method taking ImageBitmap as argument. If an ImageBitmap had been previously set against the HTMLImageElement, the method would automatically call 'close()' against it. Fundamentally there should be some easy way to repeatedly update the HTMLImageElement without having to query its previous ImageBitmap and call close() against it before setting srcObject. Would you consider copying https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal to http://wiki.whatwg.org/wiki/Category:Proposals so that it's easier to collaborate on it?
Re: [whatwg] Canvas in workers
On Tue, Oct 22, 2013 at 7:31 PM, Kenneth Russell k...@google.com wrote: Robert, please don't remove those APIs from your proposal. They're needed in order to address known use cases, and splitting them off will make it difficult to understand how they interact with WorkerCanvas later. Yes, I think it's a good idea to specify a complete set of APIs that fit together logically and if there are some we don't need, we can remove them or just delay implementing them until they're needed. I would like to suggest changing the 'srcObject' property on HTMLImageElement into some sort of method taking ImageBitmap as argument. If an ImageBitmap had been previously set against the HTMLImageElement, the method would automatically call 'close()' against it. Fundamentally there should be some easy way to repeatedly update the HTMLImageElement without having to query its previous ImageBitmap and call close() against it before setting srcObject. Hmm. I'm not sure how this should work. Maybe instead we should use canvas elements and define ImageBitmap.transferToCanvas(HTMLCanvasElement). That would mitigate Glenn's argument about having to change element types too. Would you consider copying https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal to http://wiki.whatwg.org/wiki/Category:Proposals so that it's easier to collaborate on it? No problem at all. Can you do it? I need to get a WHATWG account :-). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Tue, Oct 22, 2013 at 1:44 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Tue, Oct 22, 2013 at 7:31 PM, Kenneth Russell k...@google.com wrote: Robert, please don't remove those APIs from your proposal. They're needed in order to address known use cases, and splitting them off will make it difficult to understand how they interact with WorkerCanvas later. Yes, I think it's a good idea to specify a complete set of APIs that fit together logically and if there are some we don't need, we can remove them or just delay implementing them until they're needed. I would like to suggest changing the 'srcObject' property on HTMLImageElement into some sort of method taking ImageBitmap as argument. If an ImageBitmap had been previously set against the HTMLImageElement, the method would automatically call 'close()' against it. Fundamentally there should be some easy way to repeatedly update the HTMLImageElement without having to query its previous ImageBitmap and call close() against it before setting srcObject. Hmm. I'm not sure how this should work. Maybe instead we should use canvas elements and define ImageBitmap.transferToCanvas(HTMLCanvasElement). That would mitigate Glenn's argument about having to change element types too. Using a Canvas as the target for displaying an ImageBitmap is fraught with problems. Because Canvases are rendering targets themselves, transferring in an ImageBitmap has to interoperate somehow with the canvas's rendering context as well as other APIs like toDataURL. The DrawingBuffer proposal in http://wiki.whatwg.org/wiki/CanvasInWorkers was admittedly complex, but did properly handle fully detaching a canvas's framebuffer and attaching it to another one. ImageBitmap, being semantically read-only, isn't well suited for this task. The idea to use an HTMLImageElement to display the ImageBitmaps is elegant and I would like to see it explored, including prototyping it behind an experimental flag and seeing how well it works. Would you consider copying https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal to http://wiki.whatwg.org/wiki/Category:Proposals so that it's easier to collaborate on it? No problem at all. Can you do it? I need to get a WHATWG account :-). Yes, I'll take care of the initial draft. The whatwg was pretty responsive when I got my account though. -Ken Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] Canvas in workers
I got an account and I'm uploading the proposal now. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
Great. On Tue, Oct 22, 2013 at 2:54 PM, Robert O'Callahan rob...@ocallahan.org wrote: I got an account and I'm uploading the proposal now. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] Canvas in workers
On Tue, Oct 22, 2013 at 10:44 PM, Robert O'Callahan rob...@ocallahan.orgwrote: No problem at all. Can you do it? I need to get a WHATWG account :-). OK, I added the proposal here: http://wiki.whatwg.org/wiki/WorkerCanvas A couple of changes from the previous version: -- Added ImageBitmap.transferToImage. -- Removed HTMLImageElement.srcObject (though it may come back in other contexts) -- Made ImageBitmap Transferable. This makes it possible to ensure the worker that generated an ImageBitmap does not keep alive a reference to the buffer. -- Removed ImageBitmaps.close() since the use-cases driving this proposal probably don't need it anymore. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Sun, Oct 20, 2013 at 11:53 PM, Robert O'Callahan rob...@ocallahan.orgwrote: Glenn, taking a step back for a bit, is there anything in https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal that you would actually object to? IOW, is there anything there that you would think is completely superfluous to the platform if all your proposals were to be adopted as well? I have no objection to the overall change from CanvasProxy to WorkerCanvas, eg. the stuff in Kyle's original mail to the thread. (Being able to settle on that is one of the reasons I've tried to detach discussion for the other use cases.) I'd only recommend leaving out transferToImageBitmap, srcObject and ImageBitmap.close() parts. I do think that would be redundant with with present proposal. They can always be added later, and leaving them out keeps the WorkerCanvas proposal itself focused. -- Glenn Maynard
Re: [whatwg] Canvas in workers
On Sat, Oct 19, 2013 at 10:11 AM, Robert O'Callahan rob...@ocallahan.orgwrote: It's not clear to me how attachToCanvas works. An application like Google Maps wants to draw to multiple canvases from a worker and then render the updated canvas contents all at once, in synchrony with changes to the DOM made by the main thread. How would you do that with attachToCanvas? That's not the problem attachToCanvas tries to solve. It tries to solve rendering to multiple canvases, without requiring synchronization with the UI thread. I have a proposal for handling synchronization with DOM updates, but I'll post it in a separate thread. (To clarify, this thread is talking about three different things: rendering from a worker to the UI thread, rendering to multiple canvases, and synchronizing rendering in a worker to DOM updates in the main thread. The only reason they're in the same thread is because some proposals are trying to handle two or all three of them together. Trying to do that is leading to unwanted limitations, such as forcing synchronization when you don't want it, and it's making the conversation hard to follow. Since my proposal is orthogonal to the rest--it's separate and compatible with both WorkerCanvas and attachToCanvas--and this thread is discussing too many things at once, I'll move the third to a separate thread.) - If you're rendering in a worker and the eventual target is in the main thread, the worker needs to be careful to not start rendering again until the main thread has assigned the ImageBitmap to where it wants it, and called .close(). You'd need to send a message back to the worker going okay, you can continue now. Otherwise, you'd start rendering before a buffer has been freed up for reuse, and end up creating more backbuffers than you intended (which matters for large screens). This seems easy to get wrong, and attachToCanvas doesn't have this problem. Not if you use transferToImageBitmap. transferToImageBitmap does have this problem. If you transferToImageBitmap to detach your backing store to display it somewhere, then start rendering the next frame without waiting for the ImageBitmap to be given to the target, then as soon as you start rendering you'll end up creating a 3rd rendering buffer. (The present() proposal also has this problem, but users are only affected by it if they're actually synchronizing to DOM updates.) With attachToCanvas, you just size both canvases normally once, and switch between them with a single function call. I'm not sure how helpful this is. In the case of WebGL, the rendering context has resources that need to be sized the same as the destination color buffer, so they'll need to be resized anyway if you're actually using a single context to render to canvases of different sizes. My guess is that the advice will always be don't do that. If you think that a single context outputting to multiple canvases fundamentally won't work well with canvases of different sizes, then forget about the feature. When you attachToCanvas, you're attaching that canvas's rendering buffer, not just its color buffer. In WebGL terms, each canvas is a Framebuffer, with all its associated Renderbuffers. Attaching the context to a canvas is like using bindFramebuffer to render to its backing store. I believe these are minor changes, especially compared to moving drawing to a worker. Moving drawing to a worker is unrelated to drawing to multiple canvases. I don't agree that it's a minor change: today you can create a black-box API that renders to a context, without caring about the DOM at all. If you have to move the image to an img to display it, suddenly you have to care about the DOM to render. It breaks the basic model. -- Glenn Maynard
Re: [whatwg] Canvas in workers
On Sun, Oct 20, 2013 at 5:29 PM, Glenn Maynard gl...@zewt.org wrote: That's not the problem attachToCanvas tries to solve. It tries to solve rendering to multiple canvases, without requiring synchronization with the UI thread. I have a proposal for handling synchronization with DOM updates, but I'll post it in a separate thread. OK. - If you're rendering in a worker and the eventual target is in the main thread, the worker needs to be careful to not start rendering again until the main thread has assigned the ImageBitmap to where it wants it, and called .close(). You'd need to send a message back to the worker going okay, you can continue now. Otherwise, you'd start rendering before a buffer has been freed up for reuse, and end up creating more backbuffers than you intended (which matters for large screens). This seems easy to get wrong, and attachToCanvas doesn't have this problem. Not if you use transferToImageBitmap. transferToImageBitmap does have this problem. If you transferToImageBitmap to detach your backing store to display it somewhere, then start rendering the next frame without waiting for the ImageBitmap to be given to the target, then as soon as you start rendering you'll end up creating a 3rd rendering buffer. OK. If you think that a single context outputting to multiple canvases fundamentally won't work well with canvases of different sizes, then forget about the feature. When you attachToCanvas, you're attaching that canvas's rendering buffer, not just its color buffer. In WebGL terms, each canvas is a Framebuffer, with all its associated Renderbuffers. Attaching the context to a canvas is like using bindFramebuffer to render to its backing store. This is actually a disadvantage for attachToCanvas in some situations. If I just want to generate N different rendered images (of the same size) (e.g. http://www.turbosquid.com/Search/3D-Models/Vehicle/Car), the only thing I want N of is the final color buffer. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
Glenn, taking a step back for a bit, is there anything in https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal that you would actually object to? IOW, is there anything there that you would think is completely superfluous to the platform if all your proposals were to be adopted as well? Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Sat, Oct 19, 2013 at 1:28 AM, Glenn Maynard gl...@zewt.org wrote: I'd like to hear thoughts on the context.attachToCanvas approach. I think it has important advantages over ImageBitmap: - ImageBitmap requires the user to call close(). If the user forgets, or doesn't know, or misses it in some code paths, the problems caused aren't obvious. Worse, they may only appear in some implementations and not others, depending on GC strategies. attachToCanvas doesn't need cleanup in the first place, which is a nicer solution--there's nothing for the user to get wrong. It's not clear to me how attachToCanvas works. An application like Google Maps wants to draw to multiple canvases from a worker and then render the updated canvas contents all at once, in synchrony with changes to the DOM made by the main thread. How would you do that with attachToCanvas? - If you're rendering in a worker and the eventual target is in the main thread, the worker needs to be careful to not start rendering again until the main thread has assigned the ImageBitmap to where it wants it, and called .close(). You'd need to send a message back to the worker going okay, you can continue now. Otherwise, you'd start rendering before a buffer has been freed up for reuse, and end up creating more backbuffers than you intended (which matters for large screens). This seems easy to get wrong, and attachToCanvas doesn't have this problem. Not if you use transferToImageBitmap. - With ImageBitmap, you need to create a helper canvas, then each time you render to a new target, you need to resize the canvas to match where it'll eventually go, so the resulting ImageBitmap is the size of its destination. (This may also need to be carefully optimized, so the implementation doesn't actually resize the backing store every time its size changes.) With attachToCanvas, you just size both canvases normally once, and switch between them with a single function call. I'm not sure how helpful this is. In the case of WebGL, the rendering context has resources that need to be sized the same as the destination color buffer, so they'll need to be resized anyway if you're actually using a single context to render to canvases of different sizes. My guess is that the advice will always be don't do that. - attachToCanvas matches the way Canvas works today: you create a Canvas, put it in the document (if it's for display), and render to it. For two canvases, you'd just add a second Canvas, and toggle as needed. With ImageBitmap, you have to restructure everything as soon as you want a second canvas, since you'd want to have a single offscreen Canvas for rendering, and to have img elements in the document instead of canvases. I believe these are minor changes, especially compared to moving drawing to a worker. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Tue, Oct 15, 2013 at 5:30 PM, Kenneth Russell k...@google.com wrote: On Tue, Oct 15, 2013 at 4:41 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Wed, Oct 16, 2013 at 11:55 AM, Kenneth Russell k...@google.com wrote: On Mon, Oct 14, 2013 at 1:34 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. Note that the CanvasInWorkers draft solves some other longstanding issues not addressed by the WorkerCanvas proposal. It provides the ability to render to multiple canvases from a single context, whether workers are involved or not. That may be a useful feature, but I'd like to see it justified in its own right. There has been a lot of developer feedback on the WebGL mailing lists over the past couple of years about exactly this feature. Web sites like Turbosquid want to present lots of little thumbnails of models -- see for example http://www.turbosquid.com/Search/3D-Models/Vehicle/Car -- and have them be interactive. It's too resource-intensive to create a separate WebGL context for each. The most direct solution is to allow one context to render to multiple canvases. It achieves ideal memory utilization by being very explicit in the API, without the need for extensive and subtle optimizations behind the scenes. We can be more explicit with ImageBitmaps. We could provide WorkerCanvas.transferToImageBitmap which transfers the current canvas contents to an ImageBitmap and clears the canvas. (Canvas implementations are already optimized to support a zero-cost cleared state, because existing benchmarks require it.) Sharing ImageBitmap contents across threads during structured clone is not subtle. We can add an HTMLImageElement.srcObject attribute which could take a Blob or an ImageBitmap to enable explicit zero-copy rendering of ImageBitmaps. Would that be explicit enough for you? Yes, that generally sounds good. Personally I think high-performance manipulation of ImageBitmaps would be more generally useful than detachable DrawingBuffers, and would be easier for authors to understand. If you squint, WorkerCanvas.transferToImageBitmap is similar to detaching a DrawingBuffer. But I don't see a need to reattach a buffer to a canvas for further drawing. Do you? Not immediately. The ability to transfer out the canvas's contents, and render them in an HTMLImageElement without incurring an extra blit, should address the Maps team's requirements. Actually, adding transferToImageBitmap to HTMLCanvasElement as well would address the use case of rendering to multiple targets using one context. Instead of using multiple canvases as the targets, one would simply use multiple images. That sounds appealing. If WorkerCanvas is changed so that its width and height are mutable within the worker as you mentioned above, it sounds like it's addressing the known use cases. Capturing Glenn Maynard's feedback from the other thread started by Rik Cabanier, Glenn made a good point that there needs to be a way to explicitly deallocate the ImageBitmap. Otherwise, the JavaScript objects will have to be garbage collected before the GPU resource (texture) it references can be freed, and that will not work -- GPU resources will quickly pile up. Would it be acceptable to say that setting the 'src' property of an HTMLImageElement to an ImageBitmap neuters the incoming ImageBitmap? If not, would it be feasible to add another method to HTMLImageElement like setToImageBitmap which does this? -Ken It's worth considering whether a change to the CanvasInWorkers proposal could support presenting frames directly from the worker. Sure, by adding a commit() method to DrawingBuffer. Right? I'm not exactly sure how it would be done. In the proposal as written, the DrawingBuffer's not shared between threads, only transferred. -Ken Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] Canvas in workers
On Fri, Oct 18, 2013 at 2:06 PM, Kenneth Russell k...@google.com wrote: Capturing Glenn Maynard's feedback from the other thread started by Rik Cabanier, Glenn made a good point that there needs to be a way to explicitly deallocate the ImageBitmap. Otherwise, the JavaScript objects will have to be garbage collected before the GPU resource (texture) it references can be freed, and that will not work -- GPU resources will quickly pile up. I'd like to hear thoughts on the context.attachToCanvas approach. I think it has important advantages over ImageBitmap: - ImageBitmap requires the user to call close(). If the user forgets, or doesn't know, or misses it in some code paths, the problems caused aren't obvious. Worse, they may only appear in some implementations and not others, depending on GC strategies. attachToCanvas doesn't need cleanup in the first place, which is a nicer solution--there's nothing for the user to get wrong. - If you're rendering in a worker and the eventual target is in the main thread, the worker needs to be careful to not start rendering again until the main thread has assigned the ImageBitmap to where it wants it, and called .close(). You'd need to send a message back to the worker going okay, you can continue now. Otherwise, you'd start rendering before a buffer has been freed up for reuse, and end up creating more backbuffers than you intended (which matters for large screens). This seems easy to get wrong, and attachToCanvas doesn't have this problem. - With ImageBitmap, you need to create a helper canvas, then each time you render to a new target, you need to resize the canvas to match where it'll eventually go, so the resulting ImageBitmap is the size of its destination. (This may also need to be carefully optimized, so the implementation doesn't actually resize the backing store every time its size changes.) With attachToCanvas, you just size both canvases normally once, and switch between them with a single function call. - attachToCanvas matches the way Canvas works today: you create a Canvas, put it in the document (if it's for display), and render to it. For two canvases, you'd just add a second Canvas, and toggle as needed. With ImageBitmap, you have to restructure everything as soon as you want a second canvas, since you'd want to have a single offscreen Canvas for rendering, and to have img elements in the document instead of canvases. Here's the example from the other thread to consolidate the discussion. If you're in a worker, canvas and canvas2 can both be WorkerCanvases posted from the main thread or created directly: var canvas = document.querySelector(.canvas1); var gl = canvas.getContext(webgl); loadExpensiveResources(gl); drawStuff(gl); var canvas2 = document.querySelector(.canvas2); gl.attachToCanvas(canvas2); drawStuff(gl); // don't need to loadExpensiveResources again -- Glenn Maynard
Re: [whatwg] Canvas in workers
Le 16/10/2013 01:26, Robert O'Callahan a écrit : On Wed, Oct 16, 2013 at 11:55 AM, David Bruant bruan...@gmail.com mailto:bruan...@gmail.com wrote: If the main thread is blocked, the app drops frames anyway, no? Not necessarily. We can allow workers to present frames to the compositor without synchronizing with the main thread. ... oh... so the UI could be updated even if JS is blocking... the future is bright :-) David
Re: [whatwg] Canvas in workers
On Tue, Oct 15, 2013 at 8:30 PM, Kenneth Russell k...@google.com wrote: On Tue, Oct 15, 2013 at 4:41 PM, Robert O'Callahan rob...@ocallahan.org wrote: If you squint, WorkerCanvas.transferToImageBitmap is similar to detaching a DrawingBuffer. But I don't see a need to reattach a buffer to a canvas for further drawing. Do you? Not immediately. The ability to transfer out the canvas's contents, and render them in an HTMLImageElement without incurring an extra blit, should address the Maps team's requirements. WorkerCanvas.copyToImageBitmap could be just as effective with a proper lazy copy-on-write mechanism. It would offer the same performance in cases where you would just need to transfer (as opposed to copy) the buffer, with the added flexibility that it reattaches a new buffer to the canvas, only if needed (at next draw). Also the lazy copy can be skipped if the next draw operation to the canvas context is a clear, in which case the UA only needs to attach an uninitialized buffer. Also, because ImageBitmaps are immutable objects, the API should probably be more like var imageBitmap = createImageBitmap(myWorkerCanvas);
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 8:01 AM, Justin Novosad ju...@google.com wrote: ... oh... so the UI could be updated even if JS is blocking... the future is bright :-) If the UI is all painted in a canvas, then yes. Let's not get ahead of ourselves though. Browsers that have a compositor in a separate thread can present frames without synchronizing with the main thread, but updating a regular (DOM-based) UI would likely require style and layout calculations to be propagated from the main thread to the compositor. Right, that's the only sort of async update we're talking about here: changing what gets composited, and not anything that affects layout or is detectable by scripts. But actually, there's no disagreement. I misread Kenneth's mail as saying your proposal requires synchronization with the main thread, but he actually said this other proposal requires synchronization with the main thread (but has other benefits). I'm not sure how big a problem that synchronization will be. Posting messages should be extremely cheap: essentially free, when you're just transferring an object to a thread in the same process. The UI thread should also not be heavily loaded: you shouldn't have to wait long for the script to receive the object and push it into the canvas. But, even a cost of 2ms would be a massive hit, since that's 12% of the time you have available when rendering at 60 FPS. In practice it might make it harder to sustain smooth 60 FPS animation. For example, if the main thread occasionally spends 4ms doing GC work, you may have a rendering hitch that you wouldn't have otherwise. -- Glenn Maynard
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 5:39 AM, Justin Novosad ju...@google.com wrote: On Tue, Oct 15, 2013 at 8:30 PM, Kenneth Russell k...@google.com wrote: On Tue, Oct 15, 2013 at 4:41 PM, Robert O'Callahan rob...@ocallahan.org wrote: If you squint, WorkerCanvas.transferToImageBitmap is similar to detaching a DrawingBuffer. But I don't see a need to reattach a buffer to a canvas for further drawing. Do you? Not immediately. The ability to transfer out the canvas's contents, and render them in an HTMLImageElement without incurring an extra blit, should address the Maps team's requirements. WorkerCanvas.copyToImageBitmap could be just as effective with a proper lazy copy-on-write mechanism. It would offer the same performance in cases where you would just need to transfer (as opposed to copy) the buffer, with the added flexibility that it reattaches a new buffer to the canvas, only if needed (at next draw). Also the lazy copy can be skipped if the next draw operation to the canvas context is a clear, in which case the UA only needs to attach an uninitialized buffer. I'm assuming that transferToImageBitmap will attach a new buffer to the canvas as well. The semantic would be that the color buffer of the canvas gets transferred to the ImageBitmap, and the canvas gets a new, blank color buffer. (Any auxiliary buffers, like a depth buffer, would also be implicitly cleared.) It's easier to understand how to make the transfer operation efficient than how to optimize the copy-on-write, which requires deeper analysis of the calls made against the canvas and context in order to get peak performance. If both are present then both the createImageBitmap and transfer implementations can be really simple. Do you have a particular objection to including the transfer API? Also, because ImageBitmaps are immutable objects, the API should probably be more like var imageBitmap = createImageBitmap(myWorkerCanvas); Yes, right. That factory method is already spec'ed on the WorkerGlobalScope [1]. It actually returns a Promise, so presumably transferToImageBitmap would have to as well. [1] http://www.whatwg.org/specs/web-apps/current-work/multipage/timers.html#imagebitmapfactories -Ken
Re: [whatwg] Canvas in workers
On Thu, Oct 17, 2013 at 6:35 AM, Kenneth Russell k...@google.com wrote: Yes, right. That factory method is already spec'ed on the WorkerGlobalScope [1]. It actually returns a Promise, so presumably transferToImageBitmap would have to as well. The whole point of transferToImageBitmap is that it's really fast, so I don't see why it has to be async. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 3:53 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Thu, Oct 17, 2013 at 6:35 AM, Kenneth Russell k...@google.com wrote: Yes, right. That factory method is already spec'ed on the WorkerGlobalScope [1]. It actually returns a Promise, so presumably transferToImageBitmap would have to as well. The whole point of transferToImageBitmap is that it's really fast, so I don't see why it has to be async. True. I also wonder why all of the currently spec'ed ImageBitmap creation methods are async. I think asynchrony makes sense when creating ImageBitmaps from blobs, image elements and video elements, which may not be in an immediately accessible state, but creating an ImageBitmap from a Canvas or canvas context (or a WorkerCanvas) could be immediate.
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 1:13 PM, Justin Novosad ju...@google.com wrote: On Wed, Oct 16, 2013 at 3:53 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Thu, Oct 17, 2013 at 6:35 AM, Kenneth Russell k...@google.com wrote: Yes, right. That factory method is already spec'ed on the WorkerGlobalScope [1]. It actually returns a Promise, so presumably transferToImageBitmap would have to as well. The whole point of transferToImageBitmap is that it's really fast, so I don't see why it has to be async. True. I also wonder why all of the currently spec'ed ImageBitmap creation methods are async. I think asynchrony makes sense when creating ImageBitmaps from blobs, image elements and video elements, which may not be in an immediately accessible state, but creating an ImageBitmap from a Canvas or canvas context (or a WorkerCanvas) could be immediate. It would be fine in my opinion for transferToImageBitmap to simply return an ImageBitmap. The suggestion to return a Promise was merely for symmetry with createImageBitmap. While the Promise returned from createImageBitmap(HTMLCanvasElement) can be fulfilled immediately, is it worth introducing a special overload with a different return type? -Ken
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 9:23 PM, Kenneth Russell k...@google.com wrote: While the Promise returned from createImageBitmap(HTMLCanvasElement) can be fulfilled immediately, is it worth introducing a special overload with a different return type? Well if you want a synchronous method a promise is not going to cut it. However, please do not overload the existing method and actually provide a new one. Having one method that can be used both synchronous and asynchronous depending on its argument would be terrible. -- http://annevankesteren.nl/
Re: [whatwg] Canvas in workers
From: whatwg-boun...@lists.whatwg.org [whatwg-boun...@lists.whatwg.org] on behalf of Anne van Kesteren [ann...@annevk.nl] On Wed, Oct 16, 2013 at 9:23 PM, Kenneth Russell k...@google.com wrote: While the Promise returned from createImageBitmap(HTMLCanvasElement) can be fulfilled immediately, is it worth introducing a special overload with a different return type? Well if you want a synchronous method a promise is not going to cut it. However, please do not overload the existing method and actually provide a new one. Having one method that can be used both synchronous and asynchronous depending on its argument would be terrible. It's worth noting that immediately-fulfilled promises are pretty fast, since promises use microtasks.
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 5:29 PM, Domenic Denicola dome...@domenicdenicola.com wrote: From: whatwg-boun...@lists.whatwg.org [whatwg-boun...@lists.whatwg.org] on behalf of Anne van Kesteren [ann...@annevk.nl] On Wed, Oct 16, 2013 at 9:23 PM, Kenneth Russell k...@google.com wrote: While the Promise returned from createImageBitmap(HTMLCanvasElement) can be fulfilled immediately, is it worth introducing a special overload with a different return type? Well if you want a synchronous method a promise is not going to cut it. However, please do not overload the existing method and actually provide a new one. Having one method that can be used both synchronous and asynchronous depending on its argument would be terrible. It's worth noting that immediately-fulfilled promises are pretty fast, since promises use microtasks. Agreed, but even if the returned promise is immediately resolved, the resolution notification (and the associated state transition) will be async and will not happen at least until the current task (JS execution) ends, which is what we would like to avoid by providing a synchronous API. The objective is to remove the complexity of having to use an async API in cases where there is no benefit from being asynchronous.
Re: [whatwg] Canvas in workers
Robert, I think your argument makes sense. The DrawingBuffer mechanism described in http://wiki.whatwg.org/wiki/CanvasInWorkers can be made implicit and hidden under the hood in a way that is just as memory efficient by using the right combination of read locks and copy-on-write. However, I have concerns about how resizing would be handled by the proposed commit mechanism. I think there is no perfect solution, but whatever we do, I think animation smoothness should be a primary concern, so we should avoid blocking any threads and we should avoid dropping frames as much as possible. I like the concept of handling canvas size mismatches that is brought forward in the CanvasInWorkers proposal. If we put that idea into the WorkerCanvas proposal, here is an idea of how commits could handle resizes asynchronously: 1) Main thread executes some JS that changes the size of the canvas, and posts an event to the Worker in order to propagate the change to the WorkerCanvas. 2) On the worker thread, the WorkerCanvas stores the new size, but does not apply it right away to avoid resetting canvas contents mid frame. 3) commit() is called on the WorkerCanvas, causing the current canvas contents to be sent to the main thread for display, and the new canvas size comes into effect (lazily?) 4) The main thread receives the committed pixels. N.B.: The size of the received buffer does not match canvas element's intrinsic size 5) At paint time, the canvas contents get displayed respecting the canvas element's current size. The mismatched pixel buffer is either streched/squashed or clipped/padded to fit. An alternative that was proposed earlier in this (e-mail) thread was for the main thread to reject commits when there is a size mismatch. That does respect the desire to be asynchronous, and it avoid the inconvenience of resizing contents, but I don't like the idea because it means there can be cases where we may have a large number of consecutive dropped frames. Think of drag-resizing for example. We should avoid making that common use case behave poorly. On Mon, Oct 14, 2013 at 4:34 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. There's been some recent discussion in the WebGL WG on this topic and concerns were raised from other parties at Mozilla about the DrawingBuffer proposal above, including that it isn't possible to render from a worker without synchronizing with the main thread. This is a huge limitation. For games and other animated content, achieving a stable frame rate is super important and a key motivation for adding canvas support to workers. My vision for handling the Maps use-cases based on our proposal is this: the worker renders to one or more WorkerCanvases, then does createImageBitmap(canvasContext), then ships the ImageBitmap(s) to the main thread using postMessage, and then renders those ImageBitmaps either by drawing them to a canvas or in some more direct way. This can all be implemented in a zero-copy way with the APIs currently in the spec, though it's not trivial. You'd need to do a few optimizations: -- createImageBitmap(canvasContext) would take a lazy snapshot of the canvas contents; i.e., further modifications to the canvas would trigger a copy (on the worker), but if the canvas is untouched no copy is required. -- structured clone of the ImageBitmap would not copy. This may actually require a small spec change. -- drawing an ImageBitmap to a 2D canvas, if it covers the whole canvas, would simply replace the canvas buffer with the ImageBitmap contents (and perform copy-on-write if the script later makes changes to that canvas). These optimizations would all be useful in other contexts too. Whatever happens with canvas-in-a-worker, I bet we'll end up doing these optimizations for other reasons. It might make sense to create an API that renders an ImageBitmap more directly than drawing it to a canvas. For example we could create an API that allows an img element to render an ImageBitmap. It would be a bit simpler for authors and perhaps a bit simpler to implement than the final optimization in my list above. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 7:55 PM, David Bruant bruan...@gmail.com wrote: Not directly related, but I imagine requestAnimationFrame doesn't work in a worker. How can a script know when to stop drawing on a given canvas? rAF doesn't work in a worker, but that is easy to work around. You can have a rAF handler on the main thread that propagates the signal to your workers by posting a rAF message to them.
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 10:06 AM, Justin Novosad ju...@google.com wrote: If we put that idea into the WorkerCanvas proposal, here is an idea of how commits could handle resizes asynchronously: 1) Main thread executes some JS that changes the size of the canvas, and posts an event to the Worker in order to propagate the change to the WorkerCanvas. 2) On the worker thread, the WorkerCanvas stores the new size, but does not apply it right away to avoid resetting canvas contents mid frame. 3) commit() is called on the WorkerCanvas, causing the current canvas contents to be sent to the main thread for display, and the new canvas size comes into effect (lazily?) 4) The main thread receives the committed pixels. N.B.: The size of the received buffer does not match canvas element's intrinsic size 5) At paint time, the canvas contents get displayed respecting the canvas element's current size. The mismatched pixel buffer is either streched/squashed or clipped/padded to fit. Step 5 has an issue that different apps might want to make different choices about whether and how to scale/pad/crop. But I think here we can leverage CSS3 object-fit and object-position properties: http://dev.w3.org/csswg/css-images-3/#object-fit Let's make those apply to canvas, where the replaced content is the canvas backing store! Then we would modify your proposal a little bit. Let's say that the main thread cannot modify the canvas buffer size after the canvas has been transferred. If you want to change the buffer size, you'll have to postMessage your worker and have it update the buffer size. When the worker commits a canvas buffer with a size that doesn't match the element size, object-fit and object-position are consulted to determine what gets rendered. Those values can be sent to the compositor, so we can always render the right thing. This means a worker constantly producing frames without yielding can't be notified of a buffer size change, but I don't think that's a big deal. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 10:19 AM, Justin Novosad ju...@google.com wrote: rAF doesn't work in a worker, but that is easy to work around. You can have a rAF handler on the main thread that propagates the signal to your workers by posting a rAF message to them. That isn't a good solution since it forces the app to drop frames if the main thread is blocked. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Mon, Oct 14, 2013 at 1:34 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. Note that the CanvasInWorkers draft solves some other longstanding issues not addressed by the WorkerCanvas proposal. It provides the ability to render to multiple canvases from a single context, whether workers are involved or not. It achieves ideal memory utilization by being very explicit in the API, without the need for extensive and subtle optimizations behind the scenes. It's worth considering whether a change to the CanvasInWorkers proposal could support presenting frames directly from the worker. There's been some recent discussion in the WebGL WG on this topic and concerns were raised from other parties at Mozilla about the DrawingBuffer proposal above, including that it isn't possible to render from a worker without synchronizing with the main thread. This is a huge limitation. For games and other animated content, achieving a stable frame rate is super important and a key motivation for adding canvas support to workers. My vision for handling the Maps use-cases based on our proposal is this: the worker renders to one or more WorkerCanvases, then does createImageBitmap(canvasContext), then ships the ImageBitmap(s) to the main thread using postMessage, and then renders those ImageBitmaps either by drawing them to a canvas or in some more direct way. This can all be implemented in a zero-copy way with the APIs currently in the spec, though it's not trivial. You'd need to do a few optimizations: -- createImageBitmap(canvasContext) would take a lazy snapshot of the canvas contents; i.e., further modifications to the canvas would trigger a copy (on the worker), but if the canvas is untouched no copy is required. -- structured clone of the ImageBitmap would not copy. This may actually require a small spec change. -- drawing an ImageBitmap to a 2D canvas, if it covers the whole canvas, would simply replace the canvas buffer with the ImageBitmap contents (and perform copy-on-write if the script later makes changes to that canvas). These optimizations would all be useful in other contexts too. Whatever happens with canvas-in-a-worker, I bet we'll end up doing these optimizations for other reasons. I agree that it's probably possible to make this work. It is still worth considering whether these optimizations are going to be fragile, and whether developers will fall off a performance cliff if something subtle changes in their code or in the browser's behavior in the future. It might make sense to create an API that renders an ImageBitmap more directly than drawing it to a canvas. For example we could create an API that allows an img element to render an ImageBitmap. It would be a bit simpler for authors and perhaps a bit simpler to implement than the final optimization in my list above. That's an interesting possibility. It would probably be quite a bit simpler to both specify and implement rather than using a canvas on the receiving end. -Ken Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] Canvas in workers
Le 16/10/2013 00:12, Robert O'Callahan a écrit : On Wed, Oct 16, 2013 at 10:19 AM, Justin Novosad ju...@google.com mailto:ju...@google.com wrote: rAF doesn't work in a worker, but that is easy to work around. You can have a rAF handler on the main thread that propagates the signal to your workers by posting a rAF message to them. That isn't a good solution since it forces the app to drop frames if the main thread is blocked. If the main thread is blocked, the app drops frames anyway, no? Anyway, on not-so-good hardware (most mobiles?), workers communication cost is non-trivial and eats a bit of the 16,6ms budget. A friend told me recently that in AS3, objects on the screen can all listen to a beforeframe event (or something like that). In web platform equivalents, they have a per-element requestAnimationFrame (instead of only a global one). I found this idea interesting. Among other things, it allows implementors to not emit the event if the object isn't on the screen. It feels like it encourages good practices too; if my canvas element is too low in the document to be on screen, maybe I don't want to draw on it... I wonder how many people check if the thing they want to update is actually on screen... In the case of canvas, the UA not only could decide to not dispatch events, but could also tell precisely which part of the canvas is visible so that if the app cares or is capable or partial canvas paints, it can decide to do so. In case the main thread is blocked, events can be saved in the worker and save processing resource/battery. David
Re: [whatwg] Canvas in workers
On Mon, Oct 14, 2013 at 1:20 PM, Kenneth Russell k...@google.com wrote: 1) Rendering from a worker and displaying on the main thread with no extra blits of the rendering results 2) Allows one context to render to multiple canvases 3) Supports resizing of the drawing buffer The WorkerCanvas proposal should allow #1 and #3. (It doesn't support #3 for purely offscreen worker canvases, but that'd be easy to add.) #2 would be nice with WebGL, where setting up extra contexts can be expensive, and it may be simpler to do at the Canvas level than by mimicing OpenGL (eg. shared resources across contexts). There's been some recent discussion in the WebGL WG on this topic and concerns were raised from other parties at Mozilla about the DrawingBuffer proposal above, including that it isn't possible to render from a worker without synchronizing with the main thread. Your proposal does seem to require synchronization with the main thread, at least with double-buffering. You postMessage the DrawingBuffer to the main thread to ask it to be displayed. The worker can't start drawing the next frame until it knows that the drawing buffers have been flipped; the buffer flip happens in the main thread, when transferDrawingBufferToCanvas is called. WorkerCanvas performs the flip itself in the worker, when .commit() is called (and possibly also when the script returns). Even if the main thread is busy, the worker should be able to do this immediately. It does need to be a thread-safe operation, but it doesn't need to block if the UI thread is busy. This proposal doesn't handle synchronizing DOM updates with threaded canvas updates, but it seems like that inherently requires synchronization... -- Glenn Maynard
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 11:55 AM, David Bruant bruan...@gmail.com wrote: If the main thread is blocked, the app drops frames anyway, no? Not necessarily. We can allow workers to present frames to the compositor without synchronizing with the main thread. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Wed, Oct 16, 2013 at 11:55 AM, Kenneth Russell k...@google.com wrote: On Mon, Oct 14, 2013 at 1:34 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. Note that the CanvasInWorkers draft solves some other longstanding issues not addressed by the WorkerCanvas proposal. It provides the ability to render to multiple canvases from a single context, whether workers are involved or not. That may be a useful feature, but I'd like to see it justified in its own right. It achieves ideal memory utilization by being very explicit in the API, without the need for extensive and subtle optimizations behind the scenes. We can be more explicit with ImageBitmaps. We could provide WorkerCanvas.transferToImageBitmap which transfers the current canvas contents to an ImageBitmap and clears the canvas. (Canvas implementations are already optimized to support a zero-cost cleared state, because existing benchmarks require it.) Sharing ImageBitmap contents across threads during structured clone is not subtle. We can add an HTMLImageElement.srcObject attribute which could take a Blob or an ImageBitmap to enable explicit zero-copy rendering of ImageBitmaps. Would that be explicit enough for you? Personally I think high-performance manipulation of ImageBitmaps would be more generally useful than detachable DrawingBuffers, and would be easier for authors to understand. If you squint, WorkerCanvas.transferToImageBitmap is similar to detaching a DrawingBuffer. But I don't see a need to reattach a buffer to a canvas for further drawing. Do you? It's worth considering whether a change to the CanvasInWorkers proposal could support presenting frames directly from the worker. Sure, by adding a commit() method to DrawingBuffer. Right? Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Tue, Oct 15, 2013 at 4:41 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Wed, Oct 16, 2013 at 11:55 AM, Kenneth Russell k...@google.com wrote: On Mon, Oct 14, 2013 at 1:34 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. Note that the CanvasInWorkers draft solves some other longstanding issues not addressed by the WorkerCanvas proposal. It provides the ability to render to multiple canvases from a single context, whether workers are involved or not. That may be a useful feature, but I'd like to see it justified in its own right. There has been a lot of developer feedback on the WebGL mailing lists over the past couple of years about exactly this feature. Web sites like Turbosquid want to present lots of little thumbnails of models -- see for example http://www.turbosquid.com/Search/3D-Models/Vehicle/Car -- and have them be interactive. It's too resource-intensive to create a separate WebGL context for each. The most direct solution is to allow one context to render to multiple canvases. It achieves ideal memory utilization by being very explicit in the API, without the need for extensive and subtle optimizations behind the scenes. We can be more explicit with ImageBitmaps. We could provide WorkerCanvas.transferToImageBitmap which transfers the current canvas contents to an ImageBitmap and clears the canvas. (Canvas implementations are already optimized to support a zero-cost cleared state, because existing benchmarks require it.) Sharing ImageBitmap contents across threads during structured clone is not subtle. We can add an HTMLImageElement.srcObject attribute which could take a Blob or an ImageBitmap to enable explicit zero-copy rendering of ImageBitmaps. Would that be explicit enough for you? Yes, that generally sounds good. Personally I think high-performance manipulation of ImageBitmaps would be more generally useful than detachable DrawingBuffers, and would be easier for authors to understand. If you squint, WorkerCanvas.transferToImageBitmap is similar to detaching a DrawingBuffer. But I don't see a need to reattach a buffer to a canvas for further drawing. Do you? Not immediately. The ability to transfer out the canvas's contents, and render them in an HTMLImageElement without incurring an extra blit, should address the Maps team's requirements. Actually, adding transferToImageBitmap to HTMLCanvasElement as well would address the use case of rendering to multiple targets using one context. Instead of using multiple canvases as the targets, one would simply use multiple images. That sounds appealing. If WorkerCanvas is changed so that its width and height are mutable within the worker as you mentioned above, it sounds like it's addressing the known use cases. It's worth considering whether a change to the CanvasInWorkers proposal could support presenting frames directly from the worker. Sure, by adding a commit() method to DrawingBuffer. Right? I'm not exactly sure how it would be done. In the proposal as written, the DrawingBuffer's not shared between threads, only transferred. -Ken Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] Canvas in workers
Would you mind looking at the proposal
http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? This
was arrived at after extensive discussions with the Google Maps team,
and addresses their key use cases. Compared to the one below, it
solves the following problems:
1) Rendering from a worker and displaying on the main thread with no
extra blits of the rendering results
2) Allows one context to render to multiple canvases
3) Supports resizing of the drawing buffer
The main difference in my mind is that in the DrawingBuffer proposal,
the back buffer of the canvas can be detached, transferred via
postMessage to the other side, and attached to a Canvas, replacing its
previous back buffer. The semantics are simple, clear, avoid extra
blits of the rendered content, and support rendering into multiple
canvases from one context.
There's been some recent discussion in the WebGL WG on this topic and
concerns were raised from other parties at Mozilla about the
DrawingBuffer proposal above, including that it isn't possible to
render from a worker without synchronizing with the main thread.
Still, it seems to me it's worth considering all aspects of the
proposal, because it was motivated by a major existing web app which
is both using Canvas 2D and WebGL extensively, and desires to use
workers more heavily in its rendering pipeline.
-Ken
On Sat, Oct 12, 2013 at 9:12 PM, Kyle Huey m...@kylehuey.com wrote:
I talked at length with Robert O'Callahan about what the DOM API for
supporting canvas in web workers should look like and we came up with the
following modifications to the spec.
1. Rename CanvasProxy to WorkerCanvas and only allow it to be
transferred to workers. I don't think we're interested in supporting
cross-origin canvas via CanvasProxy (I would be curious to hear more
about what the use cases are).
2. Add a worker-only WorkerCanvas constructor that takes the desired
width/height of the drawing surface.
3. Remove the rendering context constructors and the setContext method
on WorkerCanvas (née CanvasProxy).
4. Copy all of the sensible non-node related things from
HTMLCanvasElement to WorkerCanvas. This would include
- width and height as readonly attributes
- getContext (to replace what we removed in step 3). roc prefers to
have getContext2D and getContextWebGL, and dispense with the string
argument version entirely, but I don't have strong feelings.
- toBlob. We do not intend to implement toDataURL here.
5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained
from a main thread canvas element, this would cause the buffer displayed
on screen to swap. For a WorkerCanvas created *de novo* on a worker
thread, it would do nothing. This commit method would also commit a minor
violation of run-to-completion semantics, described below.
6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
shipping them to the main thread via postMessage to allow synchronizing
canvas updates with DOM updates. We explored a couple other options but we
didn't come up with anything else that allows synchronizing updates to
multiple canvases from a worker. This isn't really sketched out here.
So the IDL would look something like:
[Constructor(unsigned long width, unsigned long height)]
interface WorkerCanvas {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
CanvasRenderingContext2D? getContext2D(any... args);
WebGLRenderingContext? getContextWebGL(any... args);
void toBlob(FileCallback? _callback, optional DOMString type, any...
arguments);
bool commit();
};
WorkerCanvas implements Transferable;
Everything would be behave pretty much as one would expect, except perhaps
for the commit method. The width and height of the canvas can be modified
on the main thread while the worker is drawing. This would fire an event
off to the worker to update the WorkerCanvas's dimensions that would be
scheduled as if the main thread had postMessage()d something to the
worker. But it's possible that the worker would attempt to draw to the
canvas before that update runs. It's also possible that the worker would
simply draw in a loop without yielding. To solve this, if commit is called
and the current dimensions on the main thread don't match the dimensions of
the WorkerCanvas it would fail (return false) and update the dimensions of
the WorkerCanvas before returning. This is technically a violation of
run-to-completion semantics, but is needed to support workers that do not
yield.
Thoughts?
- Kyle
Re: [whatwg] Canvas in workers
On Mon, Oct 14, 2013 at 2:20 PM, Kenneth Russell k...@google.com wrote: Would you mind looking at the proposal http://wiki.whatwg.org/wiki/CanvasInWorkers and commenting on it? Sure. Kyle and I looked at it while we were working on our proposal. The main issues I have with it are that rearchitecting canvas to introduce the DrawingBuffer layer of abstraction seems unnecessarily complex, and it doesn't handle direct presentation of frames from the worker, bypassing the main thread. There's been some recent discussion in the WebGL WG on this topic and concerns were raised from other parties at Mozilla about the DrawingBuffer proposal above, including that it isn't possible to render from a worker without synchronizing with the main thread. This is a huge limitation. For games and other animated content, achieving a stable frame rate is super important and a key motivation for adding canvas support to workers. My vision for handling the Maps use-cases based on our proposal is this: the worker renders to one or more WorkerCanvases, then does createImageBitmap(canvasContext), then ships the ImageBitmap(s) to the main thread using postMessage, and then renders those ImageBitmaps either by drawing them to a canvas or in some more direct way. This can all be implemented in a zero-copy way with the APIs currently in the spec, though it's not trivial. You'd need to do a few optimizations: -- createImageBitmap(canvasContext) would take a lazy snapshot of the canvas contents; i.e., further modifications to the canvas would trigger a copy (on the worker), but if the canvas is untouched no copy is required. -- structured clone of the ImageBitmap would not copy. This may actually require a small spec change. -- drawing an ImageBitmap to a 2D canvas, if it covers the whole canvas, would simply replace the canvas buffer with the ImageBitmap contents (and perform copy-on-write if the script later makes changes to that canvas). These optimizations would all be useful in other contexts too. Whatever happens with canvas-in-a-worker, I bet we'll end up doing these optimizations for other reasons. It might make sense to create an API that renders an ImageBitmap more directly than drawing it to a canvas. For example we could create an API that allows an img element to render an ImageBitmap. It would be a bit simpler for authors and perhaps a bit simpler to implement than the final optimization in my list above. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
Le 13/10/2013 06:12, Kyle Huey a écrit :
I talked at length with Robert O'Callahan about what the DOM API for
supporting canvas in web workers should look like and we came up with the
following modifications to the spec.
1. Rename CanvasProxy to WorkerCanvas and only allow it to be
transferred to workers. I don't think we're interested in supporting
cross-origin canvas via CanvasProxy (I would be curious to hear more
about what the use cases are).
2. Add a worker-only WorkerCanvas constructor that takes the desired
width/height of the drawing surface.
What is the use case for this constructor? Draw something in worker,
output an image to be sent to main thread?
3. Remove the rendering context constructors and the setContext method
on WorkerCanvas (née CanvasProxy).
4. Copy all of the sensible non-node related things from
HTMLCanvasElement to WorkerCanvas. This would include
- width and height as readonly attributes
- getContext (to replace what we removed in step 3). roc prefers to
have getContext2D and getContextWebGL, and dispense with the string
argument version entirely, but I don't have strong feelings.
For the sake of writing interoperable code in main thread and worker, I
tend to be against this sort of change. I'm aware of the ugliness of
some APIs, but consistent ugly APIs beats a mix of beautiful and ugly API.
- toBlob. We do not intend to implement toDataURL here.
5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained
from a main thread canvas element, this would cause the buffer displayed
on screen to swap. For a WorkerCanvas created *de novo* on a worker
thread, it would do nothing.
Let's have this method optional, then? Or create 2 interfaces? I'm not
sure of what can be expressed in WebIDL to solve this, but useless
methods aren't a good idea. If I can't do anything, don't give me the
method. It's like in UI. If I can't click a button, just don't show me
the button or at least grey it out.
This commit method would also commit a minor
violation of run-to-completion semantics, described below.
6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
shipping them to the main thread via postMessage to allow synchronizing
canvas updates with DOM updates. We explored a couple other options but we
didn't come up with anything else that allows synchronizing updates to
multiple canvases from a worker. This isn't really sketched out here.
So the IDL would look something like:
[Constructor(unsigned long width, unsigned long height)]
interface WorkerCanvas {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
CanvasRenderingContext2D? getContext2D(any... args);
WebGLRenderingContext? getContextWebGL(any... args);
void toBlob(FileCallback? _callback, optional DOMString type, any...
arguments);
bool commit();
Boolean as return value for success? :-s
A promise instead maybe? throw instead of false at least?
In any case, it looks like commit could be a long operation (tell me if
I'm wrong here. Do you have numbers on how long it takes/would take?),
having it async sounds reasonable.
};
WorkerCanvas implements Transferable;
Everything would be behave pretty much as one would expect, except perhaps
for the commit method. The width and height of the canvas can be modified
on the main thread while the worker is drawing. This would fire an event
off to the worker to update the WorkerCanvas's dimensions that would be
scheduled as if the main thread had postMessage()d something to the
worker. But it's possible that the worker would attempt to draw to the
canvas before that update runs. It's also possible that the worker would
simply draw in a loop without yielding. To solve this, if commit is called
and the current dimensions on the main thread don't match the dimensions of
the WorkerCanvas it would fail (return false) and update the dimensions of
the WorkerCanvas before returning. This is technically a violation of
run-to-completion semantics, but is needed to support workers that do not
yield.
I feel fairly strongly against the run-to-completion violation as it's a
foundation of how JavaScript works, how people reason about programs.
Also, TC39 is working hard to close the gap between what can be
expressed in pure ECMAScript and what the web platform does express,
it'd be inappropriate to add things that widen this gap in my opinion.
I would be much more in favor of a solution like how IndexedDB handles
transactions. Here is how it could work:
For a canvas which WorkerCanvas has been transfered, when trying to
change width/height:
*1 Store the values, but don't do it yet
*2 Send a message to the worker to re-dimension,
#1 in the worker: wait for the current message to be taken care of
#2 dispatch a dimension changed event (or equivalent)
#3 send a confirmation
Re: [whatwg] Canvas in workers
On Sat, Oct 12, 2013 at 11:12 PM, Kyle Huey m...@kylehuey.com wrote: 1. Rename CanvasProxy to WorkerCanvas and only allow it to be transferred to workers. I don't think we're interested in supporting cross-origin canvas via CanvasProxy (I would be curious to hear more about what the use cases are). You can transfer data to a worker that's cross-origin, if you have a MessagePort where the other side goes to another origin's worker (possibly given to you via eg. window.postMessage). Is the real goal here trying to limit this to threads and avoid IPC, or is this actually a cross-origin concern? 2. Add a worker-only WorkerCanvas constructor that takes the desired width/height of the drawing surface. This looks like it's trying to allow entirely off-screen rendering within a Worker, which is fine, but there's no way to resize the backing store in this mode. I don't know if that would need a separate subclass of WorkerCanvas to allow making width/height writable. - getContext (to replace what we removed in step 3). roc prefers to have getContext2D and getContextWebGL, and dispense with the string argument version entirely, but I don't have strong feelings. CanvasRenderingContext2D? getContext2D(any... args); WebGLRenderingContext? getContextWebGL(any... args); This is crazy. The platform is inconsistent enough. We have an API for this already, getContext(); don't add a different API for the exact same thing. 5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained I'm not sure what this is for. If you draw in a worker and return without calling .commit(), is the commit implicit when you return to the event loop? (See below for where this matters.) simply draw in a loop without yielding. To solve this, if commit is called and the current dimensions on the main thread don't match the dimensions of the WorkerCanvas it would fail (return false) and update the dimensions of the WorkerCanvas before returning. This is technically a violation of run-to-completion semantics, but is needed to support workers that do not yield. This sounds like it's easy to get wrong, since it'll probably be rare. An exception might be better, so if you don't handle it you at least get an error logged to the console. There will be flicker issues with this. The canvas is cleared when you change width or height. In the UI thread that's OK, since the author can synchronously redraw immediately after changing the size. Here, it's likely it won't be redrawn in time, so it'll flicker whenever the size changes, especially if it's being changed smoothly. Here's a suggestion to fix this: - When the UI thread changes Canvas.width or Canvas.height, it doesn't actually resize buffers. Instead, it sends a message to the WorkerCanvas asking for the change. Until the change actually happens, the Canvas continues to be composited as before. (However, the change to .width and .height is visible on the object immediately.) - When the WorkerCanvas's event loop receives a message asking for a size change: - Change the size of the back-buffer, and update WorkerCanvas.width and WorkerCanvas.height accordingly. - Fire onresize on the WorkerCanvas. The worker is expected to redraw here. (This is where the implicit commit matters: we want to guarantee a commit here.) - Only when the newly-redrawn buffer is committed does the front buffer's size get updated to match the back-buffer. In other words, when you change the size in the UI thread, it continues to composite the same image (possibly not filling the whole element, or being stretched) until the worker actually gets the resize and has a chance to redraw it. This also means the idea of not being able to commit because of a resize can go away, and commit() can be void, since the back-buffer size never actually changes while the worker is drawing. On Sun, Oct 13, 2013 at 11:01 AM, David Bruant bruan...@gmail.com wrote: bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. This should be synchronous and never block. Even if the Canvas is in a different process, it should be possible to do this with IPC without waiting for the other side to process the change. -- Glenn Maynard
Re: [whatwg] Canvas in workers
Le 13/10/2013 18:12, Glenn Maynard a écrit : On Sun, Oct 13, 2013 at 11:01 AM, David Bruant bruan...@gmail.com wrote: bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. This should be synchronous and never block. I might have misused the word async again, sorry about that. I think we agree. Even if the Canvas is in a different process, it should be possible to do this with IPC without waiting for the other side to process the change. How does a worker know when the changes on the screen happened? I imagine a worker would want to know that before performing other changes to the canvas. David
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 12:01 PM, David Bruant bruan...@gmail.com wrote: Le 13/10/2013 06:12, Kyle Huey a écrit : 2. Add a worker-only WorkerCanvas constructor that takes the desired width/height of the drawing surface. What is the use case for this constructor? Draw something in worker, output an image to be sent to main thread? When drawing complex stuff to a canvas (e.g. opacity groups) you often need to create a temporary canvas to hold some subset of the content and then composite that temporary canvas to the main canvas. 3. Remove the rendering context constructors and the setContext method on WorkerCanvas (née CanvasProxy). 4. Copy all of the sensible non-node related things from HTMLCanvasElement to WorkerCanvas. This would include - width and height as readonly attributes - getContext (to replace what we removed in step 3). roc prefers to have getContext2D and getContextWebGL, and dispense with the string argument version entirely, but I don't have strong feelings. For the sake of writing interoperable code in main thread and worker, I tend to be against this sort of change. I'm aware of the ugliness of some APIs, but consistent ugly APIs beats a mix of beautiful and ugly API. Fine, let's consider that suggestion dropped. - toBlob. We do not intend to implement toDataURL here. 5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained from a main thread canvas element, this would cause the buffer displayed on screen to swap. For a WorkerCanvas created *de novo* on a worker thread, it would do nothing. Let's have this method optional, then? Or create 2 interfaces? I'm not sure of what can be expressed in WebIDL to solve this, but useless methods aren't a good idea. If I can't do anything, don't give me the method. It's like in UI. If I can't click a button, just don't show me the button or at least grey it out. We could put it in a separate interface. I'm not sure that's worth it, but whatever, that's a trivial issue. bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. commit does not need to be async, it doesn't have to block in practice. I feel fairly strongly against the run-to-completion violation as it's a foundation of how JavaScript works, how people reason about programs. Also, TC39 is working hard to close the gap between what can be expressed in pure ECMAScript and what the web platform does express, it'd be inappropriate to add things that widen this gap in my opinion. Personally I don't see this as a violation of run-to-completion semantics. The important thing is that the dimensions of the WorkerCanvas (and any other state observable by the worker) do not change except during the call to commit(). Effectively commit() itself performs the change. Of course it's OK for DOM API implementations to change state :-). Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 12:12 PM, Glenn Maynard gl...@zewt.org wrote: On Sat, Oct 12, 2013 at 11:12 PM, Kyle Huey m...@kylehuey.com wrote: 1. Rename CanvasProxy to WorkerCanvas and only allow it to be transferred to workers. I don't think we're interested in supporting cross-origin canvas via CanvasProxy (I would be curious to hear more about what the use cases are). You can transfer data to a worker that's cross-origin, if you have a MessagePort where the other side goes to another origin's worker (possibly given to you via eg. window.postMessage). Is the real goal here trying to limit this to threads and avoid IPC, or is this actually a cross-origin concern? Basically it's simpler to have CanvasProxy/WorkerCanvas only supported on workers. Cross-origin isn't itself a concern. 2. Add a worker-only WorkerCanvas constructor that takes the desired width/height of the drawing surface. This looks like it's trying to allow entirely off-screen rendering within a Worker, which is fine, but there's no way to resize the backing store in this mode. We don't have a use-case for resizing the backing store of a worker-created canvas. simply draw in a loop without yielding. To solve this, if commit is called and the current dimensions on the main thread don't match the dimensions of the WorkerCanvas it would fail (return false) and update the dimensions of the WorkerCanvas before returning. This is technically a violation of run-to-completion semantics, but is needed to support workers that do not yield. This sounds like it's easy to get wrong, since it'll probably be rare. An exception might be better, so if you don't handle it you at least get an error logged to the console. Sure, that's probably better. There will be flicker issues with this. The canvas is cleared when you change width or height. In the UI thread that's OK, since the author can synchronously redraw immediately after changing the size. Here, it's likely it won't be redrawn in time, so it'll flicker whenever the size changes, especially if it's being changed smoothly. Here's a suggestion to fix this: - When the UI thread changes Canvas.width or Canvas.height, it doesn't actually resize buffers. Instead, it sends a message to the WorkerCanvas asking for the change. Until the change actually happens, the Canvas continues to be composited as before. (However, the change to .width and .height is visible on the object immediately.) - When the WorkerCanvas's event loop receives a message asking for a size change: - Change the size of the back-buffer, and update WorkerCanvas.width and WorkerCanvas.height accordingly. - Fire onresize on the WorkerCanvas. The worker is expected to redraw here. (This is where the implicit commit matters: we want to guarantee a commit here.) - Only when the newly-redrawn buffer is committed does the front buffer's size get updated to match the back-buffer. In other words, when you change the size in the UI thread, it continues to composite the same image (possibly not filling the whole element, or being stretched) until the worker actually gets the resize and has a chance to redraw it. This also means the idea of not being able to commit because of a resize can go away, and commit() can be void, since the back-buffer size never actually changes while the worker is drawing. There is the slight problem that changing both width and height would fire two events. A bigger problem is that your approach isn't compatible with a worker that draws frames in a loop without yielding. I'm uncertain how important that is, so I'll wait for Kyle to address that. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
Le 13/10/2013 23:33, Robert O'Callahan a écrit : bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. commit does not need to be async, it doesn't have to block in practice. Just so that I better understand how commit works, what happens in this case: // draw things wc.commit(); // draw other things wc.commit(); ? I feel fairly strongly against the run-to-completion violation as it's a foundation of how JavaScript works, how people reason about programs. Also, TC39 is working hard to close the gap between what can be expressed in pure ECMAScript and what the web platform does express, it'd be inappropriate to add things that widen this gap in my opinion. Personally I don't see this as a violation of run-to-completion semantics. The important thing is that the dimensions of the WorkerCanvas (and any other state observable by the worker) do not change except during the call to commit(). Effectively commit() itself performs the change. Hmm... Interesting point of view... That's the only way to learn that the dimensions have changed? Via commit() throwing an error/returning false? It feels very ad-hoc. It doesn't seem to scale very well to other reasons why commit() might fail. I also imagine that the normal pattern will be draw then commit, so it means that you always spend efforts drawing without ever knowing if what you're drawing is ever going to end up on screen; which feels like a waste of resources. Of course it's OK for DOM API implementations to change state :-). I don't understand what you mean here. What happens to WorkerCanvas in the canvas element is removed from the document or display:hidden-ed? David
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 6:07 PM, David Bruant bruan...@gmail.com wrote: Le 13/10/2013 23:33, Robert O'Callahan a écrit : bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. commit does not need to be async, it doesn't have to block in practice. Just so that I better understand how commit works, what happens in this case: // draw things wc.commit(); // draw other things wc.commit(); Two frames get queued as eligible for presentation. Which ones reach the screen is unspecified of course. Personally I don't see this as a violation of run-to-completion semantics. The important thing is that the dimensions of the WorkerCanvas (and any other state observable by the worker) do not change except during the call to commit(). Effectively commit() itself performs the change. Hmm... Interesting point of view... That's the only way to learn that the dimensions have changed? Via commit() throwing an error/returning false? It feels very ad-hoc. It doesn't seem to scale very well to other reasons why commit() might fail. What other reasons are there? I also imagine that the normal pattern will be draw then commit, so it means that you always spend efforts drawing without ever knowing if what you're drawing is ever going to end up on screen; which feels like a waste of resources. I don't see this as being a problem, or at least no more of a problem than it already is. Of course it's OK for DOM API implementations to change state :-). I don't understand what you mean here. It's OK for calling DOM methods to change state visible to script. What happens to WorkerCanvas in the canvas element is removed from the document or display:hidden-ed? It's unaffected. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Canvas in workers
Le 14/10/2013 01:27, Robert O'Callahan a écrit : On Sun, Oct 13, 2013 at 6:07 PM, David Bruant bruan...@gmail.com mailto:bruan...@gmail.com wrote: Le 13/10/2013 23:33, Robert O'Callahan a écrit : bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. commit does not need to be async, it doesn't have to block in practice. Just so that I better understand how commit works, what happens in this case: // draw things wc.commit(); // draw other things wc.commit(); Two frames get queued as eligible for presentation. Which ones reach the screen is unspecified of course. Is there a way to know if/when a frame reaches the screen? Wouldn't this information be useful at least to know when it's safe to commit again? Not directly related, but I imagine requestAnimationFrame doesn't work in a worker. How can a script know when to stop drawing on a given canvas? Personally I don't see this as a violation of run-to-completion semantics. The important thing is that the dimensions of the WorkerCanvas (and any other state observable by the worker) do not change except during the call to commit(). Effectively commit() itself performs the change. Hmm... Interesting point of view... That's the only way to learn that the dimensions have changed? Via commit() throwing an error/returning false? It feels very ad-hoc. It doesn't seem to scale very well to other reasons why commit() might fail. What other reasons are there? Won't there ever be? I also imagine that the normal pattern will be draw then commit, so it means that you always spend efforts drawing without ever knowing if what you're drawing is ever going to end up on screen; which feels like a waste of resources. I don't see this as being a problem, or at least no more of a problem than it already is. What do you mean by than it already is? Currently, if one calls a method on a context2d, the result is guaranteed to appear on screen eventually, no? It never happens that you start drawing and all of it gets dropped because of external conditions, right? Of course it's OK for DOM API implementations to change state :-). I don't understand what you mean here. It's OK for calling DOM methods to change state visible to script. That's not the problem. The semantics of commit is send this frame and maybe change dimensions. It's an awkward semantics mixing largely unrelated things. It's like if the send() method of a worker's XHR didn't work when the base changed in the parent page. It's also very dependent on timing and may cause bugs that are very hard to debug. David
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 11:22 AM, David Bruant bruan...@gmail.com wrote: bool commit(); Boolean as return value for success? :-s A promise instead maybe? throw instead of false at least? In any case, it looks like commit could be a long operation (tell me if I'm wrong here. Do you have numbers on how long it takes/would take?), having it async sounds reasonable. This should be synchronous and never block. I might have misused the word async again, sorry about that. I think we agree. Sorry, I also stated this imprecisely (block is somewhat overloaded). Specifically, this function never has to sit around and wait for the corresponding Canvas thread to become available; it can perform the flip while the other thread is working. And if it really did need to block--we're in a worker anyway, so that's OK. Even if the Canvas is in a different process, it should be possible to do this with IPC without waiting for the other side to process the change. How does a worker know when the changes on the screen happened? I imagine a worker would want to know that before performing other changes to the canvas. If the Canvas wants to change its size, then it should ask the owner of the WorkerCanvas to make the change, which means the canvas size will never change while the worker is drawing. The WorkerCanvas would never change size while the worker is rendering, and it would know that the canvas changed size when it receives an onresize event. On Sun, Oct 13, 2013 at 4:42 PM, Robert O'Callahan rob...@ocallahan.orgwrote: 1. Rename CanvasProxy to WorkerCanvas and only allow it to be transferred to workers. I don't think we're interested in supporting cross-origin canvas via CanvasProxy (I would be curious to hear more about what the use cases are). Basically it's simpler to have CanvasProxy/WorkerCanvas only supported on workers. Cross-origin isn't itself a concern. I think restricting this to workers is fine. A simple way is for calls to any methods throw an exception if you're not in a worker. This keeps structured clone and transfer simple: the rules for passing it around are the same as anything else, you just can't use it if you're not a Worker. If https://www.w3.org/Bugs/Public/show_bug.cgi?id=23358 is implemented, that'd give an easy way to define this. 2. Add a worker-only WorkerCanvas constructor that takes the desired width/height of the drawing surface. This looks like it's trying to allow entirely off-screen rendering within a Worker, which is fine, but there's no way to resize the backing store in this mode. We don't have a use-case for resizing the backing store of a worker-created canvas. I suspect this will come up with WebGL, since recreating the context from scratch can be a lot more expensive there. We should at least make sure this is possible later. Actually, there is a way: change the width and height on the CanvasRenderingContext2D you get from getContext, which isn't readonly. I'm guessing that would actually want to be read-only in this proposal, since it allows making changes that are visible to scripts in the UI thread. There is the slight problem that changing both width and height would fire two events. Changes to width and height should probably only be applied when the script returns to its event loop. A bigger problem is that your approach isn't compatible with a worker that draws frames in a loop without yielding. I'm uncertain how important that is, so I'll wait for Kyle to address that. OK. What are the use cases for doing that? Being able to do complex work in a worker in a linear, non-event-based manner is an important use case for workers in general, but I can't think of any way this applies to drawing successive frames to a canvas. -- Glenn Maynard
[whatwg] Canvas in workers
I talked at length with Robert O'Callahan about what the DOM API for
supporting canvas in web workers should look like and we came up with the
following modifications to the spec.
1. Rename CanvasProxy to WorkerCanvas and only allow it to be
transferred to workers. I don't think we're interested in supporting
cross-origin canvas via CanvasProxy (I would be curious to hear more
about what the use cases are).
2. Add a worker-only WorkerCanvas constructor that takes the desired
width/height of the drawing surface.
3. Remove the rendering context constructors and the setContext method
on WorkerCanvas (née CanvasProxy).
4. Copy all of the sensible non-node related things from
HTMLCanvasElement to WorkerCanvas. This would include
- width and height as readonly attributes
- getContext (to replace what we removed in step 3). roc prefers to
have getContext2D and getContextWebGL, and dispense with the string
argument version entirely, but I don't have strong feelings.
- toBlob. We do not intend to implement toDataURL here.
5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained
from a main thread canvas element, this would cause the buffer displayed
on screen to swap. For a WorkerCanvas created *de novo* on a worker
thread, it would do nothing. This commit method would also commit a minor
violation of run-to-completion semantics, described below.
6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
shipping them to the main thread via postMessage to allow synchronizing
canvas updates with DOM updates. We explored a couple other options but we
didn't come up with anything else that allows synchronizing updates to
multiple canvases from a worker. This isn't really sketched out here.
So the IDL would look something like:
[Constructor(unsigned long width, unsigned long height)]
interface WorkerCanvas {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
CanvasRenderingContext2D? getContext2D(any... args);
WebGLRenderingContext? getContextWebGL(any... args);
void toBlob(FileCallback? _callback, optional DOMString type, any...
arguments);
bool commit();
};
WorkerCanvas implements Transferable;
Everything would be behave pretty much as one would expect, except perhaps
for the commit method. The width and height of the canvas can be modified
on the main thread while the worker is drawing. This would fire an event
off to the worker to update the WorkerCanvas's dimensions that would be
scheduled as if the main thread had postMessage()d something to the
worker. But it's possible that the worker would attempt to draw to the
canvas before that update runs. It's also possible that the worker would
simply draw in a loop without yielding. To solve this, if commit is called
and the current dimensions on the main thread don't match the dimensions of
the WorkerCanvas it would fail (return false) and update the dimensions of
the WorkerCanvas before returning. This is technically a violation of
run-to-completion semantics, but is needed to support workers that do not
yield.
Thoughts?
- Kyle
Re: [whatwg] Canvas in workers
I'm referring to Web Workers.
http://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html
https://developer.mozilla.org/en-US/docs/Web/Guide/Performance/Using_web_workers
- Kyle
On Sun, Oct 13, 2013 at 12:20 PM, Michael Norton no...@me.com wrote:
Hi Kyle,
I have read the editor's draft of canvas 2d context and I have some
familiarity with DOM. But I do not understand what you are referencing to
worker, missing some lingo here. Can you expand on that?
Cheers,
Mike
Sent by the hope boat.
On Oct 13, 2013, at 12:12 AM, Kyle Huey m...@kylehuey.com wrote:
I talked at length with Robert O'Callahan about what the DOM API for
supporting canvas in web workers should look like and we came up with
the
following modifications to the spec.
1. Rename CanvasProxy to WorkerCanvas and only allow it to be
transferred to workers. I don't think we're interested in supporting
cross-origin canvas via CanvasProxy (I would be curious to hear more
about what the use cases are).
2. Add a worker-only WorkerCanvas constructor that takes the desired
width/height of the drawing surface.
3. Remove the rendering context constructors and the setContext method
on WorkerCanvas (née CanvasProxy).
4. Copy all of the sensible non-node related things from
HTMLCanvasElement to WorkerCanvas. This would include
- width and height as readonly attributes
- getContext (to replace what we removed in step 3). roc prefers to
have getContext2D and getContextWebGL, and dispense with the string
argument version entirely, but I don't have strong feelings.
- toBlob. We do not intend to implement toDataURL here.
5. Add a commit method to WorkerCanvas. For a WorkerCanvas obtained
from a main thread canvas element, this would cause the buffer
displayed
on screen to swap. For a WorkerCanvas created *de novo* on a worker
thread, it would do nothing. This commit method would also commit a
minor
violation of run-to-completion semantics, described below.
6. We would rely on extracting ImageBitmaps from the WorkerCanvas and
shipping them to the main thread via postMessage to allow synchronizing
canvas updates with DOM updates. We explored a couple other options
but we
didn't come up with anything else that allows synchronizing updates to
multiple canvases from a worker. This isn't really sketched out here.
So the IDL would look something like:
[Constructor(unsigned long width, unsigned long height)]
interface WorkerCanvas {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
CanvasRenderingContext2D? getContext2D(any... args);
WebGLRenderingContext? getContextWebGL(any... args);
void toBlob(FileCallback? _callback, optional DOMString type, any...
arguments);
bool commit();
};
WorkerCanvas implements Transferable;
Everything would be behave pretty much as one would expect, except
perhaps
for the commit method. The width and height of the canvas can be
modified
on the main thread while the worker is drawing. This would fire an event
off to the worker to update the WorkerCanvas's dimensions that would be
scheduled as if the main thread had postMessage()d something to the
worker. But it's possible that the worker would attempt to draw to the
canvas before that update runs. It's also possible that the worker
would
simply draw in a loop without yielding. To solve this, if commit is
called
and the current dimensions on the main thread don't match the dimensions
of
the WorkerCanvas it would fail (return false) and update the dimensions
of
the WorkerCanvas before returning. This is technically a violation of
run-to-completion semantics, but is needed to support workers that do not
yield.
Thoughts?
- Kyle
Re: [whatwg] Canvas in workers
On Sun, Oct 13, 2013 at 12:50 PM, Michael Norton no...@me.com wrote: Thank you! Read over the 1st link you provided, interesting. A worker then seems similar to a runtime process in an os environment - is it a derivation of RPC? In a manner of speaking. Workers are an abstraction allowing for parallel computing on the web without any shared mutable state or locking. They communicate via message passing, which can be thought of as an asynchronous RPC mechanism. Also, Delegation [Section 9.1.2.5] sounds like that SETI screensaver project years ago, for example. Is that an accurate description of that section's use? I am not familiar with this project. As for canvas, in Firefox I don't think you need it if you utilize XUL which has similar elements essentially describing the same process(es), for example employing hbox and vbox with various calls to and fro iFrame and DOM elements via event listeners, etc. This mailing list is for discussing standards track features for the web, so the capabilities of proprietary Mozilla features such as XUL have little relevance here, except perhaps as sources to draw inspiration from. - Kyle
Re: [whatwg] Canvas in Workers
On Thu, Jan 3, 2013 at 7:46 PM, Gregg Tavares g...@google.com wrote:
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
discussion seems to have died down here but I'd like to bring up
another
issue
In WebGL land we have creation attributes on the drawingbuffer made
for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on it
would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
Right now by 2d canvases are effectively single buffered. At the
appropriate time a copy
of the canvas is made and passed to the compositor. This copy is slow,
especially on
mobile.
Currently, to lower the VRAM footprint and improve performance, we don't do
a copy in 2d canvas. We temporarily transfer ownership of the texture to
the compositor at commit time, and block the renderer until the composite
is complete. That may change, however.
Stephen
Apple requested that for WebGL the default be for double buffering. When
double buffered, when the canvas is composited (when the current JavaScript
event exits)
the canvas's buffer is given to the compositor and the canvas is given a
new buffer (or
an old one). That new buffer is cleared, meaning the contents is gone. It's
up to the app to
draw stuff into again. If nothing is drawn the compositor will continue to
use the
buffer it acquired earlier.
In WebGL you can opt into the slower copy path. For Canvas 2D while the
default
has to remain the slow copy path it would be nice to be able to opt into
the faster
swap double buffered path.
In the 2D canvas, whenever you bind to a new canvas, the context is reset
to its default state, the context's hit region list is reset, and the
context's bitmap is reset. The next time the context is flushed, the
canvas itself is always reset (since flushing the context causes the
bitmap and hit region list to be pushed to the canvas, replacing whatever
was there before).
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Wed, 9 Jan 2013, Stephen White wrote: Right now by 2d canvases are effectively single buffered. At the appropriate time a copy of the canvas is made and passed to the compositor. This copy is slow, especially on mobile. Currently, to lower the VRAM footprint and improve performance, we don't do a copy in 2d canvas. We temporarily transfer ownership of the texture to the compositor at commit time, and block the renderer until the composite is complete. That may change, however. Good to know, thanks. Should this change, or should there be a desire for a mode where it looks like after the commit the buffer is cleared, please do let me know. Would still love input from any other vendors, too. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Wed, Jan 9, 2013 at 2:50 PM, Stephen White senorbla...@chromium.orgwrote: Currently, to lower the VRAM footprint and improve performance, we don't do a copy in 2d canvas. We temporarily transfer ownership of the texture to the compositor at commit time, and block the renderer until the composite is complete. That may change, however. Stephen To be more accurate, the renderer only blocks if it needs to access the texture, for example if getImageData is called before the composite is complete. All write operations are non-blocking because they can be recorded an deferred until the composite is complete.
Re: [whatwg] Canvas in Workers
On Wed, 9 Jan 2013, James Robinson wrote: On Wed, Jan 9, 2013 at 11:59 AM, Ian Hickson i...@hixie.ch wrote: On Wed, 9 Jan 2013, Stephen White wrote: Right now by 2d canvases are effectively single buffered. At the appropriate time a copy of the canvas is made and passed to the compositor. This copy is slow, especially on mobile. Currently, to lower the VRAM footprint and improve performance, we don't do a copy in 2d canvas. We temporarily transfer ownership of the texture to the compositor at commit time, and block the renderer until the composite is complete. That may change, however. Good to know, thanks. Should this change, or should there be a desire for a mode where it looks like after the commit the buffer is cleared, please do let me know. We (chromium) would really appreciate a way for the author to express clear-after-commit. I think this would match up with what authors frequently want, since many canvas pages clear the canvas at the start of the next frame anyway, but allow for more flexibility and optimizations in our implementation. Noted: http://wiki.whatwg.org/wiki/New_Features_Awaiting_Implementation_Interest -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Thu, 3 Jan 2013, Gregg Tavares wrote:
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
discussion seems to have died down here but I'd like to bring up
another issue
In WebGL land we have creation attributes on the drawingbuffer made
for a canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the
case where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on
it would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
Right now by 2d canvases are effectively single buffered. At the
appropriate time a copy of the canvas is made and passed to the
compositor. This copy is slow, especially on mobile.
Apple requested that for WebGL the default be for double buffering. When
double buffered, when the canvas is composited (when the current
JavaScript event exits) the canvas's buffer is given to the compositor
and the canvas is given a new buffer (or an old one). That new buffer is
cleared, meaning the contents is gone. It's up to the app to draw stuff
into again. If nothing is drawn the compositor will continue to use the
buffer it acquired earlier.
I think you mean page flipping, not double buffering.
Supporting page flipping in 2D canvas would be fine too, but I don't see
why it would need a change to the API... you would just make commit()
flip which page was active for the context API and clear the newly active
page in one operation.
On Thu, 3 Jan 2013, Gregg Tavares wrote:
So I've been thinking more about this and I'm wondering, for the time
being, why have canvas.setContext and why expose the
Canvas2DRenderingContext constructor?
Well the constructor is needed so that there's a way to do an entirely
off-screen bitmap, for when you just want to do some image work that isn't
going to be displayed.
setContext() is only needed so that you can use one context with multiple
canvases, which is primarily intended to address the WebGL case of having
one context used to render to multiple views with different settings (the
settings being themselves set on the canvas or canvas proxy).
That means we can solve the 1 context multiple canvases issue later
making this a minimal api change?
I thought the 1 context multiple canvases issue was a higher priority
than the canvas on workers issue. Is this wrong?
Is there some reason that won't work?
Well I'd rather not design something that doesn't address a known issue
and then find we have painted ourselves into a corner with respect to that
other issue. Hence trying to solve all the issues at once, or at least
solve them in a way that is compatible with future solutions.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Tue, Jan 8, 2013 at 11:12 AM, Ian Hickson i...@hixie.ch wrote:
On Thu, 3 Jan 2013, Gregg Tavares wrote:
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
discussion seems to have died down here but I'd like to bring up
another issue
In WebGL land we have creation attributes on the drawingbuffer made
for a canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false
});
We're working out the details on how to set those options for the
case where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on
it would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
Right now by 2d canvases are effectively single buffered. At the
appropriate time a copy of the canvas is made and passed to the
compositor. This copy is slow, especially on mobile.
Apple requested that for WebGL the default be for double buffering. When
double buffered, when the canvas is composited (when the current
JavaScript event exits) the canvas's buffer is given to the compositor
and the canvas is given a new buffer (or an old one). That new buffer is
cleared, meaning the contents is gone. It's up to the app to draw stuff
into again. If nothing is drawn the compositor will continue to use the
buffer it acquired earlier.
I think you mean page flipping, not double buffering.
Supporting page flipping in 2D canvas would be fine too, but I don't see
why it would need a change to the API... you would just make commit()
flip which page was active for the context API and clear the newly active
page in one operation.
How would you choose flip vs copy with just commit?
Just to be clear we're on the same page. I want to be able to do this (not
related to workers)
// create a 2d context that flips buffers instead of copies them
var ctx = canvas.getContext(2d, { preserveDrawingBuffer: false });
But, related to workers, if CanvasProxy is truly a proxy for the canvas
then I could do this
// create a 2d context that flips buffers instead of copies them
var ctx = canvasProxy.getContext(2d, { preserveDrawingBuffer: false });
On Thu, 3 Jan 2013, Gregg Tavares wrote:
So I've been thinking more about this and I'm wondering, for the time
being, why have canvas.setContext and why expose the
Canvas2DRenderingContext constructor?
Well the constructor is needed so that there's a way to do an entirely
off-screen bitmap, for when you just want to do some image work that isn't
going to be displayed.
Agreed but that's a separate problem
Problem #1) Allow a worker to render to a canvas
Problem #2) Allow a worker to render offscreen (without communicating with
the main page)
I'm suggesting we only solve problem #1 for now. To do that, all we need is
CanvasProxy to truly be a proxy for the canvas.
setContext() is only needed so that you can use one context with multiple
canvases, which is primarily intended to address the WebGL case of having
one context used to render to multiple views with different settings (the
settings being themselves set on the canvas or canvas proxy).
Right, but since it doesn't doesn't seem to work for WebGL's needs why spec
it now when we can solve problem #1 today and worry about the other
problems later?
That means we can solve the 1 context multiple canvases issue later
making this a minimal api change?
I thought the 1 context multiple canvases issue was a higher priority
than the canvas on workers issue. Is this wrong?
I don't know if it's higher priority. It seemed to inform the worker
related design so it was important to look at.
Is there some reason that won't work?
Well I'd rather not design something that doesn't address a known issue
and then find we have painted ourselves into a corner with respect to that
other issue. Hence trying to solve all the issues at once, or at least
solve them in a way that is compatible with future solutions.
Agreed. I'm just trying to make forward progress. The whole setContext vs
DrawingBuffer etc stuff seems possibly orthogonal to the issue of rendering
to a canvas from a worker. I'm wondering if we can separate the 2 issues by
just making CanvasProxy be a proxy for the canvas with the same API as
Canvas. Since Canvas already is what it is, any solution for single context
multiple canvases has to work with Canvas. So adding a new object,
CanvasProxy, which is really just a representation for Canvas with the same
API doesn't seem like it adds any new issues.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer.
Re: [whatwg] Canvas in Workers
On Tue, 8 Jan 2013, Gregg Tavares wrote:
Supporting page flipping in 2D canvas would be fine too, but I don't
see why it would need a change to the API... you would just make
commit() flip which page was active for the context API and clear
the newly active page in one operation.
How would you choose flip vs copy with just commit?
An argument to the context constructor, probably.
Just to be clear we're on the same page. I want to be able to do this
(not related to workers)
// create a 2d context that flips buffers instead of copies them
var ctx = canvas.getContext(2d, { preserveDrawingBuffer: false });
I would do it like this:
var ctx = new CanvasRenderingContext2D(100, 100, { mode: 'pageflip' });
canvas.setContext(ctx);
ctx.fillRect(10,10,10,10);
ctx.commit(); // implies clearRect(0,0,100,100);
// later
ctx.fillRect(11,11,10,10);
ctx.commit(); // implies clearRect(0,0,100,100);
But, related to workers, if CanvasProxy is truly a proxy for the canvas
then I could do this
// create a 2d context that flips buffers instead of copies them
var ctx = canvasProxy.getContext(2d, { preserveDrawingBuffer: false });
CanvasProxy doesn't have getContext(), because the direct mode doesn't
make sense cross-process. You would do:
var ctx = new CanvasRenderingContext2D(100, 100, { mode: 'pageflip' });
canvasProxy.setContext(ctx);
// draw...
Agreed but that's a separate problem
These are all related problems that contribute to the design of the same
API.
Problem #1) Allow a worker to render to a canvas
Problem #2) Allow a worker to render offscreen (without communicating with
the main page)
I'm suggesting we only solve problem #1 for now. To do that, all we need
is CanvasProxy to truly be a proxy for the canvas.
I'm solving #2 now too. (Actually it's already solved in the spec.) I'm
happy to only discuss #1 if you want, but #2 influences the design of the
API for #1 so the conversation will be weird if we ignore #2.
setContext() is only needed so that you can use one context with
multiple canvases, which is primarily intended to address the WebGL
case of having one context used to render to multiple views with
different settings (the settings being themselves set on the canvas or
canvas proxy).
Right, but since it doesn't doesn't seem to work for WebGL's needs why
spec it now when we can solve problem #1 today and worry about the other
problems later?
It seems to work fine as far as I can tell, as shown by the examples in my
recent e-mail.
Is there some reason that won't work?
Well I'd rather not design something that doesn't address a known
issue and then find we have painted ourselves into a corner with
respect to that other issue. Hence trying to solve all the issues at
once, or at least solve them in a way that is compatible with future
solutions.
Agreed. I'm just trying to make forward progress.
At this point I'm not aware of any problems with the current proposal
(what's in the HTML spec plus adding setSettings() in WebGL, making the
WebGLRenderingContext constructable, and providing an in-worker
alternative destination for WebGL for when you don't want to draw to a
canvas, as discussed in my earlier e-mail today).
I'm wondering if we can separate the 2 issues by just making CanvasProxy
be a proxy for the canvas with the same API as Canvas.
That doesn't work for 2D. The original API (getContext) relies on implicit
commits, and we can't do implicit commits sanely in a worker. That's the
main reason for having a different mechanism (setContext).
Anyway, I don't really understand what's wrong with the proposal that
addresses all the problems that have been raised at once. I don't see a
need to do this bit by bit when we've already got a full solution.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On 2013/01/04 9:46, Gregg Tavares wrote:
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
In WebGL land we have creation attributes on the drawingbuffer made for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on it
would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
The reason for this attribute and defaulting it to false, is so that
tiled renderers can avoid copying the area of the previously-rendered
drawing buffer corresponding to the current tile, into tile memory prior
to rendering. Avoiding this copy is a big performance and power win for
tiled renderers. As far as I am aware every mobile GPU, with the
exception of NVIDIA's Tegra, is a tiled renderer so supporting them is
very important.
I would expect these wins to be equally important when doing animation
in Canvas2D as I would double buffering to to avoid flicker.
Regards
-Mark
--
注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
たら削除を行い配信者にご連絡をお願いいたし ます.
NOTE: This electronic mail message may contain confidential and
privileged information from HI Corporation. If you are not the intended
recipient, any disclosure, photocopying, distribution or use of the
contents of the received information is prohibited. If you have received
this e-mail in error, please notify the sender immediately and
permanently delete this message and all related copies.
Re: [whatwg] Canvas in Workers
On Wed, 9 Jan 2013, Mark Callow wrote: I would expect these wins to be equally important when doing animation in Canvas2D as I would double buffering to to avoid flicker. Are implementors interested in a way to make 2D rendering contexts use page flipping instead of double buffering? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On 2013/01/09 12:37, Ian Hickson wrote: On Wed, 9 Jan 2013, Mark Callow wrote: I would expect these wins to be equally important when doing animation in Canvas2D as I would double buffering to to avoid flicker. Are implementors interested in a way to make 2D rendering contexts use page flipping instead of double buffering? I don't understand what you mean by page flipping so, even if I were a browser implementer, I couldn't answer. Regards -Mark -- 注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合 が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情 報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし たら削除を行い配信者にご連絡をお願いいたし ます. NOTE: This electronic mail message may contain confidential and privileged information from HI Corporation. If you are not the intended recipient, any disclosure, photocopying, distribution or use of the contents of the received information is prohibited. If you have received this e-mail in error, please notify the sender immediately and permanently delete this message and all related copies.
Re: [whatwg] Canvas in Workers
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
discussion seems to have died down here but I'd like to bring up another
issue
In WebGL land we have creation attributes on the drawingbuffer made for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on it
would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
Right now by 2d canvases are effectively single buffered. At the
appropriate time a copy
of the canvas is made and passed to the compositor. This copy is slow,
especially on
mobile.
Apple requested that for WebGL the default be for double buffering. When
double buffered, when the canvas is composited (when the current JavaScript
event exits)
the canvas's buffer is given to the compositor and the canvas is given a
new buffer (or
an old one). That new buffer is cleared, meaning the contents is gone. It's
up to the app to
draw stuff into again. If nothing is drawn the compositor will continue to
use the
buffer it acquired earlier.
In WebGL you can opt into the slower copy path. For Canvas 2D while the
default
has to remain the slow copy path it would be nice to be able to opt into
the faster
swap double buffered path.
In the 2D canvas, whenever you bind to a new canvas, the context is reset
to its default state, the context's hit region list is reset, and the
context's bitmap is reset. The next time the context is flushed, the
canvas itself is always reset (since flushing the context causes the
bitmap and hit region list to be pushed to the canvas, replacing whatever
was there before).
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Tue, 11 Dec 2012, Gregg Tavares (社�~T�) wrote:
discussion seems to have died down here but I'd like to bring up another
issue
In WebGL land we have creation attributes on the drawingbuffer made for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on it
would be nice if it worked for both APIs the same.
What does it do?
In the 2D canvas, whenever you bind to a new canvas, the context is reset
to its default state, the context's hit region list is reset, and the
context's bitmap is reset. The next time the context is flushed, the
canvas itself is always reset (since flushing the context causes the
bitmap and hit region list to be pushed to the canvas, replacing whatever
was there before).
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
discussion seems to have died down here but I'd like to bring up another
issue
In WebGL land we have creation attributes on the drawingbuffer made for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for canvas
2d for mobile. Which suggests that whatever API is decided on it would be
nice if it worked for both APIs the same.
Re: [whatwg] Canvas in Workers
On Sat, Dec 1, 2012 at 2:44 AM, Ian Hickson i...@hixie.ch wrote:
On Fri, 30 Nov 2012, Gregg Tavares (社ç~T¨) wrote:
on ImageBitmap should zero size canvases just work (and create a 0 sized
ImageBitmap)?
My personal preference is for APIs that just work with zero sizes so I
don't have to write lots of special cases for handling zero.
For example [1,2,3].slice(0,0) returns []. It doesn't throw.
abc.substring(0,0) returns it doesn't throw. fillRect(x, y, 0, 0)
doesn't throw. etc...
It just makes life a lot easier
The main reason 0-sized canvases have always thrown in drawImage() is that
I couldn't work out what you would paint, nor why you'd have a zero-sized
canvas, and throwing seemed like it'd be the best way to help the author
figure out where the problem was, rather than just ignoring the call and
having the author scratch their head about why nothing was happening.
If there's cases where you would legitimately end up with zero-sized
canvases that you'd try to draw from, though, I'm happy to change it.
I don't see how zero sized canvases are any different than zero sized
arrays or empty strings. It's not a matter of use case. It's a matter of
not having to write checks everywhere for 0. If I'm writing some app that
takes a user supplied size (say a photo editing app where the user can
select a rectangle and copy and paste), why do I want to have to check for
zero?
var x = Math.min(x1, x2);
var y = Math.min(y1, y2);
var width = Math.abs(x1 - x2);
var height = Math.abs(y1 - y2);
// Do something with rect defined by x,y,width,height
This seems no different from malloc(0) in C or the other cases I've
mentioned (array size 0 and empty string). Lots of programming becomes
easier when size = 0 works. Maybe I'm animating
function draw() {
var scale = Math.sin() * 0.5 + 0.5;
var width = realWidth * scale;
var height = realHeight * scale;
// do something with width, height
}
Why do I want to have to check for zero and special case it?
You could argue that I'd have to check of negative values but that's still
nicer than checking for 0
function draw() {
var scale = Math.sin() * 0.5 + 0.5;
var width = Math.max(0, realWidth * scale);
var height = Math.max(0, realHeight * scale);
// do something with width, height
}
vs
function draw() {
var scale = Math.sin() * 0.5 + 0.5;
var width = realWidth * scale;
var height = realHeight * scale;
if (width = 0 || height = 0) {
// skip this step
} else {
// do something with width, height
}
}
I'm just making the case it seems like 0 should always work. That includes
ImageBitmap, Canvas and ImageData
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Fri, 30 Nov 2012, Gregg Tavares (社�~T�) wrote: on ImageBitmap should zero size canvases just work (and create a 0 sized ImageBitmap)? My personal preference is for APIs that just work with zero sizes so I don't have to write lots of special cases for handling zero. For example [1,2,3].slice(0,0) returns []. It doesn't throw. abc.substring(0,0) returns it doesn't throw. fillRect(x, y, 0, 0) doesn't throw. etc... It just makes life a lot easier The main reason 0-sized canvases have always thrown in drawImage() is that I couldn't work out what you would paint, nor why you'd have a zero-sized canvas, and throwing seemed like it'd be the best way to help the author figure out where the problem was, rather than just ignoring the call and having the author scratch their head about why nothing was happening. If there's cases where you would legitimately end up with zero-sized canvases that you'd try to draw from, though, I'm happy to change it. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Fri, Nov 16, 2012 at 3:25 PM, Ian Hickson i...@hixie.ch wrote: var proxy = canvas.transferControlToProxy()); bikeshed Proxy with a capital P is an API in ES6 for creating Proxy objects. While I recognize that this is obviously just a conceptual sketch of your idea and that you're merely following the camel case tradition, now is probably the best time to be informed of potential naming conflicts. Rick
Re: [whatwg] Canvas in Workers
On Fri, Nov 16, 2012 at 1:03 PM, Oliver Hunt oli...@apple.com wrote: postMessage already has an ownership transfer list, we could always extend the set of transferable types to include a canvas like thing (the context or some such). Transferring contexts is tolerable for 2d, but a non-starter for webgl - there's too much stuff that can be loaded into a context that would have to be transferred cross-process. Hixie's approach uses a lightweight canvas proxy instead. IIRC the last time I suggested having canvas (or at least canvas-like) APIs available in workers it turned into a spec for rotating and scaling images (which i still don't fully understand). That was an attempt to solve the use-cases without doing the work to actually port canvas to the worker. Hixie's now done the latter. ~TJ
Re: [whatwg] Canvas in Workers
On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 3:25 PM, Ian Hickson i...@hixie.ch wrote: var proxy = canvas.transferControlToProxy()); bikeshed Proxy with a capital P is an API in ES6 for creating Proxy objects. While I recognize that this is obviously just a conceptual sketch of your idea and that you're merely following the camel case tradition, now is probably the best time to be informed of potential naming conflicts. The word Proxy is used all over the place on the Web platform, I'm not too worried about that. Right now the object returned above is CanvasProxy, which is the kind of proxy that it refers to. I'm certainly open to another name, any suggestions? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Fri, 16 Nov 2012, Oliver Hunt wrote: postMessage already has an ownership transfer list, we could always extend the set of transferable types to include a canvas like thing (the context or some such). That's what CanvasProxy is. IIRC the last time I suggested having canvas (or at least canvas-like) APIs available in workers it turned into a spec for rotating and scaling images (which i still don't fully understand). Yeah, this is intended to make that kind of thing unnecessary. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
CanvasProxy is a fine disambiguation. Rick On Fri, Nov 16, 2012 at 4:11 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 3:25 PM, Ian Hickson i...@hixie.ch wrote: var proxy = canvas.transferControlToProxy()); bikeshed Proxy with a capital P is an API in ES6 for creating Proxy objects. While I recognize that this is obviously just a conceptual sketch of your idea and that you're merely following the camel case tradition, now is probably the best time to be informed of potential naming conflicts. The word Proxy is used all over the place on the Web platform, I'm not too worried about that. Right now the object returned above is CanvasProxy, which is the kind of proxy that it refers to. I'm certainly open to another name, any suggestions? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 4:11 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 3:25 PM, Ian Hickson i...@hixie.ch wrote: var proxy = canvas.transferControlToProxy()); bikeshed Proxy with a capital P is an API in ES6 for creating Proxy objects. While I recognize that this is obviously just a conceptual sketch of your idea and that you're merely following the camel case tradition, now is probably the best time to be informed of potential naming conflicts. The word Proxy is used all over the place on the Web platform, I'm not too worried about that. Right now the object returned above is CanvasProxy, which is the kind of proxy that it refers to. I'm certainly open to another name, any suggestions? CanvasProxy is a fine disambiguation. Do you think we should rename the method above to say that too? canvas.transferControlToCanvasProxy() is a bit verbose. :-) I had shortened it to just canvas.transferControlToProxy on the assumption that the fact that it was called on a canvas would disambiguate that it was a canvas proxy, but I'm certainly open to other suggestions. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas in Workers
On Fri, Nov 16, 2012 at 4:48 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 4:11 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 16 Nov 2012, Rick Waldron wrote: On Fri, Nov 16, 2012 at 3:25 PM, Ian Hickson i...@hixie.ch wrote: var proxy = canvas.transferControlToProxy()); bikeshed Proxy with a capital P is an API in ES6 for creating Proxy objects. While I recognize that this is obviously just a conceptual sketch of your idea and that you're merely following the camel case tradition, now is probably the best time to be informed of potential naming conflicts. The word Proxy is used all over the place on the Web platform, I'm not too worried about that. Right now the object returned above is CanvasProxy, which is the kind of proxy that it refers to. I'm certainly open to another name, any suggestions? CanvasProxy is a fine disambiguation. Do you think we should rename the method above to say that too? canvas.transferControlToCanvasProxy() is a bit verbose. :-) I had shortened it to just canvas.transferControlToProxy on the Is there any other thing that control can be transferred to? canvas.transferControl() would be sufficient... Rick assumption that the fact that it was called on a canvas would disambiguate that it was a canvas proxy, but I'm certainly open to other suggestions. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
