Re: [whatwg] Counterproposal for canvas in workers
On Sun, Oct 20, 2013 at 7:09 PM, Kyle Huey wrote: > I think most of these have been said already, but my issues with (my > understand of?) this proposal are: > > 1. It requires defining a new script execution environment different from > both the main thread and workers. That seems like a lot of unnecessary > work. And this execution environment is either going to duplicate > much of web workers or it will lack a lot of functionality that people want. > Why would that be a lot of work? It would be almost identical to a worker environment if not the same apart from some extra interfaces. > 2. It is mostly stateless, requiring any global one time setup to happen > per task. But it also maintains a relatively heavy amount of state (a > script execution environment and global object for each task on each > canvas) which will persist for the lifetime of the canvas. Executing a > "drawSplashScreen" task at the beginning of a long-running game means you > would carry that task's memory overhead for the entire game. Developers > won't actually want to do that so they'll either have to segregate tasks > onto different canvases based on task lifetime or they will have to limit > the number of tasks they use. > True, tasks will be carried around by the canvas that created them. I don't understand why this would be harder than manually managing the state. Could you not manage this per task? (A 'detroyTask' function would cover this too) > 3. 'synchronized' false seems pretty useless. When would it be useful > to execute a bunch of tasks out of order and update the screen whenever any > of them finish? You would have essentially no guarantees on what appears > on screen. > I had the page that Ken pointed to in mind: http://www.turbosquid.com/Search/3D-Models/Vehicle/Car What if you're drawing a bunch of complex models to 1 canvas where you know that each result is in a separate location? (Could also be used for sprite sheets) > 4. I'm not sure whether these tasks are intended to be spawned from the > main thread or from a worker thread. If the former, this proposal does not > solve the "render things without waiting on the main thread at all" goal. > If the latter, it's unclear how synchronization with main thread DOM > updates works. > both. I think I went over how the synchronization works in earlier emails. Basically in a task, everything is queued so if you fire a task B in task A, all the operations of task B will be inserted atomically in the right location. (My naive polyfill doesn't do this yet but it would be easy to implement) > It seems like we're getting towards consensus on the other thread. I > would prefer to drag that proposal over the finish line since this will > require a lot more work. > I don't think it does require a lot more work (although people seem to disagree). This is a model that covers the following: - drawing is synchronized with the main thread (Glenn started this as a new thread) - you can use APIs that are resticted for workers (such as text) in a task - you can use content while is not constructed yet - you can draw to the same canvas from multiple threads - no need for triple buffering - transparent for the author (No need to create workers or figure out how many workers is optimal. No complicated message passing. No need to change your use of canvas apart from reading pixels which they shouldn't do for perf anyway) Even if my proposal doesn't go anywhere (and it certainly looks like that :-) ); it will highlight possible issues that need to be addressed or limitations that authors will have to work around.
Re: [whatwg] Counterproposal for canvas in workers
On Sun, Oct 20, 2013 at 12:28 AM, Robert O'Callahan wrote:
> On Fri, Oct 18, 2013 at 6:50 AM, Rik Cabanier wrote:
>
>> Extra methods on the canvas API:
>>
>> Promise setTaskScript(DOMString script); // can error be in promise?
>> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
>> true); // Transferable elements allowed in dictionary
>>
>> Object that is active in the task:
>>
>> interface CanvasTask {
>>
>> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
>> attribute Function onTask;
>>
>> }
>>
>> CanvasTask implements HTMLCanvasElement;
>>
>
> It looks like you intend CanvasTask to be the global object for the task
> script? So it's not a Worker and you don't get anything from
> WorkerGlobalScope? That's extremely limiting and also adds a lot of
> complexity by introducing a new kind of script global. You really would
> want to reuse workers here.
>
I have not really given this much thought. CanvasTask could be implemented
on top of a worker (like my naive polyfill)
> Also, making the HTMLCanvasElement API accessible from non-main threads is
> a big no-no. You can't let people do CanvasTask.document.window.whatever
> from non-main threads.
>
Yes, you can with my proposal.
The object that you get back with 'createCanvas' will be different under
the hood than the one you get in the main thread, but its *interface* will
be the same. Since all the calls can be marshalled to the main thread,
features like text and cors keep working. It should also be possible to use
images and canvaselement from the main thread since you don't have to
transfer their data.
Re: [whatwg] Counterproposal for canvas in workers
I think most of these have been said already, but my issues with (my understand of?) this proposal are: 1. It requires defining a new script execution environment different from both the main thread and workers. That seems like a lot of unnecessary work. And this execution environment is either going to duplicate much of web workers or it will lack a lot of functionality that people want. 2. It is mostly stateless, requiring any global one time setup to happen per task. But it also maintains a relatively heavy amount of state (a script execution environment and global object for each task on each canvas) which will persist for the lifetime of the canvas. Executing a "drawSplashScreen" task at the beginning of a long-running game means you would carry that task's memory overhead for the entire game. Developers won't actually want to do that so they'll either have to segregate tasks onto different canvases based on task lifetime or they will have to limit the number of tasks they use. 3. 'synchronized' false seems pretty useless. When would it be useful to execute a bunch of tasks out of order and update the screen whenever any of them finish? You would have essentially no guarantees on what appears on screen. 4. I'm not sure whether these tasks are intended to be spawned from the main thread or from a worker thread. If the former, this proposal does not solve the "render things without waiting on the main thread at all" goal. If the latter, it's unclear how synchronization with main thread DOM updates works. It seems like we're getting towards consensus on the other thread. I would prefer to drag that proposal over the finish line since this will require a lot more work. - Kyle
Re: [whatwg] Counterproposal for canvas in workers
On Sun, Oct 20, 2013 at 2:22 AM, Robert O'Callahan wrote: > > On Fri, Oct 18, 2013 at 3:10 PM, Glenn Maynard wrote: >> >> Also, with the "transferToImageBuffer" approach, if you want to render >> >> from a worker into multiple canvases in the UI thread, you have to post >> >> those ImageBuffers over to the main thread each frame, which has the >> same >> >> (potential) synchronization issues as the transferDrawingBufferToCanvas >> >> proposal. >> > I'm confused here. You said "if you want to render from a worker into > multiple canvases in the UI thread", which I took to mean that you wanted > to synchronize canvas updates from workers with DOM changes made by the UI > thread. But now you're saying you don't want to do that. So I don't know > what you meant. > This has nothing to do with synchronizing to DOM updates. The point is to be able to render from a single WebGL context to multiple canvanses, without having to create multiple WebGL contexts and upload a second copy of textures, vertex programs, etc. into it, which is very expensive. Doing that efficiently and asynchronously is what this is trying to solve. (The particular problem I pointed out is specific to doing that from Workers with canvases in the UI thread, but the goal itself is not.) -- Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 6:50 AM, Rik Cabanier wrote:
> Extra methods on the canvas API:
>
> Promise setTaskScript(DOMString script); // can error be in promise?
> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
> true); // Transferable elements allowed in dictionary
>
> Object that is active in the task:
>
> interface CanvasTask {
>
> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
> attribute Function onTask;
>
> }
>
> CanvasTask implements HTMLCanvasElement;
>
It looks like you intend CanvasTask to be the global object for the task
script? So it's not a Worker and you don't get anything from
WorkerGlobalScope? That's extremely limiting and also adds a lot of
complexity by introducing a new kind of script global. You really would
want to reuse workers here.
Also, making the HTMLCanvasElement API accessible from non-main threads is
a big no-no. You can't let people do CanvasTask.document.window.whatever
from non-main threads.
Rob
--
Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w *
*
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 4:17 PM, Glenn Maynard wrote: > On Thu, Oct 17, 2013 at 10:25 PM, Robert O'Callahan >wrote: > > On Fri, Oct 18, 2013 at 3:10 PM, Glenn Maynard wrote: > >> Also, with the "transferToImageBuffer" approach, if you want to render > >> from a worker into multiple canvases in the UI thread, you have to post > >> those ImageBuffers over to the main thread each frame, which has the > same > >> (potential) synchronization issues as the transferDrawingBufferToCanvas > >> proposal. > > > > What are those issues? You can do a single postMessage passing a complete > > set of ImageBItmaps. > > > > See > http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0193.html > . > I don't know the answer to this; my feeling is that posting to the UI > thread and scripts in the UI thread may or may not have > (performance/smoothness) issues, but doing it all in the worker avoids any > potential for this issue. > I'm confused here. You said "if you want to render from a worker into multiple canvases in the UI thread", which I took to mean that you wanted to synchronize canvas updates from workers with DOM changes made by the UI thread. But now you're saying you don't want to do that. So I don't know what you meant. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
Hi Justin,
I will do reach out to some developers to see if they think it's useful.
As an experiment, I wrote a javascript polyfill that implements a rough
version of my proposal.
I took a box 2d example and ported it over. Since it draws to canvas under
the hood, I didn't have to change any code to make it work.
Original example: http://cabanier.github.io/CanvasWorker/test2.html
Example that uses a worker: http://cabanier.github.io/CanvasWorker/test.html
I added an js-driven animation to show how the original example is hogging
the main thread.
Unfortunately, on Firefox there's a lot of GC overhead so it's better to
use Safari or Chrome.
On Fri, Oct 18, 2013 at 11:30 AM, Justin Novosad wrote:
>
>
>
> On Fri, Oct 18, 2013 at 12:36 PM, Rik Cabanier wrote:
>
>> Hi Justin,
>>
>> no, everything is running synchronized and there is no added latency.
>>
>> API calls on canvas will be recorded if there are outstanding tasks.
>> So, for this API call:
>>
>> ctx.drawImage(MinionCanvas, ...)
>>
>> Since it is happening in a task, drawImage will be recorded. It will only
>> execute once the task and its subtasks (ie
>> MinionCanvas.executeTask("drawMinion",
>> {}) ) are done.
>>
>> It sounds complicated but I think it's much easier for an author than
>> having to send bitmaps and message back and forth.
>>
>
> Ok, got it. I think this is compelling for many use cases, but I am not
> sure whether it is as generally useful/usable as WorkerCanvas. It would be
> great to get more web developer feedback. In particular from graphics
> intensive game and interactive app people.
>
>
>> On Fri, Oct 18, 2013 at 6:48 AM, Justin Novosad wrote:
>>
>>> Rik, I don't think the nested tasks in your example are a good use case.
>>> That workflow adds a frame of latency to the sub tasks. This is a problem
>>> because the drawImage call would be drawing from a source canvas that it
>>> out of phase with the mainscene context. To synchronize the content, I
>>> think the drawImage calls would have to be placed in a Promise resolution
>>> handler that gets invoked once all the executeTask Promises for the
>>> subtasks are resolved. That means that the drawImage calls necessary for
>>> drawing "mainscene" would end-up executing asynchronously, therefore
>>> outside the scope of the "mainscene" task function, which is a problem. So
>>> I don't think the executeTask proposal is well suited for sharding
>>> rendering jobs, at least not the way you illustrated it with that example.
>>>
>>>
>>>
>>> On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier wrote:
>>>
Extra methods on the canvas API:
Promise setTaskScript(DOMString script); // can error be in promise?
Promise executeTask(DOMString id, dictionary json, boolean synchronized
= true); // Transferable elements allowed in dictionary
Object that is active in the task:
interface CanvasTask {
HTMLCanvasElement createCanvas(unsigned long width, unsigned long
height);
attribute Function onTask;
}
CanvasTask implements HTMLCanvasElement;
Example code:
var c = document.getElementById("gameCanvas");
var gameState = {};
window.addEventListener("load", function(){
c.setTaskScript("gameLogic.js").then(function(){
c.executeTask("mainscene", gameState);
});
});
window.requestAnimationFrame(function(){
c.executeTask("mainscene", gameState);
}
Example code for gameLogic.js:
var ctx = getContext("2d");
onTask = function(DOMString id, dictionary json) {
if(id == "mainscene") {
if(typeof(MinionCanvas)=="Undefined") {
MinionCanvas = createCanvas(200, 300);
MinionCanvas.executeTask("drawMinion", {}) // creates promise under
the hood
}
if(typeof(SpaceShipCanvas)=="Undefined")
SpaceShipCanvas = createCanvas(300, 300);
SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
spaceship
executeTask("drawBackDrop", gameState); // in other task
executeTask("drawBoss", gameState); // lots of js to draw the boss so
better done in task
for(...) //for each minion {
... // set the matrix
ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
minion might still be drawing in the other thread
}
for(...) //for each spaceship {
..// set the matrix
ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still
be drawing in the other task
}
.. // other drawing commands for score, controls, etc
} else if(id == "drawMinion") {
...
} else if(id == "drawSpaceShip") {
... // set up tasks to draw parts of the ship?
} ...
}
On Thu, Oct 17, 2013 at 8:10 PM, Rik
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 12:36 PM, Rik Cabanier wrote:
> Hi Justin,
>
> no, everything is running synchronized and there is no added latency.
>
> API calls on canvas will be recorded if there are outstanding tasks.
> So, for this API call:
>
> ctx.drawImage(MinionCanvas, ...)
>
> Since it is happening in a task, drawImage will be recorded. It will only
> execute once the task and its subtasks (ie
> MinionCanvas.executeTask("drawMinion",
> {}) ) are done.
>
> It sounds complicated but I think it's much easier for an author than
> having to send bitmaps and message back and forth.
>
Ok, got it. I think this is compelling for many use cases, but I am not
sure whether it is as generally useful/usable as WorkerCanvas. It would be
great to get more web developer feedback. In particular from graphics
intensive game and interactive app people.
> On Fri, Oct 18, 2013 at 6:48 AM, Justin Novosad wrote:
>
>> Rik, I don't think the nested tasks in your example are a good use case.
>> That workflow adds a frame of latency to the sub tasks. This is a problem
>> because the drawImage call would be drawing from a source canvas that it
>> out of phase with the mainscene context. To synchronize the content, I
>> think the drawImage calls would have to be placed in a Promise resolution
>> handler that gets invoked once all the executeTask Promises for the
>> subtasks are resolved. That means that the drawImage calls necessary for
>> drawing "mainscene" would end-up executing asynchronously, therefore
>> outside the scope of the "mainscene" task function, which is a problem. So
>> I don't think the executeTask proposal is well suited for sharding
>> rendering jobs, at least not the way you illustrated it with that example.
>>
>>
>>
>> On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier wrote:
>>
>>> Extra methods on the canvas API:
>>>
>>> Promise setTaskScript(DOMString script); // can error be in promise?
>>> Promise executeTask(DOMString id, dictionary json, boolean synchronized
>>> = true); // Transferable elements allowed in dictionary
>>>
>>> Object that is active in the task:
>>>
>>> interface CanvasTask {
>>>
>>> HTMLCanvasElement createCanvas(unsigned long width, unsigned long
>>> height);
>>> attribute Function onTask;
>>>
>>> }
>>>
>>> CanvasTask implements HTMLCanvasElement;
>>>
>>> Example code:
>>>
>>> var c = document.getElementById("gameCanvas");
>>>
>>> var gameState = {};
>>>
>>> window.addEventListener("load", function(){
>>>
>>> c.setTaskScript("gameLogic.js").then(function(){
>>>
>>> c.executeTask("mainscene", gameState);
>>>
>>> });
>>>
>>> });
>>>
>>>
>>> window.requestAnimationFrame(function(){
>>>
>>> c.executeTask("mainscene", gameState);
>>>
>>> }
>>>
>>>
>>> Example code for gameLogic.js:
>>>
>>> var ctx = getContext("2d");
>>>
>>> onTask = function(DOMString id, dictionary json) {
>>>
>>> if(id == "mainscene") {
>>>
>>> if(typeof(MinionCanvas)=="Undefined") {
>>>
>>> MinionCanvas = createCanvas(200, 300);
>>>
>>> MinionCanvas.executeTask("drawMinion", {}) // creates promise under
>>> the hood
>>>
>>> }
>>>
>>> if(typeof(SpaceShipCanvas)=="Undefined")
>>>
>>> SpaceShipCanvas = createCanvas(300, 300);
>>>
>>>
>>> SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
>>> spaceship
>>>
>>>
>>> executeTask("drawBackDrop", gameState); // in other task
>>>
>>> executeTask("drawBoss", gameState); // lots of js to draw the boss so
>>> better done in task
>>>
>>>
>>> for(...) //for each minion {
>>>
>>> ... // set the matrix
>>> ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
>>> minion might still be drawing in the other thread
>>>
>>> }
>>> for(...) //for each spaceship {
>>>
>>> ..// set the matrix
>>>
>>> ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still
>>> be drawing in the other task
>>>
>>> }
>>>
>>> .. // other drawing commands for score, controls, etc
>>>
>>> } else if(id == "drawMinion") {
>>>
>>> ...
>>>
>>> } else if(id == "drawSpaceShip") {
>>>
>>> ... // set up tasks to draw parts of the ship?
>>>
>>> } ...
>>>
>>> }
>>>
>>>
>>>
>>> On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier wrote:
>>>
On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan <
rob...@ocallahan.org> wrote:
> On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote:
>
>> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote:
>>
>>> Creating temporary canvases is still possible. I'm unsure how it
>>> would be different from a worker.
>>> An advantage would be that you can draw to the temporary canvases in
>>> parallel to using them. Only PIXEL access is disallowed, you can still
>>> call
>>> drawImage using a canvas that has outstanding tasks.
>>>
>>
>> Right. The write-only restriction would only apply to canvas contexts
>> that commit (push their tasks) directly down to the compositor. You could
>> still create a canvas that is local to the worker, rasterize it in the
>>
Re: [whatwg] Counterproposal for canvas in workers
Hi Justin,
no, everything is running synchronized and there is no added latency.
API calls on canvas will be recorded if there are outstanding tasks.
So, for this API call:
ctx.drawImage(MinionCanvas, ...)
Since it is happening in a task, drawImage will be recorded. It will only
execute once the task and its subtasks (ie
MinionCanvas.executeTask("drawMinion",
{}) ) are done.
It sounds complicated but I think it's much easier for an author than
having to send bitmaps and message back and forth.
On Fri, Oct 18, 2013 at 6:48 AM, Justin Novosad wrote:
> Rik, I don't think the nested tasks in your example are a good use case.
> That workflow adds a frame of latency to the sub tasks. This is a problem
> because the drawImage call would be drawing from a source canvas that it
> out of phase with the mainscene context. To synchronize the content, I
> think the drawImage calls would have to be placed in a Promise resolution
> handler that gets invoked once all the executeTask Promises for the
> subtasks are resolved. That means that the drawImage calls necessary for
> drawing "mainscene" would end-up executing asynchronously, therefore
> outside the scope of the "mainscene" task function, which is a problem. So
> I don't think the executeTask proposal is well suited for sharding
> rendering jobs, at least not the way you illustrated it with that example.
>
>
>
> On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier wrote:
>
>> Extra methods on the canvas API:
>>
>> Promise setTaskScript(DOMString script); // can error be in promise?
>> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
>> true); // Transferable elements allowed in dictionary
>>
>> Object that is active in the task:
>>
>> interface CanvasTask {
>>
>> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
>> attribute Function onTask;
>>
>> }
>>
>> CanvasTask implements HTMLCanvasElement;
>>
>> Example code:
>>
>> var c = document.getElementById("gameCanvas");
>>
>> var gameState = {};
>>
>> window.addEventListener("load", function(){
>>
>> c.setTaskScript("gameLogic.js").then(function(){
>>
>> c.executeTask("mainscene", gameState);
>>
>> });
>>
>> });
>>
>>
>> window.requestAnimationFrame(function(){
>>
>> c.executeTask("mainscene", gameState);
>>
>> }
>>
>>
>> Example code for gameLogic.js:
>>
>> var ctx = getContext("2d");
>>
>> onTask = function(DOMString id, dictionary json) {
>>
>> if(id == "mainscene") {
>>
>> if(typeof(MinionCanvas)=="Undefined") {
>>
>> MinionCanvas = createCanvas(200, 300);
>>
>> MinionCanvas.executeTask("drawMinion", {}) // creates promise under the
>> hood
>>
>> }
>>
>> if(typeof(SpaceShipCanvas)=="Undefined")
>>
>> SpaceShipCanvas = createCanvas(300, 300);
>>
>>
>> SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
>> spaceship
>>
>>
>> executeTask("drawBackDrop", gameState); // in other task
>>
>> executeTask("drawBoss", gameState); // lots of js to draw the boss so
>> better done in task
>>
>>
>> for(...) //for each minion {
>>
>> ... // set the matrix
>> ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
>> minion might still be drawing in the other thread
>>
>> }
>> for(...) //for each spaceship {
>>
>> ..// set the matrix
>>
>> ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still
>> be drawing in the other task
>>
>> }
>>
>> .. // other drawing commands for score, controls, etc
>>
>> } else if(id == "drawMinion") {
>>
>> ...
>>
>> } else if(id == "drawSpaceShip") {
>>
>> ... // set up tasks to draw parts of the ship?
>>
>> } ...
>>
>> }
>>
>>
>>
>> On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier wrote:
>>
>>>
>>>
>>>
>>> On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan >> > wrote:
>>>
On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote:
> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote:
>
>> Creating temporary canvases is still possible. I'm unsure how it
>> would be different from a worker.
>> An advantage would be that you can draw to the temporary canvases in
>> parallel to using them. Only PIXEL access is disallowed, you can still
>> call
>> drawImage using a canvas that has outstanding tasks.
>>
>
> Right. The write-only restriction would only apply to canvas contexts
> that commit (push their tasks) directly down to the compositor. You could
> still create a canvas that is local to the worker, rasterize it in the
> worker and do readbacks in the worker, create ImageBitmaps from it, etc.
>
I'm not sure that you and Rik are talking about the same thing, since
he's still talking about "outstanding tasks". If you are talking about the
same thing, I don't know what it is. I'd like to see some concrete details
for what you'd change in the current WorkerCanvas proposal. For the sake of
clarity I've put (my understand of) it here:
https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal
>>>
>>> I
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 10:25 PM, Robert O'Callahan wrote: > On Fri, Oct 18, 2013 at 3:10 PM, Glenn Maynard wrote: > >> "transferToImageBuffer" looks like it would create a new ImageBuffer for >> each frame, so you'd need to add a close() method to make sure they don't >> accumulate due to GC lag, >> > > That's a good point. We will need something like that. It would only > neuter that thread's (main thread or worker thread) version of the > ImageBitmap. > But don't forget that this is a cost to authors, who now have to .close() the object. If they forget, or don't know they need to do that, or miss some code paths, then there are no blatant side-effects--things are just mysteriously slower, and probably with more of an effect in some implementations than others (which is never good). With attachToCanvas, this can't happen. and it seems like turning this into a fast buffer swap under the hood >> would be harder. >> > > I don't see why. > To me it seems obviously more complicated, but I guess I'll leave that evaluation to implementors. > Also, with the "transferToImageBuffer" approach, if you want to render >> from a worker into multiple canvases in the UI thread, you have to post >> those ImageBuffers over to the main thread each frame, which has the same >> (potential) synchronization issues as the transferDrawingBufferToCanvas >> proposal. >> > > What are those issues? You can do a single postMessage passing a complete > set of ImageBItmaps. > See http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0193.html. I don't know the answer to this; my feeling is that posting to the UI thread and scripts in the UI thread may or may not have (performance/smoothness) issues, but doing it all in the worker avoids any potential for this issue. On Thu, Oct 17, 2013 at 10:48 PM, Rik Cabanier wrote: > This proposal implies an extra buffer for the 2d context. My proposal >>> doesn't require that so it's more memory efficient + you can draw in >>> parallel. >>> >> >> You always need at least two buffers: a back-buffer for drawing and a >> front-buffer for display (compositing). Otherwise, as soon as you start >> drawing the next frame, the old frame is gone, so you won't be able to >> recomposite (on reflow, CSS filter changes, etc). Double-buffering at a >> minimum is pretty standard, even for native applications (with none of this >> Web complexity in the way). >> > > Won't you need another front-buffer for the worker to draw to? > I don't see why. You just use double-buffering as always: the worker draws to the backbuffer, then the drawing buffer (back-buffer) and the buffer being displayed (front-buffer) are flipped and you start over. I don't think there's any difference in this between native OpenGL, today-WebGL, and WorkerCanvas-WebGL. (I realize I'm looking at this from a WebGL-biased perspective, which clears the buffer between presentations unless you tell it not to. This is specifically to allow this sort of fast buffer flipping. 2d canvas doesn't do that, so to allow copy-free display it'd need a flag like WebGL's preserveDrawingBuffer = false. This applies to any API trying to get buffer flipping out of 2d canvas, though--something has to be added or changed. We don't need to address this here.) -- Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
Rik, I don't think the nested tasks in your example are a good use case.
That workflow adds a frame of latency to the sub tasks. This is a problem
because the drawImage call would be drawing from a source canvas that it
out of phase with the mainscene context. To synchronize the content, I
think the drawImage calls would have to be placed in a Promise resolution
handler that gets invoked once all the executeTask Promises for the
subtasks are resolved. That means that the drawImage calls necessary for
drawing "mainscene" would end-up executing asynchronously, therefore
outside the scope of the "mainscene" task function, which is a problem. So
I don't think the executeTask proposal is well suited for sharding
rendering jobs, at least not the way you illustrated it with that example.
On Fri, Oct 18, 2013 at 12:50 AM, Rik Cabanier wrote:
> Extra methods on the canvas API:
>
> Promise setTaskScript(DOMString script); // can error be in promise?
> Promise executeTask(DOMString id, dictionary json, boolean synchronized =
> true); // Transferable elements allowed in dictionary
>
> Object that is active in the task:
>
> interface CanvasTask {
>
> HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
> attribute Function onTask;
>
> }
>
> CanvasTask implements HTMLCanvasElement;
>
> Example code:
>
> var c = document.getElementById("gameCanvas");
>
> var gameState = {};
>
> window.addEventListener("load", function(){
>
> c.setTaskScript("gameLogic.js").then(function(){
>
> c.executeTask("mainscene", gameState);
>
> });
>
> });
>
>
> window.requestAnimationFrame(function(){
>
> c.executeTask("mainscene", gameState);
>
> }
>
>
> Example code for gameLogic.js:
>
> var ctx = getContext("2d");
>
> onTask = function(DOMString id, dictionary json) {
>
> if(id == "mainscene") {
>
> if(typeof(MinionCanvas)=="Undefined") {
>
> MinionCanvas = createCanvas(200, 300);
>
> MinionCanvas.executeTask("drawMinion", {}) // creates promise under the
> hood
>
> }
>
> if(typeof(SpaceShipCanvas)=="Undefined")
>
> SpaceShipCanvas = createCanvas(300, 300);
>
>
> SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw
> spaceship
>
>
> executeTask("drawBackDrop", gameState); // in other task
>
> executeTask("drawBoss", gameState); // lots of js to draw the boss so
> better done in task
>
>
> for(...) //for each minion {
>
> ... // set the matrix
> ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
> minion might still be drawing in the other thread
>
> }
> for(...) //for each spaceship {
>
> ..// set the matrix
>
> ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still be
> drawing in the other task
>
> }
>
> .. // other drawing commands for score, controls, etc
>
> } else if(id == "drawMinion") {
>
> ...
>
> } else if(id == "drawSpaceShip") {
>
> ... // set up tasks to draw parts of the ship?
>
> } ...
>
> }
>
>
>
> On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier wrote:
>
>>
>>
>>
>> On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan
>> wrote:
>>
>>> On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote:
>>>
On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote:
> Creating temporary canvases is still possible. I'm unsure how it would
> be different from a worker.
> An advantage would be that you can draw to the temporary canvases in
> parallel to using them. Only PIXEL access is disallowed, you can still
> call
> drawImage using a canvas that has outstanding tasks.
>
Right. The write-only restriction would only apply to canvas contexts
that commit (push their tasks) directly down to the compositor. You could
still create a canvas that is local to the worker, rasterize it in the
worker and do readbacks in the worker, create ImageBitmaps from it, etc.
>>>
>>> I'm not sure that you and Rik are talking about the same thing, since
>>> he's still talking about "outstanding tasks". If you are talking about the
>>> same thing, I don't know what it is. I'd like to see some concrete details
>>> for what you'd change in the current WorkerCanvas proposal. For the sake of
>>> clarity I've put (my understand of) it here:
>>> https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal
>>>
>>
>> I'll work on drawing up an example of my proposal.
>>
>> With WorkerCanvas and transferToImageBitmap, you can draw multiple layers
>>> in parallel (and actually draw, not just queue drawing commands) by
>>> creating multiple workers, having them each produce an ImageBitmap, and
>>> compositing those ImageBitmaps together by stacking elements or
>>> drawing them all to a single canvas. It uses more memory but you get more
>>> parallelism.
>>>
>>
>> They would still have to wait for each other so the images are composited
>> in-order. If you don't care about that, the 'synchronized' option would let
>> you draw as soon as you exit the task (which is how Chrome always draws
>> since it's faster)
>>
>> In fact, an implementat
Re: [whatwg] Counterproposal for canvas in workers
Extra methods on the canvas API:
Promise setTaskScript(DOMString script); // can error be in promise?
Promise executeTask(DOMString id, dictionary json, boolean synchronized =
true); // Transferable elements allowed in dictionary
Object that is active in the task:
interface CanvasTask {
HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
attribute Function onTask;
}
CanvasTask implements HTMLCanvasElement;
Example code:
var c = document.getElementById("gameCanvas");
var gameState = {};
window.addEventListener("load", function(){
c.setTaskScript("gameLogic.js").then(function(){
c.executeTask("mainscene", gameState);
});
});
window.requestAnimationFrame(function(){
c.executeTask("mainscene", gameState);
}
Example code for gameLogic.js:
var ctx = getContext("2d");
onTask = function(DOMString id, dictionary json) {
if(id == "mainscene") {
if(typeof(MinionCanvas)=="Undefined") {
MinionCanvas = createCanvas(200, 300);
MinionCanvas.executeTask("drawMinion", {}) // creates promise under the hood
}
if(typeof(SpaceShipCanvas)=="Undefined")
SpaceShipCanvas = createCanvas(300, 300);
SpaceShipCanvas.executeTask("drawSpaceShip", gameState); // redraw spaceship
executeTask("drawBackDrop", gameState); // in other task
executeTask("drawBoss", gameState); // lots of js to draw the boss so
better done in task
for(...) //for each minion {
... // set the matrix
ctx.drawImage(MinionCanvas, ...); // draw the minion <- note that the
minion might still be drawing in the other thread
}
for(...) //for each spaceship {
..// set the matrix
ctx.drawImage(SpaceShipCanvas); // draw the spaceship <- it might still be
drawing in the other task
}
.. // other drawing commands for score, controls, etc
} else if(id == "drawMinion") {
...
} else if(id == "drawSpaceShip") {
... // set up tasks to draw parts of the ship?
} ...
}
On Thu, Oct 17, 2013 at 8:10 PM, Rik Cabanier wrote:
>
>
>
> On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan
> wrote:
>
>> On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote:
>>
>>> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote:
>>>
Creating temporary canvases is still possible. I'm unsure how it would
be different from a worker.
An advantage would be that you can draw to the temporary canvases in
parallel to using them. Only PIXEL access is disallowed, you can still call
drawImage using a canvas that has outstanding tasks.
>>>
>>> Right. The write-only restriction would only apply to canvas contexts
>>> that commit (push their tasks) directly down to the compositor. You could
>>> still create a canvas that is local to the worker, rasterize it in the
>>> worker and do readbacks in the worker, create ImageBitmaps from it, etc.
>>>
>>
>> I'm not sure that you and Rik are talking about the same thing, since
>> he's still talking about "outstanding tasks". If you are talking about the
>> same thing, I don't know what it is. I'd like to see some concrete details
>> for what you'd change in the current WorkerCanvas proposal. For the sake of
>> clarity I've put (my understand of) it here:
>> https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal
>>
>
> I'll work on drawing up an example of my proposal.
>
> With WorkerCanvas and transferToImageBitmap, you can draw multiple layers
>> in parallel (and actually draw, not just queue drawing commands) by
>> creating multiple workers, having them each produce an ImageBitmap, and
>> compositing those ImageBitmaps together by stacking elements or
>> drawing them all to a single canvas. It uses more memory but you get more
>> parallelism.
>>
>
> They would still have to wait for each other so the images are composited
> in-order. If you don't care about that, the 'synchronized' option would let
> you draw as soon as you exit the task (which is how Chrome always draws
> since it's faster)
>
> In fact, an implementation could choose to take the deferred-drawing
>> approach instead. You would queue up drawing commands in the WorkerCanvas
>> (or the drawing context), and then transferToImageBitmap would not
>> immediately render but produce an ImageBitmap implementation encapsulating
>> the list of drawing commands to be drawn later, wherever/whenever that
>> ImageBitmap ended up being used. I think for commit() the implementation
>> would always want to force rasterization on the worker (or possibly some
>> dedicated canvas-rendering thread); you could forward a list of drawing
>> commands to the compositor thread for rasterization but I don't think
>> there's any reason to do that (and some good reasons not to).
>>
>
> Can you tell me how you can ensure that you don't do too much work?
> Drawing in a continuous loop using 'Commit' would waste a lot of resources.
>
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 8:21 PM, Robert O'Callahan wrote: > On Fri, Oct 18, 2013 at 4:10 PM, Rik Cabanier wrote: > >> They would still have to wait for each other so the images are composited >> in-order. If you don't care about that, the 'synchronized' option would >> let >> you draw as soon as you exit the task (which is how Chrome always draws >> since it's faster) >> > > What do you mean "wait for each other"? You only have to wait until > they're all finished. The cost of actually compositing the images is low. > That is true. This would only work for regular source-over operations. If certain compositing operations are used, they will display incorrectly. > > In fact, an implementation could choose to take the deferred-drawing >> > approach instead. You would queue up drawing commands in the >> WorkerCanvas >> > (or the drawing context), and then transferToImageBitmap would not >> > immediately render but produce an ImageBitmap implementation >> encapsulating >> > the list of drawing commands to be drawn later, wherever/whenever that >> > ImageBitmap ended up being used. I think for commit() the implementation >> > would always want to force rasterization on the worker (or possibly some >> > dedicated canvas-rendering thread); you could forward a list of drawing >> > commands to the compositor thread for rasterization but I don't think >> > there's any reason to do that (and some good reasons not to). >> > >> >> Can you tell me how you can ensure that you don't do too much work? >> Drawing >> in a continuous loop using 'Commit' would waste a lot of resources. >> > > How to throttle drawing of frames using "commit()" is a completely > separate issue. Any API that allows direct publishing of frames from > workers to the compositor will have to deal with it, in roughly the same > way. > Wouldn't it be good to solve that at the same time?
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 4:32 PM, Glenn Maynard wrote:
> On Thu, Oct 17, 2013 at 5:14 PM, Rik Cabanier wrote:
>
>> Compositors are often already threaded, so synchronizing a buffer flip
>>> with the compositor doesn't seem too far out there.)
>>>
>>
>> This proposal implies an extra buffer for the 2d context. My proposal
>> doesn't require that so it's more memory efficient + you can draw in
>> parallel.
>>
>
> You always need at least two buffers: a back-buffer for drawing and a
> front-buffer for display (compositing). Otherwise, as soon as you start
> drawing the next frame, the old frame is gone, so you won't be able to
> recomposite (on reflow, CSS filter changes, etc). Double-buffering at a
> minimum is pretty standard, even for native applications (with none of this
> Web complexity in the way).
>
Won't you need another front-buffer for the worker to draw to?
>
> I think WorkerCanvas (as well as CanvasProxy that's in the spec
> today--this isn't new to WorkerCanvas) allows full parallelism in drawing,
> both between the script and the GPU and between the worker and the main UI
> thread.
>
>
>> I don't remember "multiple workers accessing the same canvas" and I'm
>>> not quite sure what it means. I do remember "a single (WebGL) context
>>> rendering to multiple canvases". Is that what you're thinking of?
>>>
>>
>> I went back over the history and that was indeed his use case.
>>
>
> That's a good use case, I've wanted to do that myself. We haven't tried
> very hard to fit it into the WorkerCanvas approach yet, and it may also be
> that the best way to do that is orthogonal to the whole "canvas in workers"
> use case.
>
> The obvious approach is to add a new method on the context,
> "attachToCanvas(Canvas or WorkerCanvas)", which would just take the context
> and cause its output to be directed to a new Canvas (or WorkerCanvas),
> probably clearing the contents of the new canvas as a side-effect. (This
> could be added to both CanvasRenderingContext2D and WebGLRenderingContext,
> though I suspect this is only really useful for WebGL. There's no
> expensive resource loading with 2d canvas.)
>
> var canvas = document.querySelector(".canvas1");
> var gl = canvas.getContext("webgl");
> loadExpensiveResources(gl);
> drawStuff(gl);
> var canvas2 = document.querySelector(".canvas2");
> gl.attachToCanvas(canvas2);
> drawStuff(gl); // don't need to loadExpensiveResources again
>
> I think that's by far the most straightforward approach for users. Maybe
> there are implementation issues that make this hard, but if so I think they
> would apply to every approach to this use case (they're really all
> different interfaces to the same functionality)...
>
> --
> Glenn Maynard
>
>
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 3:10 PM, Glenn Maynard wrote: > "transferToImageBuffer" looks like it would create a new ImageBuffer for > each frame, so you'd need to add a close() method to make sure they don't > accumulate due to GC lag, > That's a good point. We will need something like that. It would only neuter that thread's (main thread or worker thread) version of the ImageBitmap. and it seems like turning this into a fast buffer swap under the hood would > be harder. > I don't see why. > Also, with the "transferToImageBuffer" approach, if you want to render > from a worker into multiple canvases in the UI thread, you have to post > those ImageBuffers over to the main thread each frame, which has the same > (potential) synchronization issues as the transferDrawingBufferToCanvas > proposal. > What are those issues? You can do a single postMessage passing a complete set of ImageBItmaps. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 4:10 PM, Rik Cabanier wrote: > They would still have to wait for each other so the images are composited > in-order. If you don't care about that, the 'synchronized' option would let > you draw as soon as you exit the task (which is how Chrome always draws > since it's faster) > What do you mean "wait for each other"? You only have to wait until they're all finished. The cost of actually compositing the images is low. In fact, an implementation could choose to take the deferred-drawing > > approach instead. You would queue up drawing commands in the WorkerCanvas > > (or the drawing context), and then transferToImageBitmap would not > > immediately render but produce an ImageBitmap implementation > encapsulating > > the list of drawing commands to be drawn later, wherever/whenever that > > ImageBitmap ended up being used. I think for commit() the implementation > > would always want to force rasterization on the worker (or possibly some > > dedicated canvas-rendering thread); you could forward a list of drawing > > commands to the compositor thread for rasterization but I don't think > > there's any reason to do that (and some good reasons not to). > > > > Can you tell me how you can ensure that you don't do too much work? Drawing > in a continuous loop using 'Commit' would waste a lot of resources. > How to throttle drawing of frames using "commit()" is a completely separate issue. Any API that allows direct publishing of frames from workers to the compositor will have to deal with it, in roughly the same way. Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 4:01 PM, Robert O'Callahan wrote: > On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote: > >> On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote: >> >>> Creating temporary canvases is still possible. I'm unsure how it would >>> be different from a worker. >>> An advantage would be that you can draw to the temporary canvases in >>> parallel to using them. Only PIXEL access is disallowed, you can still call >>> drawImage using a canvas that has outstanding tasks. >>> >> >> Right. The write-only restriction would only apply to canvas contexts >> that commit (push their tasks) directly down to the compositor. You could >> still create a canvas that is local to the worker, rasterize it in the >> worker and do readbacks in the worker, create ImageBitmaps from it, etc. >> > > I'm not sure that you and Rik are talking about the same thing, since he's > still talking about "outstanding tasks". If you are talking about the same > thing, I don't know what it is. I'd like to see some concrete details for > what you'd change in the current WorkerCanvas proposal. For the sake of > clarity I've put (my understand of) it here: > https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal > I'll work on drawing up an example of my proposal. With WorkerCanvas and transferToImageBitmap, you can draw multiple layers > in parallel (and actually draw, not just queue drawing commands) by > creating multiple workers, having them each produce an ImageBitmap, and > compositing those ImageBitmaps together by stacking elements or > drawing them all to a single canvas. It uses more memory but you get more > parallelism. > They would still have to wait for each other so the images are composited in-order. If you don't care about that, the 'synchronized' option would let you draw as soon as you exit the task (which is how Chrome always draws since it's faster) In fact, an implementation could choose to take the deferred-drawing > approach instead. You would queue up drawing commands in the WorkerCanvas > (or the drawing context), and then transferToImageBitmap would not > immediately render but produce an ImageBitmap implementation encapsulating > the list of drawing commands to be drawn later, wherever/whenever that > ImageBitmap ended up being used. I think for commit() the implementation > would always want to force rasterization on the worker (or possibly some > dedicated canvas-rendering thread); you could forward a list of drawing > commands to the compositor thread for rasterization but I don't think > there's any reason to do that (and some good reasons not to). > Can you tell me how you can ensure that you don't do too much work? Drawing in a continuous loop using 'Commit' would waste a lot of resources.
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 8:22 PM, Robert O'Callahan wrote: > That's not really a use-case. What would you actually be trying to do? > IIUC Ken agreed that his use-cases that appeared to require a single > context rendering to multiple canvases would be addressed just as easily > (or better) by using multiple image elements, a single canvas, and doing > "image.srcObject = canvas.transferToImageBuffer()". > I wasn't arguing a use case, I was agreeing with a feature. I think the use cases for rendering to multiple DOM elements (canvases or otherwise) using WebGL are already well-established (less so for 2d canvas). "transferToImageBuffer" looks like it would create a new ImageBuffer for each frame, so you'd need to add a close() method to make sure they don't accumulate due to GC lag, and it seems like turning this into a fast buffer swap under the hood would be harder. If you just point the context at the final canvas in the first place, it can render directly into that canvas's backbuffer, so the buffer flipping mechanics are identical to when it isn't being used at all. Also, with the "transferToImageBuffer" approach, if you want to render from a worker into multiple canvases in the UI thread, you have to post those ImageBuffers over to the main thread each frame, which has the same (potential) synchronization issues as the transferDrawingBufferToCanvas proposal. With attachToCanvas, it's just like WorkerCanvas: the buffer flipping can happen entirely within the worker. -- Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 12:32 PM, Glenn Maynard wrote: > On Thu, Oct 17, 2013 at 5:14 PM, Rik Cabanier wrote: > > > I don't remember "multiple workers accessing the same canvas" and I'm not > >> quite sure what it means. I do remember "a single (WebGL) context > >> rendering to multiple canvases". Is that what you're thinking of? > >> > > > > I went back over the history and that was indeed his use case. > > > > That's a good use case, I've wanted to do that myself. We haven't tried > very hard to fit it into the WorkerCanvas approach yet, and it may also be > that the best way to do that is orthogonal to the whole "canvas in workers" > use case. > That's not really a use-case. What would you actually be trying to do? IIUC Ken agreed that his use-cases that appeared to require a single context rendering to multiple canvases would be addressed just as easily (or better) by using multiple image elements, a single canvas, and doing "image.srcObject = canvas.transferToImageBuffer()". Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 5:14 PM, Rik Cabanier wrote:
> Compositors are often already threaded, so synchronizing a buffer flip
>> with the compositor doesn't seem too far out there.)
>>
>
> This proposal implies an extra buffer for the 2d context. My proposal
> doesn't require that so it's more memory efficient + you can draw in
> parallel.
>
You always need at least two buffers: a back-buffer for drawing and a
front-buffer for display (compositing). Otherwise, as soon as you start
drawing the next frame, the old frame is gone, so you won't be able to
recomposite (on reflow, CSS filter changes, etc). Double-buffering at a
minimum is pretty standard, even for native applications (with none of this
Web complexity in the way).
I think WorkerCanvas (as well as CanvasProxy that's in the spec today--this
isn't new to WorkerCanvas) allows full parallelism in drawing, both between
the script and the GPU and between the worker and the main UI thread.
> I don't remember "multiple workers accessing the same canvas" and I'm not
>> quite sure what it means. I do remember "a single (WebGL) context
>> rendering to multiple canvases". Is that what you're thinking of?
>>
>
> I went back over the history and that was indeed his use case.
>
That's a good use case, I've wanted to do that myself. We haven't tried
very hard to fit it into the WorkerCanvas approach yet, and it may also be
that the best way to do that is orthogonal to the whole "canvas in workers"
use case.
The obvious approach is to add a new method on the context,
"attachToCanvas(Canvas or WorkerCanvas)", which would just take the context
and cause its output to be directed to a new Canvas (or WorkerCanvas),
probably clearing the contents of the new canvas as a side-effect. (This
could be added to both CanvasRenderingContext2D and WebGLRenderingContext,
though I suspect this is only really useful for WebGL. There's no
expensive resource loading with 2d canvas.)
var canvas = document.querySelector(".canvas1");
var gl = canvas.getContext("webgl");
loadExpensiveResources(gl);
drawStuff(gl);
var canvas2 = document.querySelector(".canvas2");
gl.attachToCanvas(canvas2);
drawStuff(gl); // don't need to loadExpensiveResources again
I think that's by far the most straightforward approach for users. Maybe
there are implementation issues that make this hard, but if so I think they
would apply to every approach to this use case (they're really all
different interfaces to the same functionality)...
--
Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad wrote: > On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote: > >> Creating temporary canvases is still possible. I'm unsure how it would be >> different from a worker. >> An advantage would be that you can draw to the temporary canvases in >> parallel to using them. Only PIXEL access is disallowed, you can still call >> drawImage using a canvas that has outstanding tasks. >> > > Right. The write-only restriction would only apply to canvas contexts that > commit (push their tasks) directly down to the compositor. You could still > create a canvas that is local to the worker, rasterize it in the worker and > do readbacks in the worker, create ImageBitmaps from it, etc. > I'm not sure that you and Rik are talking about the same thing, since he's still talking about "outstanding tasks". If you are talking about the same thing, I don't know what it is. I'd like to see some concrete details for what you'd change in the current WorkerCanvas proposal. For the sake of clarity I've put (my understand of) it here: https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal With WorkerCanvas and transferToImageBitmap, you can draw multiple layers in parallel (and actually draw, not just queue drawing commands) by creating multiple workers, having them each produce an ImageBitmap, and compositing those ImageBitmaps together by stacking elements or drawing them all to a single canvas. It uses more memory but you get more parallelism. In fact, an implementation could choose to take the deferred-drawing approach instead. You would queue up drawing commands in the WorkerCanvas (or the drawing context), and then transferToImageBitmap would not immediately render but produce an ImageBitmap implementation encapsulating the list of drawing commands to be drawn later, wherever/whenever that ImageBitmap ended up being used. I think for commit() the implementation would always want to force rasterization on the worker (or possibly some dedicated canvas-rendering thread); you could forward a list of drawing commands to the compositor thread for rasterization but I don't think there's any reason to do that (and some good reasons not to). Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 3:01 PM, Glenn Maynard wrote: > On Thu, Oct 17, 2013 at 4:50 PM, Rik Cabanier wrote: > >> It seemed like that proposal was harder. Synchronization with the main >> > drawing thread seemed and the continuous committing seemed difficult too. >> > > Have implementors said that synchronizing the flip is (unreasonably) hard > to implement? (I'm not an implementor, but this proposal feels > unimplementable to me, or at least catastrophically difficult for WebGL. > That could be. I'm not all that familiar with WebGL. > Compositors are often already threaded, so synchronizing a buffer flip > with the compositor doesn't seem too far out there.) > This proposal implies an extra buffer for the 2d context. My proposal doesn't require that so it's more memory efficient + you can draw in parallel. > > >> In addition, Ken wanted multiple workers access the same canvas which I >> didn't see addressed (unless I missed it). >> > > I don't remember "multiple workers accessing the same canvas" and I'm not > quite sure what it means. I do remember "a single (WebGL) context > rendering to multiple canvases". Is that what you're thinking of? > I went back over the history and that was indeed his use case. > > On Thu, Oct 17, 2013 at 4:51 PM, Rik Cabanier wrote: > >> Thanks Glenn! >> With that info, will there ever be a way to use WebGL in different >> workers but going to the same webgl context? >> > > Sorry, which use case is this for? I'm not sure why you'd want to do > that, and it sounds like it would expose thread-safety issues to the > platform. (I'm not sure if you mean the same thing here and above--they > sound similar, but you said "canvas" in one place and "WebGL context" in > the other.) > > (Sorry if I'm forgetting things, the subject has been busy and a little > bit noisy...) > Yes. Sorry to add to the noise :-)
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 4:50 PM, Rik Cabanier wrote: > It seemed like that proposal was harder. Synchronization with the main > drawing thread seemed and the continuous committing seemed difficult too. > Have implementors said that synchronizing the flip is (unreasonably) hard to implement? (I'm not an implementor, but this proposal feels unimplementable to me, or at least catastrophically difficult for WebGL. Compositors are often already threaded, so synchronizing a buffer flip with the compositor doesn't seem too far out there.) > In addition, Ken wanted multiple workers access the same canvas which I > didn't see addressed (unless I missed it). > I don't remember "multiple workers accessing the same canvas" and I'm not quite sure what it means. I do remember "a single (WebGL) context rendering to multiple canvases". Is that what you're thinking of? On Thu, Oct 17, 2013 at 4:51 PM, Rik Cabanier wrote: > Thanks Glenn! > With that info, will there ever be a way to use WebGL in different workers > but going to the same webgl context? > Sorry, which use case is this for? I'm not sure why you'd want to do that, and it sounds like it would expose thread-safety issues to the platform. (I'm not sure if you mean the same thing here and above--they sound similar, but you said "canvas" in one place and "WebGL context" in the other.) (Sorry if I'm forgetting things, the subject has been busy and a little bit noisy...) -- Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier wrote: > > > > On Thu, Oct 17, 2013 at 2:28 PM, Robert O'Callahan > wrote: > >> On Fri, Oct 18, 2013 at 6:57 AM, Justin Novosad wrote: >> >>> Here is similar concept, but with an API more like WokerCanvas: >>> The CanvasRenderingContext2D associated with a WorkerCanvas would only >>> record draw commands, without executing them. The context would be >>> write-only. When you call commit on the WorkerCanvas, the block of >>> recorded >>> draw commands would be posted back to the main thread or directly to the >>> compositor. >> >> >> Which? They are observably different. >> >> >>> What I like about this approach is that it is always just >>> pushing data downstream, thus eliminating buffer synchronization issues >>> as >>> well as the need for double buffering canvas backing stores. >>> >> >> The write-only restriction is a problem. Also, it's really important that >> the worker be able to create temporary canvases for its own use (pdf.js for >> example needs this), and this doesn't really support that. >> > > Creating temporary canvases is still possible. I'm unsure how it would be > different from a worker. > An advantage would be that you can draw to the temporary canvases in > parallel to using them. Only PIXEL access is disallowed, you can still call > drawImage using a canvas that has outstanding tasks. > Right. The write-only restriction would only apply to canvas contexts that commit (push their tasks) directly down to the compositor. You could still create a canvas that is local to the worker, rasterize it in the worker and do readbacks in the worker, create ImageBitmaps from it, etc. > > >> >> I think we have already converged on a WorkerCanvas design that everyone >> (on this thread so far) is happy with, using ImageBitmaps to synchronize >> with the main thread as needed. Is there some problem with that proposal >> that warrants introducing the complexity of Rik's 'task' system or the >> limitations of your proposal? >> > > It seemed like that proposal was harder. Synchronization with the main > drawing thread seemed and the continuous committing seemed difficult too. > In addition, Ken wanted multiple workers access the same canvas which I > didn't see addressed (unless I missed it). > > If the other proposal is better, we can drop this one. >
Re: [whatwg] Counterproposal for canvas in workers
Thanks Glenn! With that info, will there ever be a way to use WebGL in different workers but going to the same webgl context? On Thu, Oct 17, 2013 at 2:48 PM, Glenn Maynard wrote: > On Wed, Oct 16, 2013 at 9:34 PM, Rik Cabanier wrote: > >> When drawing to canvas, Chrome stores the drawing commands in a buffer and >> > executes them when the main function returns (or access to pixel data is >> requested). >> It occurred to me that this could be re-purposed for canvas workers. A >> worker could create a list of drawing commands and if the worker is done, >> this list is executed either on the main thread or the worker or a >> compositor thread depending on what your architecture supports. >> The worker would not be allowed to read pixels or resize the canvas but >> all >> other operations would be allowed. >> > > This sounds like it serializes setting up the queue, and actually drawing > the queue. OpenGL doesn't do that: it starts sending drawing commands to > the GPU as soon as you make them, so the CPU can be setting up rendering of > the same scene while the GPU is rendering earlier commands. It only needs > to buffer if you send commands faster than the GPU can process them (the > specific details of this are internal driver magic, but that's the gist). > Waiting until all rendering commands have been called before starting to > render would be catastrophic for performance, since it would prevent > parallelism between the CPU and GPU. > > On Thu, Oct 17, 2013 at 3:35 PM, Rik Cabanier wrote: > > I'm unsure how this would work for WebGL since I'm not all that familiar >> > with its architecture. However, it seems that the end result of a webgl >> application, is a series of commands that are sent to the graphics chip. >> In >> theory, this should be compatible. >> > > All of that happens inside the OpenGL driver, which browsers have no > control over. > > -- > Glenn Maynard > >
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 2:28 PM, Robert O'Callahan wrote: > On Fri, Oct 18, 2013 at 6:57 AM, Justin Novosad wrote: > >> Here is similar concept, but with an API more like WokerCanvas: >> The CanvasRenderingContext2D associated with a WorkerCanvas would only >> record draw commands, without executing them. The context would be >> write-only. When you call commit on the WorkerCanvas, the block of >> recorded >> draw commands would be posted back to the main thread or directly to the >> compositor. > > > Which? They are observably different. > > >> What I like about this approach is that it is always just >> pushing data downstream, thus eliminating buffer synchronization issues as >> well as the need for double buffering canvas backing stores. >> > > The write-only restriction is a problem. Also, it's really important that > the worker be able to create temporary canvases for its own use (pdf.js for > example needs this), and this doesn't really support that. > Creating temporary canvases is still possible. I'm unsure how it would be different from a worker. An advantage would be that you can draw to the temporary canvases in parallel to using them. Only PIXEL access is disallowed, you can still call drawImage using a canvas that has outstanding tasks. > > I think we have already converged on a WorkerCanvas design that everyone > (on this thread so far) is happy with, using ImageBitmaps to synchronize > with the main thread as needed. Is there some problem with that proposal > that warrants introducing the complexity of Rik's 'task' system or the > limitations of your proposal? > It seemed like that proposal was harder. Synchronization with the main drawing thread seemed and the continuous committing seemed difficult too. In addition, Ken wanted multiple workers access the same canvas which I didn't see addressed (unless I missed it). If the other proposal is better, we can drop this one.
Re: [whatwg] Counterproposal for canvas in workers
On Wed, Oct 16, 2013 at 9:34 PM, Rik Cabanier wrote: > When drawing to canvas, Chrome stores the drawing commands in a buffer and > executes them when the main function returns (or access to pixel data is > requested). > It occurred to me that this could be re-purposed for canvas workers. A > worker could create a list of drawing commands and if the worker is done, > this list is executed either on the main thread or the worker or a > compositor thread depending on what your architecture supports. > The worker would not be allowed to read pixels or resize the canvas but all > other operations would be allowed. > This sounds like it serializes setting up the queue, and actually drawing the queue. OpenGL doesn't do that: it starts sending drawing commands to the GPU as soon as you make them, so the CPU can be setting up rendering of the same scene while the GPU is rendering earlier commands. It only needs to buffer if you send commands faster than the GPU can process them (the specific details of this are internal driver magic, but that's the gist). Waiting until all rendering commands have been called before starting to render would be catastrophic for performance, since it would prevent parallelism between the CPU and GPU. On Thu, Oct 17, 2013 at 3:35 PM, Rik Cabanier wrote: > I'm unsure how this would work for WebGL since I'm not all that familiar > with its architecture. However, it seems that the end result of a webgl > application, is a series of commands that are sent to the graphics chip. In > theory, this should be compatible. > All of that happens inside the OpenGL driver, which browsers have no control over. -- Glenn Maynard
Re: [whatwg] Counterproposal for canvas in workers
On Fri, Oct 18, 2013 at 6:57 AM, Justin Novosad wrote: > Here is similar concept, but with an API more like WokerCanvas: > The CanvasRenderingContext2D associated with a WorkerCanvas would only > record draw commands, without executing them. The context would be > write-only. When you call commit on the WorkerCanvas, the block of recorded > draw commands would be posted back to the main thread or directly to the > compositor. Which? They are observably different. > What I like about this approach is that it is always just > pushing data downstream, thus eliminating buffer synchronization issues as > well as the need for double buffering canvas backing stores. > The write-only restriction is a problem. Also, it's really important that the worker be able to create temporary canvases for its own use (pdf.js for example needs this), and this doesn't really support that. I think we have already converged on a WorkerCanvas design that everyone (on this thread so far) is happy with, using ImageBitmaps to synchronize with the main thread as needed. Is there some problem with that proposal that warrants introducing the complexity of Rik's 'task' system or the limitations of your proposal? Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 2:08 PM, Rik Cabanier wrote: > > > > On Thu, Oct 17, 2013 at 2:03 PM, Kenneth Russell wrote: >> >> On Wed, Oct 16, 2013 at 10:26 PM, Robert O'Callahan >> wrote: >> > On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier >> > wrote: >> > >> >> The tasks themselves can also launch synchronized/unsynchronized >> >> subtasks >> >> with promises. A task is considered "done" if it exits and all its >> >> promises >> >> are fulfilled. >> >> >> > >> > It seems that tasks are like workers, but different, and you'd have to >> > do a >> > lot of extra work to precisely define the execution environment of the >> > task >> > script. >> > >> > It also seems that you have to precisely define how different tasks >> > interact. For example is the current path left in the canvas by task 1 >> > usable by the code in task 2? You also have to define how this works in >> > WebGL. >> > >> > I don't think this supports a worker/task generating a steady stream of >> > frames, e.g. for a 3D game. Does it? >> > >> > I'm not all that enthusiastic :-) >> >> Sorry, neither am I. OpenGL (and WebGL) applications do a lot of >> one-time setup, and then repeatedly redraw using the previously >> uploaded objects. This "stateless" drawing model isn't compatible with >> that structure. > > > Every task per ID and per canvas has access to its own state/VM. > So, the first time a task is executed (or we could provide an 'init' phase), > it could do setup which will be maintained between tasks. OK, I see. Sorry for misinterpreting. It seems to me that this proposal would restrict even further what the worker executing the task can do, and be harder to program to than the existing worker model. I'm interested in pursuing the other discussion around a WorkerCanvas rather than this one. -Ken
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 2:03 PM, Kenneth Russell wrote: > On Wed, Oct 16, 2013 at 10:26 PM, Robert O'Callahan > wrote: > > On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier > wrote: > > > >> The tasks themselves can also launch synchronized/unsynchronized > subtasks > >> with promises. A task is considered "done" if it exits and all its > promises > >> are fulfilled. > >> > > > > It seems that tasks are like workers, but different, and you'd have to > do a > > lot of extra work to precisely define the execution environment of the > task > > script. > > > > It also seems that you have to precisely define how different tasks > > interact. For example is the current path left in the canvas by task 1 > > usable by the code in task 2? You also have to define how this works in > > WebGL. > > > > I don't think this supports a worker/task generating a steady stream of > > frames, e.g. for a 3D game. Does it? > > > > I'm not all that enthusiastic :-) > > Sorry, neither am I. OpenGL (and WebGL) applications do a lot of > one-time setup, and then repeatedly redraw using the previously > uploaded objects. This "stateless" drawing model isn't compatible with > that structure. > Every task per ID and per canvas has access to its own state/VM. So, the first time a task is executed (or we could provide an 'init' phase), it could do setup which will be maintained between tasks.
Re: [whatwg] Counterproposal for canvas in workers
On Wed, Oct 16, 2013 at 10:26 PM, Robert O'Callahan wrote: > On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier wrote: > >> The tasks themselves can also launch synchronized/unsynchronized subtasks >> with promises. A task is considered "done" if it exits and all its promises >> are fulfilled. >> > > It seems that tasks are like workers, but different, and you'd have to do a > lot of extra work to precisely define the execution environment of the task > script. > > It also seems that you have to precisely define how different tasks > interact. For example is the current path left in the canvas by task 1 > usable by the code in task 2? You also have to define how this works in > WebGL. > > I don't think this supports a worker/task generating a steady stream of > frames, e.g. for a 3D game. Does it? > > I'm not all that enthusiastic :-) Sorry, neither am I. OpenGL (and WebGL) applications do a lot of one-time setup, and then repeatedly redraw using the previously uploaded objects. This "stateless" drawing model isn't compatible with that structure. -Ken
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 10:57 AM, Justin Novosad wrote: > Here is similar concept, but with an API more like WokerCanvas: > The CanvasRenderingContext2D associated with a WorkerCanvas would only > record draw commands, without executing them. The context would be > write-only. When you call commit on the WorkerCanvas, the block of recorded > draw commands would be posted back to the main thread or directly to the > compositor. What I like about this approach is that it is always just > pushing data downstream, thus eliminating buffer synchronization issues as > well as the need for double buffering canvas backing stores. > That is basically what I'm proposing, except that the "commit" is a return for the task. Continually drawing in a worker seems like it would suck up a lot of power and cpu resources... > > > On Thu, Oct 17, 2013 at 1:26 AM, Robert O'Callahan > wrote: > >> On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier wrote: >> >> > The tasks themselves can also launch synchronized/unsynchronized >> subtasks >> > with promises. A task is considered "done" if it exits and all its >> promises >> > are fulfilled. >> > >> >> It seems that tasks are like workers, but different, and you'd have to do >> a >> lot of extra work to precisely define the execution environment of the >> task >> script. >> >> It also seems that you have to precisely define how different tasks >> interact. For example is the current path left in the canvas by task 1 >> usable by the code in task 2? You also have to define how this works in >> WebGL. >> >> I don't think this supports a worker/task generating a steady stream of >> frames, e.g. for a 3D game. Does it? >> >> I'm not all that enthusiastic :-) >> >> Rob >> -- >> Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni >> le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids >> teoa >> stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg >> iyvoeunr, >> 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp >> waanndt wyeonut thoo mken.o w * >> * >> > >
Re: [whatwg] Counterproposal for canvas in workers
On Wed, Oct 16, 2013 at 10:26 PM, Robert O'Callahan wrote: > On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier wrote: > >> The tasks themselves can also launch synchronized/unsynchronized subtasks >> with promises. A task is considered "done" if it exits and all its >> promises >> are fulfilled. >> > > It seems that tasks are like workers, but different, and you'd have to do > a lot of extra work to precisely define the execution environment of the > task script. > Every task would start with a 'clean' graphics state. We could define ways to pass other canvas contexts into a task. Those other contexts could even be drawn in other tasks (so you could use them while they are being drawn) > > It also seems that you have to precisely define how different tasks > interact. For example is the current path left in the canvas by task 1 > usable by the code in task 2? You also have to define how this works in > WebGL. > No. Every task acts like a unique canvas context; there's no sharing of state. I'm unsure how this would work for WebGL since I'm not all that familiar with its architecture. However, it seems that the end result of a webgl application, is a series of commands that are sent to the graphics chip. In theory, this should be compatible. > > I don't think this supports a worker/task generating a steady stream of > frames, e.g. for a 3D game. Does it? > It should. When the promise of a task if fulfilled, the main thread could schedule another task. I think the scheduling should be done inside a requestAnimationFrame for best performance. Since there can only be 1 task with the same id active at a time, an author could just continuously schedule tasks in requestAnimationFrame. In that case if the tasks need to be synchronized, I think the author would create 1 main task that generates subtasks to draw different parts of a scene. Otherwise, there wouldn't be a "flush" to the screen since tasks would be scheduled all the time. > > I'm not all that enthusiastic :-) > That's OK!
Re: [whatwg] Counterproposal for canvas in workers
Here is similar concept, but with an API more like WokerCanvas: The CanvasRenderingContext2D associated with a WorkerCanvas would only record draw commands, without executing them. The context would be write-only. When you call commit on the WorkerCanvas, the block of recorded draw commands would be posted back to the main thread or directly to the compositor. What I like about this approach is that it is always just pushing data downstream, thus eliminating buffer synchronization issues as well as the need for double buffering canvas backing stores. On Thu, Oct 17, 2013 at 1:26 AM, Robert O'Callahan wrote: > On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier wrote: > > > The tasks themselves can also launch synchronized/unsynchronized subtasks > > with promises. A task is considered "done" if it exits and all its > promises > > are fulfilled. > > > > It seems that tasks are like workers, but different, and you'd have to do a > lot of extra work to precisely define the execution environment of the task > script. > > It also seems that you have to precisely define how different tasks > interact. For example is the current path left in the canvas by task 1 > usable by the code in task 2? You also have to define how this works in > WebGL. > > I don't think this supports a worker/task generating a steady stream of > frames, e.g. for a 3D game. Does it? > > I'm not all that enthusiastic :-) > > Rob > -- > Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni > le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa > stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, > 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp > waanndt wyeonut thoo mken.o w * > * >
Re: [whatwg] Counterproposal for canvas in workers
On Thu, Oct 17, 2013 at 3:34 PM, Rik Cabanier wrote: > The tasks themselves can also launch synchronized/unsynchronized subtasks > with promises. A task is considered "done" if it exits and all its promises > are fulfilled. > It seems that tasks are like workers, but different, and you'd have to do a lot of extra work to precisely define the execution environment of the task script. It also seems that you have to precisely define how different tasks interact. For example is the current path left in the canvas by task 1 usable by the code in task 2? You also have to define how this works in WebGL. I don't think this supports a worker/task generating a steady stream of frames, e.g. for a 3D game. Does it? I'm not all that enthusiastic :-) Rob -- Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * *
[whatwg] Counterproposal for canvas in workers
All,
I have another proposal on how we can do canvas in workers. If it's not
practical or too complex, feel free to dismiss it. :-)
When drawing to canvas, Chrome stores the drawing commands in a buffer and
executes them when the main function returns (or access to pixel data is
requested).
It occurred to me that this could be re-purposed for canvas workers. A
worker could create a list of drawing commands and if the worker is done,
this list is executed either on the main thread or the worker or a
compositor thread depending on what your architecture supports.
The worker would not be allowed to read pixels or resize the canvas but all
other operations would be allowed.
The following new API's would be on the 2d context:
void setTaskScript(DOMString); // takes a url that contains the script for
the tasks
Promise executeTask(DOMString id, dictionary json, boolean synchronized =
true);
An author can execute as many tasks as he wants, but there can be only 1
task with the same id active at a time. (Calling executeTask 2 times in a
row with the same id will drop the second task.)
If synchronized is true, the UA will ensure that the drawing commands of
the task are executed in order. For instance:
var p1 = ctx.executeTask("drawBackground");
var p2 = ctx.executeTask("drawScene");
var p3 = ctx.executeTask("drawControls");
will draw a game interface using 3 different tasks
It is up to the UA to determine how many tasks it can run in parallel. It
is also up to the UA on how/where the actual drawing command are resolved,
but the commands of every task (which include the main thread) have to be
executed atomically so there's no interleaving.
Every task has access to a canvas context-like object and the dictionary
that is passed in. It can not read pixel data or resize the canvas. Every
task per canvas and per ID also has access to its own VM.
The main thread can use the canvas as usual. Accessing pixel data or
resizing will block until all tasks are completed.
If 'synchronized' is set to true:
- the canvas bitmap won't be updated until all its tasks are done
- the tasks will execute in order
If 'synchronized' is set to false:
- the canvas bitmap is updated as soon as a task is ready
- tasks can draw out of order
The tasks themselves can also launch synchronized/unsynchronized subtasks
with promises. A task is considered "done" if it exits and all its promises
are fulfilled.
Thoughts?
