Re: Synchronous workers
How common are the blocking API calls? Can we somehow predict whether the SWF will need that, and optionally use a worker/non-worker context based on that? We can do whatever we like in Gecko, but I'd imagine that any proposal to allow blocking on a worker will meet significant resistance in the WG (which is, IMO, justified). bholley On Mon, Oct 14, 2013 at 5:03 PM, Benjamin Smedberg benja...@smedbergs.us wrote: One of the shumway goals has been to run most or all of shumway in a worker, so that it can run in parallel with a web page the same way Flash currently does. This will significantly help solve jank associated with shumway loading. However, we still need to support the Flash externalInterface scripting layer, which is synchronous with respect to content. This exposes race conditions between the SWF runtime and content; we have already solved this for out-of-process plugins by introducing an explicit race resolution policy (the plugin wins races). Having this blocking interface will also support blocking shumway on graphics rendering; there are a fair number of SWF files that use an API to get a synchronous bitmap of their stage, which requires a blocking call from shumway to the main thread even after we get canvas-on-a-worker, which explicitly will not support that blocking case. I would like to know whether it is possible to support blocking between content and a worker to support this scripting use case. I'm fine with supporting this only for shumway as a special case, but I'm interested in discussing whether we can expose this to workers in general. We don't want to make blocking on workers a way to bypass asynchronous I/O checks: for instance normally workers can do synchronous indexeddb calls while the main thread can't. But perhaps we can expose a flag when you create a worker which allows it to *either* have blocking calls *or* blocking I/O, but not both? In general we expect shumway to be mainly non-blocking, so we'd usually get parallelism improvements from having it on a worker, while retaining support for the special cases where blocking is necessary for compatibility. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On 10/14/2013 12:06 PM, Bobby Holley wrote: How common are the blocking API calls? Expect them to be very common or almost universal for real Flash. Much less common for simple display ads. Can we somehow predict whether the SWF will need that, and optionally use a worker/non-worker context based on that? Not effectively, no. SWF content can and very often does load subcontent similar to iframes, so statically checking whether it imports ExternalInterface won't give you a useful answer. We can do whatever we like in Gecko, but I'd imagine that any proposal to allow blocking on a worker will meet significant resistance in the WG (which is, IMO, justified). I'm primarily interested in solving the shumway use-case. However, once we have implementation experience, I'm not convinced that blocking workers is inherently a bad thing, and we should consider proposing it at that point. Sites want to ship shumway to their own users (Flash on ipads) and it would IMO be silly to require them to run on the main thread just to maintain the purity of an entirely async worker design. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
- Original Message - How common are the blocking API calls? Can we somehow predict whether the SWF will need that, and optionally use a worker/non-worker context based on that? Fairly common: many SWFs use ExternalInterface at least once, and even more use BitmapData#draw, which allows synchronously rendering arbitrary parts of the display list into a buffer. And no, we can't predict if either functionality is used. At least not with 100% accuracy, which we'd need to make decisions about which kind of context to create based on it. Additionally, we'd need to do a full-SWF analysis for any analysis whatsoever. Which we can't in the general case, because SWF is a streaming format that can display already-loaded content while other parts are still loading. We can do whatever we like in Gecko, but I'd imagine that any proposal to allow blocking on a worker will meet significant resistance in the WG (which is, IMO, justified). Nobody's proposing to expose any of this to web content. The question is whether we want to fully expose it as an official capability of Gecko, to be used by other parts of the browser and maybe even addons, or if we want to go to some lengths to hide it as best as we can. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
Didn't see this before now, sorry. On Mon, Oct 14, 2013 at 6:15 PM, Benjamin Smedberg benja...@smedbergs.us wrote: On 10/14/2013 12:06 PM, Bobby Holley wrote: How common are the blocking API calls? Expect them to be very common or almost universal for real Flash. Much less common for simple display ads. Can we somehow predict whether the SWF will need that, and optionally use a worker/non-worker context based on that? Not effectively, no. SWF content can and very often does load subcontent similar to iframes, so statically checking whether it imports ExternalInterface won't give you a useful answer. See my previousy message: there are even more reasons why this won't work. We can do whatever we like in Gecko, but I'd imagine that any proposal to allow blocking on a worker will meet significant resistance in the WG (which is, IMO, justified). I'm primarily interested in solving the shumway use-case. However, once we have implementation experience, I'm not convinced that blocking workers is inherently a bad thing, and we should consider proposing it at that point. Sites want to ship shumway to their own users (Flash on ipads) and it would IMO be silly to require them to run on the main thread just to maintain the purity of an entirely async worker design. I'm not convinced we can get a synch worker proposal accepted. If we attempt to do so, we should see if we can coordinate with the asm.js efforts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 6:21 PM, Till Schneidereit t...@mozilla.com wrote: Nobody's proposing to expose any of this to web content. The question is whether we want to fully expose it as an official capability of Gecko, to be used by other parts of the browser and maybe even addons, or if we want to go to some lengths to hide it as best as we can. Oh, ok. That wasn't clear from Benjamin's original email (also because I thought we intended to implement Shumway with pure JS). In that case, I don't really have a strong opinion. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 6:30 PM, Bobby Holley bobbyhol...@gmail.com wrote: On Mon, Oct 14, 2013 at 6:21 PM, Till Schneidereit t...@mozilla.com wrote: Nobody's proposing to expose any of this to web content. The question is whether we want to fully expose it as an official capability of Gecko, to be used by other parts of the browser and maybe even addons, or if we want to go to some lengths to hide it as best as we can. Oh, ok. That wasn't clear from Benjamin's original email (also because I thought we intended to implement Shumway with pure JS). In that case, I don't really have a strong opinion. The first step would be to do this chrome-only. But see my follow-up. There are some Flash features that we can't implement with web platform features. E.g., Copying to clipboard and cross-domain loading of some resources requires privileged code no matter what we do. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 11:03 AM, Benjamin Smedberg benja...@smedbergs.uswrote: Having this blocking interface will also support blocking shumway on graphics rendering; there are a fair number of SWF files that use an API to get a synchronous bitmap of their stage, which requires a blocking call from shumway to the main thread even after we get canvas-on-a-worker, which explicitly will not support that blocking case. I do not understand this. Current proposals for canvas-on-a-worker allow reading the contents of a canvas into a bitmap. What's the problem here? Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * * ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On 10/14/2013 4:12 PM, Robert O'Callahan wrote: On Mon, Oct 14, 2013 at 11:03 AM, Benjamin Smedberg benja...@smedbergs.us mailto:benja...@smedbergs.us wrote: Having this blocking interface will also support blocking shumway on graphics rendering; there are a fair number of SWF files that use an API to get a synchronous bitmap of their stage, which requires a blocking call from shumway to the main thread even after we get canvas-on-a-worker, which explicitly will not support that blocking case. I do not understand this. Current proposals for canvas-on-a-worker allow reading the contents of a canvas into a bitmap. What's the problem here? Is canvas-on-a-worker a short-term project? Also from reading the whatwg discussion, I thought you had proposed .toBlob but not .toDataURI specifically so that the bitmap was not synchronously available (the blob would mediate asynchronously fetching the data). It's still not clear to me whether with canvas-on-a-worker you were planning on actually doing the graphics on the worker, or just batching drawing instructions and sending them over to a separate graphics thread to actually be executed. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 10:16 PM, Benjamin Smedberg benja...@smedbergs.us wrote: On 10/14/2013 4:12 PM, Robert O'Callahan wrote: On Mon, Oct 14, 2013 at 11:03 AM, Benjamin Smedberg benja...@smedbergs.us mailto:benja...@smedbergs.us wrote: Having this blocking interface will also support blocking shumway on graphics rendering; there are a fair number of SWF files that use an API to get a synchronous bitmap of their stage, which requires a blocking call from shumway to the main thread even after we get canvas-on-a-worker, which explicitly will not support that blocking case. I do not understand this. Current proposals for canvas-on-a-worker allow reading the contents of a canvas into a bitmap. What's the problem here? Is canvas-on-a-worker a short-term project? From talking to Kyle, it sounds like getting WebGL into workers is much closer than getting all of Context2D, too. Specifically, we need text rendering to work, because we have to be able to render device fonts. Into bitmaps. Synchronously, because everything else would be way too easy. To reiterate, we have two non-negotiable requirements for enabling off-mainthread usage of Shumway: - synchronous drawing to canvas, including text rendering - synchronous two-way communication with surrounding JS, with arbitrarily-recursive calls between page JS and Shumway Also from reading the whatwg discussion, I thought you had proposed .toBlob but not .toDataURI specifically so that the bitmap was not synchronously available (the blob would mediate asynchronously fetching the data). It's still not clear to me whether with canvas-on-a-worker you were planning on actually doing the graphics on the worker, or just batching drawing instructions and sending them over to a separate graphics thread to actually be executed. --BDS ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 4:16 PM, Benjamin Smedberg benja...@smedbergs.uswrote: Is canvas-on-a-worker a short-term project? WebGL is at least. Also from reading the whatwg discussion, I thought you had proposed .toBlob but not .toDataURI specifically so that the bitmap was not synchronously available (the blob would mediate asynchronously fetching the data). toBlob is async also. We didn't propose toDataURI mainly because it's not very useful in workers and not a good way to get the bits. The data is synchronously available. These APIs are only async because it's a bit slow to do them synchronously --- because we have to read back from the GPU in some cases, and run the image encoder. I don't know how Flash bitmaps work in Shumway, but instead of toBlob (which gives you data in an image format) you probably want getImageData, which gives you the data in an array, uncompressed, and is synchronous. It's still not clear to me whether with canvas-on-a-worker you were planning on actually doing the graphics on the worker, or just batching drawing instructions and sending them over to a separate graphics thread to actually be executed. The former, but later the latter. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * * ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 4:29 PM, Till Schneidereit t...@tillschneidereit.net wrote: From talking to Kyle, it sounds like getting WebGL into workers is much closer than getting all of Context2D, too. Specifically, we need text rendering to work, because we have to be able to render device fonts. Into bitmaps. Synchronously, because everything else would be way too easy. To reiterate, we have two non-negotiable requirements for enabling off-mainthread usage of Shumway: - synchronous drawing to canvas, including text rendering Getting text rendering to work with 2D canvas in a worker is tricky and we may want to implement it by doing blocking proxying calls to the main thread. - synchronous two-way communication with surrounding JS, with arbitrarily-recursive calls between page JS and Shumway That seems like the big problem here. Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w * * ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Synchronous workers
On Mon, Oct 14, 2013 at 11:31 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Mon, Oct 14, 2013 at 4:29 PM, Till Schneidereit t...@tillschneidereit.net wrote: From talking to Kyle, it sounds like getting WebGL into workers is much closer than getting all of Context2D, too. Specifically, we need text rendering to work, because we have to be able to render device fonts. Into bitmaps. Synchronously, because everything else would be way too easy. To reiterate, we have two non-negotiable requirements for enabling off-mainthread usage of Shumway: - synchronous drawing to canvas, including text rendering Getting text rendering to work with 2D canvas in a worker is tricky and we may want to implement it by doing blocking proxying calls to the main thread. - synchronous two-way communication with surrounding JS, with arbitrarily-recursive calls between page JS and Shumway That seems like the big problem here. Exactly. That's the issue that got this whole discussion going. The fact that having a synchronous worker-mainthread channel also enables using it for rendering is the less important issue. Although the text rendering issue makes it pretty attractive. The way I see it, we should do asynchronous drawing by default. Either in yet another worker, or on the main thread as long as we don't have everything we need to do it in a worker. Only for the rendering-to-bitmapdata case should we use the sync channel to do the drawing. This makes sense from a responsiveness POV, in any case. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
