Re: [whatwg] onclose events for MessagePort
On Sat, Oct 19, 2013 at 2:26 AM, Jonas Sicking jo...@sicking.cc wrote: What I think might work is to say that as long as a channeldropped event listener is registered with a port, that is equivalent to holding a strong reference to the port. I.e. that prevents the channel from being GCed. Even if no references are held to either port. In other words, we'd give up 3, but only when 2 is actively used. Agreed - this was my proposal here: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-October/041068.html in response to your earlier email. We would also need to define that the channeldropped event is never fired on a port if that port's *owning* window has been navigated away from. Otherwise we could *never* GC a channel between two windows that has channeldropped registered on either side. Makes sense, although I'm a bit fuzzy about the rules around MessagePorts and window navigation (for example, if I navigate a window, is all content in that window now shutdown/discarded, even though I could in theory get back to the window by immediately clicking back)? So the expectation is that a caller should only register a channeldropped event if it's actively waiting for a response from that thread. It's not something that you should register when creating the port and then never unregister. Having a channeldropped event listener registered can cause a whole worker thread to be kept alive longer than it otherwise would. How does this work - imagine that I have a reference to a MessagePort, but I'm not actively waiting for any response on the port so I don't have a channeldropped event listener. Now, the remote side of the port crashes. I send a message on the port and add a channeldropped event handler - are you saying that adding a channeldropped event handler should trigger a channeldropped event if the other side has already crashed? If not, then how do I find out that the channel has been dropped if I don't keep the event handler registered all the time?
Re: [whatwg] Canvas in workers
On Sun, Oct 20, 2013 at 11:53 PM, Robert O'Callahan rob...@ocallahan.orgwrote: Glenn, taking a step back for a bit, is there anything in https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal that you would actually object to? IOW, is there anything there that you would think is completely superfluous to the platform if all your proposals were to be adopted as well? I have no objection to the overall change from CanvasProxy to WorkerCanvas, eg. the stuff in Kyle's original mail to the thread. (Being able to settle on that is one of the reasons I've tried to detach discussion for the other use cases.) I'd only recommend leaving out transferToImageBitmap, srcObject and ImageBitmap.close() parts. I do think that would be redundant with with present proposal. They can always be added later, and leaving them out keeps the WorkerCanvas proposal itself focused. -- Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Sun, Oct 20, 2013 at 11:16 PM, Robert O'Callahan
rob...@ocallahan.orgwrote:
With all these proposals I think it's OK to allow the main thread to do
(e.g.) a toDataURL and read what the current contents of the canvas is,
We can defer this discussion, since it's not something new to this proposal
(or any other proposal we're discussing).
On Sun, Oct 20, 2013 at 11:33 PM, Robert O'Callahan rob...@ocallahan.orgwrote:
To me, passing the image data explicitly in an ImageBuffer along with the
present message seems like a better fit to the workers message-passing
model than this proposal, where the data is stored as hidden state in the
canvas element with (effectively) a setter in the worker and a getter in
the main thread, and that setting and getting has to be coordinated with
postMessage for synchronization. The relationship between a commit and
its present has to be deduced by reasoning about the timing of messages,
rather than by just reasoning about JS data flow through postMessage.
Using ImageBitmap for this has a lot of issues. It requires synchronizing
with scripts in the UI thread. It requires manualling resize your canvas
repeatedly to fit different destinations. It also may potentially create
lots of backbuffers. Here's an example of code using ImageBitmap
incorrectly, leading to excess memory allocation:
function render()
{
var canvas = myWorkerCanvas;
renderTo(canvas);
var buffer = canvas.transferToImageBitmap();
postMessage(buffer);
}
setTimeout(render, 1);
We start with one backbuffer available, render to it (renderTo), peel it
off the canvas to be displayed somewhere, and toss it off to the main
thread. (For the sake of the example, the main thread is busy and doesn't
process it immediately.) The worker enters render() again, and when it
gets to renderTo, a new backbuffer has to be allocated, since the one
buffer we have is still used by the ImageBuffer and can't be changed. This
happens repeatedly, creating new backbuffers each time, since none of them
can be reused.
This is an extreme example, but if this ever happens even once, it means
potentially allocating an extra backbuffer.
This proposal also requires that whenever a worker is going to return
image data to the main thread, the main thread must start things off by
creating a canvas element. It's also not possible for a worker to spawn off
sub-workers to do drawing (at least, not without some really ugly
coordination with the main thread.)
Sure it is. If you want an offscreen buffer, you just new
WorkerCanvas(). This is unrelated to offscreen drawing.
--
Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 3:39 PM, Glenn Maynard gl...@zewt.org wrote:
On Sun, Oct 20, 2013 at 11:16 PM, Robert O'Callahan rob...@ocallahan.org
wrote:
With all these proposals I think it's OK to allow the main thread to do
(e.g.) a toDataURL and read what the current contents of the canvas is,
We can defer this discussion, since it's not something new to this proposal
(or any other proposal we're discussing).
On Sun, Oct 20, 2013 at 11:33 PM, Robert O'Callahan rob...@ocallahan.org
wrote:
To me, passing the image data explicitly in an ImageBuffer along with the
present message seems like a better fit to the workers message-passing
model than this proposal, where the data is stored as hidden state in the
canvas element with (effectively) a setter in the worker and a getter in the
main thread, and that setting and getting has to be coordinated with
postMessage for synchronization. The relationship between a commit and its
present has to be deduced by reasoning about the timing of messages,
rather than by just reasoning about JS data flow through postMessage.
Using ImageBitmap for this has a lot of issues. It requires synchronizing
with scripts in the UI thread.
This isn't difficult, and amounts to a few additional lines of code in
the main thread's onmessage handler.
The ImageBitmap style proposal has another significant advantage in
that it allows a single canvas context to present results in multiple
output regions on the page.
It requires manualling resize your canvas
repeatedly to fit different destinations. It also may potentially create
lots of backbuffers. Here's an example of code using ImageBitmap
incorrectly, leading to excess memory allocation:
function render()
{
var canvas = myWorkerCanvas;
renderTo(canvas);
var buffer = canvas.transferToImageBitmap();
postMessage(buffer);
}
setTimeout(render, 1);
We start with one backbuffer available, render to it (renderTo), peel it off
the canvas to be displayed somewhere, and toss it off to the main thread.
(For the sake of the example, the main thread is busy and doesn't process it
immediately.) The worker enters render() again, and when it gets to
renderTo, a new backbuffer has to be allocated, since the one buffer we have
is still used by the ImageBuffer and can't be changed. This happens
repeatedly, creating new backbuffers each time, since none of them can be
reused.
This is an extreme example, but if this ever happens even once, it means
potentially allocating an extra backbuffer.
This sort of resource exhaustion is certainly possible, but I view
this downside as smaller than the upside of addressing both of the
above use cases.
-Ken
This proposal also requires that whenever a worker is going to return
image data to the main thread, the main thread must start things off by
creating a canvas element. It's also not possible for a worker to spawn off
sub-workers to do drawing (at least, not without some really ugly
coordination with the main thread.)
Sure it is. If you want an offscreen buffer, you just new WorkerCanvas().
This is unrelated to offscreen drawing.
--
Glenn Maynard
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 6:08 PM, Kenneth Russell k...@google.com wrote: Using ImageBitmap for this has a lot of issues. It requires synchronizing with scripts in the UI thread. This isn't difficult, and amounts to a few additional lines of code in the main thread's onmessage handler. Synchronization with the UI thread isn't bad because it's difficult. Avoiding synchronization with the main thread has been raised as a desirable goal: http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0152.htmlincluding that it isn't possible to render from a worker without synchronizing with the main thread. (My previous comments on this are here: http://www.mail-archive.com/whatwg@lists.whatwg.org/msg35959.html) The ImageBitmap style proposal has another significant advantage in that it allows a single canvas context to present results in multiple output regions on the page. You can do that. You just create a WorkerCanvas for each canvas you want to present to, hand them to the worker, then attachToCanvas in the worker to switch from canvas to canvas. (That's orthogonal to explicitpresent.) This sort of resource exhaustion is certainly possible, but I view this downside as smaller than the upside of addressing both of the above use cases. I can only find one thing above that you might be referring to as a use case (the one I replied to immediately above). What was the other? -- Glenn Maynard
Re: [whatwg] onclose events for MessagePort
On Fri, Oct 18, 2013 at 8:26 PM, Jonas Sicking jo...@sicking.cc wrote: On Thu, Oct 17, 2013 at 2:08 PM, Ehsan Akhgari eh...@mozilla.com wrote: It occurs to me that all of the proposals here does expose some amount of GC behavior. Even a channeldropped message which is sent only when the other side crashes exposes GC behavior. If GC happens to run before the crash and then collect the MessageChannel ports, then no channel exists at the time of crash, and thus no event is sent. However if the GC runs later, or if it doesn't successfully collect the MessageChannel ports, then the channeldropped event does fire. I'm not sure if I understand this. If the MessagePort exists on the side that is interested to handle the event, then it can't be GCed on the other side either, right? I thought the proposal was to not fire channeldropped when the channel is GCed. Thus allowing channels with both message and channeldropped event listeners on either side to still be GCed. Is that correct? Yes, that's correct. If so, that exposes GC behavior. If at some point both pages that hold on to an endpoint of a message channel drop their references the channel can get GCed. If it is GCed no events fire. However if the page holding on to either port crashes before the GC happens, then the channeldropped event is fired on the other port. Hence the timing of the GC affects whether channeldropped is fired. Hence GC behavior is exposed. Yeah, I see the problem now. That's not to say that this solution wouldn't work. Exposing some amount of GC behavior might be ok. But it does mean that we should have a realistic bar as we discuss expanding the event to more situations than just process crashes. One solution which I think would never expose GC behavior is to simply have a property on the MessagePort which indicates if the owner of the other side has been killed or navigated away from. No event would fire as that property changes value. Since it's a property, it can only be read if a reference to the MessagePort is being held. As long as such a reference exists neither side of the MessageChannel can be GCed. Exposing this state as a property will make people who have use cases such as Update the UI if this other tab/page/app/etc is killed poll the attribute, which seems non-ideal to me. I don't see a way to simultaneously fulfill all of the following constraints: 1. Don't expose GC. 2. Provide a callback when the other side crashes or lives in a thread that dies. 3. Enable GC of channels where neither side referenced. What I think might work is to say that as long as a channeldropped event listener is registered with a port, that is equivalent to holding a strong reference to the port. I.e. that prevents the channel from being GCed. Even if no references are held to either port. In other words, we'd give up 3, but only when 2 is actively used. And then define that channeldropped is fired on a port whenever the other side is killed due to either living in a process that crashes, or living in a thread that is closed or terminated. We would also need to define that the channeldropped event is never fired on a port if that port's *owning* window has been navigated away from. Otherwise we could *never* GC a channel between two windows that has channeldropped registered on either side. This sounds good to me. So the expectation is that a caller should only register a channeldropped event if it's actively waiting for a response from that thread. It's not something that you should register when creating the port and then never unregister. Having a channeldropped event listener registered can cause a whole worker thread to be kept alive longer than it otherwise would. Right. We could further define that channeldropped is fired when the owner of the *other side* is navigated away from. This would mean that events can be received even after a channeldropped event has been fired since other windows could still hold a reference to the port and send messages through it. However it would allow us to release the strong reference that the channeldropped event listener implicitly creates as soon as either side is navigated away from. But what if the page is navigated back to? I think this implies that having fired a channeldropped event because of this reason means that the UA needs to make it impossible to navigate back to the same window, which means disabling optimizations such as Gecko's back-forward cache, which sucks. Also, having a setTimeout(0) loop which polls the attribute would open us to the exact same risks as the event would, I think. How do you mean? Which risks in particular are you referring to? I was talking about GC observabiity, but I think I was wrong since the attribute that you were proposing could not be queried if you lost the reference to it from js... Cheers, -- Ehsan
Re: [whatwg] onclose events for MessagePort
On Mon, Oct 21, 2013 at 3:19 AM, Andrew Wilson atwil...@google.com wrote: On Sat, Oct 19, 2013 at 2:26 AM, Jonas Sicking jo...@sicking.cc wrote: What I think might work is to say that as long as a channeldropped event listener is registered with a port, that is equivalent to holding a strong reference to the port. I.e. that prevents the channel from being GCed. Even if no references are held to either port. In other words, we'd give up 3, but only when 2 is actively used. Agreed - this was my proposal here: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-October/041068.html in response to your earlier email. Yeah, you're right, sorry for overlooking this the first time around! So the expectation is that a caller should only register a channeldropped event if it's actively waiting for a response from that thread. It's not something that you should register when creating the port and then never unregister. Having a channeldropped event listener registered can cause a whole worker thread to be kept alive longer than it otherwise would. How does this work - imagine that I have a reference to a MessagePort, but I'm not actively waiting for any response on the port so I don't have a channeldropped event listener. Now, the remote side of the port crashes. I send a message on the port and add a channeldropped event handler - are you saying that adding a channeldropped event handler should trigger a channeldropped event if the other side has already crashed? If not, then how do I find out that the channel has been dropped if I don't keep the event handler registered all the time? I think we may need to mandate that a channeldropped eventis fired when you register a handler on a port with the other side having already crashed. Cheers, -- Ehsan http://ehsanakhgari.org/
Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread
On Mon, Oct 21, 2013 at 4:34 PM, Glenn Maynard gl...@zewt.org wrote: On Mon, Oct 21, 2013 at 6:08 PM, Kenneth Russell k...@google.com wrote: Using ImageBitmap for this has a lot of issues. It requires synchronizing with scripts in the UI thread. This isn't difficult, and amounts to a few additional lines of code in the main thread's onmessage handler. Synchronization with the UI thread isn't bad because it's difficult. Avoiding synchronization with the main thread has been raised as a desirable goal: http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0152.html including that it isn't possible to render from a worker without synchronizing with the main thread. (My previous comments on this are here: http://www.mail-archive.com/whatwg@lists.whatwg.org/msg35959.html) The ImageBitmap style proposal has another significant advantage in that it allows a single canvas context to present results in multiple output regions on the page. You can do that. You just create a WorkerCanvas for each canvas you want to present to, hand them to the worker, then attachToCanvas in the worker to switch from canvas to canvas. (That's orthogonal to explicitpresent.) OK, I misunderstood that part of your attachToCanvas proposal. There are some unexpected consequences of the attachToCanvas API style. For example, what if two contexts use attachToCanvas to target the same canvas? What if one of those contexts is 2D and the other is WebGL? Currently it's illegal to try to fetch two different context types for a single Canvas. The current CanvasProxy spec contains several complex rules for these cases, and they're not easy to understand. Will it be guaranteed that if you have a WebGL context, attachToCanvas to canvas1, do some rendering, and then attachToCanvas to canvas2, that the only remaining buffer in canvas1 is its color buffer? No depth buffers, multisample buffers, etc. will have to remain for some reason? How would WebGL's preserveDrawingBuffer attribute, which is a property of the context, interact with directing its output to multiple canvases? Fundamentally I think the behavior is easier to spec, and the implementation is easier to make correct, if the ultimate destination is an image rather than a canvas, and the color buffer is transferred out of the WorkerCanvas in an explicit step. -Ken This sort of resource exhaustion is certainly possible, but I view this downside as smaller than the upside of addressing both of the above use cases. I can only find one thing above that you might be referring to as a use case (the one I replied to immediately above). What was the other? -- Glenn Maynard
