from:"Robert O'Callahan"

On Tue, Jul 14, 2015 at 10:33 PM, Philip Jägenstedt phil...@opera.com
wrote:

 I see, is this a policy that's applied even with a single video in a
 page, or is it something like a limit on the total number of players
 that can be created, or limits on how much stuff (decoders) they set
 up internally? Presto actually had a limit on the number of players on
 32-bit systems, as one might otherwise run out of virtual memory with
 just a few hundred video elements...


There are various kinds of resource limits. The problem with those is that
they're fragile and make behavior unpredictable.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

On Tue, Jul 14, 2015 at 8:51 PM, Philip Jägenstedt phil...@opera.com
wrote:

 Would it solve the same practical problems if auto were redefined to
 mean that the goal readyState is HAVE_ENOUGH_DATA?


Probably.


 Is there a reason it doesn't do this in mobile Firefox?


We don't want a page with many video preload elements to collapse. Maybe
we shouldn't worry about that.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

On Tue, Jul 14, 2015 at 11:13 PM, Philip Jägenstedt phil...@opera.com
wrote:

 If the behavior could be made interoperable for when resources are not
 a problem, that would be a good start at least. The spec can say that
 the browser should at least try to ready HAVE_ENOUGH_DATA for
 preload=auto, to rule out artificial limits if any browser has them.


If that behavior is required for Web compatibility, then we should do it.
However I also think there's value in having a preload value that preloads
aggressively on some platforms but not on others.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

On Tue, Jul 14, 2015 at 11:24 PM, Philip Jägenstedt phil...@opera.com
wrote:

 Basically a I trust the browser to decide and promise not to assume
 anything state? The auto state is already suitable named and
 defined for that, so if implementing auto as anything other than
 try to reach HAVE_ENOUGH_DATA is in fact web compatible, a new state
 for that might make sense.

 Practically speaking, though, what would be the policy for auto that
 differs between platforms, in the case of a single video element in a
 page?


Our current policy, which is basically HAVE_ENOUGH_DATA on desktop and
HAVE_METADATA on mobile.

If the different policies could be made explicit, it might be
 nice to have explicit preload states for them and a way for scripts to
 know the effective preload state.


Maybe that's overengineering.

I propose we leave 'auto' as-is for now until we have more data, and first
do the other work I mentioned.

Then a possible next step would be to determine how much preloading
preload=auto must do for Web compatibility (i.e., which readyState it
must reach), and define preload=auto to have that as the goal readyState,
*but* we could also say that it preloads as much data as possible given
browser-specific policy (while not actually advancing the readyState beyond
the goal readyState until the autoplaying flag is false).

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

2015-07-13 Thread Robert O'Callahan

Apart from the problems already discussed here, there is currently no
specced or interoperably implemented way to set a preload value that
guarantees HAVE_CURRENT_DATA, HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA will be
reached ... auto may do one of those things, but it doesn't have to, and
in fact on mobile Firefox we don't guarantee any of those things.

So I suggest we revamp preload as follows:
preload=none: the goal readyState is HAVE_NOTHING.
preload=loadedmetadata: the goal readyState is HAVE_METADATA.
preload=metadata: the goal readyState is HAVE_METADATA (or some higher
value if needed for Web compatbility).
preload=loadeddata: the goal readyState is HAVE_CURRENT_DATA.
preload=canplay: the goal readyState is HAVE_FUTURE_DATA.
preload=canplaythrough: the goal readyState is HAVE_ENOUGH_DATA.
(Thus the expectation for authors is that if you set preload=X then event
X will eventually fire.)

I would spec that while the autoplaying flag is true, the readyState of the
element is limited to less than or equal to the goal readyState if there is
one. (Note that the autoplaying flag value does not depend on the presence
of the autoplay attribute.)

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

2015-07-07 Thread Robert O'Callahan

On Tue, Jul 7, 2015 at 9:41 PM, Philip Jägenstedt phil...@opera.com wrote:

 Unsurprisingly, I like this, as it's basically what Presto did. However, I
 think preload=metadata should reach HAVE_CURRENT_DATA, because you do
 want to decode the first frame. The naming mismatch is weird, but oh well.


OK.


  The question is, would Chrome(ium) be willing to implement that
 reasonably
  quickly? I'm not optimistic given the canplaythrough debacle...

 Do you have links to previous debacles, so that I can see what the
 constraints are, or have been?


The constraint just seems to be a lack of interest in fixing interop issues
in this area:
http://code.google.com/p/chromium/issues/detail?id=73609


 Given that Blink's default preload state is
 auto that ought to limit the breakage somewhat, but I wouldn't really be
 surprised to find sites that use preload=metadata and yet assume that
 canplay will fire, or similar.

 Whatever kind of mess we're in, it would be nice to aim for
 interoperability.


Indeed.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

2015-07-06 Thread Robert O'Callahan

On Mon, Jul 6, 2015 at 11:17 PM, Philip Jägenstedt phil...@opera.com
wrote:

 On Wed, Jun 10, 2015 at 6:53 AM, Robert O'Callahan rob...@ocallahan.org
 wrote:
  In Gecko, video preload=metadata doesn't fire canplay. This is
  allowed (encouraged, even) by the spec, since we can efficiently satisfy
  preload=metadata by stopping decoding after one frame, and if we only
  decode one frame then readyState will not advance beyond
 HAVE_CURRENT_DATA.
  However, this may be counterintuitive for Web developers. Also, Chrome
  fires canplay in this situation (although that's probably part of the
  Chrome fires canplay and canplaythrough willy-nilly bug). Anyone else
  have an opinion on this?

 Are you having site compat problems related to this?


There's https://bugzilla.mozilla.org/show_bug.cgi?id=1165203 (the same bug
as in the other thread), thought that involves load() as well. I don't know
of any sites where this causes problems in the absence of load(), though I
wouldn't be surprised if they exist. I think it would be a good idea to
align browser behaviors or the spec.

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

2015-07-06 Thread Robert O'Callahan

On Tue, Jul 7, 2015 at 1:44 AM, Philip Jägenstedt phil...@opera.com wrote:

 On Mon, Jul 6, 2015 at 2:19 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

  On Mon, Jul 6, 2015 at 11:17 PM, Philip Jägenstedt phil...@opera.com
  wrote:
 
  On Wed, Jun 10, 2015 at 6:53 AM, Robert O'Callahan 
 rob...@ocallahan.org
  wrote:
   In Gecko, video preload=metadata doesn't fire canplay. This is
   allowed (encouraged, even) by the spec, since we can efficiently
 satisfy
   preload=metadata by stopping decoding after one frame, and if we
 only
   decode one frame then readyState will not advance beyond
  HAVE_CURRENT_DATA.
   However, this may be counterintuitive for Web developers. Also, Chrome
   fires canplay in this situation (although that's probably part of
 the
   Chrome fires canplay and canplaythrough willy-nilly bug). Anyone
 else
   have an opinion on this?
 
  Are you having site compat problems related to this?
 
  There's https://bugzilla.mozilla.org/show_bug.cgi?id=1165203 (the same
  bug as in the other thread), thought that involves load() as well. I
 don't
  know of any sites where this causes problems in the absence of load(),
  though I wouldn't be surprised if they exist. I think it would be a good
  idea to align browser behaviors or the spec.

 Yeah, I agree. In Presto I added some explicit limits to readyState based
 on preload so that even for data: URLs or fully cached resources, it
 wouldn't progress beyond HAVE_CURRENT_DATA. (However, buffered wasn't
 censored, so it could look a bit strange if inspected closely.) Is this how
 it works in Gecko too, or is it actually the media framework that makes the
 guarantee without any clamping on top of it?


We don't do any clamping of the state, so given preload=metadata you may
or may not reach HAVE_CURRENT_DATA or HAVE_FUTURE_DATA (or even
HAVE_ENOUGH_DATA).

My previous attempts to nail down the precise behavior or preload haven't
 gone well, as Hixie considers preload as a hint where anything goes. Even
 so, specifying the normal behavior seems like a good idea.

 In the specific case of preload=metadata, what would you like the spec to
 say?


Clamping the readyState would be a good way to get reliable behavior and
interop while keeping preload useful. We'd spec some kind of goal
readyState associated with a resource load, and spec that readyState is
clamped to the goal readyState. For preload=none that's HAVE_NOTHING, for
preload=metadata that's HAVE_METADATA, for other preload values it would
be HAVE_ENOUGH_DATA. Method calls like play() or setting autoplay would
force it to HAVE_ENOUGH_DATA and it could never decrease.

The question is, would Chrome(ium) be willing to implement that reasonably
quickly? I'm not optimistic given the canplaythrough debacle...

Rob
-- 
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn

Re: [whatwg] Apple's new link rel=icon mask not-quite-favicon syntax causing problems in other browsers; needs standardization?

2015-06-14 Thread Robert O'Callahan

On Mon, Jun 15, 2015 at 12:40 PM, Daniel Holbert dholb...@mozilla.com
wrote:

 There was a suggestion on one of the bugs that we simply ignore link
 tags that have a mask attribute; this may be what we want to do to
 avoid having a zillion site-compat conversations, but I wanted to get
 other vendors/spec-authors' opinions before considering this option too
 seriously.


Let's do this now. We can't wait for Apple and/or Web sites to get their
act together.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

[whatwg] preload=metadata elements don't necessarily fire canplay

2015-06-09 Thread Robert O'Callahan

In Gecko, video preload=metadata doesn't fire canplay. This is
allowed (encouraged, even) by the spec, since we can efficiently satisfy
preload=metadata by stopping decoding after one frame, and if we only
decode one frame then readyState will not advance beyond HAVE_CURRENT_DATA.
However, this may be counterintuitive for Web developers. Also, Chrome
fires canplay in this situation (although that's probably part of the
Chrome fires canplay and canplaythrough willy-nilly bug). Anyone else
have an opinion on this?

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] High-density canvases

2015-04-16 Thread Robert O'Callahan

On Thu, Apr 16, 2015 at 7:07 AM, Justin Novosad ju...@google.com wrote:

 In the interest of moving forward with this, I began an experimental
 implementation of the renderedPixelWidth/Height propsal (
 https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize) in Blink.  I ran
 into
 some issues, which I documented in the issues section of the proposal.  I
 would like to draw your attentions to them at this point:

 1. Hard to determine a rendered size when the canvas is not attached to the
 DOM. Perhaps in that case the current intrinsic size should be returned?


I think in this case you should estimate it based on the current zoom level
of the document, i.e. just assume no CSS transforms will be applied to the
element.


 2. After a layout change that affects rendered pixel size, there is no
 guarantee that the size change event will be handled before the layout
 change is propagated to screen, so the content may be temporarily displayed
 in an inconsistent state. Note: the same issue exists with existing methods
 that may be based on mutation observers or window.onresize, for example.
 Though it is not the stated objective of this proposal to solve this
 synchronization problem, there may be an opportunity to do so.
 3. Accessing rendered pixel size is layout-inducing. To avoid layout
 thrashing, we should consider making this an asynchronous getter (e.g.
 asyncGetBoundignClientRect). This would also prevent renderedsizechanged
 events from firing from within the evaluation of the
 renderedPixelWidth/Height
 attributes, which is weird.


renderedsizechange isn't exactly the same as resize. For example during
zooming (any kind of zooming), an element with a fixed CSS px size does not
need to receive a resize event, but it should probably receive a
renderedsizechange.

I think we should keep renderedsizechange as a separate event, but we do
need to adjust how it fires. I think we should flush renderedsizechange
events before painting, just like we do for onscroll events.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-14 Thread Robert O'Callahan

On Wed, Apr 15, 2015 at 1:15 PM, Kenneth Russell k...@google.com wrote:

 If repeatedly transferring ImageBitmaps of the same size into the same
 img won't cause repeated re-layouts, that alleviates my concern
 about efficiency of using images as the output path. I expect that a
 single OffscreenCanvas will be used to produce output to multiple
 areas on the page (whether into images or canvases) but that it will
 be sized large enough so that all of the ImageBitmaps it produces will
 cover the largest of those areas, avoiding repeatedly resizing the
 OffscreenCanvas.


So you want to produce multiple ImageBitmaps of the same (largest) size and
then render them to multiple elements in the page, each element cropping
its ImageBitmap to a (potentially) different size? Isn't that going to be
wasteful of memory?

There is one other problem which has come up repeatedly in canvas
 applications: needing to be able to accurately measure the number of
 device pixels covered by a canvas, so pixel-accurate rendering can be
 done. https://wiki.whatwg.org/wiki/CanvasRenderedPixelSize addresses
 this and it's currently being implemented in Chromium. There is no
 such mechanism for images. It would be necessary to understand exactly
 how many device pixels the output image is covering in the document so
 that the OffscreenCanvas can be sized appropriately. Using a canvas
 element for the display of these ImageBitmaps avoids this problem. It
 also makes the API more symmetric; a canvas-like object produces the
 ImageBitmap, and a canvas element is used to view it. Do you have any
 suggestion for how this issue would be solved with img?


We could support renderedPixelWidth/Height on img as well as canvas.
But this is a valid argument in favor of rendering those ImageBitmaps
through canvas.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-14 Thread Robert O'Callahan

I guess there are really two different sets of use-cases:
1) Use-cases where the ImageBitmap is sized to fill a particular area of
the screen.
2) Use-cases where the ImageBitmap is sized subject to some other
constraints, e.g. you're processing an existing image.

For #2, you want the ImageBitmap's size to be the intrinsic size of the
element.

For #1, you don't care about having an intrinsic size. Instead the
element's size is set by CSS and you want the ImageBitmap to fill the
element, and you want to be able to use renderedPixelWidth/Height to size
the ImageBitmap. However, for these use-cases I think the behavior
currently specced in the OffscreenBitmap proposal doesn't make sense:
  // The ImageBitmap, when displayed, is clipped to the rectangle
  // defined by the canvas's instrinsic width and height. Pixels that
  // would be covered by the canvas's bitmap which are not covered by
  // the supplied ImageBitmap are rendered transparent black.
I'm not sure what this means now. Suppose the canvas element has CSS
width:100px; height:100px and the canvas rendered size is 200x200. If the
application creates a 200x200 ImageBitmap and installs it in the canvas, as
specced isn't this just going to display some 100x100 subrectangle of the
ImageBitmap, cropping out the rest?

What if instead of a new context type, we had a method on the canvas
element itself to install a new ImageBitmap, which replaces the backbuffer
and sets the intrinsic size of the canvas element to the ImageBitmap's
size, so that 'width' and 'height' are thereafter ignored? Then if the
canvas has non-auto CSS 'width' and 'height', the image buffer will be
scaled to fit the CSS size and the example above will work as expected. CSS
object-fit will then also work, so if the ImageBitmap producer lags behind
canvas resizing for some reason, authors can use object-fit:contain (or
cover) to display the mis-sized images in a reasonable way. (object-fit is
also important for the case where a Worker is presenting images through an
OffscreenCanvas and its image production may lag behind resizing of the
canvas element on another thread; we can pass the object-fit value through
to the compositor to ensure the temporary results look OK.)

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-13 Thread Robert O'Callahan

On Tue, Apr 14, 2015 at 9:03 AM, Justin Novosad ju...@google.com wrote:

 On Sun, Apr 12, 2015 at 5:41 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

  On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell k...@google.com wrote:
 
 
  3. Are onload and onerror events fired? This question applies both to
  the in-progress download and to the transferred-in ImageBitmap.
 
 
  No.
 

 From the web developer's perspective wouldn't that lead to
 non-deterministic behavior?  What I mean is that if the onload listener was
 not yet called at the time an ImageBitmap is transferred in, the onload
 listener may still get called depending on whether the onload event was
 already dispatched when the ImageBitmap transfer occurred.  That's a race.
 This makes me wonder... is there a precedent for recalling dispatched
 events?


Yes. Media elements, for example, dispatch tasks to an element-specific
task source, which is emptied when a new load starts:
https://html.spec.whatwg.org/multipage/embedded-content.html#dom-media-load

Having said that, the sort of races that you've identified are endemic to
the Web platform. For example the spec (and implementations) allow an img
load event to fire after a different image has already started loading in
the img. So I don't think there's a new problem here.

Having said that^2, I should probably amend my suggestion so that we
dispatch the error event per section 9.2, with the condition amended to
if the element has a src attribute or a srcset attribute or a parent that
is a picture element *and the has-an-ImageBitmap flag was not already
set*, so dispatch of events for non-ImageBitmap-loads is not affected.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-12 Thread Robert O'Callahan

On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell k...@google.com wrote:

 Suppose src=myimage.png is set on an image element and then, while
 it is downloading, an ImageBitmap is transferred into it.


This should behave as if src was removed just before the ImageBitmap is
transferred into it.

In spec terms, I think we'd define a new state flag has an ImageBitmap.
Transferring an ImageBitmap into an img element would set that flag and
trigger update the image data. After step 2 of update the image data,
when the flag is set, we'd synchronously do what update the image data
would do with no source: clear last selected source (step 4), and do
steps 9.1 and 9.2 (but never fire the error event). Then, transferring
the ImageBitmap would do the load complete work of step 14: set the
completely available state, set the image data, and add the image to the
list of available images.

Note that this is also exactly what we'd need to do to support srcObject on
img, which would be nice to have so you can render a Blob with an
HTMLImageElement without dealing with the annoying createObjectURL lifetime
issues.

Thus:

1. What is displayed in the webpage as myimage.png is downloading?


As normal before an ImageBitmap is transferred, the ImageBitmap afterward.

2. Do the downloaded bits overwrite what you transferred or do we stop
 the download?


Stop the download.


 3. Are onload and onerror events fired? This question applies both to
 the in-progress download and to the transferred-in ImageBitmap.


No.


 4. What should the 'complete' property and 'currentSrc' attribute reflect?


True and the empty string respectively.


 5. Can the developer go back to having the img element contain what
 was set on 'src' and not what was transferred in?


As written, yes, by performing another relevant mutation to trigger update
the image data again.

srcset and the picture element make the situation even more complex.


I think the above covers it.


 In comparison, displaying ImageBitmaps with a custom canvas context
 offers simple semantics and avoids other problems raised on this
 thread like requiring a layout, or firing events, upon recipt of a new
 ImageBitmap.


AFAIK there aren't any issues with events. The issue with layout is simply
whether or not we want the element receiving an ImageBitmap to have the
expected intrinsic size. AFAICT for unnecessary layouts to occur would
require repeated ImageBitmap transfers into the same HTMLImageElement,
where the ImageBitmaps have varying sizes and the element has auto 'width'
or 'height', but the author actually intends that the images all be scaled
or cropped to the same size, and doesn't notice this is not happening. Is
there a more realistic scenario?

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-12 Thread Robert O'Callahan

On Mon, Apr 13, 2015 at 9:41 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Sat, Apr 11, 2015 at 1:49 PM, Kenneth Russell k...@google.com wrote:

 5. Can the developer go back to having the img element contain what
 was set on 'src' and not what was transferred in?


 As written, yes, by performing another relevant mutation to trigger
 update the image data again.


Sorry, this is incorrect. As written, no --- it's not possible to stop
displaying an ImageBitmap and return to displaying 'src'. We could provide
API for that if needed, though I don't think it is.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-10 Thread Robert O'Callahan

On Sat, Apr 11, 2015 at 2:18 AM, Justin Novosad ju...@google.com wrote:

 Riddle me this: What would be the value an HTMLImageElement's src attribute
 after an ImageBitmap is transferred in? A data URL? What if the ImageBitmap
 was sourced from an actual resource with a URL, would we pipe that
 through?  In cases where there is no tractable URL (ex: ImageBitmap was
 grabbed fram a tainted canvas), then what?


We have the same issue for HTMLMediaElement.srcObject:
http://dev.w3.org/2011/webrtc/editor/archives/20140619/getusermedia.html#widl-HTMLMediaElement-srcObject
The current approach is to leave 'src' unchanged, and the spec says that if
there is a current srcObject (or ImageBitmap in our case), render that and
ignore 'src'.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-08 Thread Robert O'Callahan

On Thu, Apr 9, 2015 at 9:44 AM, Kenneth Russell k...@google.com wrote:

 On Tue, Apr 7, 2015 at 6:42 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:
  As far as I can tell, the only significant difference between
  this and WorkerCanvas is using an HTMLCanvasElement bitmaprenderer as a
  zero-copy way to render an ImageBitmap, instead of HTMLImageElement.

 That's the main difference. Another is that OffscreenCanvas can be
 directly instantiated from both the main thread and workers.


Oh, it was always intended that WorkerCanvas be instantiable from a Worker,
it just wasn't written down (and Exposed hadn't been invented yet :-).)


 Can you explain what the problem was with using HTMLImageElement?

 One browser implementer pointed out that HTMLImageElement has several
 complex internal states related to loading, and they do not want to
 conflate that with the display of ImageBitmaps. I've documented this
 in the OffscreenCanvas proposal. (I originally thought Canvas had more
 complex internal state, but now think that it can be easier to define
 new behavior against a new canvas context than img.)


I like your original thought better :-). I don't think circumventing the
HTMLImageElement states is an issue, in Gecko at least. To me it seems
desirable for an element displaying an ImageBitmap to get its intrinsic
size from the ImageBitmap, which fits better with HTMLImageElement than
HTMLCanvasElement. For example this would make object-fit and
object-position useful on such elements.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Recorder audio and save to server

2015-04-07 Thread Robert O'Callahan

On Tue, Apr 7, 2015 at 11:40 PM, Ashley Gullen ash...@scirra.com wrote:

 I've brought this up on other lists, but I think it's worth bringing up
 again here: it's a significant limitation in MediaRecorder that it can only
 work in real-time. If a user has a pre-recorded WAV file which is one hour
 long and they want to upload that or encode it to a different format,
 MediaRecorder requires that the encoding happen in real-time. So the user
 has to sit there and wait for an hour when it could be seconds to minutes
 if it could work faster. So I think there are still important use cases for
 uploading audio that MediaRecorder doesn't cover.

 My preferred solution would be to allow MediaRecorder to accept an
 AudioBuffer instead of a MediaStream, and in this case allow it to encode
 as fast as possible.


In Gecko, MediaRecorder has a constructor that accepts an AudioNode. This
AudioNode can belong to an OfflineAudioContext, which solves your use-case.
This extension was proposed and accepted on public-media-capture though the
spec hasn't been updated yet.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-04-07 Thread Robert O'Callahan

On Wed, Apr 8, 2015 at 11:41 AM, Kenneth Russell k...@google.com wrote:

 I apologize for taking so long to update this proposal, but it's now
 in a reasonable state:
 https://wiki.whatwg.org/wiki/OffscreenCanvas

 (Renamed per feedback from Anne -- thanks.)

 Please offer your feedback. Multiple browser implementers agreed on
 this form of the API at a recent face-to-face meeting of the WebGL
 working group, and the proposal should be able to address multiple use
 cases. I'm very interested in prototyping it to see if it can offer
 better parallelism.


It seems fine. As far as I can tell, the only significant difference
between this and WorkerCanvas is using an HTMLCanvasElement
bitmaprenderer as a zero-copy way to render an ImageBitmap, instead of
HTMLImageElement. Can you explain what the problem was with using
HTMLImageElement?

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Recorder audio and save to server

2015-04-06 Thread Robert O'Callahan

On Sat, Apr 4, 2015 at 9:28 PM, Narendra Sisodiya 
naren...@narendrasisodiya.com wrote:

 Hi,
 I am trying to do simple audio recording service.
 with help of navigator.getUserMedia, I am able to record audio in wav
 format.


MediaRecorder is designed for exactly this.
https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder_API

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-25 Thread Robert O'Callahan

On Thu, Mar 26, 2015 at 9:43 AM, Justin Novosad ju...@google.com wrote:

 createImageBitmap(myImageSource).then(function (image) {
 image.toBlob().then(admireYourShinyNewBlob); });


Oh, right. Sorry! Of course :-).

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-25 Thread Robert O'Callahan

On Wed, Mar 25, 2015 at 11:41 PM, Anne van Kesteren ann...@annevk.nl
wrote:

 On Fri, Mar 20, 2015 at 11:15 PM, Robert O'Callahan
 rob...@ocallahan.org wrote:
  My understanding is that the current consensus proposal for canvas in
  Workers is not what's in the spec, but this:
  https://wiki.whatwg.org/wiki/WorkerCanvas
  See Canvas in Workers threads from October 2013 for the discussion. svn
  is failing me but the CanvasProxy proposal in the spec definitely
 predates
  those threads.
 
  Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy
 stuff
  from the spec to avoid confusion.
 
  That proposal contains WorkerCanvas.toBlob, which needs to be updated to
  use promises.

 There's also https://wiki.whatwg.org/wiki/WorkerCanvas2 it seems. It
 would be interesting to know what the latest on this is.


Right now that appears to be just a copy of WorkerCanvas with some
boilerplate text added. I assume someone's working on it and will mention
it on this list if/when they're ready to discuss it :-).

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-25 Thread Robert O'Callahan

On Thu, Mar 26, 2015 at 8:40 AM, Justin Novosad ju...@google.com wrote:

 One idea we've been brewing that has not surfaced on this thread so far is
 that of having an ImageBitmap.toBlob(), which would return a Promise. Also,
 ImageBitmap should be transferable. This idea is just partially baked right
 now, but I wanted to throw it out into the open to get people thinking
 about it. The rationale behind this is that ImageBitmap is basically a
 lightweight read-only proxy for pretty much anything that can be used as an
 image source. So this single API entry-point can perform as well as a
 direct toBlob() method hanging off of any other type of object that an
 ImageBitmap can be created from. Because it is an opaque and immutable
 object, it gives browser vendors a lot of latitude for implementing high
 performance code paths. It allows pixel data from the DOM to cross Worker
 boundaries without making copies (at least in cases where the source is a
 static image). It allows interactions with blob storage to be driven from a
 worker (vs. canvas.toBlob), hence avoiding jank on the main thread. Raw
 image data generated in JS would not have to transit through a canvas (just
 create an ImageBitmap from an ImageData object), which eliminates a lot of
 unnecessary pixel pushing.


That makes sense to me.

If we do that, and also add HTMLCanvasElement.transferToImageBitmap, then
one-shot draw into a canvas and create a blob applications could have
really minimal memory usage. That method would detach the backbuffer and
clear the canvas (which I think all browsers are optimizing to no
backbuffer).

Eliminating the transit through canvas also
 allows for fast paths for capturing blobs from video, img, svg, URLs,
 to name a few.


With new APIs on those elements, you mean?

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-22 Thread Robert O'Callahan

I'm getting confused as to what the problem is.

To restate the advantages of async toBlob: suppose I have code that does
  drawStuffTo(context);
  context.canvas.toBlob().then(callback);
  // no further drawing to the canvas

This has a few advantages over using getImageData and then encoding to JPEG
via JS in a Worker:
1) getImageData will stall the main thread until all canvas rasterization
is finished and the data has been read back from GPU memory. toBlob lets
the main thread run while those things happen. This is the biggest issue.
2) The first stage of JPEG compression is to convert the data to YUV 4:2:0,
which can be done on the GPU and produces a buffer which is only half the
size, so even after you've copied it by reading it back this uses no more
CPU+GPU memory, less GPU-to-CPU bandwidth, and will probably be faster. You
may be able to do even more JPEG compression on the GPU, depending on the
underlying architecture.
3) Using the browser's native JPEG encoder is probably a bit faster and
lighter-weight than JS in a Worker, especially due to SIMD. Browsers must
already have a native JPEG encoder to support canvas.toDataURL().

In the case where someone does
  drawStuffTo(context);
  context.canvas.toBlob().then(callback);
  drawStuffTo(context);
the browser has a few options. After rasterizing the first set of commands
you can make a copy in GPU memory. Or, you can apply #2 above and make a
YUV 4:2:0 copy using half the memory. Or, you can start the readback, and
delay rasterizing the second set of commands until the readback has
finished. You can vary the strategy depending on the amount of memory
available.

If the browser is ever really low on memory, it can always stall everything
until JPEG encoding is done and intermediate buffers released. So there can
never be a memory-use advantage to making the promise fail.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-22 Thread Robert O'Callahan

On Sun, Mar 22, 2015 at 6:45 PM, Rik Cabanier caban...@gmail.com wrote:

 On Sat, Mar 21, 2015 at 1:44 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

  On Sun, Mar 22, 2015 at 7:16 AM, Rik Cabanier caban...@gmail.com
 wrote:
 
  Justin is worried that in order to make this asynchronous, Chrome has to
  create a snapshot of the canvas bits which is slow if it resides on the
  GPU.
  Of course, his workaround to use getImageData is just as slow since it
 has
  to do the same operation.
 
 
  One of the advantages of having a native async toBlob API is that the
  browser can asynchronously read back from GPU memory (when the graphics
 API
  permits this --- D3D11 does, at least). Gecko currently doesn't take
  advantage of this, however.
 

 You would need a copy in GPU memory first to do the async readback on.


Not necessarily.


 There are many scenarios (ie fullscreen hidi canvas) where this might fill
 the GPU's memory.


Unlikely in practice.

 To alleviate this, I have 2 proposals:
  - After calling toBlob, the canvas is read-only until the promise is
  fulfilled
  - If the user changes the canvas after calling toBlob, the promise is
  cancelled.
 
  Maybe we should only allow 1 outstanding toBlob per canvas element too.
 
 
  I don't think we should impose any of these restrictions. They're not
  necessary.
 

 How else would you avoid making a copy?


It depends on lots of variables, but there are certainly scenarios when you
can do async readback without making a copy.

Even if you have to make a copy in GPU memory, that's not a big problem
most of the time, not compared to doing a synchronous readback.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-21 Thread Robert O'Callahan

On Sat, Mar 21, 2015 at 5:45 PM, Rik Cabanier caban...@gmail.com wrote:

 Ah, OK. I thought we were changing it for both cases. This will cause a lot
 of confusion...


If we want to keep HTMLCanvasElement and WorkerCanvas in sync, we can.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-20 Thread Robert O'Callahan

On Sat, Mar 21, 2015 at 3:38 PM, Rik Cabanier caban...@gmail.com wrote:

 Do you know how many site use toBlob in Firefox?
 A quick search on github shows a very high number of pages [1] so it might
 be too late to change.

 Maybe you can keep the callback and return a promise?


None of them use WorkerCanvas.toBlob since we don't implement that yet.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas image to blob/dataurl within Worker

2015-03-20 Thread Robert O'Callahan

On Sat, Mar 21, 2015 at 1:13 AM, Jake Archibald jaffathec...@gmail.com
wrote:

 Receiving a push message results in a 'push' event within the
 ServiceWorker. The likely action at this point is to show a notification.
 It should be possible to generate an image to use as an icon for this
 notification (
 https://notifications.spec.whatwg.org/#dom-notificationoptions-icon).

 This could be a badge showing some kind of unread count, some combination
 of app icon  avatar, or even a default avatar (Google Inbox generates an
 avatar from the senders names first letter).

 This is also useful for generating images to go into the cache API, or
 transforming images as they pass through the ServiceWorker.

 API:

 Almost all the pieces already exist, except a way to get the image data of
 a CanvasRenderingContext2D into a format that can be read from a
 url. ImageBitmap seems like a good fit for such an API:

 var context = new CanvasRenderingContext2D(192, 192);

 Promise.all(
   caches.match('/avatars/ben.png')
 .then(r = r.blob())
 .then(b = createImageBitmap(b)),
   caches.match('/avatars/julie.png')
 .then(r = r.blob())
 .then(b = createImageBitmap(b)),
 ).then(([ben, julie]) = {
   context.drawImage(ben, 0, 0);
   context.drawImage(julie, 96, 96);
   return createImageBitmap(context);
 }).then(
   //  and here's the bit we're missing… 
   image = image.toDataURL()
 ).then(icon = {
   self.registration.showNotificaiton(Hello!, {icon});
 });


My understanding is that the current consensus proposal for canvas in
Workers is not what's in the spec, but this:
https://wiki.whatwg.org/wiki/WorkerCanvas
See Canvas in Workers threads from October 2013 for the discussion. svn
is failing me but the CanvasProxy proposal in the spec definitely predates
those threads.

Ian, unless I'm wrong, it would be helpful to remove the CanvasProxy stuff
from the spec to avoid confusion.

That proposal contains WorkerCanvas.toBlob, which needs to be updated to
use promises.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] HTML video and IE, Safari, Firefox, Chrome

2014-11-04 Thread Robert O'Callahan

On Wed, Nov 5, 2014 at 3:46 AM, Martin Hoernig hoer...@in.tum.de wrote:

 Thank you Rob!

  I'm not sure what this means:

  Firefox fires the canplaythrough event after buffering is completed or
 halted instead of a bandwidth depending solution

  Do you mean that even when the data is arriving at a very high rate, we
 don't fire canplaythrough until all the data is downloaded?

 Yep.


I have a fix for that bug now. Thanks again.



  You might want to test canplaythrough a bit more carefully; your tests
 didn't find http://code.google.com/p/chromium/issues/detail?id=73609
 (canplaythrough is basically unimplemented in Chrome; Chrome fires it
 immediately after canplay in all circumstances, which causes problems for
 other browsers).

 We have performed most of our tests in high bandwidth scenarios (Since
 we know the network bit rate and the video bit rate, we can take the
 canplaythrough event as mandatory as well.). The only situation in which a
 difference exists is (4) reset, where playing has to be fired after canplay
 and right before canplaythrough. But, you are right, additional low
 bandwidth tests are a good addition. I'm thinking for example of a stall
 test (The stall timeout is a user-agent defined length of time, which
 should be about three seconds.)


Low-bandwidth tests are particularly interesting to test because browser
developers tend to operate in high-bandwidth environments so don't pay
enough attention to low-bandwidth environments :-).

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] HTML video and IE, Safari, Firefox, Chrome

2014-11-03 Thread Robert O'Callahan

Thanks for the testing! Please file bugs against browsers where you feel
they're not following the spec.

I'm not sure what this means:

 Firefox fires the canplaythrough event after buffering is completed or
 halted instead of a bandwidth depending solution

Do you mean that even when the data is arriving at a very high rate, we
don't fire canplaythrough until all the data is downloaded?

You might want to test canplaythrough a bit more carefully; your tests
didn't find http://code.google.com/p/chromium/issues/detail?id=73609
(canplaythrough is basically unimplemented in Chrome; Chrome fires it
immediately after canplay in all circumstances, which causes problems for
other browsers).

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] HTML video and IE, Safari, Firefox, Chrome

2014-11-03 Thread Robert O'Callahan

On Tue, Nov 4, 2014 at 10:54 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 I'm not sure what this means:

 Firefox fires the canplaythrough event after buffering is completed or
 halted instead of a bandwidth depending solution

 Do you mean that even when the data is arriving at a very high rate, we
 don't fire canplaythrough until all the data is downloaded?


I guess that's what you mean, since I can reproduce something like that
locally. Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1093399.
Thanks!!!

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] New feature: better integration with browser find interface

2014-10-29 Thread Robert O'Callahan

On Thu, Oct 30, 2014 at 9:15 AM, Peter Kasting pkast...@google.com wrote:

 If browsers had to retry open finds every time the page content changed,
 then leaving one's find bar open could have very large negative performance
 effects, even if the browser focused only on the modified pieces of the
 page.  Is there an implementation idea I'm missing for how browsers could
 fix this bug in a performant way?  Otherwise I can't see us doing what it
 seems like you want.


It seems possible to me:
1) You can do it lazily, during idle time (though some apps don't have any)
2) You can do it incrementally
3) You can start with the visible part of the page
Having said that, it would be complex enough I don't know if it would ever
be worth implementing.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Gapless playback problems with web audio standards

2014-10-27 Thread Robert O'Callahan

On Sun, Oct 26, 2014 at 2:30 AM, David Kendal m...@dpk.io wrote:

 Hi,

 http://w3.org/mid/10b10a1d-8b84-4015-8d49-a45b87e4b...@dpk.io


Web Audio should be usable for this. If it doesn't work due to brower bugs,
we should fix the browser bugs. I know we've fixed bugs related to this in
Firefox this year so I'd like to know if obvious techniques still don't
work.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Gapless playback problems with web audio standards

2014-10-27 Thread Robert O'Callahan

On Tue, Oct 28, 2014 at 1:13 AM, David Kendal m...@dpk.io wrote:

 On 27 Oct 2014, at 12:08, Robert O'Callahan rob...@ocallahan.org wrote:

  Web Audio should be usable for this. If it doesn't work due to brower
 bugs, we should fix the browser bugs. I know we've fixed bugs related to
 this in Firefox this year so I'd like to know if obvious techniques still
 don't work.

 Can you suggest an ‘obvious technique’ I don’t describe in my email? Given
 that none of the techniques there work even remotely directly with samples,
 but rather with approximate durations, I don’t see how it’s possible to get
 sample-accurate transitions.


Durations are floating point, and browsers should round them to the nearest
sample boundary, so there shouldn't be a problem there.

If you have a set of AudioBuffers that you want to play gaplessly, you
should just be able to do this:
var audioBuffers = ...;
var audioContext = new AudioContext();
var t = 0;
for (var i = 0; i  audioBuffers.length; ++i) {
  var node = audioContext.createBufferSource();
  node.buffer = audioBuffers[i];
  node.start(t);
  t += buffer.duration;
}

For best results the audioBuffers' sample rate should match the sample rate
of audioContext, though I think Firefox should work even when that's not
the case.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Support filters in Canvas

2014-09-30 Thread Robert O'Callahan

On Tue, Sep 30, 2014 at 12:45 AM, Rik Cabanier caban...@gmail.com wrote:

 Are you proposing that this is developed as a separate spec and not as an
 addition to the canvas specification?


No, not really.

I want to make sure that we have a pathway to standardize SVG filter
support in canvas. If we restrict the canvas spec to features for which
multiple implementations are imminent, and SVG filters aren't such a
feature, then we'll need another way to record and discuss proposals for
SVG filters in canvas.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Support filters in Canvas

2014-09-29 Thread Robert O'Callahan

On Mon, Sep 29, 2014 at 2:12 PM, Rik Cabanier caban...@gmail.com wrote:

 Can we limit it to just the set of CSS filter shorthands for now?
 I think other UA's are further behind in their implementation of
 integrating SVG filters in their rendering pipeline.


How about we put CSS + SVG filters into the one spec draft for now, and
split them up into two drafts if we actually get to the point where vendors
want to ship one and not the other? It seems premature to create both HTML
Canvas Filters Level 1 and HTML Canvas Filters Level 2 at the same time.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] [2D Canvas] Proposal: batch variants of drawImage

2014-08-04 Thread Robert O'Callahan

It looks reasonable to me.

How do these calls interact with globalAlpha etc? You talk about
decomposing them to individual drawImage calls; does that mean each image
draw is treated as a separate composite operation?

Currently you have to choose between using a single image or passing an
array with one element per image-draw. It seems to me it would be more
flexible to always pass an array but allow the parameters array to refer to
an image by index. Did you consider that approach?

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] Canvas-Only Document Type

2014-07-07 Thread Robert O'Callahan

If you want jank-free canvas rendering, you probably really want WebGL
drawing from a Worker.

Rob
-- 
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

Re: [whatwg] High-density canvases

2014-07-02 Thread Robert O'Callahan

On Thu, Jul 3, 2014 at 3:03 AM, Justin Novosad ju...@google.com wrote:

 On Fri, Jun 27, 2014 at 6:42 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

 This behavior seems sound at first glance, but because that arithmetic may
  induce a change to the intrinsic aspect ratio due to rounding, step 3)
 may
  not jump out of the loop during the second iteration, thus allowing the
  feedback loop to keep spinning.
 
 
  What are the specified values of CSS 'width' and 'height' on the canvas
 in
  your example?
 

 Anything but 'auto'.


If both 'width' and 'height' are not 'auto' then changing the canvas width
and height will not trigger a change in the canvas content-box, so
renderedPixelWidth/Height should not change, so there's no loop here.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-27 Thread Robert O'Callahan

On Sat, Jun 28, 2014 at 3:41 AM, Justin Novosad ju...@google.com wrote:

 Example of a problematic renderedsizechange event listener:

 canvas.width = Math.floor(canvas.renderedPixelWidth * myPixelScale);
 canvas.height = Math.floor(canvas.renderedPixelHeight * myPixelScale);


I don't know why anyone would actually want to do this.

This behavior seems sound at first glance, but because that arithmetic may
 induce a change to the intrinsic aspect ratio due to rounding, step 3) may
 not jump out of the loop during the second iteration, thus allowing the
 feedback loop to keep spinning.


What are the specified values of CSS 'width' and 'height' on the canvas in
your example?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-26 Thread Robert O'Callahan

On Fri, Jun 27, 2014 at 2:00 AM, Justin Novosad ju...@google.com wrote:

 Hadn't thought of that. object-fit seems smells dangerous. I think we may
 need to define explicit behaviors for renderedPixelWidth/Height for the
 different object fit modes.


I don't think so. Given renderedPixelWidth/Height returns the size of the
content box, and the element's CSS width and height are not 'auto', then
renderedPixelWidth/Height are not affected by object-fit or the intrinsic
size, so there is no feedback loop.

For example, with 'object-fit: contains', will the renderedPixelWidth/Height
 be computed in a way to fill the element's content area, or will it
 preserve the aspect ratio of the original intrinsic size?


The former.


 Also, with object fit triggering a renderedsizechange event, the event
 listener will presumably change the intrinsic size of the canvas, which
 will invalidate style (because the object-fit computation depends on the
 intrinsic size), and that causes a style invalidation feedback loop.


We don't implement object-fit in Gecko yet, but when we do the change to
the intrinsic size will trigger a relayout that will end up not change
anything so there is no feedback loop.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-25 Thread Robert O'Callahan

On Thu, Jun 26, 2014 at 2:49 AM, Justin Novosad ju...@google.com wrote:

 I was wondering if there is anything we can do with this feature to prevent
 web apps from creating layout feedback loops.  What I mean is that if the
 event listener for renderedsizechange changes the layout of the page in a
 way that does not preserve renderedPixelWidth/Height, we can get into a lot
 of trouble. Even if the resizing arithmetic in the event listener is sound,
 if the rounding logic does not match exactly what the browser does,
 fractional scales may cause instabilities.

 The main source of of the problem is that intrinsic size affects CSS size
 when width or height are set to 'auto', which is the default.  Some ideas
 that come to mind for breaking the feedback loop:
 1) renderedPixelWidth/Height return the intrinsic size when CSS
 width/height are set to 'auto'
 2) renderedPixelWidth/Height do not change when CSS width/height are set to
 'auto'


We could do this and I think it would work. It's arguable whether the
complexity is justified, but I'd be OK with doing it.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-25 Thread Robert O'Callahan

On Thu, Jun 26, 2014 at 10:10 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 26, 2014 at 2:49 AM, Justin Novosad ju...@google.com wrote:

 I was wondering if there is anything we can do with this feature to
 prevent
 web apps from creating layout feedback loops.  What I mean is that if the
 event listener for renderedsizechange changes the layout of the page in a
 way that does not preserve renderedPixelWidth/Height, we can get into a
 lot
 of trouble. Even if the resizing arithmetic in the event listener is
 sound,
 if the rounding logic does not match exactly what the browser does,
 fractional scales may cause instabilities.

 The main source of of the problem is that intrinsic size affects CSS size
 when width or height are set to 'auto', which is the default.  Some ideas
 that come to mind for breaking the feedback loop:
 1) renderedPixelWidth/Height return the intrinsic size when CSS
 width/height are set to 'auto'
 2) renderedPixelWidth/Height do not change when CSS width/height are set
 to
 'auto'


 We could do this and I think it would work. It's arguable whether the
 complexity is justified, but I'd be OK with doing it.


To clarify, I think a version of option #1 would be the best way to go.
renderedPixelWidth would return the current canvas buffer width when the
canvas element's CSS specified value for 'width' is 'auto', and similar for
height.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-25 Thread Robert O'Callahan

On Thu, Jun 26, 2014 at 10:13 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 To clarify, I think a version of option #1 would be the best way to go.
 renderedPixelWidth would return the current canvas buffer width when the
 canvas element's CSS specified value for 'width' is 'auto', and similar for
 height.


Actually, that would break some use-cases involving object-fit, so instead
renderedPixelWidth/Height should return the current canvas buffer width
only when *both* 'width' and 'height' are 'auto'.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-24 Thread Robert O'Callahan

On Wed, Jun 25, 2014 at 3:30 PM, Rik Cabanier caban...@gmail.com wrote:

 Add a new event renderedsizechange to HTMLCanvasElement. This event does
 not bubble and is not cancelable. Whenever the value that would be returned
 by renderedPixelWidth orrenderedPixelHeight changes, queue a task to fire
 renderedsizechange at the HTMLCanvasElement if there is not already a task
 pending to fire such an event at that element.


 - If there's a transition or animation that affects the canvas element,
 should it receive resize events at 60fps?


UA dependent. In practice I would change the renderedPixelWidth/Height
properties whenever Gecko rerasterizes the surrounding CSS content with a
new resolution.


 - will CSS 3D transforms affect the rendered canvas size? If so, what would
 the optimal resolution be if there's a rotation


UA dependent. Again, I would change the renderedPixelWidth/Height
properties whenever Gecko rerasterizes the surrounding CSS content with a
new resolution, based on that resolution.


 - what happens if the canvas element is not in the document? Will it just
 return the pixel width/height?


UA dependent. I'd probably just leave it unchanged while it's not in the
document (and assume 1 CSS pixel per device pixel if it's never been in a
document with a screen presentation).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-23 Thread Robert O'Callahan

On Tue, Jun 24, 2014 at 11:44 AM, Kenneth Russell k...@google.com wrote:

 If there's no concern about potential duplicated functionality then
 let's add the attributes to the canvas element. They'll be easier for
 developers to understand and easier to verify as obviously correct.

 How should we proceed? Would you add an entry to
 http://wiki.whatwg.org/wiki/Category:Proposals ?


I'll do that now.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-23 Thread Robert O'Callahan

On Tue, Jun 24, 2014 at 12:27 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 I'll do that now.


Done.
http://wiki.whatwg.org/wiki/CanvasRenderedPixelSize

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] SVG cloning elements from HTML5

2014-06-23 Thread Robert O'Callahan

On Thu, Jun 19, 2014 at 4:33 AM, Tab Atkins Jr. jackalm...@gmail.com
wrote:

 Yes, increasing the set of name-alikes between html and svg is absolutely
 not something we'll support. (I've unfortunately been out of the loop with
 the svgwg for a while, or else I would have prevented this from showing up
 in the first place.)

 Allowing html directly in svg is definitely the right answer. Parsing
 shouldn't be too hard, and defining the layout model would be pretty
 trivial.


I think we should actually figure this out.

I'm not an expert on the HTML parser, but it seems to me there's already
some support for what we need. E.g. given
!DOCTYPE HTML
svg
span id=s/span
div id=i/div
the elements s and i are put in the HTML namespace already.

For layout, we could do this:
1) When an HTML element is a child of an SVG element, perform CSS layout of
the HTML element treating the nearest SVG viewport as the containing block.
Its user-space width and height become the width and height of the
containing block in CSS pixels.
2) Treat such HTML elements as position:relative.
3) Add x and y attributes to HTMLElement and have them map to left
and top CSS properties. (There's a crazy legacy compat issue for
HTMLImageElement.x/y but we can hack around that.)
4) Add a transform attribute to HTMLElement and have it map to the
transform CSS property.

#3 and #4 are optional, there are pros and cons to having them. Using the
nearest SVG viewport in #1 is because other SVG container elements don't
have a defined size (and we can't use getBBox because that depends on the
layout of the children).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-19 Thread Robert O'Callahan

On Fri, Jun 20, 2014 at 7:32 AM, Justin Novosad ju...@google.com wrote:

 +1 from me too. But I have one concern in terms of future proofing, because
 I would like to keep the door open for auto-resizing as a possible future
 improvement. In an eventual auto-resize proposal, we will want to make
 the preferredsizechange
 event cancelable.


Why?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-19 Thread Robert O'Callahan

On Fri, Jun 20, 2014 at 6:06 AM, Kenneth Russell k...@google.com wrote:

 It is a little unfortunate that a canvas-specific solution is needed.
 Is it known whether document.getBoxQuads solves this problem in
 Firefox?


It doesn't.

Gecko (and I assume other engines) typically snaps CSS box edges to device
pixel edges at render time, so that CSS backgrounds and borders look crisp.
So if a canvas has a CSS background exactly filling its content box, our
snapping of the content box is what determines the ideal device pixel size
of the canvas buffer.

Authors can estimate the canvas device pixel size using getBoxQuads to get
the canvas content box in fractional CSS pixels and multiplying by
devicePixelRatio (assuming no CSS transforms or other effects present). But
if that estimate results in a non-integral device pixel size, there's no
way for authors to know how we will snap that to an integral size.

We could move these attributes up the element hierarchy, or better still to
a new generic API that works on all elements (e.g. some variant of
getBoxQuads). (CSS fragmentation might split an element into multiple boxes
with different sizes.) I don't know of any good use-cases for that, but
maybe there are some?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

[whatwg] SVG cloning elements from HTML5

2014-06-18 Thread Robert O'Callahan

I just discovered
https://svgwg.org/svg2-draft/embedded.html
This looks very problematic. It ties SVG and HTML5 together in
uncomfortable ways. For example, SVGIframeElement as specced has two
width attributes. It's unclear how to keep this making sense as SVG and
HTML5 evolve, e.g. as new attributes get added to HTML5 elements. It will
be easy to change HTML5 in ways that break this SVG spec (and possibly vice
versa). SVGIframeElement implements HTMLIFrameElement creates a multiple
inheritance situation that I don't think any engine would really want to
implement, even if it made sense, which I don't think it does. I'm pretty
sure no-one who actually works on HTML5 has approved of this. Yes, it's
only in a draft, but people are already taking that as a cue to start
implementing. I think we need to take a step back from this, figure out
what the goals are and then work together on solving them the right way.

It seems to me that if we possibly can, we should solve this by reusing the
actual HTML elements so much less spec synchronization is needed. We
probably should find a way to support actual HTML elements as direct
children of SVG elements with no intervening foreignObject. We should
make sure the parser can handle that too, at least for a whitelisted set of
elements.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-18 Thread Robert O'Callahan

On Thu, Jun 19, 2014 at 3:30 AM, Justin Novosad ju...@google.com wrote:

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer.


There must be a cliff with that approach, where if you issue a sufficient
number of drawing commands without clearing the canvas you have to
rasterize, otherwise memory usage will grow without bound. That seems like
a problem for authors. I think it's reasonably easy for authors to
understand and control canvas memory usage at the moment, and I'd like to
not make it worse.

I can see that the command buffer approach would be useful for many
applications though.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-18 Thread Robert O'Callahan

On Thu, Jun 19, 2014 at 11:52 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Thu, Jun 19, 2014 at 3:30 AM, Justin Novosad ju...@google.com wrote:

 I am currently trying an experimental approach where canvases that are
 drawn to, but never read from (no toDataURL or getImageData calls) have
 their contents stored as a command buffer, rather than a pixel buffer.


 There must be a cliff with that approach, where if you issue a sufficient
 number of drawing commands without clearing the canvas you have to
 rasterize, otherwise memory usage will grow without bound. That seems like
 a problem for authors. I think it's reasonably easy for authors to
 understand and control canvas memory usage at the moment, and I'd like to
 not make it worse.


Also, you would have the problem of how correctly size temporary offscreen
canvases and variable-resolution image assets that Ian's approach had. And
without adding new API, there remains the problem of getImageData needing
to return 1 pixel per canvas coordinate unit for compatibility, thus
requiring some kind of getImageDataHD for authors who want more ... thus
requiring new API.

So while I applaud experimentation with retaining canvas command buffers, I
don't want it to block consideration of alternatives for explicit canvas
buffer sizing.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] High-density canvases

2014-06-12 Thread Robert O'Callahan

[Resurrecting old thread]

On Tue, Sep 10, 2013 at 12:00 PM, Ian Hickson i...@hixie.ch wrote:

 It would be nice to fix these all at once, and I think we can, by
 introducing a configuration option on getContext(), in the style of WebGL:

getContext('2d', { density: 'autosize' });

 This would trigger the following behaviour: When the context is created,
 and subsequently when the canvas changes size (e.g. due to being sized
 with CSS relative units and the element they're relative to changing), or
 when the display density changes size (e.g. due to page zoom), then:

- the width and height of the canvas bitmaps get updated to match the
  new native size of the canvas, at native density.

- the coordinate space of the canvas (context.width/context.height)
  gets updated to match the size of the canvas in CSS pixel units.

- a 'resize' event gets fired at the canvas.

 We would dump the *HD versions of the methods, and make the regular ones
 go back to returning the actual raw pixels, since that would now work fine
 and still provide HD-quality content everywhere it's available.

 What do people think?


I think I'd rather not take control of canvas resizing away from
applications, even opt-in. That leads to complexity such as extra API for
slaving other canvases. I also think we'd be better off sticking to the
invariant that the units of canvas coordinate space are single pixels in
the canvas bitmap; I think that simplifies things for implementers and
authors.

Here's an alternative proposal which I think is a bit simpler and more
flexible:
Expose two new DOM attributes on HTMLCanvasElement:
readonly attribute long preferredWidth;
readonly attribute long preferredHeight;
These attributes are the UA's suggested canvas size for optimizing output
quality. It's basically what Ian's proposal would have set as the automatic
size. We would also add a preferredsizechange event when those attributes
change.

Applications that want DPI-aware canvases would read those attributes and
use them to set the size, just once if they only want to paint once, during
each requestAnimationFrame callback for games and other animations, and in
the preferredsizechange event handler if they are willing to paint multiple
times but aren't actually animating. The application would be responsible
for scaling its output to the new canvas coordinate space size. Giving the
application control of the size change simplifies things for the browser
and gives applications maximum flexibility, e.g. resizing ancillary
resources as needed, or imposing constraints on the chosen size.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Proposal: toDataURL “image/png” compression control

2014-05-31 Thread Robert O'Callahan

On Sat, May 31, 2014 at 3:44 AM, Justin Novosad ju...@google.com wrote:

 My point is, we need a proper litmus test for the just do it in script
 argument because, let's be honnest, a lot of new features being added to
 the Web platform could be scripted efficiently, and that does not
 necessarily make them bad features.


Which ones?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] ImageBitmap feature requests

2014-05-18 Thread Robert O'Callahan

On Sat, May 17, 2014 at 4:18 AM, Anne van Kesteren ann...@annevk.nl wrote:

 Maybe we should have img.srcObject similar to what we're doing for
 media elements. img.src can simply return about:imagebitmap or some
 such. That way you can also assign a Blob to an img element without
 having to do the weird createObjectURL() hack that might leak memory
 if you're not careful.


I like this approach, but I think it's simpler to continue to have
HTMLImageElement.src reflect the src content attribute.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] addPath and CanvasPathMethods

2014-03-20 Thread Robert O'Callahan

On Fri, Mar 21, 2014 at 8:15 AM, Justin Novosad ju...@google.com wrote:

 Sorry for the confusion, the point I was trying to make was unrelated to
 the CTM question (almost). My point is that the tesselation of a path is
 something that can be cached in a Path2D object.

 If you do this, you can take advantage of the cached tessellation:
 (apply tranform 1 to ctx)
 ctx.fill(path1)
 (apply tranform 2 to ctx)
 ctx.fill(path2)

 If you do it this way, the aggregated path needs to be re-tesselated each
 time because the winding rule would need to be re-applied:
 (apply tranform 1 to ctx)
 ctx.addPath(path1)
 (apply tranform 2 to ctx)
 ctx.addPath(path2)
 ctx.fill();

 Technically, these two ways of drawing are not equivalent (depends on
 compositing mode, transparency, and winding rule, overlaps between paths),
 but they can be used to achieve similar things.  Nonetheless the second way
 is detrimental to performance, and we'd be encouraging it by providing an
 addPath method on the context.  Besides, if the dev really needs to add
 paths together, it can be done inside an intermediate path object.


An implementation can turn #2 into #1 if the paths obviously don't overlap.
If they might overlap, the author probably shouldn't be doing the latter!
TBH I don't see why authors would choose the latter approach.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Hit regions and events

2014-03-05 Thread Robert O'Callahan

On Wed, Mar 5, 2014 at 5:11 PM, Rik Cabanier caban...@gmail.com wrote:

 On Tue, Mar 4, 2014 at 6:51 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:
  The question remains: what should happen in Rik's example? More
 generally,
  is this event rerouting supposed to be able to trigger browser default
  event handling behavior, or only DOM event dispatch?

 The spec implies that default event handling is supposed to take place. ie:

 With a control, they can make routing events to DOM elements automatic,
 allowing e.g. clicks on a canvas to automatically submit a form via a
 button element.


 Likewise, if the region is backed by an a element that represents a
 hyperlink, clicking on that region should navigate you to that link.
 I don't really understand why that would make the implementation more
 difficult though. Wouldn't we get this for free by retargeting the event?


The problem is that if these retargeted events can trigger default browser
behavior, the browser has to be able to compute the position of the event
relative to the new target DOM node, and it's not clear how to do that.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Hit regions and events

2014-03-04 Thread Robert O'Callahan

On Wed, Mar 5, 2014 at 12:53 PM, Ian Hickson i...@hixie.ch wrote:

 On Fri, 28 Feb 2014, Rik Cabanier wrote:
  For instance, if the fallback is an edit control and the user
  drag-selects some text on the canvas, is it expected that this text is
  also selected in the edit control?

 You can't validly include a text field in canvas fallback precisely
 because of this kind of thing. See:

http://whatwg.org/html#best-practices


The question remains: what should happen in Rik's example? More generally,
is this event rerouting supposed to be able to trigger browser default
event handling behavior, or only DOM event dispatch?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Proposal: requestBackgroundProcessing()

2014-02-20 Thread Robert O'Callahan

On Fri, Feb 21, 2014 at 11:32 AM, Ashley Gullen ash...@scirra.com wrote:

 One solution is to make web workers able to access many of the same APIs
 the UI thread can, especially with WebGL, Web Audio, WebRTC and rAF. Then
 it might be practical to move a full game engine in to a worker. If that is
 planned, I guess there is no need for a new feature.


Having (some form of) all these APIs available in a Worker is a long-term
goal, but as Glenn just pointed out, it sounds like you could just run the
server in a Worker and the rest of the engine on the main thread.

Having said that, your server would want WebRTC DataChannels on the worker,
and we don't have that yet. You can work around it by proxying messages to
the main thread; postMessage from a Worker to the main thread is usually
not throttled.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

[whatwg] HTML fieldset rendering

2014-01-15 Thread Robert O'Callahan

The rendering of fieldsets is under-specified at the moment, even taking
into account
http://www.whatwg.org/specs/web-apps/current-work/multipage/rendering.html#the-fieldset-and-legend-elements
.

1) No spec describes how browsers move the top border of the fieldset down
by about half the height of the rendered legend.

2) It's unclear how 'overflow' should work on a fieldset.
2a) For overflow:hidden, Blink seems to clip to something outside the
fieldset's border (actually I think it clips to the inside of where the
border would be if the legend hadn't forced the top edge to move down), and
clips the rendered legend. Gecko clips to the inside of the fieldset's
border and does not clip the legend.
(I think a strict reading of the spec would suggest that you should clip to
the inside of the border and the legend should be clipped like any other
descendant, but that would make overflow:hidden useless with legends, and
it would break existing content.)
2b) For overflow:auto/scroll, the scrollport corresponds to the cliprect
for 'overflow:hidden' in both browsers. Blink scrolls all the children of
the fieldset (but not its border). (This looks very strange when the legend
scrolls away from the border.) Gecko scrolls everything except the rendered
legend.

3) It's unclear how absolutely positioned children work when a fieldset is
an abs-pos container.
Blink and Gecko agree that the abs-pos container bounds for absolutely
positioned regular children as well as abs-pos descendants of the rendered
legend match the cliprect used for overflow:hidden (though they disagree on
what that is), and that if the fieldset is scrollable then all abs-pos
descendants scroll with it.
However, Gecko's behavior here is just broken when an abs-pos child of the
rendered legend has auto left/top and you scroll the fieldset, and it's
deeply unsatisfying to have the grandchild of the fieldset be scrolled by
the fieldset while the parent in the middle is not.

Blink's behavior is simple from an implementation point of view: you
pretend the top border-edge didn't move down (well, except for background
positioning and maybe other things) and just draw the top edge in a
different place, and everything else is unchanged. Gecko's behavior is much
more like you've actually moved the top border-edge down, but that approach
requires you to exclude the rendered legend from overflow clipping and
scrolling, and gets tricky when the fieldset is an abs-pos container. OTOH
for scrolling fieldsets Blink's behavior is a bit farcical.

I'm not sure what to do.

Rob
--
Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w

Re: [whatwg] Proposal: downsample while decoding image blobs in createImageBitmap()

2013-12-18 Thread Robert O'Callahan

Yes. Especially on these devices, JPEG decoding is undue latency.

Also, Rik's suggestion makes an ImageBitmap just a thin wrapper around a
Blob, with decoding performed every time we do a drawImage call. That's bad
for performance in many cases. David's proposal lets the author be a little
bit more explicit and the UA doesn't have to guess what to do.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Questions regarding Path object

2013-11-13 Thread Robert O'Callahan

On Wed, Nov 13, 2013 at 12:12 PM, Jussi Kalliokoski 
jussi.kallioko...@gmail.com wrote:

 Path is also too generic even in the context of graphics. If we later on
 want to add a path object for 3-dimensional paths, you end up with Path and
 Path3D? Yay for consistency. Path2D would immediately inform what
 dimensions we're dealing with and also that this is to do with graphics,
 and thus sounds like a good name to me.


Sounds good to me.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] CanvasRenderingContext2D with addPath, currentPath

2013-11-04 Thread Robert O'Callahan

If you return a path in user-space, what do you get if you call
getCurrentPath with a singular transform?
  ctx.moveTo(0,0);
  ctx.lineTo(1,1);
  ctx.scale(0,0);
  var p = ctx.getCurrentPath();
?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] CanvasRenderingContext2D with addPath, currentPath

2013-11-02 Thread Robert O'Callahan

On Sat, Nov 2, 2013 at 1:01 PM, Rik Cabanier caban...@gmail.com wrote:

 The latest Safari is shipping currentPath and Blink has implemented it
 behind a runtime flag.
 Could we put this in the specification?

 the proposal is for a new attribute on the 2d canvas rendering context:

 attribute Path currentPath;


 This attribute would give you a non-live/copy of the current path in device
 space.


Does this mean that ctx.currentPath != ctx.currentPath? That's bad!

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] CanvasRenderingContext2D with addPath, currentPath

2013-11-02 Thread Robert O'Callahan

On Sun, Nov 3, 2013 at 3:03 PM, Rik Cabanier caban...@gmail.com wrote:

 On Sat, Nov 2, 2013 at 1:01 AM, Robert O'Callahan rob...@ocallahan.orgwrote:

 Does this mean that ctx.currentPath != ctx.currentPath?


 Yes


  That's bad!


 Why would it be bad (apart from being different)?


It means that currentPath isn't behaving anything like a data member.

I'm not super familiar with why this became such an important design
principle, but I know that it is one. For example, this is exactly the
reason why the WebIDL spec prohibits attributes from returning dictionaries.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w

Re: [whatwg] Canvas in workers

2013-10-24 Thread Robert O'Callahan

We talked through this proposal with a lot of Mozilla people in a meeting
and collectively decided that we don't care about the case of workers that
commit multiple frames to a canvas without yielding --- at least for now.
So we want to remove commit() and copy the main-thread semantics that a
canvas frame is eligible for presentation whenever script is not running in
the worker.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread

On Tue, Oct 22, 2013 at 12:39 AM, Glenn Maynard gl...@zewt.org wrote:

 Using ImageBitmap for this has a lot of issues.  It requires synchronizing
 with scripts in the UI thread.  It requires manualling resize your canvas
 repeatedly to fit different destinations.  It also may potentially create
 lots of backbuffers. Here's an example of code using ImageBitmap
 incorrectly, leading to excess memory allocation:

 function render()
 {
 var canvas = myWorkerCanvas;
 renderTo(canvas);
 var buffer = canvas.transferToImageBitmap();
 postMessage(buffer);
 }
 setTimeout(render, 1);



This code actually does something potentially useful which can't easily be
done with attachToCanvas: generating a series of images as fast as possible
which will be processed on another thread in some way other than just
rendering them on the screen. (E.g., be encoded into an animated image or
video file.)

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread

On Tue, Oct 22, 2013 at 4:37 PM, Glenn Maynard gl...@zewt.org wrote:

 On Tue, Oct 22, 2013 at 2:48 AM, Robert O'Callahan rob...@ocallahan.org
 wrote:

  This code actually does something potentially useful which can't easily
 be
  done with attachToCanvas: generating a series of images as fast as
 possible
  which will be processed on another thread in some way other than just
  rendering them on the screen. (E.g., be encoded into an animated image or
  video file.)
 

 (This is a proposal for attachToCanvas--now setContext--not against
 transferToImageBitmap, if there are use cases that transferToImageBitmap
 solves best in its own right.  It seems like toBlob already handles this,
 though.)


It doesn't, because toBlob encodes to an image format.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

On Tue, Oct 22, 2013 at 7:31 PM, Kenneth Russell k...@google.com wrote:

 Robert, please don't remove those APIs from your proposal. They're
 needed in order to address known use cases, and splitting them off
 will make it difficult to understand how they interact with
 WorkerCanvas later.


Yes, I think it's a good idea to specify a complete set of APIs that fit
together logically and if there are some we don't need, we can remove them
or just delay implementing them until they're needed.

I would like to suggest changing the 'srcObject' property on
 HTMLImageElement into some sort of method taking ImageBitmap as
 argument. If an ImageBitmap had been previously set against the
 HTMLImageElement, the method would automatically call 'close()'
 against it. Fundamentally there should be some easy way to repeatedly
 update the HTMLImageElement without having to query its previous
 ImageBitmap and call close() against it before setting srcObject.


Hmm. I'm not sure how this should work.

Maybe instead we should use canvas elements and define
ImageBitmap.transferToCanvas(HTMLCanvasElement). That would mitigate
Glenn's argument about having to change element types too.

Would you consider copying
 https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal to
 http://wiki.whatwg.org/wiki/Category:Proposals so that it's easier to
 collaborate on it?


No problem at all. Can you do it? I need to get a WHATWG account :-).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

I got an account and I'm uploading the proposal now.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

On Tue, Oct 22, 2013 at 10:44 PM, Robert O'Callahan rob...@ocallahan.orgwrote:

 No problem at all. Can you do it? I need to get a WHATWG account :-).


OK, I added the proposal here:
http://wiki.whatwg.org/wiki/WorkerCanvas
A couple of changes from the previous version:
-- Added ImageBitmap.transferToImage.
-- Removed HTMLImageElement.srcObject (though it may come back in other
contexts)
-- Made ImageBitmap Transferable. This makes it possible to ensure the
worker that generated an ImageBitmap does not keep alive a reference to the
buffer.
-- Removed ImageBitmaps.close() since the use-cases driving this proposal
probably don't need it anymore.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 4:17 PM, Glenn Maynard gl...@zewt.org wrote:

 On Thu, Oct 17, 2013 at 10:25 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:
  On Fri, Oct 18, 2013 at 3:10 PM, Glenn Maynard gl...@zewt.org wrote:
   Also, with the transferToImageBuffer approach, if you want to render
  from a worker into multiple canvases in the UI thread, you have to post
  those ImageBuffers over to the main thread each frame, which has the
 same
  (potential) synchronization issues as the transferDrawingBufferToCanvas
  proposal.
 
  What are those issues? You can do a single postMessage passing a complete
  set of ImageBItmaps.
 

 See
 http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Oct/0193.html
 .
 I don't know the answer to this; my feeling is that posting to the UI
 thread and scripts in the UI thread may or may not have
 (performance/smoothness) issues, but doing it all in the worker avoids any
 potential for this issue.


I'm confused here. You said if you want to render from a worker into
multiple canvases in the UI thread, which I took to mean that you wanted
to synchronize canvas updates from workers with DOM changes made by the UI
thread. But now you're saying you don't want to do that. So I don't know
what you meant.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 6:50 AM, Rik Cabanier caban...@gmail.com wrote:

 Extra methods on the canvas API:

 Promise setTaskScript(DOMString script); // can error be in promise?
 Promise executeTask(DOMString id, dictionary json, boolean synchronized =
 true); // Transferable elements allowed in dictionary

 Object that is active in the task:

 interface CanvasTask {

 HTMLCanvasElement createCanvas(unsigned long width, unsigned long height);
 attribute Function onTask;

 }

 CanvasTask implements HTMLCanvasElement;


It looks like you intend CanvasTask to be the global object for the task
script? So it's not a Worker and you don't get anything from
WorkerGlobalScope? That's extremely limiting and also adds a lot of
complexity by introducing a new kind of script global. You really would
want to reuse workers here.

Also, making the HTMLCanvasElement API accessible from non-main threads is
a big no-no. You can't let people do CanvasTask.document.window.whatever
from non-main threads.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread

On Mon, Oct 21, 2013 at 4:46 AM, Glenn Maynard gl...@zewt.org wrote:

 On Sun, Oct 20, 2013 at 9:26 PM, Kyle Huey m...@kylehuey.com wrote:

  On Sun, Oct 20, 2013 at 11:33 PM, Glenn Maynard gl...@bluegoji.com
 wrote:
 
  It must not be possible for the UI thread to detect whether present()
 did
  anything--if there's no frame in the ready buffer, nothing changes and
 the
  UI thread can't detect this.  Similarly, it must not be possible for the
  rendering thread to detect if the ready frame has been presented.  These
  rules are to prevent exposing asynchronous behavior.
 
 
  Well you can readback from canvases, so how is that going to work?

 However it works today, since CanvasProxy needs the same thing.  If a
 CanvasProxy/WorkerCanvas exists for a canvas, you should have to use a
 toBlob method on that, and calls to that (and earlier calls in progress) on
 the Canvas itself should fail.  (If CanvasProxy isn't doing that it seems
 like a bug.)


With all these proposals I think it's OK to allow the main thread to do
(e.g.) a toDataURL and read what the current contents of the canvas is, and
possibly observe that they have changed. This is already possible today
with drawing video elements to a canvas; you can observe that the current
video frame sometimes changes. (It is of course essential that you don't
read a partially updated frame.)

If we don't want that, one way to fix it is to lock the frame used by a
particular script execution. So the first time you do a drawImage(video),
or a toDataURL within a particular script execution, we lock the current
contents of the video element or the canvas element for the purposes of
that script execution and ensure that any further usage of that element
within this script execution returns the same data.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Synchronizing Canvas updates in a worker to DOM changes in the UI thread

On Sun, Oct 20, 2013 at 5:33 PM, Glenn Maynard gl...@bluegoji.com wrote:

 Example:

 canvas id=canvas explicitpresent
 script
 var canvas = document.querySelector(#canvas);
 var worker = createWorker();
 worker.postMessage({
 cmd: init,
 canvas: canvas.getWorkerCanvas(),
 });

 worker.onmessage = function(e)
 {
 // The worker told us that a frame has been committed.  Present it for
 display.
 canvas.present();

 // Tell the worker that it should start rendering the next frame.
 worker.postMessage({cmd: update});

 // Do any DOM changes here, to synchronize them with displaying the new
 canvas.
 updateUI();
 }
 /script

 Worker:

 onmessage = function(e)
 {
 // On initialization only:
 if(e.data.cmd == init)
 canvas = e.data.canvas;

 // Render our scene.
 renderFrame(canvas);

 // Commit the scene.
 canvas.commit();

 // Tell the main thread that the frame is ready.
 postMessage(present);
 }
 function renderFrame(workerCanvas) { }


To me, passing the image data explicitly in an ImageBuffer along with the
present message seems like a better fit to the workers message-passing
model than this proposal, where the data is stored as hidden state in the
canvas element with (effectively) a setter in the worker and a getter in
the main thread, and that setting and getting has to be coordinated with
postMessage for synchronization. The relationship between a commit and
its present has to be deduced by reasoning about the timing of messages,
rather than by just reasoning about JS data flow through postMessage.

This proposal also requires that whenever a worker is going to return image
data to the main thread, the main thread must start things off by creating
a canvas element. It's also not possible for a worker to spawn off
sub-workers to do drawing (at least, not without some really ugly
coordination with the main thread.)

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

On Sun, Oct 20, 2013 at 5:29 PM, Glenn Maynard gl...@zewt.org wrote:

 That's not the problem attachToCanvas tries to solve.  It tries to solve
 rendering to multiple canvases, without requiring synchronization with the
 UI thread.  I have a proposal for handling synchronization with DOM
 updates, but I'll post it in a separate thread.


OK.


   - If you're rendering in a worker and the eventual target is in the main
  thread, the worker needs to be careful to not start rendering again
 until
  the main thread has assigned the ImageBitmap to where it wants it, and
  called .close().  You'd need to send a message back to the worker going
  okay, you can continue now.  Otherwise, you'd start rendering before a
  buffer has been freed up for reuse, and end up creating more backbuffers
  than you intended (which matters for large screens).  This seems easy to
  get wrong, and attachToCanvas doesn't have this problem.
 
 
  Not if you use transferToImageBitmap.

 transferToImageBitmap does have this problem.  If you transferToImageBitmap
 to detach your backing store to display it somewhere, then start rendering
 the next frame without waiting for the ImageBitmap to be given to the
 target, then as soon as you start rendering you'll end up creating a 3rd
 rendering buffer.


OK.

If you think that a single context outputting to multiple canvases
 fundamentally won't work well with canvases of different sizes, then forget
 about the feature.

 When you attachToCanvas, you're attaching that canvas's rendering buffer,
 not just its color buffer.  In WebGL terms, each canvas is a Framebuffer,
 with all its associated Renderbuffers.  Attaching the context to a canvas
 is like using bindFramebuffer to render to its backing store.


This is actually a disadvantage for attachToCanvas in some situations. If I
just want to generate N different rendered images (of the same size) (e.g.
http://www.turbosquid.com/Search/3D-Models/Vehicle/Car), the only thing I
want N of is the final color buffer.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

Glenn, taking a step back for a bit, is there anything in
https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal that you would
actually object to? IOW, is there anything there that you would think is
completely superfluous to the platform if all your proposals were to be
adopted as well?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Canvas in workers

2013-10-19 Thread Robert O'Callahan

On Sat, Oct 19, 2013 at 1:28 AM, Glenn Maynard gl...@zewt.org wrote:

 I'd like to hear thoughts on the context.attachToCanvas approach.  I
 think it has important advantages over ImageBitmap:

 - ImageBitmap requires the user to call close().  If the user forgets, or
 doesn't know, or misses it in some code paths, the problems caused aren't
 obvious.  Worse, they may only appear in some implementations and not
 others, depending on GC strategies.  attachToCanvas doesn't need cleanup in
 the first place, which is a nicer solution--there's nothing for the user to
 get wrong.


It's not clear to me how attachToCanvas works. An application like Google
Maps wants to draw to multiple canvases from a worker and then render the
updated canvas contents all at once, in synchrony with changes to the DOM
made by the main thread. How would you do that with attachToCanvas?


 - If you're rendering in a worker and the eventual target is in the main
 thread, the worker needs to be careful to not start rendering again until
 the main thread has assigned the ImageBitmap to where it wants it, and
 called .close().  You'd need to send a message back to the worker going
 okay, you can continue now.  Otherwise, you'd start rendering before a
 buffer has been freed up for reuse, and end up creating more backbuffers
 than you intended (which matters for large screens).  This seems easy to
 get wrong, and attachToCanvas doesn't have this problem.


Not if you use transferToImageBitmap.


 - With ImageBitmap, you need to create a helper canvas, then each time you
 render to a new target, you need to resize the canvas to match where it'll
 eventually go, so the resulting ImageBitmap is the size of its
 destination.  (This may also need to be carefully optimized, so the
 implementation doesn't actually resize the backing store every time its
 size changes.)  With attachToCanvas, you just size both canvases normally
 once, and switch between them with a single function call.


I'm not sure how helpful this is. In the case of WebGL, the rendering
context has resources that need to be sized the same as the destination
color buffer, so they'll need to be resized anyway if you're actually using
a single context to render to canvases of different sizes. My guess is that
the advice will always be don't do that.

- attachToCanvas matches the way Canvas works today: you create a Canvas,
 put it in the document (if it's for display), and render to it.  For two
 canvases, you'd just add a second Canvas, and toggle as needed.  With
 ImageBitmap, you have to restructure everything as soon as you want a
 second canvas, since you'd want to have a single offscreen Canvas for
 rendering, and to have img elements in the document instead of canvases.


I believe these are minor changes, especially compared to moving drawing to
a worker.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 6:57 AM, Justin Novosad ju...@google.com wrote:

 Here is similar concept, but with an API more like WokerCanvas:
 The CanvasRenderingContext2D associated with a WorkerCanvas would only
 record draw commands, without executing them. The context would be
 write-only. When you call commit on the WorkerCanvas, the block of recorded
 draw commands would be posted back to the main thread or directly to the
 compositor.


Which? They are observably different.


 What I like about this approach is that it is always just
 pushing data downstream, thus eliminating buffer synchronization issues as
 well as the need for double buffering canvas backing stores.


The write-only restriction is a problem. Also, it's really important that
the worker be able to create temporary canvases for its own use (pdf.js for
example needs this), and this doesn't really support that.

I think we have already converged on a WorkerCanvas design that everyone
(on this thread so far) is happy with, using ImageBitmaps to synchronize
with the main thread as needed. Is there some problem with that proposal
that warrants introducing the complexity of Rik's 'task' system or the
limitations of your proposal?

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 10:56 AM, Justin Novosad ju...@google.com wrote:

 On Thu, Oct 17, 2013 at 5:50 PM, Rik Cabanier caban...@gmail.com wrote:

 Creating temporary canvases is still possible. I'm unsure how it would be
 different from a worker.
 An advantage would be that you can draw to the temporary canvases in
 parallel to using them. Only PIXEL access is disallowed, you can still call
 drawImage using a canvas that has outstanding tasks.


 Right. The write-only restriction would only apply to canvas contexts that
 commit (push their tasks) directly down to the compositor. You could still
 create a canvas that is local to the worker, rasterize it in the worker and
 do readbacks in the worker, create ImageBitmaps from it, etc.


I'm not sure that you and Rik are talking about the same thing, since he's
still talking about outstanding tasks. If you are talking about the same
thing, I don't know what it is. I'd like to see some concrete details for
what you'd change in the current WorkerCanvas proposal. For the sake of
clarity I've put (my understand of) it here:
https://wiki.mozilla.org/User:Roc/WorkerCanvasProposal

With WorkerCanvas and transferToImageBitmap, you can draw multiple layers
in parallel (and actually draw, not just queue drawing commands) by
creating multiple workers, having them each produce an ImageBitmap, and
compositing those ImageBitmaps together by stacking img elements or
drawing them all to a single canvas. It uses more memory but you get more
parallelism.

In fact, an implementation could choose to take the deferred-drawing
approach instead. You would queue up drawing commands in the WorkerCanvas
(or the drawing context), and then transferToImageBitmap would not
immediately render but produce an ImageBitmap implementation encapsulating
the list of drawing commands to be drawn later, wherever/whenever that
ImageBitmap ended up being used. I think for commit() the implementation
would always want to force rasterization on the worker (or possibly some
dedicated canvas-rendering thread); you could forward a list of drawing
commands to the compositor thread for rasterization but I don't think
there's any reason to do that (and some good reasons not to).

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 12:32 PM, Glenn Maynard gl...@zewt.org wrote:

 On Thu, Oct 17, 2013 at 5:14 PM, Rik Cabanier caban...@gmail.com wrote:

  I don't remember multiple workers accessing the same canvas and I'm not
  quite sure what it means.  I do remember a single (WebGL) context
  rendering to multiple canvases.  Is that what you're thinking of?
 
 
  I went back over the history and that was indeed his use case.
 

 That's a good use case, I've wanted to do that myself.  We haven't tried
 very hard to fit it into the WorkerCanvas approach yet, and it may also be
 that the best way to do that is orthogonal to the whole canvas in workers
 use case.


That's not really a use-case. What would you actually be trying to do? IIUC
Ken agreed that his use-cases that appeared to require a single context
rendering to multiple canvases would be addressed just as easily (or
better) by using multiple image elements, a single canvas, and doing
image.srcObject = canvas.transferToImageBuffer().

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers

On Fri, Oct 18, 2013 at 4:10 PM, Rik Cabanier caban...@gmail.com wrote:

 They would still have to wait for each other so the images are composited
 in-order. If you don't care about that, the 'synchronized' option would let
 you draw as soon as you exit the task (which is how Chrome always draws
 since it's faster)


What do you mean wait for each other? You only have to wait until they're
all finished. The cost of actually compositing the images is low.

In fact, an implementation could choose to take the deferred-drawing
  approach instead. You would queue up drawing commands in the WorkerCanvas
  (or the drawing context), and then transferToImageBitmap would not
  immediately render but produce an ImageBitmap implementation
 encapsulating
  the list of drawing commands to be drawn later, wherever/whenever that
  ImageBitmap ended up being used. I think for commit() the implementation
  would always want to force rasterization on the worker (or possibly some
  dedicated canvas-rendering thread); you could forward a list of drawing
  commands to the compositor thread for rasterization but I don't think
  there's any reason to do that (and some good reasons not to).
 

 Can you tell me how you can ensure that you don't do too much work? Drawing
 in a continuous loop using 'Commit' would waste a lot of resources.


How to throttle drawing of frames using commit() is a completely separate
issue. Any API that allows direct publishing of frames from workers to the
compositor will have to deal with it, in roughly the same way.

Rob
-- 
Jtehsauts  tshaei dS,o n Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w  *
*

Re: [whatwg] Counterproposal for canvas in workers