Re: [whatwg] Canvas-Only Document Type
On Tue, Jul 8, 2014 at 7:37 AM, Brian Blakely anewpage.me...@gmail.com wrote: I agree, that would be valuable. This proposal is coming from the direction that sweeping out the main thread, and leaving only what is necessary for a full Canvas experience, would benefit such experiences. Blink doesn't actually schedule anything related to CSS or layout on the main thread if all you're doing is drawing to canvas so you wouldn't be saving any main thread time. I doubt having those features is causing you any jank. They do use some memory, but for a document with nothing but a canvas I suspect it's very little. - E
Re: [whatwg] Canvas-Only Document Type
Thank you for those informative comments, Elliott. My own tests in Firefox reflect yours and Boris's feedback. While there is clearly overhead from non-Canvas vectors, it's pretty minor. On Wed, Jul 9, 2014 at 12:43 PM, Elliott Sprehn espr...@gmail.com wrote: On Tue, Jul 8, 2014 at 7:37 AM, Brian Blakely anewpage.me...@gmail.com wrote: I agree, that would be valuable. This proposal is coming from the direction that sweeping out the main thread, and leaving only what is necessary for a full Canvas experience, would benefit such experiences. Blink doesn't actually schedule anything related to CSS or layout on the main thread if all you're doing is drawing to canvas so you wouldn't be saving any main thread time. I doubt having those features is causing you any jank. They do use some memory, but for a document with nothing but a canvas I suspect it's very little. - E
Re: [whatwg] Canvas-Only Document Type
The DOM and CSS obviously have a lot of utility that goes unused in Canvas apps, but I think that, overall, the Web can benefit by empowering Canvas. It was originally brought to the platform with a mentality influenced by embedded plugins (Flash), but now fullscreen or full viewport experiences are much more prevalent and important for high-impact content. The platform must shift to regard those experiences as the rule, not the exception. As for accessibility, the Web is well overdue for a paradigm that can handle more than document-oriented content with more than alt content. That said, it would be simple to modify this proposal to handle accessibility just as well as the Web does today for the same use cases. For example, one idea that is interesting to me is more established use of the Speech Synthesis API for very interactive content. On Mon, Jul 7, 2014 at 7:27 PM, Katelyn Gadd k...@luminance.org wrote: Too many use cases right now depend on the use of the DOM and CSS. Introducing this document model would require introducing a whole new set of APIs for solving those problems just for this particular corner case. Furthermore, removing the 'document' object throws away features you actually want... Some examples: CSS will still exist in this model since Canvas uses CSS for specification of things like fonts and colors. Web font loading is typically done at the document level in a stylesheet before you use the webfont in a canvas. Getting full dimensions for text is not (iirc) possible without using the DOM to create an invisible element containing the text, since the canvas measurement api only returns a width. Some important event listeners are registered on the document and/or elements, not the window. Various important content types are loaded via the DOM and (iirc) loads may not start or complete until they are added to the document's DOM tree. Without a document it would not be possible to load this content. (Scripts are the most obvious example; without a document how would you programmatically add scripts? IIRC video/audio/img have certain behaviors that only kick in once they are added to the DOM.) Pointer lock and fullscreen are defined in terms of document/elements, not the window. P.S. Handwaving in that fashion about accessibility is not a good idea. There's a huge subset of internet users out there that have real accessibility needs and it is not wise - or in some cases, even LEGAL - to dismiss them out of hand without careful consideration. The current web may not be IDEAL for accessibility, but throwing out the DOM is going to make your apps even worse in this regard. On Mon, Jul 7, 2014 at 4:06 PM, Brian M. Blakely anewpage.me...@gmail.com wrote: The use cases involved are mostly intended for 60fps high-load graphics, so the kind of a11y conditions served by the DOM (screen readers) aren't as helpful in these apps. I don't think the DOM makes videogames more accessible. The Web Platform doesn't serve that case well; it isn't something this is trying to solve. On Jul 7, 2014, at 6:54 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Has anyone considered the accessibility implications of this? IIUC accessibility for canvas is provided through extra dom elements. So, this would defeat that purpose. Silvia. On Tue, Jul 8, 2014 at 8:39 AM, Brian M. Blakely anewpage.me...@gmail.com wrote: Hi Ashley, With the budding of Canvas 2D and WebGL UI frameworks, I believe that, in a couple years' time, the role of CSS in the cases I described will diminish drastically. A lot of this was kind of waiting for Apple to give the OK before people began committing their hearts to WebGL. On Jul 7, 2014, at 5:17 PM, Ashley Gullen ash...@scirra.com wrote: Having developed a major HTML5 game engine, and given this appears to be aimed at a gaming use case, I feel qualified to offer my opinion: I'm not sure this is a good idea. Despite being 99% canvas and javascript, we use CSS to implement some useful scaling modes (like letterbox fullscreen). We also use the DOM for many useful features, such as form controls, divs, Twitter or Facebook buttons and so on, which are positioned over the canvas. In particular text inputs are useful for things like name entry or logins even for games, and are typically difficult and error-prone to reimplement in only canvas and javascript. Is there any evidence that such a mode would actually improve performance? Are there benchmarks indicating the existence of a DOM, even if inert, harms performance in any way? Ashley Gullen Scirra.com On 7 July 2014 21:35, Brian Blakely anewpage.me...@gmail.com wrote: Floating a concept for a document mode which eschews CSS and the DOM to enable a more jank-free Canvas surface. Depending on how this allows for optimization, might be used well for games, VR, wearables, and ultra-portable or high-performance apps. Probably most
Re: [whatwg] Canvas-Only Document Type
Thank you for this information, Boris. Did you collect this with Firefox's own devtools? I'd like to set up a full-view Canvas document and see how it profiles. FWIW, memory is one of several optimizations one could conjure, and it is the culmination of these that (in concept) creates a more polished experience. For example, if a user changes their device orientation, the application could perhaps respond in a more jank-free fashion since the UA can skip refreshing the layout tree. On Mon, Jul 7, 2014 at 9:47 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/7/14, 6:58 PM, Brian M. Blakely wrote: I am not a browser developer, I am a Web developer, but it stands to reason that (as stated in the OP) there may at least be a fruitful memory savings I just measured in Firefox 30, and a bare-bones document (about:blank) uses approximately 550KB of memory. This breaks down as about: * 330KB of JS heap (which presumably would be there anyway for your special document), * 10KB DOM data structures * 210KB layout data (mostly style data, looks like). So you could plausibly save about 220KB of memory. I encourage you to do similar measurements in other browsers as desired. For comparison, a 320x426 canvas (and I doubt you're going to be using ones smaller than that in this special mode) will have a backing store that takes about 545KB for a 2D canvas. The sort of devices where you're likely to actually use WebGL will probably have 4MB+ canvas backing stores, and probably comparable for WebGL. On the memory front, this could be especially useful when sharing space with the 30+ tabs That would translate into a savings of about 7MB. Not nothing, but just to put that in perspective simply loading http://code.jquery.com/jquery-1.11.1.min.js into those blank tabs above makes them use 1.43MB of memory instead of 550KB. That's without _doing_ anything; just loading the library. So I suspect in practice the script state and canvas backing stores in those 30+ tabs would totally swamp the 7MB of layout/DOM data. -Boris
Re: [whatwg] Canvas-Only Document Type
I agree, that would be valuable. This proposal is coming from the direction that sweeping out the main thread, and leaving only what is necessary for a full Canvas experience, would benefit such experiences. On Mon, Jul 7, 2014 at 11:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: If you want jank-free canvas rendering, you probably really want WebGL drawing from a Worker. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
Re: [whatwg] Canvas-Only Document Type
On 7/8/14, 10:28 AM, Brian Blakely wrote: Thank you for this information, Boris. Did you collect this with Firefox's own devtools? With about:memory, to be precise. I'd like to set up a full-view Canvas document and see how it profiles. Please do! FWIW, memory is one of several optimizations one could conjure, and it is the culmination of these that (in concept) creates a more polished experience. For example, if a user changes their device orientation, the application could perhaps respond in a more jank-free fashion since the UA can skip refreshing the layout tree. Refreshing a layout tree with three boxes in it doesn't take very long. -Boris
Re: [whatwg] Canvas-Only Document Type
Do you have any data to back the proposal? For example, how much overhead
do you expect this to save compared to a document that contains just a
full-screen canvas? Is the HTML parsing, style calculation and DOM layout
overhead really that high for a document that has nothing but a canvas
element in it? If it is, perhaps that can be optimized without changing
the spec.
On Mon, Jul 7, 2014 at 4:35 PM, Brian Blakely anewpage.me...@gmail.com
wrote:
Floating a concept for a document mode which eschews CSS and the DOM
to enable a more jank-free Canvas surface.
Depending on how this allows for optimization, might be used well for
games, VR, wearables, and ultra-portable or high-performance apps.
Probably most beneficial to memory usage and first paint time. Would
appreciate if some vendor engineers who might be reading could chime
in on this point.
Strawman:
Document only contains !doctype canvas-[2d|3d] and script elements.
Everything else is ignored. document object is gone.
A Canvas drawing surface consumes the entire viewport. It always has
an opaque backing store, same as specifying getContext('2d', { alpha:
false }).
UA provides:
* A host object representing surface's CanvasRenderingContext2D or
WebGLRenderingContext (depending on specified doctype).
* In lieu of DOM, an API for creating offscreen canvases (actually,
this abstraction should probably exist anyway). This might live on
the Context host obj, which may open a beneficial performance
relationship between onscreen canvas and offscreen children.
Re: [whatwg] Canvas-Only Document Type
Having developed a major HTML5 game engine, and given this appears to be
aimed at a gaming use case, I feel qualified to offer my opinion: I'm not
sure this is a good idea.
Despite being 99% canvas and javascript, we use CSS to implement some
useful scaling modes (like letterbox fullscreen). We also use the DOM for
many useful features, such as form controls, divs, Twitter or Facebook
buttons and so on, which are positioned over the canvas. In particular text
inputs are useful for things like name entry or logins even for games, and
are typically difficult and error-prone to reimplement in only canvas and
javascript.
Is there any evidence that such a mode would actually improve performance?
Are there benchmarks indicating the existence of a DOM, even if inert,
harms performance in any way?
Ashley Gullen
Scirra.com
On 7 July 2014 21:35, Brian Blakely anewpage.me...@gmail.com wrote:
Floating a concept for a document mode which eschews CSS and the DOM
to enable a more jank-free Canvas surface.
Depending on how this allows for optimization, might be used well for
games, VR, wearables, and ultra-portable or high-performance apps.
Probably most beneficial to memory usage and first paint time. Would
appreciate if some vendor engineers who might be reading could chime
in on this point.
Strawman:
Document only contains !doctype canvas-[2d|3d] and script elements.
Everything else is ignored. document object is gone.
A Canvas drawing surface consumes the entire viewport. It always has
an opaque backing store, same as specifying getContext('2d', { alpha:
false }).
UA provides:
* A host object representing surface's CanvasRenderingContext2D or
WebGLRenderingContext (depending on specified doctype).
* In lieu of DOM, an API for creating offscreen canvases (actually,
this abstraction should probably exist anyway). This might live on
the Context host obj, which may open a beneficial performance
relationship between onscreen canvas and offscreen children.
Re: [whatwg] Canvas-Only Document Type
Has anyone considered the accessibility implications of this? IIUC
accessibility for canvas is provided through extra dom elements. So,
this would defeat that purpose.
Silvia.
On Tue, Jul 8, 2014 at 8:39 AM, Brian M. Blakely
anewpage.me...@gmail.com wrote:
Hi Ashley,
With the budding of Canvas 2D and WebGL UI frameworks, I believe that, in a
couple years' time, the role of CSS in the cases I described will diminish
drastically. A lot of this was kind of waiting for Apple to give the OK
before people began committing their hearts to WebGL.
On Jul 7, 2014, at 5:17 PM, Ashley Gullen ash...@scirra.com wrote:
Having developed a major HTML5 game engine, and given this appears to be
aimed at a gaming use case, I feel qualified to offer my opinion: I'm not
sure this is a good idea.
Despite being 99% canvas and javascript, we use CSS to implement some useful
scaling modes (like letterbox fullscreen). We also use the DOM for many
useful features, such as form controls, divs, Twitter or Facebook buttons
and so on, which are positioned over the canvas. In particular text inputs
are useful for things like name entry or logins even for games, and are
typically difficult and error-prone to reimplement in only canvas and
javascript.
Is there any evidence that such a mode would actually improve performance?
Are there benchmarks indicating the existence of a DOM, even if inert, harms
performance in any way?
Ashley Gullen
Scirra.com
On 7 July 2014 21:35, Brian Blakely anewpage.me...@gmail.com wrote:
Floating a concept for a document mode which eschews CSS and the DOM
to enable a more jank-free Canvas surface.
Depending on how this allows for optimization, might be used well for
games, VR, wearables, and ultra-portable or high-performance apps.
Probably most beneficial to memory usage and first paint time. Would
appreciate if some vendor engineers who might be reading could chime
in on this point.
Strawman:
Document only contains !doctype canvas-[2d|3d] and script elements.
Everything else is ignored. document object is gone.
A Canvas drawing surface consumes the entire viewport. It always has
an opaque backing store, same as specifying getContext('2d', { alpha:
false }).
UA provides:
* A host object representing surface's CanvasRenderingContext2D or
WebGLRenderingContext (depending on specified doctype).
* In lieu of DOM, an API for creating offscreen canvases (actually,
this abstraction should probably exist anyway). This might live on
the Context host obj, which may open a beneficial performance
relationship between onscreen canvas and offscreen children.
Re: [whatwg] Canvas-Only Document Type
The use cases involved are mostly intended for 60fps high-load graphics, so the
kind of a11y conditions served by the DOM (screen readers) aren't as helpful in
these apps. I don't think the DOM makes videogames more accessible. The Web
Platform doesn't serve that case well; it isn't something this is trying to
solve.
On Jul 7, 2014, at 6:54 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote:
Has anyone considered the accessibility implications of this? IIUC
accessibility for canvas is provided through extra dom elements. So,
this would defeat that purpose.
Silvia.
On Tue, Jul 8, 2014 at 8:39 AM, Brian M. Blakely
anewpage.me...@gmail.com wrote:
Hi Ashley,
With the budding of Canvas 2D and WebGL UI frameworks, I believe that, in a
couple years' time, the role of CSS in the cases I described will diminish
drastically. A lot of this was kind of waiting for Apple to give the OK
before people began committing their hearts to WebGL.
On Jul 7, 2014, at 5:17 PM, Ashley Gullen ash...@scirra.com wrote:
Having developed a major HTML5 game engine, and given this appears to be
aimed at a gaming use case, I feel qualified to offer my opinion: I'm not
sure this is a good idea.
Despite being 99% canvas and javascript, we use CSS to implement some
useful scaling modes (like letterbox fullscreen). We also use the DOM for
many useful features, such as form controls, divs, Twitter or Facebook
buttons and so on, which are positioned over the canvas. In particular text
inputs are useful for things like name entry or logins even for games, and
are typically difficult and error-prone to reimplement in only canvas and
javascript.
Is there any evidence that such a mode would actually improve performance?
Are there benchmarks indicating the existence of a DOM, even if inert,
harms performance in any way?
Ashley Gullen
Scirra.com
On 7 July 2014 21:35, Brian Blakely anewpage.me...@gmail.com wrote:
Floating a concept for a document mode which eschews CSS and the DOM
to enable a more jank-free Canvas surface.
Depending on how this allows for optimization, might be used well for
games, VR, wearables, and ultra-portable or high-performance apps.
Probably most beneficial to memory usage and first paint time. Would
appreciate if some vendor engineers who might be reading could chime
in on this point.
Strawman:
Document only contains !doctype canvas-[2d|3d] and script elements.
Everything else is ignored. document object is gone.
A Canvas drawing surface consumes the entire viewport. It always has
an opaque backing store, same as specifying getContext('2d', { alpha:
false }).
UA provides:
* A host object representing surface's CanvasRenderingContext2D or
WebGLRenderingContext (depending on specified doctype).
* In lieu of DOM, an API for creating offscreen canvases (actually,
this abstraction should probably exist anyway). This might live on
the Context host obj, which may open a beneficial performance
relationship between onscreen canvas and offscreen children.
Re: [whatwg] Canvas-Only Document Type
Brian M. Blakely anewpage.me...@gmail.com writes: The use cases involved are mostly intended for 60fps high-load graphics, so the kind of a11y conditions served by the DOM (screen readers) aren't as helpful in these apps. I don't think the DOM makes videogames more accessible. The Web Platform doesn't serve that case well; it isn't something this is trying to solve. I think no one should ever optimize for inaccessible web content. In case you think otherwise: Would you elaborate why your wish to display “60fps high-load graphics” on the web outweighs the needs of people with old or non widely used hardware or software, people with disabilities, software that has no optical sensory input (e.g. web spiders) and all those of us users who want to zoom or select text? Consider the following: If web content is not accessible by default, only a privileged class of users will be able to see and use it. We had that scenario already when “web” developers used Flash for everything. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Canvas-Only Document Type
If you want jank-free canvas rendering, you probably really want WebGL drawing from a Worker. Rob -- oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo owohooo osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o oioso oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo owohooo osoaoyoso,o o‘oYooouo ofolo!o’o owoiololo oboeo oiono odoaonogoeoro ooofo otohoeo ofoioroeo ooofo ohoeololo.
