Re: [whatwg] High-density canvases
On Thu, Jun 12, 2014 at 11:42 PM, Robert O'Callahan rob...@ocallahan.org wrote: I think I'd rather not take control of canvas resizing away from applications, even opt-in. That leads to complexity such as extra API for slaving other canvases. I also think we'd be better off sticking to the invariant that the units of canvas coordinate space are single pixels in the canvas bitmap; I think that simplifies things for implementers and authors. I agree. Here's an alternative proposal which I think is a bit simpler and more flexible: Expose two new DOM attributes on HTMLCanvasElement: readonly attribute long preferredWidth; readonly attribute long preferredHeight; These attributes are the UA's suggested canvas size for optimizing output quality. It's basically what Ian's proposal would have set as the automatic size. We would also add a preferredsizechange event when those attributes change. Applications that want DPI-aware canvases would read those attributes and use them to set the size, just once if they only want to paint once, during each requestAnimationFrame callback for games and other animations, and in the preferredsizechange event handler if they are willing to paint multiple times but aren't actually animating. The application would be responsible for scaling its output to the new canvas coordinate space size. Giving the application control of the size change simplifies things for the browser and gives applications maximum flexibility, e.g. resizing ancillary resources as needed, or imposing constraints on the chosen size. I like this new proposal. It works for both the canvas and WebGL use cases, and it puts the app in control of behaviour in a way that makes sense for immediate-mode APIs. I assume that the size change event would fire: - on browser page zoom - on pinch-zoom - when a CSS animation (e.g., scale) changes the canvas size in CSS pixels For browsers that implement the latter two off the main thread, perhaps they should only fire at end-of-gesture or end-of-animation, to avoid the rendered size being out-of-sync with scaled size by the time the canvas gets composited. I agree with Mark that the names need work. How about something that incorporates device pixel in some way, to reflect that this is roughly dpr * css scale * size? devicePixelWidth widthInDevicePixels pixelExactWidth exactPixelWidth pixelWidth pixelRatioExactWidth unscaledWidth unscaledPixelWidth nativeWidth nativePixelWidth Stephen Rob -- Jtehsauts tshaei dS,o n Wohfy Mdaon yhoaus eanuttehrotraiitny eovni le atrhtohu gthot sf oirng iyvoeu rs ihnesa.rt sS?o Whhei csha iids teoa stiheer :p atroa lsyazye,d 'mYaonu,r sGients uapr,e tfaokreg iyvoeunr, 'm aotr atnod sgaoy ,h o'mGee.t uTph eann dt hwea lmka'n? gBoutt uIp waanndt wyeonut thoo mken.o w
Re: [whatwg] High-density canvases
On Fri, Sep 27, 2013 at 5:51 PM, Ian Hickson i...@hixie.ch wrote:
On Tue, 10 Sep 2013, Stephen White wrote:
For posterity, here were our objections to the original high-DPI canvas
spec:
- It doesn't scale well to non-integer devicePixelRatios
Can you elaborate on this? I don't see why the new proposal would have
this problem, but I also don't see why the old one would, so I don't know
if it's because I don't understand the problem or if it's because I'm
missing something from the old proposal.
IIRC, the concern was that calling getImageData() would return surprising
results when resampling by non-integer ratios, where the returned result
very much depends on the resampling mode, and the border treatment. At any
rate, since getImageData() doesn't resample in the new proposal, this won't
be a problem.
Stephen
One question: now that some browsers are including browser zoom (page
zoom) in window.devicePixelRatio, will/should the new proposal
automatically cause a resize callback on page zoom, in order to preserve
1:1 device pixels?
My intent was to do so, yes. In practice I presume it'd be up to the
browser to decide how often to actually do this (e.g. if page zoom is
being smoothly animated, you may wish to only do it every few frames).
(Note that I think this is a problem with current JS-based
implementations of canvas auto-scale as well, although perhaps there's a
DOM event for this that you can listen to; I might just be showing my
ignorance here.)
Currently page zoom should trigger a 'resize' event, but I expect few
pages check.
On Tue, 10 Sep 2013, Justin Novosad wrote:
There is another closely related issue that's been discussed before:
adding a redraw callback to 2d canvas. In the past we discussed this
for solving the problem of recoverring from a gpu context loss, but it
seems there may be better reasons to consider adding a redraw callback
such as freeing memory consumed by canvas backing stores that are in
background tabs, and re-building the content when needed. This
discussion was revived in the past few days on the chromium graphics-dev
mailing list:
https://groups.google.com/a/chromium.org/forum/?fromgroups#!topic/graphics-dev/CQJXpXxO6dk
The idea is still embryonic and we're brainstorming in this chromium
issue: crbug.com/287823
I think that discussion should be merged with this thread because a
resize event is another case where one may want to redraw. It would be
great to solve all of these issues together.
I think it would make eminent sense to also fire the event ('resize', I
guess) if the context was lost or if the canvas was about to be made
visible again after the browser dropped the rendering, yes.
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] High-density canvases
For posterity, here were our objections to the original high-DPI canvas
spec:
- The API feels like a short-term hack to automagically do something
that the developer may or may not want done (e.g., if the game/app was
tuned for particular resolution, or for pixel-exact rendering) that we'll
be stuck with in the web platform long after its short-term usefulness has
expired
- It doesn't scale well to non-integer devicePixelRatios
- It is easy for developers to implement the above behaviour in JS if
desired
I think the new proposal addresses the first point, since it's opt-in. I
don't think the second point is a problem, since [get|put]ImageData() will
be back to manipulating exact backing store pixels, so no non-integer
resizing will be required. The third point becomes moot.
One question: now that some browsers are including browser zoom (page zoom)
in window.devicePixelRatio, will/should the new proposal automatically
cause a resize callback on page zoom, in order to preserve 1:1 device
pixels? (Note that I think this is a problem with current JS-based
implementations of canvas auto-scale as well, although perhaps there's a
DOM event for this that you can listen to; I might just be showing my
ignorance here.)
Stephen
On Mon, Sep 9, 2013 at 8:00 PM, Ian Hickson i...@hixie.ch wrote:
On Wed, 17 Jul 2013, Rik Cabanier wrote:
Ian wrote:
The density aspect of this might be pointless, given the failure of
getImageDataHD(); if we're dropping that one, I'll drop this one at
the same time.
Yes, please drop it since the HD methods are going away from the one
implementation.
On Tue, 9 Jul 2013, Stephen White wrote:
Conversely, if it helps to bring the spec closer to the implementations,
one thing we do not intend to implement in Chrome is the automatic
high-DPI canvas scaling (ie., auto-doubling of backing stores,
getImageDataHD(), putImageDataHD(), etc).
I believe Apple has also announced that they are dropping support for
this in Safari 7.
So my understanding is that the reason this feature failed is that there's
existing content that assumes a 1:1 ratio, and having an automatic
high-density mode was making some pages end up with canvases with four
canvas pixels per CSS pixel (linearly) -- two from the browser making a
native canvas, times two from the page scaling the canvas for high DPI
displays. This is a factor of sixteen over a 1:1 canvas, a factor of four
more than it should be for high DPI, and a big waste of resources.
As much as sites do this manually, though, it's a huge pain in the neck to
have to worry about pixel density when you're creating your canvas and
drawing on it, especially if you're not drawing sprites on it.
While we're talking about annoying things, there's also the annoyance that
canvases tend to not take zoom into account (either density-affecting zoom
like page zoom on desktop, or transparent zoom like pinch-zoom on mobile
for non-mobile-optimised sites, which the site isn't supposed to know
about): you have to remember to listen for onresize, and then manually
blow away your canvas and recreate it at the right density and then
squeeze it into place so that the coordinate space matches what your code
is expecting while the canvas is actually sized for the display.
There's also the issue of full-bleed canvases where every time the
container changes, you have to remember to re-update the canvas coordinate
space and repaint because otherwise your pretty page gets all warped.
It would be nice to fix these all at once, and I think we can, by
introducing a configuration option on getContext(), in the style of WebGL:
getContext('2d', { density: 'autosize' });
This would trigger the following behaviour: When the context is created,
and subsequently when the canvas changes size (e.g. due to being sized
with CSS relative units and the element they're relative to changing), or
when the display density changes size (e.g. due to page zoom), then:
- the width and height of the canvas bitmaps get updated to match the
new native size of the canvas, at native density.
- the coordinate space of the canvas (context.width/context.height)
gets updated to match the size of the canvas in CSS pixel units.
- a 'resize' event gets fired at the canvas.
We would dump the *HD versions of the methods, and make the regular ones
go back to returning the actual raw pixels, since that would now work fine
and still provide HD-quality content everywhere it's available.
What do people think?
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Antialiasing of line widths 1 (was Re: Blurry lines in 2D Canvas (and SVG))
On Sat, Aug 10, 2013 at 11:07 PM, Rik Cabanier caban...@gmail.com wrote: On Sat, Aug 10, 2013 at 7:50 AM, Glenn Maynard gl...@zewt.org wrote: On Sat, Aug 10, 2013 at 7:42 AM, Stephen White senorbla...@chromium.orgwrote: Chrome (well, Skia actually) uses a hairline mode for line widths 1. It draws a line of width 1, and uses the width to modulate the alpha. I think the idea is to prevent blotchiness/unevenness caused by undersampling or missed coverage (Skia uses 16 samples of AA). That sounds like it should be fine, since it should give results similar to what users would expect from simple coverage antialiasing. I'm not sure that's what I'm seeing, though. http://jsfiddle.net/eZEyH/1/ The 0.001 width stroke is being drawn solid black in the pixel-centered (left) case. In the right one, horizontally aligned to the edge of a pixel, the stroke disappears. (I left it vertically pixel-centered, so the box didn't disappear entirely.) The right is what I'd expect to always happen with a lineWidth that thin. Similar things happen with thicker widths, the 0.001 just makes it very easy to see. That is clearly a bug. :-) Yep, seems to be a bug in Skia's raster backend. I've logged it as https://code.google.com/p/skia/issues/detail?id=1505; feel free to add further comments there. Stephen This can become visible during animation, eg. http://jsfiddle.net/xSUuB/1/. In Chrome, the line flickers between solid black and grey. In Firefox, it's antialiased normally, so it consistently appears grey (actually shifting between one pixel of grey and two pixels of lighter grey). Yeah. Chrome seems to flip between no AA and AA which sounds like a bug in their algorithm.
Re: [whatwg] remove resetClip from the Canvas 2D spec
If this is strictly a performance issue, then we definitely should fix that
before adding new API, IMHO. It would be great to get some reduced test
cases where save()/restore() is a bottleneck.
(Incidentally, we did some performance fixes recently for setFont() in
Chrome.)
Stephen
On Mon, Aug 12, 2013 at 6:34 PM, Rik Cabanier caban...@gmail.com wrote:
On Mon, Aug 12, 2013 at 2:26 PM, Justin Novosad ju...@google.com wrote:
Ok, so here is a simple proposal:
IDL:
enum CanvasSaveMode { all, transform, clip, transform-and-clip };
save(optional CanvasSaveMode mode);
Modes:
all: save the entire rendering context state
transform: save only the current transform
clip: save only the current clip
if mode is not specified, the entire context state is saved (for backward
compatibility)
The restore method's interface does not change. It restores whatever
state was saved by the matching save call.
I wasn't really thinking about a new API surface :-)
Can't this be fixed under the hood? The tricks that Simon is doing, could
be done by the browser itself.
If not, this proposal looks reasonable (if you turn it into a dict like
Tab says).
On Mon, Aug 12, 2013 at 4:56 PM, Simon Sarris simon.sar...@gmail.comwrote:
Good point, I think part of the problem has to do with the fact that
save
is non-selective (saves all of the state).
Yes, since save() and restore() save and restore everything, it creates
the side effect of needing to set ctx.font/fillStyle/strokeStyle more often
than otherwise, which are slow to set, probably because of some CSS parser
activity, but I'm not wise enough to know.
If there was merely a way to save and restore the context, or perhaps
some other subset of state, that would probably work nicely too.
[whatwg] Antialiasing of line widths 1 (was Re: Blurry lines in 2D Canvas (and SVG))
On Sat, Aug 10, 2013 at 1:07 AM, Glenn Maynard gl...@zewt.org wrote: On Fri, Aug 9, 2013 at 11:07 PM, Rik Cabanier caban...@gmail.com wrote: Chrome seems ignore stroke widths that are smaller than 1 (which is reasonable). (That seems wrong to me--it should continue to draw based on pixel coverage--but that's a separate issue...) Is it? Obviously you can't draw less than a pixel, but the user did specify that he wants it too look black. strokeStyle = black doesn't mean every pixel in the stroke should be black. It's the color of the pen. If you draw over half of a pixel with a black pen, you get 50% grey. It'd be one thing if Chrome didn't antialias at all, but if Chrome is antialiasing a stroke with a lineWidth of 1.5, it doesn't make sense that it's not antialiasing a stroke with a lineWidth of 0.75. I don't think this is strictly specified; the only mention of anti-aliasing is an example of how to do it (oversampling). Chrome (well, Skia actually) uses a hairline mode for line widths 1. It draws a line of width 1, and uses the width to modulate the alpha. I think the idea is to prevent blotchiness/unevenness caused by undersampling or missed coverage (Skia uses 16 samples of AA). Here's an example: hairline on the left, .2 width path on the right: http://jsfiddle.net/FWLZt/ Stephen This is tangental, though. Might want to start another thread if you want to go over this more, or we'll derail this one... Done. -- Glenn Maynard
Re: [whatwg] remove resetClip from the Canvas 2D spec
Although Skia could support resetClip() via SkRegion::kReplace_Op, it's problematic for the API in general, and I think we should avoid it. In particular, it makes it impossible to place a display list (SkPicture in Skia parlance) inside a parent display list containing a clip and be assured that the child will not draw outside the given region, since the child display list can always resetClip() its way out of the parent's clip. It probably also prevents culling optimizations for the same reason. For example, if one used Skia to draw the entirety of a browser UI including chrome and content, the resetClip() inside the web page contents would overwrite the browser UI. Obviously we don't do that in Chrome, but it goes some idea of the problem at the API level. Stephen On Thu, Jul 18, 2013 at 1:39 PM, Ian Hickson i...@hixie.ch wrote: On Tue, 29 Jan 2013, Rik Cabanier wrote: we were looking at how resetClip could be implemented in WebKit. Looking over the Core Graphics implementation, this feature can't be implemented without significant overhead. I also found an email from 2007 where Maciej states the same concern: http://permalink.gmane.org/gmane.org.w3c.whatwg.discuss/10582 The solution on Mac is probably for Apple to update CoreGraphics to support this feature. This is a quite widely requested feature. Since no browser has implemented it, can it be removed from the spec? It's new, so no browser having implemented it is expected. If browsers don't implement it, it'll get removed in due course. But it would be sad for authors, who are the main concern here. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Blurry lines in 2D Canvas (and SVG)
I think one problem you might run into is that, if you consider a stroked line to be centered on pixel centers rather than pixel edges, then the same path when filled and stroked would touch different pixels along each edge. Consider a 10x10 rectangle, drawn at coordinates coordinates 5, 5. If filled, this would fill pixels 5-14 in X and 5-14 in Y. If stroked, this will draw 1-pixel wide rectangles centered along (5, 5) - (14, 5) - (14, 14) - (5, 5). With antialiasing this will touch pixels 4-15 in each dimension. http://jsfiddle.net/6KS4V/ If the stroke was instead drawn centered over half pixels, the stroked rects would be centered along (5.5, 5.5) - (14.5, 5.5) - (14.5, 14.5) - (14.5, 5.5) - (5.5, 5.5). This would touch pixels 5-15 in each dimension. If drawn with transparency, the resulting left and top edges would look different than the bottom and right edges. E.g., http://jsfiddle.net/9xbkX/ . (Please ignore blurriness induced by the CSS upscaling; you can remove the CSS and use a zooming tool if you prefer). Stephen On Tue, Jul 23, 2013 at 7:19 PM, Rik Cabanier caban...@gmail.com wrote: All, we've noticed that if you draw lines in canvas or SVG, they always end up blurry. For instance see this fiddle: http://jsfiddle.net/V92Gn/128/ This happens because you offset 1 pixel and then draw a half pixel stroke on each side. Since it covers only half the pixel, the color gets mapped to 50% gray. You can work around this by doing an extra offset of half the devicepixelratio, but ideally this should never happen. Is this behavior specified somewhere? Is there a way to turn this off?
Re: [whatwg] Blurry lines in 2D Canvas (and SVG)
That's an interesting idea. I suppose the fully general solution would be to have a stroke offset. E.g., with a stroke width of 4 and and offset of 2 you'd get outer, offset -2 you'd get inner, offset 1 you'd get 3 pixels outer and 1 pixel inner, etc. Dunno how useful that is, though. Stephen On Fri, Aug 9, 2013 at 5:24 PM, Glenn Maynard gl...@zewt.org wrote: On Fri, Aug 9, 2013 at 4:17 PM, Stephen White senorbla...@chromium.orgwrote: If the stroke was instead drawn centered over half pixels, the stroked rects would be centered along (5.5, 5.5) - (14.5, 5.5) - (14.5, 14.5) - (14.5, 5.5) - (5.5, 5.5). This would touch pixels 5-15 in each dimension. If drawn with transparency, the resulting left and top edges would look different than the bottom and right edges. E.g., http://jsfiddle.net/9xbkX/ My proposal addresses this, by adding an outer stroke mode. http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2013-July/040252.html -- Glenn Maynard
Re: [whatwg] Blurry lines in 2D Canvas (and SVG)
On Fri, Aug 9, 2013 at 8:16 PM, Rik Cabanier caban...@gmail.com wrote: On Fri, Aug 9, 2013 at 2:17 PM, Stephen White senorbla...@chromium.orgwrote: I think one problem you might run into is that, if you consider a stroked line to be centered on pixel centers rather than pixel edges, then the same path when filled and stroked would touch different pixels along each edge. Consider a 10x10 rectangle, drawn at coordinates coordinates 5, 5. If filled, this would fill pixels 5-14 in X and 5-14 in Y. If stroked, this will draw 1-pixel wide rectangles centered along (5, 5) - (14, 5) - (14, 14) - (5, 5). With antialiasing this will touch pixels 4-15 in each dimension. http://jsfiddle.net/6KS4V/ If the stroke was instead drawn centered over half pixels, the stroked rects would be centered along (5.5, 5.5) - (14.5, 5.5) - (14.5, 14.5) - (14.5, 5.5) - (5.5, 5.5). This would touch pixels 5-15 in each dimension. If drawn with transparency, the resulting left and top edges would look different than the bottom and right edges. E.g., http://jsfiddle.net/9xbkX/. (Please ignore blurriness induced by the CSS upscaling; you can remove the CSS and use a zooming tool if you prefer). Yes, I agree that strokes and fills should be treated the same to avoid this problem. I'm unsure if Glenn's suggestion for an outer and inner stroke fix the problem. An inner or outer stroke will look very different. Sure, but it's up to the developer to choose. It looks like Illustrator has something very similar; see the Align Stroke buttons here: http://help.adobe.com/en_US/illustrator/cs/using/WSA1E31D7D-13E6-41ac-AA8C-4AD129B9FC1Ca.html. I don't have a copy of Illustrator to try it, though. In addition if the corners of the path don't align with the grid, you will get a blurry outline again. Well, as long as you choose the right mitering style, I think the corners would be fine. It's basically the same algorithm as normal path stroking, except instead of translating half the stroke width on either side of the path along the tangent, you'd just use the path as one edge and translate the other the full stroke width. As an experiment, I drew 4 rectangles in JSFiddle with stroke width of .5, .75, 1, 1.5 and 2: http://jsfiddle.net/6KS4V/2/ I aligned them to the grid as Glenn suggested. This is a blown up screenshot from IE (Firefox looked the same): http://bit.ly/16FVCKd and here's one from Chrome: http://bit.ly/19Tf9Ko The rectangle that's 2 points wide is somewhat blurry, but the one that is 1.5 is very bad. Chrome seems ignore stroke widths that are smaller than 1 (which is reasonable). I think this more like what the outer stroke style would do: http://jsfiddle.net/ZrbQh/ The interior rectangle is the same in each one, with the outer edge nudged outwards along the tangent (the sizes may be wrong for the 1 lineWidth cases; I didn't spend too long on it). Stephen I then recreated the content in Illustrator, opened the file in Acrobat and took a screenshot: http://bit.ly/15loBhu As you can see, Acrobat doesn't apply any anti-aliasing. It seems to 'snap' the points to the grid and adjust the stroke width so it fills whole pixels. I ran some experiments and I *think* this is also what CSS does. I believe that this could be accomplished with a new attribute in the graphics state. Doing it programmatically should also work but is quite difficult. I wonder if our mozilla friends that work on shumway and pdf.js are running into this...
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Tue, Jul 16, 2013 at 6:41 PM, Ian Hickson i...@hixie.ch wrote:
This thread was gigantic and involved many proposals. I've only included
the last one below, since it seemed to take into account the most of the
feedback mentioned on the thread; I haven't responded to all the
intermediate e-mails which were mainly just a discussion amongst
contributors, and not direct feedback on the spec itself.
I haven't yet changed the spec. The main thrust of the feedback below ends
with the proposal to use WebGL's 'alpha' feature for the 2D context; is
this what implementors want to do?
We're implementing this in Chromium (currently behind the experimental
canvas features flag).
[...]
On Fri, 15 Feb 2013, Stephen White wrote (with roc's annotations inline
prefixed with | and mine inline not prefixed):
So let me take a stab at a brief summary of the proposals so far, and
the pros and cons of each (correct me if I missed anything):
opaque attribute or matteColor property
pro: fairly easy to implement
pro: no performance hit over regular rendering
pro: many opportunities for optimization
pro: catches all in-canvas cases of color fringing
con: does not handle any out-of-canvas color fringing
con: opt-in
| con: requires changes to canvas compositing spec and possibly
| implementations.
automatic opacity detection
pro: catches most (all?) cases of in-canvas color fringing
pro: some opportunties for optimization (must be conservative in some
cases)
con: does not catch color fringing on CSS transforms, canvas - WebGL,
etc
context.textAntialising = { 'none', 'grayscale', 'subpixel' }
pro: very easy to implement
pro: no performance hit
con: does not catch any cases of color fringing; completely up to web
developer
con: opt-in
| con: requires specification and implementation of what happens when
| subpixel AA is drawn over transparent background.
collect commands into a buffer, flush buffer only when compositing
canvas to page, and decide on subpixel AA at that point.
pro: catches all cases of color fringing
con: in some cases, requires an infinite buffer (e.g., a canvas that
never clears, and only accumulates drawing frame-to-frame means
you must accumulate commands indefinitely)
or giving up and using grayscale at some point
con: difficult to implement (e.g., canvas-to-canvas drawImage(), etc)
con: may introduce performance hit due to re-rendering with and without
subpixel AA (in cases where you would rather have just gone
without)
con: doesn't handle pixel manipulation cases (since you can't return
two sets of pixels and you can't regenerate the stuff that script
is generating based on the returned pixels)
two buffers (one grayscale, one LCD AA)
pro: handles all cases of color fringing
pro: moderately easy to implement
con: RAM (or VRAM) usage is doubled
con: possibly-unnecessary performance hit
con: must be opt-in
[...]
On Wed, 20 Feb 2013, Rik Cabanier wrote:
So now we have:
- don't do this on pinch-zoom devices
- don't do this for HW accelerated canvases
- don't do this if the canvas dpi doesn't match the screen
- don't do this if there are transforms
- authors will have to be very careful when using this feature since it
can
turn on or off or cause rendering glitches.
Is it still worth pursuing this?
On Thu, 21 Feb 2013, Stephen White wrote:
I believe it is. Even with those constraints, there are a large number
of applications which can benefit from text which looks as good as the
native platform can provide.
That said, I also think Robert is right that we should not spec out
precisely when subpixel AA text will occur in any of these automatic
modes, since:
1) there are some platforms/devices which don't do LCD text at all
2) It may be too restrictive for the browser implementor, e.g., they
may be essentially required to implement deferred rendering or two
backing stores in order to meet the resulting spec, which seems
onerous
Subpixel AA text aside, I still think it's worth spec'ing out mozOpaque,
if only just for the optimization opportunities that we don't get with
an automatic solution (e.g., putImageData). Its implementation is
fairly straightforward (much more so than the other options above), and
it won't break any existing content.
To me, the it breaks compositing argument falls into the doctor, it
hurts when I do this category: the user is specifically opting into an
opaque backing store, and so the changes in behaviour for compositing
modes which reference destination alpha are expected, just as they are
when using DST_ALPHA blending modes in a WebGL context created with the
alpha attribute set to false.
On Fri, 22 Feb 2013, Robert O'Callahan wrote:
I think Rik is convincing me that we shouldn't
Re: [whatwg] Adding 2D Canvas features (Was: Grouping in canvas 2d)
Conversely, if it helps to bring the spec closer to the implementations, one thing we do not intend to implement in Chrome is the automatic high-DPI canvas scaling (ie., auto-doubling of backing stores, getImageDataHD(), putImageDataHD(), etc). I believe Apple has also announced that they are dropping support for this in Safari 7. Stephen On Fri, Jun 28, 2013 at 3:30 PM, Tom Wiltzius wiltz...@chromium.org wrote: The only major Canvas2D features being actively developed in Chromium right now are: - having a canvas context alpha attribute - text decoration - compositing and blending - canvas access in workers (easily referenced from http://www.chromestatus.com/features) It is concerning to me that the list of other unimplemented features that aren't being worked on could block the standardization of the above (all of which have been discussed on this list at one point, but not all of which are in the spec yet). How can we help reconcile this discrepancy? On Fri, Jun 14, 2013 at 11:54 AM, Rik Cabanier caban...@gmail.com wrote: I agree that hit regions should be high on the priority list. They've been in the spec for a while and are absolutely required for accessibility. I will try to follow up on this feature with the browsers. We recently added a basic Path object to WebKit and I know that mozilla is looking at the path object. At this point, I wouldn't ask to add begin/endLayer to the spec. Instead, we will talk to the browser vendors and work on implementing the feature. Just putting it in the spec is not enough to get an implementation... On Fri, Jun 14, 2013 at 10:42 AM, Ian Hickson i...@hixie.ch wrote: On Fri, 14 Jun 2013, Brian Salomon wrote: As an implementor, we would prefer the layer approach. This would have lower overhead in Chromium/Skia. We can make better decisions about caching and deferred rendering. It also seems like a really handy API for devs, especially the ability to inherit the graphics state. Would the spec have anything to say about beginLayer()/endLayer() balancing, especially with respect to RAF? I have no ojection to adding this to the spec, but right now the spec has a bunch of features that aren't implemented, and there's a long list of other features people want that aren't yet specced. I'm very hesitant to get the spec further and further away from implementations. For example, here are some of the bug numbers for canvas feature requests: 11399 canvas Locking individual color channels (e.g. drawing to alpha only) 21835 canvas Path object should have a way to add paths keeping only the union given a fill rule 21939 canvas Path objects would be much more useful if their individual commands (moveTo, lineTo, etc.) could be inspected from JavaScript [...] 8794canvas lineWidth = 'hairline' 11739 canvas clearPath() that clears pixels the way clearRect() does, but using a path 9236canvas Detecting the intersection of Path objects 9235canvas perspective transformations 18751 canvas a way to get the coordinate of the last point in a path 21346 canvas Have ImageBitmap expose height and width attributes (Bugs accessible from https://www.w3.org/Bugs/Public/) There's also the printCallback API proposal from Mozilla: http://lists.w3.org/Archives/Public/public-whatwg-archive/2012Sep/0371.html Adding a parameter to drawImage for sprite sheets to avoid bleeding, proposal from Chrome: http://lists.w3.org/Archives/Public/public-whatwg-archive/2012Dec/0088.html Stroke alignment: http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Jul/0238.html Page flipping instead of double buffering: http://lists.w3.org/Archives/Public/public-whatwg-archive/2013Jan/0073.html Inner shadows: http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-November/038079.html Plus, as I mentioned, things in the spec that aren't implemented widely: Right now, the things in the spec that aren't widely implemented are the things that were needed for accessibility (hit regions) and the things that are the basis for some of the most-requested features (Paths). I think before we add more features, it's important that we figure out which browsers want to implement which features, and that we start with the highest priority ones. -- Ian Hickson U+1047E)\._.,--,'``. fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
Thanks for your comments.
On Fri, Apr 19, 2013 at 12:53 PM, Rik Cabanier caban...@gmail.com wrote:
Nice!
The behaviour of putImageData() and putImageDataHD() is to premultiply the
RGB components by the alpha component as usual, but write 1.0 into
destination alpha. In other words, if (r, g, b, a) are the component values
in a given pixel passed to putImageData[HD](), then r' = ar, g' = ag, b' =
ab are the colour components of the resulting canvas pixel, and (r', g',
b', 1.0) is written to the canvas backing store.
I think that's confusing. Maybe cut that sentence since it isn't needed.
(Having the black backing store implies this behavior)
Do you mean, remove both sentences, or just the last one? I wanted to make
it clear that the RGB is still premultiplied with the original alpha, even
though the alpha is subsequently ignored.
attribute boolean alpha;
Should it be read-only?
Hmm, I have no idea. WebGL's doesn't have it, but I'm not enough of an IDL
expert to know whether it's necessary.
Canvas2DContextAttributes getContextAttributes();
Maybe make it an attribute:
attribute Canvas2DContextAttributes attributes:
I'm following WebGL syntax here as well. Apparently WebGL's is nullable,
though, so I've added ? here.
Stephen
On Fri, Apr 19, 2013 at 9:13 AM, Stephen White
senorbla...@chromium.orgwrote:
Here's a short proposal I've written up for the getContext('2d', { alpha:
false } ) version of this idea (much of it culled from the mega-thread
above).
http://wiki.whatwg.org/wiki/CanvasOpaque
Comments are welcome.
Stephen
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
Would Mozilla (or other browser vendors) be interested in implementing the
hint as Gregg described above?
If so, we could break out the LCD text issue from canvas opacity, and
consider the latter on its own merits, since it has benefits apart from LCD
text (i.e., performance). Regarding that, if I'm reading correctly,
Vladimir Vukicevic has expressed support on webkit-dev for the
ctx.getContext('2d', { alpha: false }) proposal (basically, a syntactic
rewrite of canvas opaque). Does this indeed have traction with other
browser vendors?
As for naming, I would prefer that it be something like ctx.fontSmoothing
or ctx.fontSmoothingHint, to align more closely with canvas's
ctx.imageSmoothingEnabled and webkit's -webkit-font-smoothing CSS property.
-webkit-font-smoothing has none, antialiased and
subpixel-antialiased as options. I think it's ok to explicitly call out
subpixel antialiasing, even if the platform (or UA) does not support it,
especially if the attribute explicitly describes itself as a hint.
Stephen
On Sun, Mar 17, 2013 at 11:17 PM, Gregg Tavares g...@google.com wrote:
On Sun, Mar 17, 2013 at 1:40 PM, Robert O'Callahan rob...@ocallahan.org
wrote:
On Sat, Mar 16, 2013 at 5:52 PM, Gregg Tavares g...@google.com wrote:
Let me ask again in a different way ;-) Specifically about LCD style
antialiasing.
What about a context attribute antialiasRenderingQualityHint for now
with
2 settings default and displayDependent
context.antialiasRenderingQualityHint = displayDependent
How would this interact with canvas opacity? E.g. if the author uses
displayDependent and then draws text over transparent pixels in the
canvas,
what is the UA supposed to do?
Whatever the UA wants. It's a hint. From my POV, since the spec doesn't say
anything about anti-aliasing then it really doesn't matter.
My preference, if I was programming a UA, would be if the user sets
displayDependent and the UA is running on a lo-dpi machine I'd
unconditionally render LCD-AA with the assumption that the canvas is
composited on white. If they want some other color they'd fill the canvas
with as solid color first. Personally I don't think that needs to be
specced, but it would be my suggestion. As I mentioned, even without this
hint the spec doesn't prevent a UA from unconditionally using LCD-AA.
Very few developers are going to run into issues. Most developers that use
canvas aren't going to set the hint. Most developers that use canvas dont'
make it transparent nor do they CSS rotate/scale them. For those few
developers that do happen to blend and/or rotate/scale AND set the hint
they'll get probably get some fringing but there (a) there was no guarantee
they wouldn't already have that problem since as pointed out, the spec
doesn't specify AA nor what kind, and (b) if they care they'll either stop
using the hint or they'll search for why is my canvas fringy and the
answer will pop up on stackoverlow and they can choose one of the
solutions.
Rob
--
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe
fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Apr 3, 2013 at 12:04 PM, Gregg Tavares g...@google.com wrote:
On Wed, Apr 3, 2013 at 8:41 AM, Stephen White senorbla...@chromium.orgwrote:
Would Mozilla (or other browser vendors) be interested in implementing
the hint as Gregg described above?
If so, we could break out the LCD text issue from canvas opacity, and
consider the latter on its own merits, since it has benefits apart from LCD
text (i.e., performance). Regarding that, if I'm reading correctly,
Vladimir Vukicevic has expressed support on webkit-dev for the
ctx.getContext('2d', { alpha: false }) proposal (basically, a syntactic
rewrite of canvas opaque). Does this indeed have traction with other
browser vendors?
As for naming, I would prefer that it be something like ctx.fontSmoothing
or ctx.fontSmoothingHint, to align more closely with canvas's
ctx.imageSmoothingEnabled and webkit's -webkit-font-smoothing CSS property.
-webkit-font-smoothing has none, antialiased and
subpixel-antialiased as options. I think it's ok to explicitly call out
subpixel antialiasing, even if the platform (or UA) does not support it,
especially if the attribute explicitly describes itself as a hint.
Why call it Font smoothing? Shouldn't a UA be able to also render paths
using the same hint?
I think it would be better to control antialiasing for fonts and paths
independently. In addition to providing greater control, subpixel
antialiasing usually only benefits small, detailed paths such as fonts.
Large paths don't benefit greatly from it.
On a practical level, most platform graphics APIs don't provide subpixel
antialiasing for paths, only for text. So it would have to be implemented
as a custom path renderer, and it would have to have access to the LCD
subpixel structure and orientation, which many platform APIs may not
provide.
Stephen
Stephen
On Sun, Mar 17, 2013 at 11:17 PM, Gregg Tavares g...@google.com wrote:
On Sun, Mar 17, 2013 at 1:40 PM, Robert O'Callahan rob...@ocallahan.org
wrote:
On Sat, Mar 16, 2013 at 5:52 PM, Gregg Tavares g...@google.com
wrote:
Let me ask again in a different way ;-) Specifically about LCD style
antialiasing.
What about a context attribute antialiasRenderingQualityHint for now
with
2 settings default and displayDependent
context.antialiasRenderingQualityHint = displayDependent
How would this interact with canvas opacity? E.g. if the author uses
displayDependent and then draws text over transparent pixels in the
canvas,
what is the UA supposed to do?
Whatever the UA wants. It's a hint. From my POV, since the spec doesn't
say
anything about anti-aliasing then it really doesn't matter.
My preference, if I was programming a UA, would be if the user sets
displayDependent and the UA is running on a lo-dpi machine I'd
unconditionally render LCD-AA with the assumption that the canvas is
composited on white. If they want some other color they'd fill the canvas
with as solid color first. Personally I don't think that needs to be
specced, but it would be my suggestion. As I mentioned, even without this
hint the spec doesn't prevent a UA from unconditionally using LCD-AA.
Very few developers are going to run into issues. Most developers that
use
canvas aren't going to set the hint. Most developers that use canvas
dont'
make it transparent nor do they CSS rotate/scale them. For those few
developers that do happen to blend and/or rotate/scale AND set the hint
they'll get probably get some fringing but there (a) there was no
guarantee
they wouldn't already have that problem since as pointed out, the spec
doesn't specify AA nor what kind, and (b) if they care they'll either
stop
using the hint or they'll search for why is my canvas fringy and the
answer will pop up on stackoverlow and they can choose one of the
solutions.
Rob
--
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng
nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe
fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq
gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Mar 13, 2013 at 10:28 PM, Gregg Tavares g...@google.com wrote: On Wed, Mar 13, 2013 at 1:18 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Thu, Mar 14, 2013 at 8:04 AM, Gregg Tavares g...@google.com wrote: It seems like an opaque canvas should be an orthogonal issue to subpixel-aa. Subpixel AA seems like it should be a Canvas2DRenderingContext setting though maybe with a name like ctx.antialiasingRenderQuality = With options of none grayscale bestForDeviceIfAxisAlignedAndNotScaledOrBlended My mistake. They should be none alpha bestForDeviceIfNotCanvasIsNotRotatedAndCanvasIsNotScaledAndCanvasIsOpaque Don't forget AndCanvasIsNotFilteredAndCanvasIsNotDrawnViaWebGLThroughAShaderWhichModifiesFragmentColour. :) (And actually, this name sort of leads me to believe the opposite: that the API will take care of these cases for me, and I don't have to worry about them.) Naming aside, this is basically the proposal from message #1 in this thread (and mine from partway through). The objections were that this is a footgun with which web developers should not be trusted. For the record, I don't agree with that assessment. However, since it seemed that moz-opaque had at least some chance of being implemented by other browser vendors, and provides a generally useful optimization, I was pursuing that approach instead. Stephen ;-) Yes, I know that's a horrible name but it spells out the limitation of the higher quality aa needed on some devices. A dev can opt in (Since the default is alpha which is what happens today). If they opt in (a) it will look good if they follow the rules and (b) as the world transitions to HD-DPI it will end up being alpha so it's forward compatible. Ugh! This would let the developer choose. It would be clear what the limits are, when to use it, and would let the developer choose what they need, even in an opaque canvas. Then we would need to come up with a spec for what happens when you composite subpixel AA over non-opaque pixels, including how the per-channel alpha values are combined to form a single alpha value. IIRC in some cases (D2D) you just can't do it. If we said that in a non-opaque canvas, subpixel AA is treated as grayscale, that would be OK. sure. Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Mar 13, 2013 at 2:48 PM, Gregg Tavares g...@google.com wrote:
Sorry for only mentioning this so late but is there any chance to steer
this to be more inline with WebGL?
WebGL already has the option to have an opaque canvas using context
creation parameters. In WebGL it's
gl = canvas.getContext(webgl, {alpha: false});
If we go forward with an opaque attribute now you have 2 conflicting
settings.
canvas.opaque = true;
gl = canvas.getContext(webgl, {alpha: true});
Who wins that conflict? Yea, I know we could come up with rules. ( the 2
settings, etc...)
But, there are other context creation attributes we'd like to see on a 2d
canvas. One that comes to mind is 'preserveDrawingBuffer'.
preserveDrawingBuffer: false in WebGL means that the canvas is double
buffered. This is a performance win since most browsers using GPU
compositing need to copy the contents of the canvas when compositing.
Setting preseverDrawingBuffer: false (which is the default in WebGL) means
the browser can double buffer and avoid the copy. We'd like to see that
same attribute for 2D canvas/contexts to get the same perf benefit for
canvas games, etc.
So, given we want more creation attributes and given WebGL already has a
way to declare opaqueness why not follow the existing method and add
context creation parameters to 2d canvas to solve this issue rather than
make a new and conflicting 'opaque' attribute?
I have no major objections to this approach, so long as it doesn't make
this change contingent on a WebGL spec change. In particular, it's
tempting to unify the IDL between Canvas WebGL (although it may not be
necessary -- I'm far from an IDL expert.) Here, let me show you my
ignorance: can we create a
interface Canvas2DContextAttributes {
attribute boolean alpha;
}
which has no relation to WebGLContextAttributes?
Then we get at least the duck typing such that
ctx = canvas.getContext('2d', {alpha: false });
and
ctx = canvas.getContext('webgl', {alpha: false });
both work, although one is coerced into a WebGLContextAttributes and the
other to a Canvas2DContextAttributes.
Does that make sense?
Stephen
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Mon, Mar 11, 2013 at 4:32 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 12, 2013 at 8:23 AM, Stephen White senorbla...@chromium.orgwrote: On Mon, Mar 11, 2013 at 2:56 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 12, 2013 at 7:53 AM, Stephen White senorbla...@chromium.org wrote: All other canvas functionality behaves as normal, including operations which modify the alpha values of the backing store. However, any such transparency values will be ignored when compositing the canvas into the page, and the canvas will be treated as if every pixel has an alpha of 1.0. That would mean getImageData can return non-1.0 alpha values, which is probably not what you want to implement. That's what Firefox/Linux does (in fact, it always seems to return 0.0 alpha from getImageData()). We definitely shouldn't spec that! And I'm pretty sure that behavior would vary across Firefox platforms. But we need to have consistent behavior here. I considered three options: 1) Prevent non-1.0 alpha ever getting into the canvas. At a minimum, this would require the following: - For putImageData, apply premultiplication, then write 1.0 alpha into the canvas. - Change initialization and clearRect() to clear to opaque black instead of transparent black. - Modify all canvas compositing modes to leave destination alpha unchanged The latter is easy to do in OpenGL and CoreGraphics, but hard to do in Skia, and hard to do in accelerated CoreGraphics (IOSurfaces don't seem to support any opaque formats, although I could be wrong -- that was just from an hour or so of experimentation). I'm not sure about Cairo. You can always implement it slowly using readback. I think we should just spec this, and maybe note that authors shouldn't use non-over operators on opaque canvases. Over time we'll probably find a way to make it fast everywhere. I'm a little leery of spec'ing something that has negative performance implications. As an example, the darker compositing mode was removed from the spec due to hardware-accelerated performance concerns, IIRC. OTOH, unlike that change, this spec should not have performance implications for OpenGL or Direct3D acceleration, only CoreGraphics via IOSurface and skia (so far). How would you feel about simply mapping the dest-alpha-modifying compositing modes to source-over, as in proposal 2) above? Stephen Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
Here's a draft of proposal (1) above: Motivation: Compositing a canvas element into the page can be expensive, due to blending operations, and lack of opportunity for culling. Since arbitrary graphics operations can affect the opacity of the canvas, it is difficult to determine programmatically whether the canvas is opaque. Allowing the developer to explicitly mark a canvas as opaque allows the user agent to optimize blending at page composite time, as well to cull fully-obscured elements behind the canvas. Description: The opaque attribute is a boolean attribute of the canvas element, whose presence indicates that the alpha values in the canvas backing store must be 1.0 at all times. All canvas operations are modified to preserve this invariant. If the opaque attribute is not present, or if parsing its value returns an error, then the default value (false) must be used instead. When a canvas has the opaque attribute, the backing store must be initialized to opaque black (rgba(0, 0, 0, 1.0)), instead of transparent black (rgba(0, 0, 0, 0.0)). Setting, changing, removing or setting the attribute redundantly to its existing value causes the canvas to be cleared to the appropriate value. When a canvas has the opaque attribute, clearRect() clears to opaque black instead of transparent black. The behaviour of putImageData() and putImageDataHD() when a canvas has the opaque attribute is to premultiply the RGB components by the alpha component as usual, but write 1.0 into destination alpha. In other words, if (r, g, b, a) are the component values in a given pixel passed to putImageData[HD](), then r' = ar, g' = ag, b' = ab are the colour components of the resulting canvas pixel, and (r', g', b', 1.0) is written to the canvas backing store. When a canvas has the opaque attribute, all globalCompositeOperation modes behave as normal and the resulting RGB components are written to the canvas backing store, but the alpha component is left unchanged at 1.0. Stephen On Tue, Mar 12, 2013 at 12:53 PM, Stephen White senorbla...@chromium.orgwrote: On Mon, Mar 11, 2013 at 4:32 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 12, 2013 at 8:23 AM, Stephen White senorbla...@chromium.orgwrote: On Mon, Mar 11, 2013 at 2:56 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Tue, Mar 12, 2013 at 7:53 AM, Stephen White senorbla...@chromium.org wrote: All other canvas functionality behaves as normal, including operations which modify the alpha values of the backing store. However, any such transparency values will be ignored when compositing the canvas into the page, and the canvas will be treated as if every pixel has an alpha of 1.0. That would mean getImageData can return non-1.0 alpha values, which is probably not what you want to implement. That's what Firefox/Linux does (in fact, it always seems to return 0.0 alpha from getImageData()). We definitely shouldn't spec that! And I'm pretty sure that behavior would vary across Firefox platforms. But we need to have consistent behavior here. I considered three options: 1) Prevent non-1.0 alpha ever getting into the canvas. At a minimum, this would require the following: - For putImageData, apply premultiplication, then write 1.0 alpha into the canvas. - Change initialization and clearRect() to clear to opaque black instead of transparent black. - Modify all canvas compositing modes to leave destination alpha unchanged The latter is easy to do in OpenGL and CoreGraphics, but hard to do in Skia, and hard to do in accelerated CoreGraphics (IOSurfaces don't seem to support any opaque formats, although I could be wrong -- that was just from an hour or so of experimentation). I'm not sure about Cairo. You can always implement it slowly using readback. I think we should just spec this, and maybe note that authors shouldn't use non-over operators on opaque canvases. Over time we'll probably find a way to make it fast everywhere. I'm a little leery of spec'ing something that has negative performance implications. As an example, the darker compositing mode was removed from the spec due to hardware-accelerated performance concerns, IIRC. OTOH, unlike that change, this spec should not have performance implications for OpenGL or Direct3D acceleration, only CoreGraphics via IOSurface and skia (so far). How would you feel about simply mapping the dest-alpha-modifying compositing modes to source-over, as in proposal 2) above? Stephen Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
Here's a first draft of a proposal to standardize moz-opaque. (Note that Firefox/Linux and Firefox/Win differ in their implementation of moz-opaque. This proposal is most similar to the Firefox/Linux implementation, but with the canvas cleared to opaque black as in the Firefox/Win version). Motivation: Compositing a canvas element into the page can be expensive, due to blending operations, and lack of opportunity for culling. Since arbitrary graphics operations can affect the opacity of the canvas, it is difficult to determine programmatically whether the canvas is opaque. Allowing the developer to explicitly mark a canvas as opaque allows the user agent to optimize blending at page composite time, as well as culling obscured elements behind the canvas. Description: The opaque attribute is a boolean attribute of the canvas element, whose presence indicates that the alpha values in the canvas backing store must be ignored when compositing the canvas into the page. If the attribute is not present, or if parsing its value returns an error, then the default value (false) must be used instead. When a canvas has the opaque attribute, the backing store must be initialized to opaque black (rgba(0, 0, 0, 1.0)), instead of transparent black (rgba(0, 0, 0, 0.0)). Changing or removing the attribute after initialization causes the canvas to be cleared to the appropriate value. All other canvas functionality behaves as normal, including operations which modify the alpha values of the backing store. However, any such transparency values will be ignored when compositing the canvas into the page, and the canvas will be treated as if every pixel has an alpha of 1.0. Stephen On Fri, Mar 8, 2013 at 6:57 PM, Stephen White senorbla...@chromium.orgwrote: On Sat, Feb 23, 2013 at 6:48 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Sat, Feb 23, 2013 at 4:59 AM, Stephen White senorbla...@chromium.orgwrote: On Thu, Feb 21, 2013 at 7:01 PM, Rik Cabanier caban...@gmail.comwrote: On Fri, Feb 22, 2013 at 10:33 AM, Robert O'Callahan rob...@ocallahan.org wrote: I think a fully automatic solution that tries to use subpixel AA but is always able to render grayscale AA if needed is the way to go. Possibly with an author hint to suggest opting into a more expensive rendering path. Here are the problems I see with that approach: 1) In order to avoid a performance hit for existing content, it still requires a spec change (the hint) 2) Even with the hint, when the author knows they want LCD AA, they still incur a performance penalty of drawing to two buffers. 3) It still can't handle all cases, such as canvas - WebGL, which will have to remain grayscale-only, even when the author knows it would be safe for their application. I agree those are problems. All of the available options have problems. Given that that's the case, I am going to move forward with the opaque attribute, since I feel it is the lesser of all the evils presented thus far. Paying the cost of two buffers and double-rendering just isn't palatable, IMHO. Also, what form should this authoring hint take? Is it going to explicitly call out LCD AA? In that case, how is it better than an opt-in canvas attribute? If it doesn't explicitly call out LCD AA, but that's the only effect it has, what should it be called? Perhaps we could use text-rendering:optimizeLegibility on the canvas element. We also might be over-thinking the danger that LCD AA poses. Firefox/Linux and Firefox/Mac are both currently shipping with LCD AA turned on unconditionally in canvas, and it's trivial to make them expose color fringing. WebKit nightlies (Safari build) seem do the same, although Safari 6.0 doesn't. Stephen I also have concerns that the knowledge of when it's safe to use the LCD AA buffer is going to spread throughout the browser codebase, even in areas which currently have no knowledge of canvas, in order to handle all the special cases. This may just be an implementation detail (and may be avoidable, this is TBD), but it does have the potential to introduce dependencies or complicate implementation. Maybe. Maybe I'm missing something, but if we're going down the automatic road, why do we need a new function/attribute? Why not simply detect when a canvas-sized fillRect() has been performed with an opaque fillStyle? This would also allow optimization of existing content. I agree. Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Mon, Mar 11, 2013 at 2:56 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Mar 12, 2013 at 7:53 AM, Stephen White senorbla...@chromium.orgwrote: All other canvas functionality behaves as normal, including operations which modify the alpha values of the backing store. However, any such transparency values will be ignored when compositing the canvas into the page, and the canvas will be treated as if every pixel has an alpha of 1.0. That would mean getImageData can return non-1.0 alpha values, which is probably not what you want to implement. That's what Firefox/Linux does (in fact, it always seems to return 0.0 alpha from getImageData()). I considered three options: 1) Prevent non-1.0 alpha ever getting into the canvas. At a minimum, this would require the following: - For putImageData, apply premultiplication, then write 1.0 alpha into the canvas. - Change initialization and clearRect() to clear to opaque black instead of transparent black. - Modify all canvas compositing modes to leave destination alpha unchanged The latter is easy to do in OpenGL and CoreGraphics, but hard to do in Skia, and hard to do in accelerated CoreGraphics (IOSurfaces don't seem to support any opaque formats, although I could be wrong -- that was just from an hour or so of experimentation). I'm not sure about Cairo. 2) Same as (1), but force all destination-alpha-referencing compositing modes to source-over (or raise an exception). This seems somewhat draconian, and doesn't match what either Firefox implementation currently does. 3) Ignore the canvas per-pixel alpha at page composite time. This seems to be what Firefox/Linux does. Stephen Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Sat, Feb 23, 2013 at 6:48 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Sat, Feb 23, 2013 at 4:59 AM, Stephen White senorbla...@chromium.orgwrote: On Thu, Feb 21, 2013 at 7:01 PM, Rik Cabanier caban...@gmail.com wrote: On Fri, Feb 22, 2013 at 10:33 AM, Robert O'Callahan rob...@ocallahan.org wrote: I think a fully automatic solution that tries to use subpixel AA but is always able to render grayscale AA if needed is the way to go. Possibly with an author hint to suggest opting into a more expensive rendering path. Here are the problems I see with that approach: 1) In order to avoid a performance hit for existing content, it still requires a spec change (the hint) 2) Even with the hint, when the author knows they want LCD AA, they still incur a performance penalty of drawing to two buffers. 3) It still can't handle all cases, such as canvas - WebGL, which will have to remain grayscale-only, even when the author knows it would be safe for their application. I agree those are problems. All of the available options have problems. Given that that's the case, I am going to move forward with the opaque attribute, since I feel it is the lesser of all the evils presented thus far. Paying the cost of two buffers and double-rendering just isn't palatable, IMHO. Also, what form should this authoring hint take? Is it going to explicitly call out LCD AA? In that case, how is it better than an opt-in canvas attribute? If it doesn't explicitly call out LCD AA, but that's the only effect it has, what should it be called? Perhaps we could use text-rendering:optimizeLegibility on the canvas element. We also might be over-thinking the danger that LCD AA poses. Firefox/Linux and Firefox/Mac are both currently shipping with LCD AA turned on unconditionally in canvas, and it's trivial to make them expose color fringing. WebKit nightlies (Safari build) seem do the same, although Safari 6.0 doesn't. Stephen I also have concerns that the knowledge of when it's safe to use the LCD AA buffer is going to spread throughout the browser codebase, even in areas which currently have no knowledge of canvas, in order to handle all the special cases. This may just be an implementation detail (and may be avoidable, this is TBD), but it does have the potential to introduce dependencies or complicate implementation. Maybe. Maybe I'm missing something, but if we're going down the automatic road, why do we need a new function/attribute? Why not simply detect when a canvas-sized fillRect() has been performed with an opaque fillStyle? This would also allow optimization of existing content. I agree. Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Thu, Feb 21, 2013 at 7:01 PM, Rik Cabanier caban...@gmail.com wrote: On Fri, Feb 22, 2013 at 10:33 AM, Robert O'Callahan rob...@ocallahan.orgwrote: I think Rik is convincing me that we shouldn't expose mozOpaque or any other explicit subpixel AA control to the Web. It will be very easy for Web authors to test it in one place and discover that it works without realizing that they're causing problems for some users. I think a fully automatic solution that tries to use subpixel AA but is always able to render grayscale AA if needed is the way to go. Possibly with an author hint to suggest opting into a more expensive rendering path. Here are the problems I see with that approach: 1) In order to avoid a performance hit for existing content, it still requires a spec change (the hint) 2) Even with the hint, when the author knows they want LCD AA, they still incur a performance penalty of drawing to two buffers. 3) It still can't handle all cases, such as canvas - WebGL, which will have to remain grayscale-only, even when the author knows it would be safe for their application. Also, what form should this authoring hint take? Is it going to explicitly call out LCD AA? In that case, how is it better than an opt-in canvas attribute? If it doesn't explicitly call out LCD AA, but that's the only effect it has, what should it be called? I also have concerns that the knowledge of when it's safe to use the LCD AA buffer is going to spread throughout the browser codebase, even in areas which currently have no knowledge of canvas, in order to handle all the special cases. This may just be an implementation detail (and may be avoidable, this is TBD), but it does have the potential to introduce dependencies or complicate implementation. Great! I think matteColor (or matteStyle to be more consistent) can easily be implemented. We can optimize rendering later. So, if a mattecolor is set the UA can assume that: Maybe I'm missing something, but if we're going down the automatic road, why do we need a new function/attribute? Why not simply detect when a canvas-sized fillRect() has been performed with an opaque fillStyle? This would also allow optimization of existing content. Stephen - all compositing operation within the canvas can ignore background alpha - the canvas can be copied directly to the screen (unless another effect is applied to the canvas element or its ancestor) If mattecolor is set, the UA should matte with that color. If a compositing operation (that introduces alpha) is used, the matte operation needs to be repeated. Rik
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Tue, Feb 19, 2013 at 11:31 PM, Rik Cabanier caban...@gmail.com wrote: So now we have: - don't do this on pinch-zoom devices - don't do this for HW accelerated canvases - don't do this if the canvas dpi doesn't match the screen - don't do this if there are transforms - authors will have to be very careful when using this feature since it can turn on or off or cause rendering glitches. Is it still worth pursuing this? I believe it is. Even with those constraints, there are a large number of applications which can benefit from text which looks as good as the native platform can provide. That said, I also think Robert is right that we should not spec out precisely when subpixel AA text will occur in any of these automatic modes, since: 1) there are some platforms/devices which don't do LCD text at all 2) It may be too restrictive for the browser implementor, e.g., they may be essentially required to implement deferred rendering or two backing stores in order to meet the resulting spec, which seems onerous Subpixel AA text aside, I still think it's worth spec'ing out mozOpaque, if only just for the optimization opportunities that we don't get with an automatic solution (e.g., putImageData). Its implementation is fairly straightforward (much more so than the other options above), and it won't break any existing content. To me, the it breaks compositing argument falls into the doctor, it hurts when I do this category: the user is specifically opting into an opaque backing store, and so the changes in behaviour for compositing modes which reference destination alpha are expected, just as they are when using DST_ALPHA blending modes in a WebGL context created with the alpha attribute set to false. Stephen On Tue, Feb 19, 2013 at 3:40 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Tue, Feb 19, 2013 at 5:19 PM, Stephen White senorbla...@chromium.orgwrote: Even with text on an opaque background, I think you still have to worry about the case of transformed canvases. E.g., text drawn over an opaque background into a single still frame canvas, but then subsequently rotated via CSS transforms from 0 degrees through non-0. The first frame can use subpixel AA, but then subsequent frames can't. So I think you need to keep the command stream around (first case) or always render two buffers as soon as you draw text. That seems like a pretty heavy burden. For canvas-WebGL the problem becomes pretty much intractable, since there's no way to know what a given shader will do to the pixels. So I think you'd always have to give up and do grayscale AA in that case. Yes, you're quite right. Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Sat, Feb 16, 2013 at 4:09 AM, Robert O'Callahan rob...@ocallahan.orgwrote:
On Sat, Feb 16, 2013 at 11:09 AM, Stephen White
senorbla...@chromium.orgwrote:
On Fri, Feb 15, 2013 at 2:37 PM, Robert O'Callahan
rob...@ocallahan.orgwrote:
On Sat, Feb 16, 2013 at 4:35 AM, Stephen White senorbla...@chromium.org
wrote:
Even within canvas, there may be a way to break it if the LCD AA text
is drawn first and a dest-alpha compositing mode is used overtop (I haven't
verified this though).
I don't think that's a problem. All destination alpha values will be 1
even after the subpixel AA test is painted, and you'll treat it as normal.
I was thinking something like this:
- draw subpixel AA text over an opaque background
- draw a partially-transparent rect with destination-atop mode over the
subpixel AA text
Now some of the subpixel AA text is still there, but the destination
alpha is non-1, so things will go bad when you composite into the page.
(At least I think it will.. Porter-Duff makes my head spin sometimes.)
I don't see the problem. After the first step, you have an RGBA buffer
that represents exactly the result of the rendering. The second step treats
it exactly like any other RGBA buffer. Am I being dense?
No, it's probably me. I was thinking that since there was a way to
introduce non-1 alpha into the backing store after-the-fact, it would break
the previously-performed subpixel coverage blending. But perhaps not.
The suggestion on the list earlier was to keep two versions of the canvas
buffer: one with grayscale AA, another with subpixel AA, and composite with
the subpixel AA buffer when we can do that safely, otherwise use the
grayscale AA version. In many implementations there would be a performance
hit for this, so it would make sense to have authors opt-in to that
performance hit.
It would also be a needless performance hit if the developer knew that
they always wanted subpixel AA, and that their app would never fringe.
I don't think the developer can ever know that. They'd have to assume the
browser/platform does not have a quick-zoom feature, which is an assumption
developers shouldn't be making. Although I guess we could just declare that
that case isn't important enough to matter.
Well, pretty much all devices that have pinch-zoom today are high-DPI, and
don't implement subpixel AA. :) But no, I don't think that's not a good
assumption to bake into the web platform.
So let me take a stab at a brief summary of the proposals so far, and the
pros and cons of each (correct me if I missed anything):
moz-opaque
pro: fairly easy to implement
pro: no performance hit over regular rendering
pro: many opportunities for optimization
pro: catches all in-canvas cases of color fringing
con: does not handle any out-of-canvas color fringing
con: opt-in
Additional con: requires changes to canvas compositing spec and possibly
implementations.
Yes, since it's new behaviour it'll require spec changes. I guess we can
consider that as part of con of being opt-in.
automatic opacity detection
pro: catches most (all?) cases of in-canvas color fringing
pro: some opportunties for optimization (must be conservative in some
cases)
con: does not catch color fringing on CSS transforms, canvas - WebGL, etc
context attribute (something like: context.textAntialising = { 'none',
'grayscale', 'subpixel' })
pro: very easy to implement
pro: no performance hit
con: does not catch any cases of color fringing; completely up to web
developer
con: opt-in
Additional con: requires specification and implementation of what happens
when subpixel AA is drawn over transparent background.
deferred canvas rendering (collect commands into a buffer, flush buffer
only when compositing canvas to page, and decide on subpixel AA at that
point)
pro: catches all cases of color fringing
con: in some cases, requires an infinite buffer (e.g., a canvas that
never clears, and only accumulates drawing frame-to-frame means you must
accumulate commands indefinitely)
Not really true, you can just give up on the complex cases and draw
grayscale whenever you feel like it.
And leave the behaviour unspecified, I'm guessing? Doesn't that least to
inconsistent behaviour between browsers? Even within one implementation,
if you had content whose command stream varied over the maximum buffer
length threshold, it'd toggle between subpixel and grayscale AA. Not sure
if this is a case worth worrying about, but it does feel like it's exposing
more of the implementation that one would like.
Stephen
con: difficult to implement (e.g., canvas-to-canvas drawImage(), etc)
con: may introduce performance hit due to re-rendering with and without
subpixel AA (in cases where you would rather have just gone without)
two buffers (one grayscale, one LCD AA)
pro: handles all cases of color fringing
pro: moderately easy to implement
con: RAM
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Sat, Feb 16, 2013 at 4:12 AM, Robert O'Callahan rob...@ocallahan.orgwrote: On Sat, Feb 16, 2013 at 11:09 AM, Stephen White senorbla...@chromium.orgwrote: deferred canvas rendering (collect commands into a buffer, flush buffer only when compositing canvas to page, and decide on subpixel AA at that point) pro: catches all cases of color fringing con: in some cases, requires an infinite buffer (e.g., a canvas that never clears, and only accumulates drawing frame-to-frame means you must accumulate commands indefinitely) con: difficult to implement (e.g., canvas-to-canvas drawImage(), etc) con: may introduce performance hit due to re-rendering with and without subpixel AA (in cases where you would rather have just gone without) two buffers (one grayscale, one LCD AA) pro: handles all cases of color fringing pro: moderately easy to implement con: RAM (or VRAM) usage is doubled con: possibly-unnecessary performance hit con: must be opt-in Both of these schemes can actually be optimized some more: As long as no text is drawn to a canvas, you can freely rasterize (in the first case) or use just one buffer (in the second case). In fact, this is true as long as no text is drawn to a canvas over non-opaque pixels. So a lot of canvas usage could be handled with little or no performance hit. Even with text on an opaque background, I think you still have to worry about the case of transformed canvases. E.g., text drawn over an opaque background into a single still frame canvas, but then subsequently rotated via CSS transforms from 0 degrees through non-0. The first frame can use subpixel AA, but then subsequent frames can't. So I think you need to keep the command stream around (first case) or always render two buffers as soon as you draw text. That seems like a pretty heavy burden. For canvas-WebGL the problem becomes pretty much intractable, since there's no way to know what a given shader will do to the pixels. So I think you'd always have to give up and do grayscale AA in that case. Stephen Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Thu, Feb 14, 2013 at 10:21 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Thu, Feb 14, 2013 at 11:59 PM, Stephen White senorbla...@chromium.orgwrote: I think this is difficult to do in the general case, such as putImageData() or drawImage() or patterns, since you would need to examine all the pixels of the source image to determine if they contain non-1 alpha. This would be cost-prohibitive. If it's just for the purposes of optimization, you could be conservative and simply clear the flag when there's the potential for (but not certainty of) non-1 alpha. But if you're also using that flag to allow subpixel AA, the behaviour becomes quite unpredictable for the web developer and hard to spec crisply. What if we just said a) non-over operators clear the subpixel AA flag b) putImageData clears the subpixel AA flag Both of these are infrequently used AFAIK. Authors could work around most cases of b) by doing putImageData to another canvas and compositing it into the destination with 'over'. Even with these constraints, I don't think we can guarantee that it's safe to use LCD AA text. Once you've drawn with LCD AA text, even if it's safe at the time of drawing, there's no guarantee that it will be safe later. For instance, if you then drawImage() that canvas rotated into another canvas, or even just full-page-zoom it, you'll see colour fringing. Or apply CSS 2D or 3D transforms. There are also existing apps which use canvas for 2D text glyphs, and then transform and place them rotated in WebGL. Those will show colour fringing. Even within canvas, there may be a way to break it if the LCD AA text is drawn first and a dest-alpha compositing mode is used overtop (I haven't verified this though). So I'm starting to think that LCD AA text really has to be opt-in, to avoid breaking existing content. By opting it, you're agreeing that these artifacts are acceptable for your app. For example, you know that even if you're going to do a canvas-to-canvas draw, you're always going to draw at 1:1 scale and no rotation. Stephen How bad would that be? Would it help if we added an attribute to the canvas to let authors detect whether the subpixel AA flag is set? Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Fri, Feb 15, 2013 at 2:37 PM, Robert O'Callahan rob...@ocallahan.orgwrote:
On Sat, Feb 16, 2013 at 4:35 AM, Stephen White
senorbla...@chromium.orgwrote:
Even with these constraints, I don't think we can guarantee that it's
safe to use LCD AA text. Once you've drawn with LCD AA text, even if it's
safe at the time of drawing, there's no guarantee that it will be safe
later. For instance, if you then drawImage() that canvas rotated into
another canvas, or even just full-page-zoom it, you'll see colour fringing.
Or apply CSS 2D or 3D transforms. There are also existing apps which use
canvas for 2D text glyphs, and then transform and place them rotated in
WebGL. Those will show colour fringing.
This came up on the list earlier.
Even within canvas, there may be a way to break it if the LCD AA text is
drawn first and a dest-alpha compositing mode is used overtop (I haven't
verified this though).
I don't think that's a problem. All destination alpha values will be 1
even after the subpixel AA test is painted, and you'll treat it as normal.
I was thinking something like this:
- draw subpixel AA text over an opaque background
- draw a partially-transparent rect with destination-atop mode over the
subpixel AA text
Now some of the subpixel AA text is still there, but the destination alpha
is non-1, so things will go bad when you composite into the page. (At
least I think it will.. Porter-Duff makes my head spin sometimes.)
So I'm starting to think that LCD AA text really has to be opt-in, to
avoid breaking existing content. By opting it, you're agreeing that these
artifacts are acceptable for your app. For example, you know that even if
you're going to do a canvas-to-canvas draw, you're always going to draw at
1:1 scale and no rotation.
It's difficult to know that on a mobile browser or any other browser where
you have some kind of fast zoom UI.
The suggestion on the list earlier was to keep two versions of the canvas
buffer: one with grayscale AA, another with subpixel AA, and composite with
the subpixel AA buffer when we can do that safely, otherwise use the
grayscale AA version. In many implementations there would be a performance
hit for this, so it would make sense to have authors opt-in to that
performance hit.
It would also be a needless performance hit if the developer knew that they
always wanted subpixel AA, and that their app would never fringe.
Rob
--
Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur
Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl
bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat
lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir
— whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb
tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
So let me take a stab at a brief summary of the proposals so far, and the
pros and cons of each (correct me if I missed anything):
moz-opaque
pro: fairly easy to implement
pro: no performance hit over regular rendering
pro: many opportunities for optimization
pro: catches all in-canvas cases of color fringing
con: does not handle any out-of-canvas color fringing
con: opt-in
automatic opacity detection
pro: catches most (all?) cases of in-canvas color fringing
pro: some opportunties for optimization (must be conservative in some
cases)
con: does not catch color fringing on CSS transforms, canvas - WebGL, etc
context attribute (something like: context.textAntialising = { 'none',
'grayscale', 'subpixel' })
pro: very easy to implement
pro: no performance hit
con: does not catch any cases of color fringing; completely up to web
developer
con: opt-in
deferred canvas rendering (collect commands into a buffer, flush buffer
only when compositing canvas to page, and decide on subpixel AA at that
point)
pro: catches all cases of color fringing
con: in some cases, requires an infinite buffer (e.g., a canvas that never
clears, and only accumulates drawing frame-to-frame means you must
accumulate commands indefinitely)
con: difficult to implement (e.g., canvas-to-canvas drawImage(), etc)
con: may introduce performance hit due to re-rendering with and without
subpixel AA (in cases where you would rather have just gone without)
two buffers (one grayscale, one LCD AA)
pro: handles all cases of color fringing
pro: moderately easy to implement
con: RAM (or VRAM) usage is doubled
con: possibly-unnecessary performance hit
con: must be opt-in
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Feb 13, 2013 at 11:35 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Thu, Feb 14, 2013 at 5:16 PM, Rik Cabanier caban...@gmail.com wrote: Looking at the WebKit implementation, I'm unsure how 'opaque' can implemented for accelerated canvas. It might work with non-accelerated canvas but would have to run some experiments. I also look at mozilla's Core Graphics implementation and unless I'm missing something, it doesn't have special code to handle 'opaque'. When do you use this parameter? CanvasRenderingContext2D::GetSurfaceFormat is part of the process. That selects a surface format that is passed down to the graphics layer when creating the canvas surface. It's true that we don't currently do anything with that when drawing with CoreGraphics. That would need to be cleaned up before we started promoting this feature. Now that you mention it, having to modify the definition of compositing is a bit of a bummer for the 'opaque' attribute approach. I think we could do everything we want using your approach --- internally keeping a flag to indicate whether the alpha values of the canvas are all 1, setting it when the canvas is filled with a solid color and clearing it when non-over drawing (or clear()) are used. Let's try that! I think this is difficult to do in the general case, such as putImageData() or drawImage() or patterns, since you would need to examine all the pixels of the source image to determine if they contain non-1 alpha. This would be cost-prohibitive. If it's just for the purposes of optimization, you could be conservative and simply clear the flag when there's the potential for (but not certainty of) non-1 alpha. But if you're also using that flag to allow subpixel AA, the behaviour becomes quite unpredictable for the web developer and hard to spec crisply. We could consider separating the two concepts again. In an earlier thread, there was an attempt to automatically determine all the places where it's safe to enabled subpixel AA, but that seemed to result in a complex implementation, with all cases still not being covered (such as canvas-to-canvas drawImage()). The other alternative is programmatic control over subpixel AA, using a context attribute. That was the first thing that Justin proposed in the earlier thread, and would be my preference as well (a fully-loaded footgun: you can shoot yourself with it, but the behaviour and performance characteristics are very easy to understand and spec). But there didn't seem to be agreement around that either. Which is how I ended up at the moz-opaque flag. it restricts canvas to the subset of operations which result in a 1 alpha in the backing store, to allow optimizations and the use of subpixel AA. I think that is actually quite a useful subset (generally, the subset that doesn't need destination alpha). I believe the same thing can be achieved in WebGL by setting the alpha attribute to false in WebGLContextAttributes. Stephen But I think matte is unnecessarily obscure. How about adding a clear(DOMString) method that does a 'copy' of the color to the entire canvas buffer? The color could default to rgba(0,0,0,0). Rob -- Wrfhf pnyyrq gurz gbtrgure naq fnvq, “Lbh xabj gung gur ehyref bs gur Tragvyrf ybeq vg bire gurz, naq gurve uvtu bssvpvnyf rkrepvfr nhgubevgl bire gurz. Abg fb jvgu lbh. Vafgrnq, jubrire jnagf gb orpbzr terng nzbat lbh zhfg or lbhe freinag, naq jubrire jnagf gb or svefg zhfg or lbhe fynir — whfg nf gur Fba bs Zna qvq abg pbzr gb or freirq, ohg gb freir, naq gb tvir uvf yvsr nf n enafbz sbe znal.” [Znggurj 20:25-28]
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Tue, Feb 12, 2013 at 6:14 PM, Rik Cabanier caban...@gmail.com wrote: On Tue, Feb 12, 2013 at 2:56 PM, Stephen White senorbla...@chromium.orgwrote: On Fri, Nov 23, 2012 at 6:04 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 29 Mar 2012, Jeremy Apthorp wrote: On Thu, Mar 29, 2012 at 10:25 AM, Jeremy Apthorp jere...@chromium.org wrote: On Thu, Mar 29, 2012 at 8:41 AM, Ian Hickson i...@hixie.ch wrote: On Fri, 13 Jan 2012, Jeremy Apthorp wrote: I'd like to draw non-antialiased lines in a canvas. Currently it seems that the only way to do this is to directly access the pixel data. Is there a reason there's no way to turn off antialiasing? What's the use case? Pixel-art style games. Specifically: even with the new image smoothing stuff in place for drawImage, a 1:2 diagonal line will still be anti-aliased (only the antialiasing will look silly scaled up to 2x). Do you have an example of a game where lines are drawn using a line API without antialiasing, then scaled up? Most pixel art games I've seen tend to use bitmaps for that kind of thing. On Mon, 12 Nov 2012, Justin Novosad wrote: For many types of apps, DOM-based rendering is uncompetitively slow [so we should make text rendering in canvas more controllable] This seems like something we should fix, not something we should work around by having people use canvas instead. Using canvas has all kinds of terrible side-effects, like reducing the likely accessibility of the page, making searcheability much worse, etc. Also, do you have any metrics showing the magnitude of this problem on real-world sites that might consider using canvas instead? If LCD text were enable-able, authors would have to be mindful of a number of caveats in order to avoid rendering artifacts. Do we have any reason to believe the majority of authors would make the right decisions here? (The main reason we haven't provided control over things like antialiasing is that many authors tend to make terribly bad decisions.) (Before anyone gets offended, by the way: that you are reading this almost guarantees that you are above average in terms of authoring ability.) On Tue, Nov 13, 2012 at 9:37 PM, Robert O'Callahan wrote: We'd have to define what happens when you use subpixel antialiasing incorrectly, because we can be pretty sure authors will use it incorrectly and expect to get interoperable behavior. That's certainly true. Mozilla supports a mozOpaque attribute which makes the canvas buffer RGBX (initialized to solid black) and enables subpixel antialiasing for most text drawing. That might be enough to address your use-cases. I haven't specified this; if other vendors intend to implement this let me know and I can spec it. I'm not sure it's worth it though. [blowing the dust off this thread] Folks on the Chrome team are looking into implementing this attribute, and would be interested in seeing it spec'ed. What are you implementing? Initializing the canvas to black or subpixel antialiasing? We're interested in both aspects: the opportunity for culling and blending optimizations at composite time, as well as enabling subpixel AA. Stephen On Wed, 14 Nov 2012, Robert O'Callahan wrote: On Wed, Nov 14, 2012 at 8:09 AM, Justin Novosad ju...@chromium.org wrote: Are there precedents for exposing features with documented caveats? (excluding caveats that were discovered after the fact) Yes, and many of them have been extremely problematic, because Web authors will ignore the caveats. Right. I'd really like to avoid adding more if we can help it. On Wed, 14 Nov 2012, Justin Novosad wrote: There is a recent improvement in Chrome called deferred 2D canvas rendering (enabled by default as of Chrome 23). It is a mechanism that records 2d canvas commands during JS execution, and only executes them for real when the render buffer needs to be resolved (draw to screen, getImageData, toDataURL, etc.). If you want to check it out, the guts are in Skia: SkGPipe is a sort of FIFO for graphics commands, SkDeferredCanvas is a wrapper that manages the GPipe and automatically flushes it and applies some command culling optimizations. So to come back to the problem of with and without subpixel AA buffers: if rendering is deferred, the non-AA buffer would never get rasterized (and possibly never even allocated), unless it needs to be. Obviously there are practical limitations, for example we cannot store an unlimited stream of recorded commands, so if the canvas draws indefinitely without ever being cleared, at some point we have to rasterize the non-AA buffer just so that we can safely discard the recording data. Also, if at record time the necessary conditions for subpixel AA are not met, perhaps we just
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Feb 13, 2013 at 12:22 PM, Rik Cabanier caban...@gmail.com wrote: For blending optimizations, it might be better to introduce a function instead of a boolean attribute like 'opaque'. What you really want, is to matte [1] the canvas with a solid color so you can optimize compositing. How about this API: void applyMatte(DOMString color); // color is a CSS rgb color value (alpha is ignored) When you call this function, the canvas is matted with that color. If it's the first drawing call, you can just fill the canvas with that color (no compositing needed) After matting, you no longer have to read or update the alpha channel since it's always 1 which should speed up drawing. Just to be sure we're on the same page, when I mentioned compositing optimizations, I was referring to compositing the canvas backing store into the page, not compositing operations within the canvas itself. One advantage of using an element attribute is that it could be used at backing store allocation time, to allocate RGB instead of RGBA. Forcing a reallocation of the backing store on attribute change would be consistent with changing width and height of the canvas, which have the same effect. Doing so on a context operation would not. If we did use a context function approach as you suggest, how would subpixel AA be handled? Would it be enabled on first call of the function and never disabled? Is there a way to query if the canvas is opaque once it's called? (I'm assuming that all changes to canvas alpha after the first call would have to be ignored, since otherwise you'd have to sniff every operation to see if it affected alpha, and reset the bit, although perhaps I'm misunderstanding your proposal.) Stephen
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Wed, Feb 13, 2013 at 5:31 PM, Rik Cabanier caban...@gmail.com wrote: On Wed, Feb 13, 2013 at 11:25 AM, Stephen White senorbla...@chromium.org wrote: On Wed, Feb 13, 2013 at 12:22 PM, Rik Cabanier caban...@gmail.com wrote: For blending optimizations, it might be better to introduce a function instead of a boolean attribute like 'opaque'. What you really want, is to matte [1] the canvas with a solid color so you can optimize compositing. How about this API: void applyMatte(DOMString color); // color is a CSS rgb color value (alpha is ignored) When you call this function, the canvas is matted with that color. If it's the first drawing call, you can just fill the canvas with that color (no compositing needed) After matting, you no longer have to read or update the alpha channel since it's always 1 which should speed up drawing. Just to be sure we're on the same page, when I mentioned compositing optimizations, I was referring to compositing the canvas backing store into the page, not compositing operations within the canvas itself. sorry, I didn't mean to say blending. This is for optimizing compositing within the canvas and of the canvas into the page. One advantage of using an element attribute is that it could be used at backing store allocation time, to allocate RGB instead of RGBA. Forcing a reallocation of the backing store on attribute change would be consistent with changing width and height of the canvas, which have the same effect. Doing so on a context operation would not. why not? There is no reason for you to allocate the backing store until it's needed. True. I was just trying to understand when you would use this function more than once during canvas lifetime. Presumably, that's the reason for having a function, no? If so, and you don't maintain any special state about the opacity of the canvas, how is this different from an rgb() fillRect()? The strange thing with an element attribute is that you can't change it back and it's also detached from the JS code that does the drawing. You can change it back programmatically through the DOM, as you can with the element width and height. If we did use a context function approach as you suggest, how would subpixel AA be handled? Would it be enabled on first call of the function and never disabled? I did not think about subpixel AA. If I read the mozilla proposal correctly, they do AA if they know that the canvas is opaque. So, with my proposal, there would be no AA until you call 'applyMatte' (assuming you follow the mozilla way of doing AA). OK, so there would have to be a bit of state saved at that point, in order to know if subpixel AA is allowable? At any rate, I don't think this works if you subsequently modify the canvas's alpha (see below). Is there a way to query if the canvas is opaque once it's called? Wouldn't the user know that he called 'applyMatte'? Also why do you want to query it? (I'm assuming that all changes to canvas alpha after the first call would have to be ignored, since otherwise you'd have to sniff every operation to see if it affected alpha, and reset the bit, although perhaps I'm misunderstanding your proposal.) No, you don't have to do that and can still use alpha. Simple alpha compositing is defined as follows [1] (in premultiplied alpha): co = cs + cb x (1 - αs) αo = αs + αb x (1 - αs) If you know that the backdrop is matted (αb = 1), the second formula always resolves to 1 so you can skip it. The problem is, canvas supports more than simple (source-over) compositing, so there are ways of modifying the destination alpha back to non-1, e.g., a primitive with non-1 alpha drawn with source-copy, or via putImageData(). At that point, the state of the canvas's alpha is unknown, so the page compositor can make no assumptions about its opacity. Stephen
Re: [whatwg] Enabling LCD Text and antialiasing in canvas
On Fri, Nov 23, 2012 at 6:04 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 29 Mar 2012, Jeremy Apthorp wrote: On Thu, Mar 29, 2012 at 10:25 AM, Jeremy Apthorp jere...@chromium.org wrote: On Thu, Mar 29, 2012 at 8:41 AM, Ian Hickson i...@hixie.ch wrote: On Fri, 13 Jan 2012, Jeremy Apthorp wrote: I'd like to draw non-antialiased lines in a canvas. Currently it seems that the only way to do this is to directly access the pixel data. Is there a reason there's no way to turn off antialiasing? What's the use case? Pixel-art style games. Specifically: even with the new image smoothing stuff in place for drawImage, a 1:2 diagonal line will still be anti-aliased (only the antialiasing will look silly scaled up to 2x). Do you have an example of a game where lines are drawn using a line API without antialiasing, then scaled up? Most pixel art games I've seen tend to use bitmaps for that kind of thing. On Mon, 12 Nov 2012, Justin Novosad wrote: For many types of apps, DOM-based rendering is uncompetitively slow [so we should make text rendering in canvas more controllable] This seems like something we should fix, not something we should work around by having people use canvas instead. Using canvas has all kinds of terrible side-effects, like reducing the likely accessibility of the page, making searcheability much worse, etc. Also, do you have any metrics showing the magnitude of this problem on real-world sites that might consider using canvas instead? If LCD text were enable-able, authors would have to be mindful of a number of caveats in order to avoid rendering artifacts. Do we have any reason to believe the majority of authors would make the right decisions here? (The main reason we haven't provided control over things like antialiasing is that many authors tend to make terribly bad decisions.) (Before anyone gets offended, by the way: that you are reading this almost guarantees that you are above average in terms of authoring ability.) On Tue, Nov 13, 2012 at 9:37 PM, Robert O'Callahan wrote: We'd have to define what happens when you use subpixel antialiasing incorrectly, because we can be pretty sure authors will use it incorrectly and expect to get interoperable behavior. That's certainly true. Mozilla supports a mozOpaque attribute which makes the canvas buffer RGBX (initialized to solid black) and enables subpixel antialiasing for most text drawing. That might be enough to address your use-cases. I haven't specified this; if other vendors intend to implement this let me know and I can spec it. I'm not sure it's worth it though. [blowing the dust off this thread] Folks on the Chrome team are looking into implementing this attribute, and would be interested in seeing it spec'ed. Thanks, Stephen On Wed, 14 Nov 2012, Robert O'Callahan wrote: On Wed, Nov 14, 2012 at 8:09 AM, Justin Novosad ju...@chromium.org wrote: Are there precedents for exposing features with documented caveats? (excluding caveats that were discovered after the fact) Yes, and many of them have been extremely problematic, because Web authors will ignore the caveats. Right. I'd really like to avoid adding more if we can help it. On Wed, 14 Nov 2012, Justin Novosad wrote: There is a recent improvement in Chrome called deferred 2D canvas rendering (enabled by default as of Chrome 23). It is a mechanism that records 2d canvas commands during JS execution, and only executes them for real when the render buffer needs to be resolved (draw to screen, getImageData, toDataURL, etc.). If you want to check it out, the guts are in Skia: SkGPipe is a sort of FIFO for graphics commands, SkDeferredCanvas is a wrapper that manages the GPipe and automatically flushes it and applies some command culling optimizations. So to come back to the problem of with and without subpixel AA buffers: if rendering is deferred, the non-AA buffer would never get rasterized (and possibly never even allocated), unless it needs to be. Obviously there are practical limitations, for example we cannot store an unlimited stream of recorded commands, so if the canvas draws indefinitely without ever being cleared, at some point we have to rasterize the non-AA buffer just so that we can safely discard the recording data. Also, if at record time the necessary conditions for subpixel AA are not met, perhaps we just forget about it. I admit this is a complex solution for implementors, but it makes the management of subpixel-AA safety transparent to web authors. I think it'd be reasonable (for some definition of reasonable that relates to whether it's compatible with the spec, anyway) for implementors to do this today, without having to expose any control to the author. On Thu, 15 Nov 2012, Fred Andrews wrote: The canvas that scripts draw into could be over-sized with the UA down sampling
Re: [whatwg] Canvas in Workers
On Thu, Jan 3, 2013 at 7:46 PM, Gregg Tavares g...@google.com wrote:
On Tue, Dec 11, 2012 at 9:04 AM, Ian Hickson i...@hixie.ch wrote:
On Tue, 11 Dec 2012, Gregg Tavares (社ç~T¨) wrote:
discussion seems to have died down here but I'd like to bring up
another
issue
In WebGL land we have creation attributes on the drawingbuffer made
for a
canvas. Example
gl = canvas.getContext(webgl, { preserveDrawingBuffer: false });
We're working out the details on how to set those options for the case
where we have 1 context and multiple canvases.
The particular option above would apparently be a huge perf win for
canvas 2d for mobile. Which suggests that whatever API is decided on it
would be nice if it worked for both APIs the same.
What does it do?
Effectively it makes the canvas double buffered.
Right now by 2d canvases are effectively single buffered. At the
appropriate time a copy
of the canvas is made and passed to the compositor. This copy is slow,
especially on
mobile.
Currently, to lower the VRAM footprint and improve performance, we don't do
a copy in 2d canvas. We temporarily transfer ownership of the texture to
the compositor at commit time, and block the renderer until the composite
is complete. That may change, however.
Stephen
Apple requested that for WebGL the default be for double buffering. When
double buffered, when the canvas is composited (when the current JavaScript
event exits)
the canvas's buffer is given to the compositor and the canvas is given a
new buffer (or
an old one). That new buffer is cleared, meaning the contents is gone. It's
up to the app to
draw stuff into again. If nothing is drawn the compositor will continue to
use the
buffer it acquired earlier.
In WebGL you can opt into the slower copy path. For Canvas 2D while the
default
has to remain the slow copy path it would be nice to be able to opt into
the faster
swap double buffered path.
In the 2D canvas, whenever you bind to a new canvas, the context is reset
to its default state, the context's hit region list is reset, and the
context's bitmap is reset. The next time the context is flushed, the
canvas itself is always reset (since flushing the context causes the
bitmap and hit region list to be pushed to the canvas, replacing whatever
was there before).
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[whatwg] Canvas: imageSmoothingEnabled not part of the state stack?
Looking into some of the features recently added to the canvas spec, I noticed that imageSmoothingEnabled wasn't added to the list of attributes saved and restored in the canvas state (in sec. 4.8.11.1.1). This also seems to be the case for the new line dash parameters. Intentional, or oversight? Stephen
