Re: [whatwg] [canvas] imageRenderingQuality property
On Tue, 1 Jul 2008, Philip Taylor wrote: On 01/07/2008, Ian Hickson i...@hixie.ch wrote: It seems better for the browser to simply detect when the graphics burden being placed on the hardware by the page is too much to be done at high quality given the current load on the CPU, and for the browser to automatically drop down to a lower fidelity, higher speed rendering on the fly when appropriate. Sometimes the author will want to force best-quality rendering, regardless of the performance impact. E.g. a photo manipulation application might let you resize a segment of a photo, displaying a live preview (where performance is more important than quality), and then render the final resized image and store it in a canvas for future processing. That final rendering needs to be the best possible quality, so it's not acceptable for the browser to decide that it should semi-randomly drop the quality because it detected the live preview was CPU-intensive. Similarly, a pseudo-3d FPS game might load textures at runtime and perform some preprocessing (like resizing to be square, and rendering lots of smaller copies to be used as mipmaps for distant walls so they look prettier), and then draw that processed texture into the game thousands of times a second. Since the preprocessing is only done once, and its result is reused for the whole of the rest of the game, it should be done at the highest possible quality, regardless of performance. So, adaptively reducing the quality and allowing no author control seems like a bad idea. Perhaps the imageRenderingQuality property could have values 'high' and 'auto', where the default is 'high' (so that existing content continues working the same as it always has, and to avoid surprising authors by randomly switching the rendering quality when they have no reason to expect such weird behaviour), and 'auto' means 'low (but perhaps switch to high if the browser thinks it's going to be fast enough)'. That would avoid the issue of authors setting quality='low' and preventing high-speed users from getting the best quality output. I've made a note of this in the spec: // v4 we've also received requests for: attribute boolean span title=dom-context-2d-forceHighQualityforceHighQuality/span // (default false) // when enabled, it would prevent the UA from falling back on lower-quality but faster rendering routines // useful e.g. for when an image manipulation app uses canvas both for UI previews and the actual work I haven't added it yet because mozImageSmoothingEnabled doesn't seem to be used much (if at all?) and the uses of canvas so far have apparently not really run into serious performance problems yet. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] [canvas] imageRenderingQuality property
The image rendering quality property is indeed unable to hit the tradeoff between beauty of presentation and rendering speed. However, it is perfectly all right to say 'this content is some fancy GUI can be rendered downscaled without degrading the content - but that content contains engineering drawings that must be rendered as accurately as can be.' This is semantic information the browser has no way of inferring. Cheers, Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Oliver Hunt Sent: Tuesday, July 01, 2008 2:18 AM To: Mark Finkle Cc: Vladimir Vukicevic; Robert O'Callahan; WHATWG; David Hyatt; Jerason Banes; Ian Hickson; Robert O'Callahan Subject: Re: [whatwg] [canvas] imageRenderingQuality property So now we need to define levels of graphic burden? and at what level of burden does the quality suffer? Seems just as hard to define. Having the author explicit say this has to be as high quality as possible or less can be low quality seems better and we have examples of other specs offering the same kind of control. No. The whole point is that the UA is in the best position to identify what the tradeoffs are, not the author -- if you want a flag to specify the quality to be used then that would require you to determine what the tradeoffs were yourself, with no substantial knowledge of what combination any given user was actually using. You need to realise that different UAs and different platforms have substantially different performance characteristics.
Re: [whatwg] [canvas] imageRenderingQuality property
Yeah, Philip has been slowly convincing me that something like imageRenderingQuality with high and auto values* would be useful, basically for reasons similar to what you just gave. I still don't believe there should be a low option, because (as i have said before) the author is not in a position to be able to determine what performance tradeoffs are involved in the users system. --Oliver * This property was suggested by Philip Taylor in an earlier email On Jul 1, 2008, at 1:18 AM, Kristof Zelechovski wrote: The image rendering quality property is indeed unable to hit the tradeoff between beauty of presentation and rendering speed. However, it is perfectly all right to say 'this content is some fancy GUI can be rendered downscaled without degrading the content - but that content contains engineering drawings that must be rendered as accurately as can be.' This is semantic information the browser has no way of inferring. Cheers, Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Oliver Hunt Sent: Tuesday, July 01, 2008 2:18 AM To: Mark Finkle Cc: Vladimir Vukicevic; Robert O'Callahan; WHATWG; David Hyatt; Jerason Banes; Ian Hickson; Robert O'Callahan Subject: Re: [whatwg] [canvas] imageRenderingQuality property So now we need to define levels of graphic burden? and at what level of burden does the quality suffer? Seems just as hard to define. Having the author explicit say this has to be as high quality as possible or less can be low quality seems better and we have examples of other specs offering the same kind of control. No. The whole point is that the UA is in the best position to identify what the tradeoffs are, not the author -- if you want a flag to specify the quality to be used then that would require you to determine what the tradeoffs were yourself, with no substantial knowledge of what combination any given user was actually using. You need to realise that different UAs and different platforms have substantially different performance characteristics.
Re: [whatwg] [canvas] imageRenderingQuality property
Challenge accepted. THIS: the content the author wants the browser to render. HAPPEN: the sequence of operations performed by the browser in order to render THIS. QUICKLY: meeting the user's expectations how long he should wait for THIS. QUALITY: how accurately the browser follows the author's prescription. EXTRA TIME: the amount of time the rendering process is late with respect to the QUICKLY deadline. HIGHEST: most accurate. HTH, Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Charles Sent: Tuesday, July 01, 2008 1:52 AM To: 'WHATWG' Subject: Re: [whatwg] [canvas] imageRenderingQuality property How can an author know which is appropriate? Erm, presumably because they're the author -- it seems quite valid to for an author to be able to say Just make this happen quickly, I don't care about the quality or Take extra time to make this the highest quality you can. No problem, all you have to do is define: - this - happen - quickly - quality - extra time - highest Of course the hard bit is that these change for every content developer, often per-platform. -- Charles
Re: [whatwg] [canvas] imageRenderingQuality property
On Mon, 30 Jun 2008, Vladimir Vukicevic wrote: On Jun 11, 2008, at 3:34 AM, Ian Hickson wrote: On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). How can an author know which is appropriate? Erm, presumably because they're the author -- it seems quite valid to for an author to be able to say Just make this happen quickly, I don't care about the quality or Take extra time to make this the highest quality you can. But which of those to pick depends on the hardware at least as much as the complexity of the drawing code, as has been noted before. This table shows what the author would want to pick for a given page: 2008: 2008 desktop2008 mobile Complex Graphics high-qualityhigh-speed Simple Graphics high-qualityhigh-quality 2012: 2012 desktop2012 mobile2008 desktop 2018-era Graphicshigh-qualityhigh-speed high-speed Complex Graphics high-qualityhigh-quality high-quality Simple Graphics high-qualityhigh-quality high-quality This table shows the same thing again, but this time the desktop machine is also doing a massive high-CPU job in the background: 2008: 2008 desktop2008 mobile Complex Graphics high-speed high-speed Simple Graphics high-speed high-quality 2012: 2012 desktop2012 mobile2008 desktop 2018-era Graphicshigh-speed high-speed high-speed Complex Graphics high-speed high-quality high-speed Simple Graphics high-qualityhigh-quality high-speed How can the author possibly know what the right answer is? It seems better for the browser to simply detect when the graphics burden being placed on the hardware by the page is too much to be done at high quality given the current load on the CPU, and for the browser to automatically drop down to a lower fidelity, higher speed rendering on the fly when appropriate. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] [canvas] imageRenderingQuality property
How can an author know which is appropriate? Erm, presumably because they're the author -- it seems quite valid to for an author to be able to say Just make this happen quickly, I don't care about the quality or Take extra time to make this the highest quality you can. No problem, all you have to do is define: - this - happen - quickly - quality - extra time - highest Of course the hard bit is that these change for every content developer, often per-platform. -- Charles
Re: [whatwg] [canvas] imageRenderingQuality property
On Mon, Jun 30, 2008 at 7:30 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 30 Jun 2008, Vladimir Vukicevic wrote: On Jun 11, 2008, at 3:34 AM, Ian Hickson wrote: On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). How can an author know which is appropriate? Erm, presumably because they're the author -- it seems quite valid to for an author to be able to say Just make this happen quickly, I don't care about the quality or Take extra time to make this the highest quality you can. It seems better for the browser to simply detect when the graphics burden being placed on the hardware by the page is too much to be done at high quality given the current load on the CPU, and for the browser to automatically drop down to a lower fidelity, higher speed rendering on the fly when appropriate. So now we need to define levels of graphic burden? and at what level of burden does the quality suffer? Seems just as hard to define. Having the author explicit say this has to be as high quality as possible or less can be low quality seems better and we have examples of other specs offering the same kind of control.
Re: [whatwg] [canvas] imageRenderingQuality property
On Jun 30, 2008, at 4:14 PM, Vladimir Vukicevic wrote: On Jun 11, 2008, at 3:34 AM, Ian Hickson wrote: On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). How can an author know which is appropriate? Erm, presumably because they're the author -- it seems quite valid to for an author to be able to say Just make this happen quickly, I don't care about the quality or Take extra time to make this the highest quality you can. Yes they are the author, but the user is the person actually using the content and history shows that the author will make such decisions based on how *their* machine works. Even if they try to test on other platforms/UAs, you cannot expect them to test the entire gamut from handheld device through to multicore workstations, and then expect them to update their site every 18 months to allow for the increased processing power. The alternative option, is to have the UA detect when a site is attempting to do the kinds of updates that would result in performance problems and make appropriate decisions to improve the performance (eg. reducing image scaling quality). The adjusting UA option has numerous advantages -- it does not require the developer to tests 1000s of different devices; it doesn't require the developer to update their content every few months; it allows the UA to make performance decisions based on the actual hardware and resources available to it at runtime (eg. allowing it to reduce quality on low powered devices); Improvements in UA performance can immediately result in improved quality, without the developer updating the site; Quality is not unnecessarily lowered in newer/faster UAs just because the developer made decisions based on an older/slower UA. Their are disadvantages -- the UA implementation complexity may be increased, quality may be reduced even when not desired by the developer. The first of these issues is to me a non-issue, in a choice between making the UA implementation more complex (which happens once per UA) vs. increasing the workload on the developer for each site in which they use canvas (which may happen 100s or 1000s of times per developer), increasing UA complexity seems the obvious choice. The second issue is more difficult, and i don't have any immediate solution -- you *could* have a flag which forced the UA to always use its highest quality level (which is a noop on current UAs as far as i am aware), but to me it seems that that would merely result in the poor behaviour on low powered devices that the developer did not consider. Finally, in general my experience suggests that if you have such a flag the developers will end up being split into three groups, those who *always* specify fastest, those who always specify highest quality, and those who never use the flag at all. I would also not expect this flag to be used on the basis of any kind of performance or quality testing, based on what they perceive to be most important, regardless as to whether their code actually had any performance issues. (Yes, there will be a few developers who do perf test, etc, and try to make good decisions, but i would expect them to represent a very small minority of developers) --Oliver - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
On 01/07/2008, Ian Hickson [EMAIL PROTECTED] wrote: [...] It seems better for the browser to simply detect when the graphics burden being placed on the hardware by the page is too much to be done at high quality given the current load on the CPU, and for the browser to automatically drop down to a lower fidelity, higher speed rendering on the fly when appropriate. Sometimes the author will want to force best-quality rendering, regardless of the performance impact. E.g. a photo manipulation application might let you resize a segment of a photo, displaying a live preview (where performance is more important than quality), and then render the final resized image and store it in a canvas for future processing. That final rendering needs to be the best possible quality, so it's not acceptable for the browser to decide that it should semi-randomly drop the quality because it detected the live preview was CPU-intensive. Similarly, a pseudo-3d FPS game might load textures at runtime and perform some preprocessing (like resizing to be square, and rendering lots of smaller copies to be used as mipmaps for distant walls so they look prettier), and then draw that processed texture into the game thousands of times a second. Since the preprocessing is only done once, and its result is reused for the whole of the rest of the game, it should be done at the highest possible quality, regardless of performance. So, adaptively reducing the quality and allowing no author control seems like a bad idea. Perhaps the imageRenderingQuality property could have values 'high' and 'auto', where the default is 'high' (so that existing content continues working the same as it always has, and to avoid surprising authors by randomly switching the rendering quality when they have no reason to expect such weird behaviour), and 'auto' means 'low (but perhaps switch to high if the browser thinks it's going to be fast enough)'. That would avoid the issue of authors setting quality='low' and preventing high-speed users from getting the best quality output. -- Philip Taylor [EMAIL PROTECTED]
Re: [whatwg] [canvas] imageRenderingQuality property
So now we need to define levels of graphic burden? and at what level of burden does the quality suffer? Seems just as hard to define. Having the author explicit say this has to be as high quality as possible or less can be low quality seems better and we have examples of other specs offering the same kind of control. No. The whole point is that the UA is in the best position to identify what the tradeoffs are, not the author -- if you want a flag to specify the quality to be used then that would require you to determine what the tradeoffs were yourself, with no substantial knowledge of what combination any given user was actually using. You need to realise that different UAs and different platforms have substantially different performance characteristics. And that said how would you make the decision yourself anyway? Your decision will almost always hurt *some* portion of your userbase, either through lower quality than would otherwise be possible, or by making a page that is unusable for someone with a lower powered device. And then, after you've made that decision you will start getting some people saying UA1 looks better than UA2 because you underestimated performance of the UAs, and UA1 ignores the flag. Or you'll get people saying UA1 is unusable because you found it was slow in UA2, so you dropped the quality setting -- which UA1 ignores, but then a year later you get people saying UA1 looks better again because hardware is faster so it now do its higher quality rendering without any problem -- but UA2 is still using the low quality path. Finally you test on UA1 and UA2, and find your site looks good in one, but is too slow in the other, do you now lower the quality in both so that your site is usable in both, or do you try and make the decision in JS at runtime? If the latter you now have quite a bit more work to do. How do you determine the performance of a particular UA/platform combination? JS performance is not the same as canvas, so you have to manually test canvas performance, so you are now trying to hand code what could already be built into the UA. Alternatively you could just try to take the lazy path and do a UA check, and try to determine the performance based on UA but now you have *all* of the standard UA check problems, and you still can't test the real performance because you don't know what hardware you're on. If the UA makes its own decisions about performance, it can make a decision based on a wide range of information that you do not have any access to, in addition to being aware of the specific costs of certain operations (eg. a UA could choose to lower the quality of only a single operation, like image scaling, but leave other operations at their standard fidelity -- trying to do this in a flag would be both horrifically complicated for authors *and* would represent tremendous over specification by the standard) --Oliver
Re: [whatwg] [canvas] imageRenderingQuality property
On Thu, 12 Jun 2008, Robert O'Callahan wrote: On Wed, Jun 11, 2008 at 10:34 PM, Ian Hickson [EMAIL PROTECTED] wrote: Why not just have the UA run in a high quality mode the first time it is painted on, but if the script tries to paint again within a certain amount of time, switch to high speed? This makes sense. However, there is still a potential use case for speed vs quality hints: when the author wants to treat some content on the page as visually more important than other content. For example, you might have a gallery page with a bunch of unimportant bling around the actual image/canvas you care about. I'm unsure whether that's realistic enough to justify supporting hints. I think we should put off adding a feature to handle that case until we've got a site that would actually benefit from it. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] [canvas] imageRenderingQuality property
On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). How can an author know which is appropriate? This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty: attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. This seems to be something that is completely inappropriate for SVG or for canvas. The UA should make its own tradeoffs, likely tradeoffs that don't even remotely fit into the categories above (e.g. adapting to scripts that do animations to converge on the best quality possible at 30fps, or optimise for memory usage). On Mon, 2 Jun 2008, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. Why not just have the UA run in a high quality mode the first time it is painted on, but if the script tries to paint again within a certain amount of time, switch to high speed? Trusting the author to do this right seems like a fantastically bad idea. On Mon, 2 Jun 2008, David Hyatt wrote: On Jun 2, 2008, at 4:34 PM, Vladimir Vukicevic wrote: Yeah, I agree -- I thought that there was some plan somewhere to uplift a bunch of these SVG CSS properties into general usage? I know that Gecko uplifted text-rendering, we should figure out what else makes sense to pull up. (If image-rendering were uplifted, it would apply to canvas, for the scaling/transformation of the canvas element itself as opposed to the canvas rendering content.) Right, that would be my expectation as well (that the CSS property could be applied to canvas to control the quality when rendering the canvas buffer itself). That neatly moves the problem away from the canvas API, and thus it wouldn't be my problem necessarily, but I still don't think it'd be a good idea. :-) On Mon, 2 Jun 2008, Oliver Hunt wrote: That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Typically such a property would be considered a hint, and as such would likely be ignored. If honouring this property was _required_ rather than being a hint you would hit the following problems: * Low power devices would have a significant potential for poor performance if a developer found that their desktop performed well so set the requirement to high quality. * High power devices would be forced to use low quality rendering modes when perfectly capable of providing better quality without significant performance penalty. Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. The correct behaviour would be for the UA to work out itself what it can do, which it needs to be able to do anyway, in order to satisfy the auto option. That line of reasoning seems correct to me. On Mon, 2 Jun 2008, Vladimir Vukicevic wrote: If web apps misuse the property, then bugs should be filed on those apps that incorrectly use the property, and the app developer should fix them. That's naive, I'm afraid. In practice, sites are written and abandoned, and don't get fixed. On Tue, 3 Jun 2008, Robert O'Callahan wrote: These hint properties are opt-in for UAs. If you don't like the idea, just treat all values as auto. There's not much point us adding the feature if multiple UAs aren't going to implement it. Also, if it's a hint, then it's untestable, and we
Re: [whatwg] [canvas] imageRenderingQuality property
On Wed, Jun 11, 2008 at 10:34 PM, Ian Hickson [EMAIL PROTECTED] wrote: Why not just have the UA run in a high quality mode the first time it is painted on, but if the script tries to paint again within a certain amount of time, switch to high speed? This makes sense. However, there is still a potential use case for speed vs quality hints: when the author wants to treat some content on the page as visually more important than other content. For example, you might have a gallery page with a bunch of unimportant bling around the actual image/canvas you care about. I'm unsure whether that's realistic enough to justify supporting hints. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] [canvas] imageRenderingQuality property
If you don't mind someone weighing in who's been working with the Nintendo Wii for the past year and a half, I have found that the quality setting in Flash is tremendously useful. While the march of Moore's law has ensured that Flash on the desktop is zippy, it has also created a gap whereby smaller devices are powerful enough to compete with last generation desktops. For these devices, the quality setting is more than a nice to have. It's absolutely critical to obtaining decent performance. The algorithm's I've used over on WiiCade automatically adjust for high quality when on the desktop, and low to medium quality when on the Wii. This provides a more consistent experience to users of both systems. In addition, many game provide a quality setting in their options screen. (Similar to the rendering features options in most 3D games.) This allows the user to adjust the rendering speed manually if he finds his system is too slow or the game in question is more than fast enough on the Wii. I can see a quality setting in Javascript used in much the same way. The site would set the quality of the rendering based on knowledge of particular platforms. Modern desktops would default to highest quality. Furthermore, video games (and we all know that's going to be a huge use of Canvas at some point ;-)) that push the limit of Canvas may allow the user to shift down as it were to compensate for their slower machine or the slow Canvas rendering of their browser. I definitely think this setting should be a hint rather than a hard and fast set of rules. If a UA (especially desktop UAs) wants to ignore the setting, that's fine. It will cause no appreciable damage. But it will allow for slower UAs to be tuned for usage. e.g. If I'm looking at charts on my Wii, I'd rather they be high quality. If I'm playing a video game, the quality simply does not matter as much. I hope you will all keep we poor device users in mind when you come to your decision. Thanks! Jerason Banes On Mon, Jun 2, 2008 at 5:52 PM, Oliver Hunt [EMAIL PROTECTED] wrote: Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. If the quality knob is no longer necessary, why would new content perform badly? The issue is not that certain operations are slower than others, the issue is that anything that requires the developer to choose between performance/quality is going to become obsolete as the performance trade offs are constantly moving and are not the same from UA to UA, from platform to platform. I think the issue of performance is a complex one that will not benefit in the long term from a simple on off switch. Conceivably we could introduce new rendering primitives, such as CanvasSprite, CanvasLayer, or some such which would, i suspect, provide a similar benefit, but be more resilient in the face of changing performance characteristics.
Re: [whatwg] [canvas] imageRenderingQuality property
Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest-neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. - Vlad On Jun 2, 2008, at 12:25 PM, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest- neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
I like the idea of this property. I actually would love to see the SVG property applied to HTML img as well. :) dave On Jun 2, 2008, at 4:15 PM, Vladimir Vukicevic wrote: Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. - Vlad On Jun 2, 2008, at 12:25 PM, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest- neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
Yeah, I agree -- I thought that there was some plan somewhere to uplift a bunch of these SVG CSS properties into general usage? I know that Gecko uplifted text-rendering, we should figure out what else makes sense to pull up. (If image-rendering were uplifted, it would apply to canvas, for the scaling/transformation of the canvas element itself as opposed to the canvas rendering content.) - Vlad On Jun 2, 2008, at 2:26 PM, David Hyatt wrote: I like the idea of this property. I actually would love to see the SVG property applied to HTML img as well. :) dave On Jun 2, 2008, at 4:15 PM, Vladimir Vukicevic wrote: Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. - Vlad On Jun 2, 2008, at 12:25 PM, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest- neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
On Jun 2, 2008, at 4:34 PM, Vladimir Vukicevic wrote: Yeah, I agree -- I thought that there was some plan somewhere to uplift a bunch of these SVG CSS properties into general usage? I know that Gecko uplifted text-rendering, we should figure out what else makes sense to pull up. (If image-rendering were uplifted, it would apply to canvas, for the scaling/transformation of the canvas element itself as opposed to the canvas rendering content.) - Vlad Right, that would be my expectation as well (that the CSS property could be applied to canvas to control the quality when rendering the canvas buffer itself). dave
Re: [whatwg] [canvas] imageRenderingQuality property
That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Typically such a property would be considered a hint, and as such would likely be ignored. If honouring this property was _required_ rather than being a hint you would hit the following problems: * Low power devices would have a significant potential for poor performance if a developer found that their desktop performed well so set the requirement to high quality. * High power devices would be forced to use low quality rendering modes when perfectly capable of providing better quality without significant performance penalty. Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. The correct behaviour would be for the UA to work out itself what it can do, which it needs to be able to do anyway, in order to satisfy the auto option. --Oliver On Jun 2, 2008, at 2:15 PM, Vladimir Vukicevic wrote: Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. - Vlad On Jun 2, 2008, at 12:25 PM, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest- neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
On Jun 2, 2008, at 2:39 PM, Oliver Hunt wrote: That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Typically such a property would be considered a hint, and as such would likely be ignored. If honouring this property was _required_ rather than being a hint you would hit the following problems: * Low power devices would have a significant potential for poor performance if a developer found that their desktop performed well so set the requirement to high quality. * High power devices would be forced to use low quality rendering modes when perfectly capable of providing better quality without significant performance penalty. Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. If web apps misuse the property, then bugs should be filed on those apps that incorrectly use the property, and the app developer should fix them. The web platform shouldn't prevent developers from exercising control over how their content is rendered; most developers, as you say, probably shouldn't change anything from the default 'auto'. But the capability should be there. Arbitrarily deciding what developers can and can't do isn't interesting from the perspective of creating a full-featured platform, IMO. No matter how fast smooth/bilinear filtering is, something more complex is always going to be slower, and something less complex is always going to be faster. If those perf differences are significant to the web app, no matter how small, you're going to want to be able to have that control. If they're not, then you should just be using 'auto' and let the UA handle it. - Vlad On Jun 2, 2008, at 2:15 PM, Vladimir Vukicevic wrote: Sure; bilinear filtering is slower than nearest neighbour sampling, and in many cases the app author would like to be able to decide that tradeoff (or, at least, to be able to say I want this to go as fast as possible, regardless of quality). Some apps might also render to a canvas just once, and would prefer to do it at the highest quality filtering available even if it's more expensive than the default. - Vlad On Jun 2, 2008, at 12:25 PM, Oliver Hunt wrote: Um, could you actually give some kind of reasoning for these? I am not aware of any significant performance issues in Canvas that cannot be almost directly attributed to JavaScript itself rather than the canvas. --Oliver On Jun 2, 2008, at 12:19 PM, Vladimir Vukicevic wrote: I'd like to propose adding an imageRenderingQuality property on the canvas 2D context to allow authors to choose speed vs. quality when rendering images (especially transformed ones). This is modeled on the SVG image-rendering property, at http://www.w3.org/TR/SVG/painting.html#ImageRenderingProperty : attribute string imageRenderingQuality; 'auto' (default): The user agent shall make appropriate tradeoffs to balance speed and quality, but quality shall be given more importance than speed. 'optimizeQuality': Emphasize quality over rendering speed. 'optimizeSpeed': Emphasize speed over rendering quality. No specific image sampling algorithm is specified for any of these properties, with the exception that, at a minimum, nearest- neighbour resampling should be used. One alternative is to specify 'best', 'good', 'fast', with good being the default, as opposed to the SVG names; I think those names are more descriptive, but there might be value in keeping the names consistent with SVG, especially if that property bubbles up into general CSS usage. - Vlad
Re: [whatwg] [canvas] imageRenderingQuality property
On Jun 2, 2008, at 4:57 PM, Vladimir Vukicevic wrote: On Jun 2, 2008, at 2:39 PM, Oliver Hunt wrote: That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Typically such a property would be considered a hint, and as such would likely be ignored. If honouring this property was _required_ rather than being a hint you would hit the following problems: * Low power devices would have a significant potential for poor performance if a developer found that their desktop performed well so set the requirement to high quality. * High power devices would be forced to use low quality rendering modes when perfectly capable of providing better quality without significant performance penalty. Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. If web apps misuse the property, then bugs should be filed on those apps that incorrectly use the property, and the app developer should fix them. The web platform shouldn't prevent developers from exercising control over how their content is rendered; most developers, as you say, probably shouldn't change anything from the default 'auto'. But the capability should be there. Arbitrarily deciding what developers can and can't do isn't interesting from the perspective of creating a full-featured platform, IMO. No matter how fast smooth/bilinear filtering is, something more complex is always going to be slower, and something less complex is always going to be faster. If those perf differences are significant to the web app, no matter how small, you're going to want to be able to have that control. If they're not, then you should just be using 'auto' and let the UA handle it. I completely agree. Almost any feature can be abused. It's not our job to withhold features just because they can be abused. It's also worth pointing out that this is a common graphical knob supported by Cairo and CoreGraphics. It is a proprietary MS CSS property and an SVG CSS property. In other words, it seems to be a pretty widely implemented feature and as such seems like it would be a worthwhile addition to canvas. If we add this, we should also add support for text rendering quality as well, since canvas is picking up the ability to draw text. dave ([EMAIL PROTECTED])
Re: [whatwg] [canvas] imageRenderingQuality property
On Tue, Jun 3, 2008 at 9:34 AM, Vladimir Vukicevic [EMAIL PROTECTED] wrote: Yeah, I agree -- I thought that there was some plan somewhere to uplift a bunch of these SVG CSS properties into general usage? I know that Gecko uplifted text-rendering, we should figure out what else makes sense to pull up. Sort of off topic, but as you said, we uplifted 'text-rendering' in Gecko 1.9. I'm experimenting with 'clip-path', 'mask' and 'filter'. We're also looking at uplifting 'pointer-events'. For the non-hint properties we have to be more careful because it needs to be specified what they mean for non-SVG content; uplifting the hint properties is a lot easier. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] [canvas] imageRenderingQuality property
On Tue, Jun 3, 2008 at 9:39 AM, Oliver Hunt [EMAIL PROTECTED] wrote: That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Right, it is. The user-agent is free to map all property values to maximum quality. Typically such a property would be considered a hint, and as such would likely be ignored. Ignored by who? Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. If the quality knob is no longer necessary, why would new content perform badly? These hint properties are opt-in for UAs. If you don't like the idea, just treat all values as auto. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] [canvas] imageRenderingQuality property
On Jun 2, 2008, at 3:19 PM, Robert O'Callahan wrote: On Tue, Jun 3, 2008 at 9:39 AM, Oliver Hunt [EMAIL PROTECTED] wrote: That's exactly what i would be afraid of people doing. If I have a fast system why should i have to experience low quality rendering? It should be the job of the platform to determine what level of performance or quality can be achieved on a given device. Right, it is. The user-agent is free to map all property values to maximum quality. Typically such a property would be considered a hint, and as such would likely be ignored. Ignored by who? Given a few years all UA's as the majority of content saying low quality will be old, and therefore targeting slower UAs and platforms (given moore's law, and the fact that all UAs are currently getting faster this seems a reasonable assertion) so respecting it would mean the majority of sites using low quality mode would not need to. Neither of these apply if the property were just a hint, but now you have to think about what happens to content that uses this property in 18 months time. You've told the UA to use a low quality rendering when it may no longer be necessary, so now the UA has a choice it either always obeys the property meaning lower quality than is necessary so that new content performs well, or it ignores the property in which case new content performs badly. If the quality knob is no longer necessary, why would new content perform badly? The issue is not that certain operations are slower than others, the issue is that anything that requires the developer to choose between performance/quality is going to become obsolete as the performance trade offs are constantly moving and are not the same from UA to UA, from platform to platform. I think the issue of performance is a complex one that will not benefit in the long term from a simple on off switch. Conceivably we could introduce new rendering primitives, such as CanvasSprite, CanvasLayer, or some such which would, i suspect, provide a similar benefit, but be more resilient in the face of changing performance characteristics. Rob --Oliver
