Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Darin Fisher]

On Mon, Apr 16, 2012 at 3:34 PM, Maciej Stachowiak  wrote:

>
> On Apr 16, 2012, at 1:44 PM, Darin Fisher wrote:
>
> On Mon, Apr 16, 2012 at 1:42 PM, Oliver Hunt  wrote:
>
>>
>> On Apr 16, 2012, at 1:00 PM, Darin Fisher  wrote:
>>
>> On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak wrote:
>>
>>>
>>> On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:
>>>
>>> Could this be an opportunity to design an asynchronous API for fetching
>>> the pixel buffer?  It seems like many implementations are GPU backed now,
>>> and fetching the pixel buffer is an expensive (blocking) operation.  Had
>>> you considered making such a change?
>>>
>>>
>>> Adding async support was suggested on the whatwg thread about this. I
>>> think async support is useful, but should not be tied to high DPI support.
>>> In particular, we shouldn't, in my opinion, require authors to rewrite
>>> their existing sync code to an async model just to properly support higher
>>> resolutions.
>>>
>>> In addition, the whatwg thread revealed that there are many hidden
>>> complexities in the design of get/putImageData, in particular designing how
>>> they work in the face of sychronous drawing operations to the same canvas.
>>> The HiDPI problem is much more straightforward and does not need to be
>>> gated on resolving the async design issues.
>>>
>>>
>> I think you are giving up a good opportunity.  The barriers to an async
>> API are more readily overcome when there are extra benefits to developers.
>>  HiDPI could be a great way to attract developers to a better API.
>>
>> I've addressed those other concerns on the WhatWG thread.
>>
>>
>> No, gating HiDPI on rewriting your code into a more complex, and
>> generally slower model seems like a great way to encourage developers to
>> ignore high dpi.
>>
>> --Oliver
>>
>>
> I'm not sure why developers would choose to ignore HiDPI.  It seems like
> it helps their apps/pages look better!
>
> You really feel like you need to "kowtow" to developers to get them to
> adopt HiDPI?
>
>
> Different developers will have different priorities. HD image data and
> async readback both have potential benefits in image quality and
> nonblocking responsiveness respectively. Here is an example of an
> application using getImageData which would clearly benefit from HD, but
> it's not obvious that async would be useful:
>
> http://mudcu.be/sketchpad/
>
> Here is another where HD helps but benefits of async are unclear, since it
> does a pixel read-write-modify cycle:
>
> http://nerget.com/pressure/pressure.html
>
> Tying HD to async may hurt the adoption of both by requiring developers to
> take two different code change hits when they only care about one. In
> particular, the async change is likely to be more invasive to code
> structure.  If developers are discouraged, they may end up using neither.
> Thus, in my opinion, these two enhancements to ImageData should stand and
> fall on their own merits.
>
>
Hi Maciej,

I was thinking that an asynchronous version of getImageData could be used
to fetch either the HD or non-HD backing.

At any rate, it seems like my motivation is clear.  Blocking getImageData
is a performance penalty for implementations that use deferred rendering.
 If not a penalty for the caller, then surely a penalty for other pages
that may share the same thread.  Making such an API return even larger
amounts of memory seems like it only increases the penalty.

I understand your desire to separate these concerns.  I'm just worried that
we are missing an opportunity to guide developers to a better place.

Regards,
-Darin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Dirk Schulze]

On Apr 16, 2012, at 4:30 PM, Maciej Stachowiak wrote:

> 
> On Apr 16, 2012, at 4:03 PM, Dirk Schulze wrote:
> 
>>> 
>>> Different developers will have different priorities. HD image data and 
>>> async readback both have potential benefits in image quality and 
>>> nonblocking responsiveness respectively. Here is an example of an 
>>> application using getImageData which would clearly benefit from HD, but 
>>> it's not obvious that async would be useful:
>>> 
>>> http://mudcu.be/sketchpad/
>> 
>> This seems to be a use case for SVG and not canvas :)
> 
> It's a raster-oriented paint app, not a vector-oriented draw app, so I don't 
> think SVG would be helpful.
This is off topic, but I disagree. The only thing that you want is a rasterized 
result at the end. In the meantime it can be object and vector based (and in 
high resolution) on the screen. This would even allow you to create high 
resolution PNGs. I see a lot more benefits, but we can discuss it on another 
place.

Dirk

> 
> Regards,
> Maciej
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Maciej Stachowiak]

On Apr 16, 2012, at 4:03 PM, Dirk Schulze wrote:

>> 
>> Different developers will have different priorities. HD image data and async 
>> readback both have potential benefits in image quality and nonblocking 
>> responsiveness respectively. Here is an example of an application using 
>> getImageData which would clearly benefit from HD, but it's not obvious that 
>> async would be useful:
>> 
>> http://mudcu.be/sketchpad/
> 
> This seems to be a use case for SVG and not canvas :)

It's a raster-oriented paint app, not a vector-oriented draw app, so I don't 
think SVG would be helpful.

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Charles Pritchard]

On 4/16/2012 3:34 PM, Maciej Stachowiak wrote:


On Apr 16, 2012, at 1:44 PM, Darin Fisher wrote:

On Mon, Apr 16, 2012 at 1:42 PM, Oliver Hunt > wrote:



On Apr 16, 2012, at 1:00 PM, Darin Fisher mailto:da...@chromium.org>> wrote:


On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak
mailto:m...@apple.com>> wrote:


On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:


Could this be an opportunity to design an asynchronous API
for fetching the pixel buffer?  It seems like many
implementations are GPU backed now, and fetching the pixel
buffer is an expensive (blocking) operation.  Had you
considered making such a change?


Adding async support was suggested on the whatwg thread
about this. I think async support is useful, but should not
be tied to high DPI support. In particular, we shouldn't, in
my opinion, require authors to rewrite their existing sync
code to an async model just to properly support higher
resolutions.

In addition, the whatwg thread revealed that there are many
hidden complexities in the design of get/putImageData, in
particular designing how they work in the face of sychronous
drawing operations to the same canvas. The HiDPI problem is
much more straightforward and does not need to be gated on
resolving the async design issues.


I think you are giving up a good opportunity.  The barriers to
an async API are more readily overcome when there are extra
benefits to developers.  HiDPI could be a great way to attract
developers to a better API.

I've addressed those other concerns on the WhatWG thread.


No, gating HiDPI on rewriting your code into a more complex, and
generally slower model seems like a great way to encourage
developers to ignore high dpi.

--Oliver


I'm not sure why developers would choose to ignore HiDPI.  It seems 
like it helps their apps/pages look better!


You really feel like you need to "kowtow" to developers to get them 
to adopt HiDPI?


Different developers will have different priorities. HD image data and 
async readback both have potential benefits in image quality and 
nonblocking responsiveness respectively. Here is an example of an 
application using getImageData which would clearly benefit from HD, 
but it's not obvious that async would be useful:


http://mudcu.be/sketchpad/


Developers are rarely targeting existing HiDPI with iOS via 
window.devicePixelRatio. They have learned though, word spreads around.
That's just an issue of project and community maturity. Canvas on iOS 
requires a lot of advanced techniques for good operation.


I was heavily involved in Sketchpad as well as a Canvas-based image 
editor, Darkroom.

http://www.youtube.com/watch?v=4MrEtsplano#t=2m

In a subsequent version we did enable arbitrary blend modes in drawing. 
Michael put together several dozen modes.
Blocking was and is a big issue. Especially with high resolution devices 
like Wacom pen-tablets.


If there's any blocking due to heavy computations, we end up losing a 
lot of precision of movement and pen pressure.
Some drawing applications have a quick render, which is then followed up 
by a high quality composite (running on pixels).


The actual getImageData call is not a big deal, but regardless, we 
already face the need of an async loop, of possibly dropping frames and 
of keeping various state data around.
In the sketchpad project as you are looking at it, it is not using 
getImageData. It's using a lot of drawImage calls.



Here is another where HD helps but benefits of async are unclear, 
since it does a pixel read-write-modify cycle:


http://nerget.com/pressure/pressure.html

Tying HD to async may hurt the adoption of both by requiring 
developers to take two different code change hits when they only care 
about one. In particular, the async change is likely to be more 
invasive to code structure.  If developers are discouraged, they may 
end up using neither. Thus, in my opinion, these two enhancements to 
ImageData should stand and fall on their own merits.




That project has one getImageData call, it's kept in memory and it's 
initialized from a blank canvas.
It would only need an additional line of code (and argument) to support 
async getHD. It's not a separate code path.


I am trying to say this: an async getImageDataHD will not "hurt" 
developers nor discourage them.


The fluid dynamics demo would benefit from using requestAnimationFrame 
instead of setInterval, and using

webkitPostMessage to do the heavy lifting in the displayDensity method.

It could benefit from multiple-cores (in theory) by using several buffers.

I think there are excellent points to discuss; points that have been 
discussed in this thread. There were calls for benchmarking, weighing up 
the technical overhead.


I want to make it clear that developers would not be

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Dirk Schulze]

> 
> Different developers will have different priorities. HD image data and async 
> readback both have potential benefits in image quality and nonblocking 
> responsiveness respectively. Here is an example of an application using 
> getImageData which would clearly benefit from HD, but it's not obvious that 
> async would be useful:
> 
> http://mudcu.be/sketchpad/

This seems to be a use case for SVG and not canvas :)

Dirk

> 
> Here is another where HD helps but benefits of async are unclear, since it 
> does a pixel read-write-modify cycle:
> 
> http://nerget.com/pressure/pressure.html
> 
> Tying HD to async may hurt the adoption of both by requiring developers to 
> take two different code change hits when they only care about one. In 
> particular, the async change is likely to be more invasive to code structure. 
>  If developers are discouraged, they may end up using neither. Thus, in my 
> opinion, these two enhancements to ImageData should stand and fall on their 
> own merits.
> 
> Regards,
> Maciej
> 
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Maciej Stachowiak]

On Apr 16, 2012, at 1:44 PM, Darin Fisher wrote:

> On Mon, Apr 16, 2012 at 1:42 PM, Oliver Hunt  wrote:
> 
> On Apr 16, 2012, at 1:00 PM, Darin Fisher  wrote:
> 
>> On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak  wrote:
>> 
>> On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:
>> 
>>> Could this be an opportunity to design an asynchronous API for fetching the 
>>> pixel buffer?  It seems like many implementations are GPU backed now, and 
>>> fetching the pixel buffer is an expensive (blocking) operation.  Had you 
>>> considered making such a change?
>> 
>> Adding async support was suggested on the whatwg thread about this. I think 
>> async support is useful, but should not be tied to high DPI support. In 
>> particular, we shouldn't, in my opinion, require authors to rewrite their 
>> existing sync code to an async model just to properly support higher 
>> resolutions.
>> 
>> In addition, the whatwg thread revealed that there are many hidden 
>> complexities in the design of get/putImageData, in particular designing how 
>> they work in the face of sychronous drawing operations to the same canvas. 
>> The HiDPI problem is much more straightforward and does not need to be gated 
>> on resolving the async design issues.
>> 
>> 
>> I think you are giving up a good opportunity.  The barriers to an async API 
>> are more readily overcome when there are extra benefits to developers.  
>> HiDPI could be a great way to attract developers to a better API.
>> 
>> I've addressed those other concerns on the WhatWG thread.
> 
> No, gating HiDPI on rewriting your code into a more complex, and generally 
> slower model seems like a great way to encourage developers to ignore high 
> dpi.
> 
> --Oliver
> 
> 
> I'm not sure why developers would choose to ignore HiDPI.  It seems like it 
> helps their apps/pages look better!
> 
> You really feel like you need to "kowtow" to developers to get them to adopt 
> HiDPI?

Different developers will have different priorities. HD image data and async 
readback both have potential benefits in image quality and nonblocking 
responsiveness respectively. Here is an example of an application using 
getImageData which would clearly benefit from HD, but it's not obvious that 
async would be useful:

http://mudcu.be/sketchpad/

Here is another where HD helps but benefits of async are unclear, since it does 
a pixel read-write-modify cycle:

http://nerget.com/pressure/pressure.html

Tying HD to async may hurt the adoption of both by requiring developers to take 
two different code change hits when they only care about one. In particular, 
the async change is likely to be more invasive to code structure.  If 
developers are discouraged, they may end up using neither. Thus, in my opinion, 
these two enhancements to ImageData should stand and fall on their own merits.

Regards,
Maciej



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Maciej Stachowiak]

On Apr 16, 2012, at 1:48 PM, Oliver Hunt wrote:

> 
> On Apr 16, 2012, at 1:44 PM, Darin Fisher  wrote:
>> I'm not sure why developers would choose to ignore HiDPI.  It seems like it 
>> helps their apps/pages look better!
>> 
>> You really feel like you need to "kowtow" to developers to get them to adopt 
>> HiDPI?
> 
> The existing API supported high dpi, no one wrote there code to handle it 
> because no one really uses it.  It will be decades before high dpi is the 
> norm (eg. it will remain a "no one really uses it" feature), so why would you 
> go for the more complex and slower code path to support something that will 
> in general not effect anyone?

Apple ships systems with a non 1-to-1 ratio of CSS pixels to device pixels 
(iPhone 4, iPhone 4S, the new iPad), though presently none of them scale canvas 
backing stores by default. Part of the reason it was necessary to stick with 
unscaled canvas backing store is compat issues which are addressed by the HiDPI 
proposal. It seems likely to me that more of the industry will move to high 
resolution displays over time.

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Oliver Hunt]

On Apr 16, 2012, at 1:44 PM, Darin Fisher  wrote:
> I'm not sure why developers would choose to ignore HiDPI.  It seems like it 
> helps their apps/pages look better!
> 
> You really feel like you need to "kowtow" to developers to get them to adopt 
> HiDPI?

The existing API supported high dpi, no one wrote there code to handle it 
because no one really uses it.  It will be decades before high dpi is the norm 
(eg. it will remain a "no one really uses it" feature), so why would you go for 
the more complex and slower code path to support something that will in general 
not effect anyone?

--Oliver

> 
> -Darin
> 
>  
>> 
>> -Darin
>> 
>>  
>>> 
>>> Regards,
>>> -Darin
>>> 
>>> 
>>> 
>>> On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:
>>> [This is actually an enhancement announcement to an existing feature]
>>> 
>>> Over at 
>>> ,
>>>  Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to 
>>> allow authors to take full advantage of high-resolution backing stores, 
>>> when available (whereas the existing API hides the fact that the backing 
>>> store resolution is not CSS pixel resolution, for compatibility with 
>>> existing content). The enhancements include a backingStorePixelRatio 
>>> attribute and {get,put}ImageDataHD functions on CanvasRenderingContext2D.
>>> 
>>> Through  and 
>>> , I am making these 
>>> enhancements, with the names prefixed with “webkit”.
>>> 
>>> There is no build-time option to disable these enhancements. Ports that 
>>> don’t opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version 
>>> of the additional API. Ports that opt into high-DPI canvas need to enhance 
>>> their ImageBuffer implementation to fully support the resolutionScale and 
>>> CoordinateSystem parameters.
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>> 
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>> 
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> 
> 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Darin Fisher]

On Mon, Apr 16, 2012 at 1:42 PM, Oliver Hunt  wrote:

>
> On Apr 16, 2012, at 1:00 PM, Darin Fisher  wrote:
>
> On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak  wrote:
>
>>
>> On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:
>>
>> Could this be an opportunity to design an asynchronous API for fetching
>> the pixel buffer?  It seems like many implementations are GPU backed now,
>> and fetching the pixel buffer is an expensive (blocking) operation.  Had
>> you considered making such a change?
>>
>>
>> Adding async support was suggested on the whatwg thread about this. I
>> think async support is useful, but should not be tied to high DPI support.
>> In particular, we shouldn't, in my opinion, require authors to rewrite
>> their existing sync code to an async model just to properly support higher
>> resolutions.
>>
>> In addition, the whatwg thread revealed that there are many hidden
>> complexities in the design of get/putImageData, in particular designing how
>> they work in the face of sychronous drawing operations to the same canvas.
>> The HiDPI problem is much more straightforward and does not need to be
>> gated on resolving the async design issues.
>>
>>
> I think you are giving up a good opportunity.  The barriers to an async
> API are more readily overcome when there are extra benefits to developers.
>  HiDPI could be a great way to attract developers to a better API.
>
> I've addressed those other concerns on the WhatWG thread.
>
>
> No, gating HiDPI on rewriting your code into a more complex, and generally
> slower model seems like a great way to encourage developers to ignore high
> dpi.
>
> --Oliver
>
>
I'm not sure why developers would choose to ignore HiDPI.  It seems like it
helps their apps/pages look better!

You really feel like you need to "kowtow" to developers to get them to
adopt HiDPI?

-Darin



>
> -Darin
>
>
>
>>
>> Regards,
>> -Darin
>>
>>
>>
>> On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:
>>
>>> [This is actually an enhancement announcement to an existing feature]
>>>
>>> Over at <
>>> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-March/035112.html>,
>>> Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to
>>> allow authors to take full advantage of high-resolution backing stores,
>>> when available (whereas the existing API hides the fact that the backing
>>> store resolution is not CSS pixel resolution, for compatibility with
>>> existing content). The enhancements include a backingStorePixelRatio
>>> attribute and {get,put}ImageDataHD functions on CanvasRenderingContext2D.
>>>
>>> Through  and <
>>> https://bugs.webkit.org/show_bug.cgi?id=83836>, I am making these
>>> enhancements, with the names prefixed with “webkit”.
>>>
>>> There is no build-time option to disable these enhancements. Ports that
>>> don’t opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version
>>> of the additional API. Ports that opt into high-DPI canvas need to enhance
>>> their ImageBuffer implementation to fully support the resolutionScale and
>>> CoordinateSystem parameters.
>>> ___
>>> webkit-dev mailing list
>>> webkit-dev@lists.webkit.org
>>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>>
>>
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>>
>>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Oliver Hunt]

On Apr 16, 2012, at 1:00 PM, Darin Fisher  wrote:

> On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak  wrote:
> 
> On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:
> 
>> Could this be an opportunity to design an asynchronous API for fetching the 
>> pixel buffer?  It seems like many implementations are GPU backed now, and 
>> fetching the pixel buffer is an expensive (blocking) operation.  Had you 
>> considered making such a change?
> 
> Adding async support was suggested on the whatwg thread about this. I think 
> async support is useful, but should not be tied to high DPI support. In 
> particular, we shouldn't, in my opinion, require authors to rewrite their 
> existing sync code to an async model just to properly support higher 
> resolutions.
> 
> In addition, the whatwg thread revealed that there are many hidden 
> complexities in the design of get/putImageData, in particular designing how 
> they work in the face of sychronous drawing operations to the same canvas. 
> The HiDPI problem is much more straightforward and does not need to be gated 
> on resolving the async design issues.
> 
> 
> I think you are giving up a good opportunity.  The barriers to an async API 
> are more readily overcome when there are extra benefits to developers.  HiDPI 
> could be a great way to attract developers to a better API.
> 
> I've addressed those other concerns on the WhatWG thread.

No, gating HiDPI on rewriting your code into a more complex, and generally 
slower model seems like a great way to encourage developers to ignore high dpi.

--Oliver

> 
> -Darin
> 
>  
>> 
>> Regards,
>> -Darin
>> 
>> 
>> 
>> On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:
>> [This is actually an enhancement announcement to an existing feature]
>> 
>> Over at 
>> ,
>>  Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to 
>> allow authors to take full advantage of high-resolution backing stores, when 
>> available (whereas the existing API hides the fact that the backing store 
>> resolution is not CSS pixel resolution, for compatibility with existing 
>> content). The enhancements include a backingStorePixelRatio attribute and 
>> {get,put}ImageDataHD functions on CanvasRenderingContext2D.
>> 
>> Through  and 
>> , I am making these 
>> enhancements, with the names prefixed with “webkit”.
>> 
>> There is no build-time option to disable these enhancements. Ports that 
>> don’t opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version 
>> of the additional API. Ports that opt into high-DPI canvas need to enhance 
>> their ImageBuffer implementation to fully support the resolutionScale and 
>> CoordinateSystem parameters.
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>> 
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Charles Pritchard]

On 4/16/2012 12:55 PM, Maciej Stachowiak wrote:

On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:

Could this be an opportunity to design an asynchronous API for 
fetching the pixel buffer?  It seems like many implementations are 
GPU backed now, and fetching the pixel buffer is an expensive 
(blocking) operation.  Had you considered making such a change?


Adding async support was suggested on the whatwg thread about this. I 
think async support is useful, but should not be tied to high DPI 
support. In particular, we shouldn't, in my opinion, require authors 
to rewrite their existing sync code to an async model just to properly 
support higher resolutions.


TL;DR: Canvas authors will largely be using web workers (thus async) 
regardless of this proposal.


Oliver Hunt is suggests that an async fetch would be of "significant 
additional complexity for content authors"; Maciej, you're also saying 
that "authors [would have] to rewrite their existing sync code".


ImageData operations are very much a niche, a very small niche, inside 
of yet another very small niche. Canvas is an expensive low-level API to 
work with. Cheaper than SVG support, but still expensive.

Authors working with Canvas have taken on complexity already.

All authors working with Canvas ImageData are looking at it from an 
async perspective because they'd like to offload the work to another thread.
We've had to wait awhile for it but webkitPostMessage and 
Uint8ClampedArray are coming along. Those are already an async pairing.


Further, we can do little optimizations to treat the typed array as a 
Float32. Once we're knee-deep in ImageData, everything is on the table.


The "code complexity" of async vs sync on ImageData is minor. Authors 
are looking at using async anyway so that they have the option to move 
the heavy lifting off-thread so as not to block the UI.


I just have to disagree with Oliver and Maciej here on the "complexity" 
and "rewrite" portions. 1) It's already being done as common practice so 
as not to block the UI, 2) It's not complex nor a rewrite, it's just a 
simple layer
of code, and 3) we're already looking at far more complex things in 
Canvas such as simply supporting requestAnimationFrame.



-Charles









___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Darin Fisher]

On Mon, Apr 16, 2012 at 12:55 PM, Maciej Stachowiak  wrote:

>
> On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:
>
> Could this be an opportunity to design an asynchronous API for fetching
> the pixel buffer?  It seems like many implementations are GPU backed now,
> and fetching the pixel buffer is an expensive (blocking) operation.  Had
> you considered making such a change?
>
>
> Adding async support was suggested on the whatwg thread about this. I
> think async support is useful, but should not be tied to high DPI support.
> In particular, we shouldn't, in my opinion, require authors to rewrite
> their existing sync code to an async model just to properly support higher
> resolutions.
>
> In addition, the whatwg thread revealed that there are many hidden
> complexities in the design of get/putImageData, in particular designing how
> they work in the face of sychronous drawing operations to the same canvas.
> The HiDPI problem is much more straightforward and does not need to be
> gated on resolving the async design issues.
>
>
I think you are giving up a good opportunity.  The barriers to an async API
are more readily overcome when there are extra benefits to developers.
 HiDPI could be a great way to attract developers to a better API.

I've addressed those other concerns on the WhatWG thread.

-Darin



>
> Regards,
> -Darin
>
>
>
> On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:
>
>> [This is actually an enhancement announcement to an existing feature]
>>
>> Over at <
>> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-March/035112.html>,
>> Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to
>> allow authors to take full advantage of high-resolution backing stores,
>> when available (whereas the existing API hides the fact that the backing
>> store resolution is not CSS pixel resolution, for compatibility with
>> existing content). The enhancements include a backingStorePixelRatio
>> attribute and {get,put}ImageDataHD functions on CanvasRenderingContext2D.
>>
>> Through  and <
>> https://bugs.webkit.org/show_bug.cgi?id=83836>, I am making these
>> enhancements, with the names prefixed with “webkit”.
>>
>> There is no build-time option to disable these enhancements. Ports that
>> don’t opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version
>> of the additional API. Ports that opt into high-DPI canvas need to enhance
>> their ImageBuffer implementation to fully support the resolutionScale and
>> CoordinateSystem parameters.
>> ___
>> webkit-dev mailing list
>> webkit-dev@lists.webkit.org
>> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Maciej Stachowiak]

On Apr 16, 2012, at 10:52 AM, Darin Fisher wrote:

> Could this be an opportunity to design an asynchronous API for fetching the 
> pixel buffer?  It seems like many implementations are GPU backed now, and 
> fetching the pixel buffer is an expensive (blocking) operation.  Had you 
> considered making such a change?

Adding async support was suggested on the whatwg thread about this. I think 
async support is useful, but should not be tied to high DPI support. In 
particular, we shouldn't, in my opinion, require authors to rewrite their 
existing sync code to an async model just to properly support higher 
resolutions.

In addition, the whatwg thread revealed that there are many hidden complexities 
in the design of get/putImageData, in particular designing how they work in the 
face of sychronous drawing operations to the same canvas. The HiDPI problem is 
much more straightforward and does not need to be gated on resolving the async 
design issues.

 - Maciej

> 
> Regards,
> -Darin
> 
> 
> 
> On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:
> [This is actually an enhancement announcement to an existing feature]
> 
> Over at 
> , 
> Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to 
> allow authors to take full advantage of high-resolution backing stores, when 
> available (whereas the existing API hides the fact that the backing store 
> resolution is not CSS pixel resolution, for compatibility with existing 
> content). The enhancements include a backingStorePixelRatio attribute and 
> {get,put}ImageDataHD functions on CanvasRenderingContext2D.
> 
> Through  and 
> , I am making these 
> enhancements, with the names prefixed with “webkit”.
> 
> There is no build-time option to disable these enhancements. Ports that don’t 
> opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version of the 
> additional API. Ports that opt into high-DPI canvas need to enhance their 
> ImageBuffer implementation to fully support the resolutionScale and 
> CoordinateSystem parameters.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Darin Fisher]

Could this be an opportunity to design an asynchronous API for fetching the
pixel buffer?  It seems like many implementations are GPU backed now, and
fetching the pixel buffer is an expensive (blocking) operation.  Had you
considered making such a change?

Regards,
-Darin



On Thu, Apr 12, 2012 at 6:17 PM, Dan Bernstein  wrote:

> [This is actually an enhancement announcement to an existing feature]
>
> Over at <
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-March/035112.html>,
> Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to
> allow authors to take full advantage of high-resolution backing stores,
> when available (whereas the existing API hides the fact that the backing
> store resolution is not CSS pixel resolution, for compatibility with
> existing content). The enhancements include a backingStorePixelRatio
> attribute and {get,put}ImageDataHD functions on CanvasRenderingContext2D.
>
> Through  and <
> https://bugs.webkit.org/show_bug.cgi?id=83836>, I am making these
> enhancements, with the names prefixed with “webkit”.
>
> There is no build-time option to disable these enhancements. Ports that
> don’t opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version
> of the additional API. Ports that opt into high-DPI canvas need to enhance
> their ImageBuffer implementation to fully support the resolutionScale and
> CoordinateSystem parameters.
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Feature announcement: canvas pixel access API additions for high-resolution backing stores

[Dan Bernstein]

[This is actually an enhancement announcement to an existing feature]

Over at 
, 
Edward O’Connor has proposed enhancements to CanvasRenderingContext2D to allow 
authors to take full advantage of high-resolution backing stores, when 
available (whereas the existing API hides the fact that the backing store 
resolution is not CSS pixel resolution, for compatibility with existing 
content). The enhancements include a backingStorePixelRatio attribute and 
{get,put}ImageDataHD functions on CanvasRenderingContext2D.

Through  and 
, I am making these 
enhancements, with the names prefixed with “webkit”.

There is no build-time option to disable these enhancements. Ports that don’t 
opt into ENABLE_HIGH_DPI_CANVAS get a working, albeit boring, version of the 
additional API. Ports that opt into high-DPI canvas need to enhance their 
ImageBuffer implementation to fully support the resolutionScale and 
CoordinateSystem parameters.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev