Re: [whatwg] WebGL and ImageBitmaps

2014-05-18 Thread K. Gadd
I'd expect that the error might not accumulate for most color values.
Rounding would potentially kick in once you get the first loss of
precision. I've only historically seen color shifts upon repeated
rendering in scenarios where you're losing lots of precision, or
losing energy (bad RGB - HSV conversions, for example) - you don't
actually need a lot of precision to fix that as long as your
coefficients are right.

On Fri, May 16, 2014 at 8:41 PM, Rik Cabanier caban...@gmail.com wrote:



 On Fri, May 16, 2014 at 3:06 PM, Justin Novosad ju...@google.com wrote:

 On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com wrote:


 Is the Web page not composited in sRGB? If so, it seems the backing store
 should be sRGB too.



 The web page is not composited in sRGB. It is composited in the output
 device's color space, which is often sRGB or close to sRGB, but not always.
 A notable significant exception is pre Snow Leopard Macs that use a gamma
 1.8 transfer curve.
 By the way, sniffing the display color profile through getImageData is a
 known fingerprinting technique. This factor alone can be sufficient to
 fingerprint a user who has a calibrated monitor.


 I'm unable to reproduce what you're describing. So, if I fill with a color
 and repeatedly do a getImageData/putImageData, should I see color shifts?



Re: [whatwg] WebGL and ImageBitmaps

2014-05-18 Thread Rik Cabanier
On Sun, May 18, 2014 at 2:15 AM, K. Gadd k...@luminance.org wrote:

 I'd expect that the error might not accumulate for most color values.
 Rounding would potentially kick in once you get the first loss of
 precision.


That doesn't make sense. If this is a shift because of color management, it
should happen for pretty much all values.
I changed my profile to generate wild color shifts and tried random color
values but don't see any changes in any browser.

Could this just be happening with images that have profiles?


 I've only historically seen color shifts upon repeated
 rendering in scenarios where you're losing lots of precision, or
 losing energy (bad RGB - HSV conversions, for example) - you don't
 actually need a lot of precision to fix that as long as your
 coefficients are right.
 On Fri, May 16, 2014 at 8:41 PM, Rik Cabanier caban...@gmail.com wrote:
 
 
 
  On Fri, May 16, 2014 at 3:06 PM, Justin Novosad ju...@google.com
 wrote:
 
  On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com
 wrote:
 
 
  Is the Web page not composited in sRGB? If so, it seems the backing
 store
  should be sRGB too.
 
 
 
  The web page is not composited in sRGB. It is composited in the output
  device's color space, which is often sRGB or close to sRGB, but not
 always.
  A notable significant exception is pre Snow Leopard Macs that use a
 gamma
  1.8 transfer curve.
  By the way, sniffing the display color profile through getImageData is a
  known fingerprinting technique. This factor alone can be sufficient to
  fingerprint a user who has a calibrated monitor.
 
 
  I'm unable to reproduce what you're describing. So, if I fill with a
 color
  and repeatedly do a getImageData/putImageData, should I see color shifts?
 



Re: [whatwg] WebGL and ImageBitmaps

2014-05-18 Thread K. Gadd
The point I was trying to make there is that for many format
conversions or encoding conversions (RGB-YUV, RGB-HSL), not all
input values are degraded equally. The amount of error introduced
depends on the inputs. There are going to be some values for which the
conversion is more or less accurate - for example, in most cases I
would expect black and white to convert without any error. As a
result, you can't just pick a few random colors and fill a canvas with
them and decide based on that whether or not error is being
introduced. At a minimum, you should use a couple test pattern bitmaps
and do a comparison of the result. Keep in mind that all the
discussions of profile conversion so far have been about bitmaps, not
synthesized solid colors.

I am, of course, not an expert - but I have observed this with
repeated RGB-HSL conversions in the past (testing poor
implementations that introduced accumulated error against relatively
good implementations that did not accumulate very much error over
time.)

http://en.wikipedia.org/wiki/SRGB#Specification_of_the_transformation

Note that as described there, clipping and rounding may occur and
linear - gamma-corrected conversions may also occur. We also can't
know what color profile configuration your machine happens to be using
when you run these tests, and what browser you're using. Both of those
are important when saying that you can/can't reproduce the issue.

On Sun, May 18, 2014 at 8:22 AM, Rik Cabanier caban...@gmail.com wrote:



 On Sun, May 18, 2014 at 2:15 AM, K. Gadd k...@luminance.org wrote:

 I'd expect that the error might not accumulate for most color values.
 Rounding would potentially kick in once you get the first loss of
 precision.


 That doesn't make sense. If this is a shift because of color management, it
 should happen for pretty much all values.
 I changed my profile to generate wild color shifts and tried random color
 values but don't see any changes in any browser.

 Could this just be happening with images that have profiles?


 I've only historically seen color shifts upon repeated
 rendering in scenarios where you're losing lots of precision, or
 losing energy (bad RGB - HSV conversions, for example) - you don't
 actually need a lot of precision to fix that as long as your
 coefficients are right.
 On Fri, May 16, 2014 at 8:41 PM, Rik Cabanier caban...@gmail.com wrote:
 
 
 
  On Fri, May 16, 2014 at 3:06 PM, Justin Novosad ju...@google.com
  wrote:
 
  On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com
  wrote:
 
 
  Is the Web page not composited in sRGB? If so, it seems the backing
  store
  should be sRGB too.
 
 
 
  The web page is not composited in sRGB. It is composited in the output
  device's color space, which is often sRGB or close to sRGB, but not
  always.
  A notable significant exception is pre Snow Leopard Macs that use a
  gamma
  1.8 transfer curve.
  By the way, sniffing the display color profile through getImageData is
  a
  known fingerprinting technique. This factor alone can be sufficient to
  fingerprint a user who has a calibrated monitor.
 
 
  I'm unable to reproduce what you're describing. So, if I fill with a
  color
  and repeatedly do a getImageData/putImageData, should I see color
  shifts?
 




Re: [whatwg] WebGL and ImageBitmaps

2014-05-18 Thread Rik Cabanier
On Sun, May 18, 2014 at 8:10 PM, K. Gadd k...@luminance.org wrote:

 The point I was trying to make there is that for many format
 conversions or encoding conversions (RGB-YUV, RGB-HSL), not all
 input values are degraded equally. The amount of error introduced
 depends on the inputs. There are going to be some values for which the
 conversion is more or less accurate - for example, in most cases I
 would expect black and white to convert without any error. As a
 result, you can't just pick a few random colors and fill a canvas with
 them and decide based on that whether or not error is being
 introduced. At a minimum, you should use a couple test pattern bitmaps
 and do a comparison of the result. Keep in mind that all the
 discussions of profile conversion so far have been about bitmaps, not
 synthesized solid colors.

 I am, of course, not an expert - but I have observed this with
 repeated RGB-HSL conversions in the past (testing poor
 implementations that introduced accumulated error against relatively
 good implementations that did not accumulate very much error over
 time.)

 http://en.wikipedia.org/wiki/SRGB#Specification_of_the_transformation

 Note that as described there, clipping and rounding may occur and
 linear - gamma-corrected conversions may also occur.


But that was not was Justin said. He said they didn't know what profile was
used so sometimes things are not color managed correctly.
It was also possible to infer what machine you were running based on the
detected profile.

Color managment is not equal to the simple color transformations you
descirbe.


 We also can't
 know what color profile configuration your machine happens to be using
 when you run these tests, and what browser you're using. Both of those
 are important when saying that you can/can't reproduce the issue.


I think you have things backwards. YOU raised an issue with color
management and I'm trying to reproduce it but I'm failing because you
didn't give us enough information.
I tried:
- many different colors
- different profiles
- repeated put/getImageData calls on the same canvas
and can't reproduce.
When and on what platform/browser does this problem occur?


 On Sun, May 18, 2014 at 8:22 AM, Rik Cabanier caban...@gmail.com wrote:
 
 
 
  On Sun, May 18, 2014 at 2:15 AM, K. Gadd k...@luminance.org wrote:
 
  I'd expect that the error might not accumulate for most color values.
  Rounding would potentially kick in once you get the first loss of
  precision.
 
 
  That doesn't make sense. If this is a shift because of color management,
 it
  should happen for pretty much all values.
  I changed my profile to generate wild color shifts and tried random color
  values but don't see any changes in any browser.
 
  Could this just be happening with images that have profiles?
 
 
  I've only historically seen color shifts upon repeated
  rendering in scenarios where you're losing lots of precision, or
  losing energy (bad RGB - HSV conversions, for example) - you don't
  actually need a lot of precision to fix that as long as your
  coefficients are right.
  On Fri, May 16, 2014 at 8:41 PM, Rik Cabanier caban...@gmail.com
 wrote:
  
  
  
   On Fri, May 16, 2014 at 3:06 PM, Justin Novosad ju...@google.com
   wrote:
  
   On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com
   wrote:
  
  
   Is the Web page not composited in sRGB? If so, it seems the backing
   store
   should be sRGB too.
  
  
  
   The web page is not composited in sRGB. It is composited in the
 output
   device's color space, which is often sRGB or close to sRGB, but not
   always.
   A notable significant exception is pre Snow Leopard Macs that use a
   gamma
   1.8 transfer curve.
   By the way, sniffing the display color profile through getImageData
 is
   a
   known fingerprinting technique. This factor alone can be sufficient
 to
   fingerprint a user who has a calibrated monitor.
  
  
   I'm unable to reproduce what you're describing. So, if I fill with a
   color
   and repeatedly do a getImageData/putImageData, should I see color
   shifts?
  
 
 



Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Justin Novosad
On Thu, May 15, 2014 at 12:27 AM, Rik Cabanier caban...@gmail.com wrote:

 On Wed, May 14, 2014 at 7:45 PM, Glenn Maynard gl...@zewt.org wrote:


 Yes, color correction from sRGB to your monitor should not affect drawing
 on canvas. (What if you had multiple monitors :-))


This is a longstanding issue with 2D canvas that several developers have
complained about over the years.  The color space of the canvas backing
store is currently unspecified. Blink/WebKit uses output-referred color
space, which is bad for some inter-op cases, but good for performance.
Calling drawImage will produce inter-operable behavior as far as visual
output is concerned, but getImageData will yield values that have the
display color profile baked in. Some web developers have worked around this
by reverse-engineering the client-specific canvas to sRGB colorspace
transform by running a test pattern through drawImage+getImageData.  It is
horrible that we are making devs resort to this.

Adding a colorspace option to createImageBitmap is not enough IMHO. I think
we need a more global color-management approach for canvas. I commented on
https://www.w3.org/Bugs/Public/show_bug.cgi?id=25643


Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Ian Hickson
On Fri, 16 May 2014, Justin Novosad wrote:
 
 This is a longstanding issue with 2D canvas that several developers have
 complained about over the years.  The color space of the canvas backing
 store is currently unspecified.

It's defined that the colour space must be the same as used for CSS and 
img, but yeah the final colour space is left up to the UA:

   http://whatwg.org/html#color-spaces-and-color-correction


 Blink/WebKit uses output-referred color space, which is bad for some 
 inter-op cases, but good for performance. Calling drawImage will produce 
 inter-operable behavior as far as visual output is concerned, but 
 getImageData will yield values that have the display color profile baked 
 in.

I'm not quite sure what you mean here. If you mean that you can set 
'fillStyle' to a colour, fillRect the canvas, and the get the data for 
that pixel and find that it's a different colour, then that's 
non-conforming. If you mean that you can take a bitmap image without 
colour-profile information and draw it on a canvas and then getImageData() 
will return different results, then again, that's non-conforming.

If you mean that drawing an image with a color profile will result in 
getImageData() returning different colour pixels on different systems, 
then that's allowed, because the colour space of the canvas (and the rest 
of the Web platform, which must be the same colour space) is not defined.


 Some web developers have worked around this by reverse-engineering the 
 client-specific canvas to sRGB colorspace transform by running a test 
 pattern through drawImage+getImageData.  It is horrible that we are 
 making devs resort to this.

I'm not really sure what this work around achieves. Can you elaborate?

If you just want to do everything in sRGB, then putting all your images in 
sRGB but without giving color space information (or setting the option to 
'strip', if we add these createImageBitmap() options) would result in what 
you want, no?

You'd have to manually (or on the server) convert images that were in 
other colour spaces, though.


 Adding a colorspace option to createImageBitmap is not enough IMHO. I 
 think we need a more global color-management approach for canvas.

If we need colour management, we need it for the Web as a whole, not just 
for canvas. So far, though, implementations have not been implementing the 
features that have been proposed, so...:

   http://www.w3.org/TR/css3-color/#dropped

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'


Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Kenneth Russell
On Fri, May 16, 2014 at 9:27 AM, Ian Hickson i...@hixie.ch wrote:
 On Fri, 16 May 2014, Justin Novosad wrote:

 This is a longstanding issue with 2D canvas that several developers have
 complained about over the years.  The color space of the canvas backing
 store is currently unspecified.

 It's defined that the colour space must be the same as used for CSS and
 img, but yeah the final colour space is left up to the UA:

http://whatwg.org/html#color-spaces-and-color-correction


 Blink/WebKit uses output-referred color space, which is bad for some
 inter-op cases, but good for performance. Calling drawImage will produce
 inter-operable behavior as far as visual output is concerned, but
 getImageData will yield values that have the display color profile baked
 in.

 I'm not quite sure what you mean here. If you mean that you can set
 'fillStyle' to a colour, fillRect the canvas, and the get the data for
 that pixel and find that it's a different colour, then that's
 non-conforming. If you mean that you can take a bitmap image without
 colour-profile information and draw it on a canvas and then getImageData()
 will return different results, then again, that's non-conforming.

 If you mean that drawing an image with a color profile will result in
 getImageData() returning different colour pixels on different systems,
 then that's allowed, because the colour space of the canvas (and the rest
 of the Web platform, which must be the same colour space) is not defined.


 Some web developers have worked around this by reverse-engineering the
 client-specific canvas to sRGB colorspace transform by running a test
 pattern through drawImage+getImageData.  It is horrible that we are
 making devs resort to this.

 I'm not really sure what this work around achieves. Can you elaborate?

 If you just want to do everything in sRGB, then putting all your images in
 sRGB but without giving color space information (or setting the option to
 'strip', if we add these createImageBitmap() options) would result in what
 you want, no?

 You'd have to manually (or on the server) convert images that were in
 other colour spaces, though.


 Adding a colorspace option to createImageBitmap is not enough IMHO. I
 think we need a more global color-management approach for canvas.

 If we need colour management, we need it for the Web as a whole, not just
 for canvas. So far, though, implementations have not been implementing the
 features that have been proposed, so...:

http://www.w3.org/TR/css3-color/#dropped

Recently there has been activity in some implementations to finally
solve the color management problem. I can't speak for Safari, but have
heard that as of Mac OS X 10.9, that implementation is rendering web
pages into the sRGB color space per the CSS spec, and converting to
the display's color profile on the way to the screen. I was told that
there was a concern there would be a compatibility impact with web
apps implicitly expecting to work in the display's color space, but
surprisingly, everything seemed to work during this transition.

In Chrome there is also work ongoing to handle multiple monitors with
different display color profiles. Noel Gordon (CC'd) is driving this
and the list of active issues can be seen at
https://code.google.com/p/chromium/issues/list?q=owner%3Anoel%40chromium.org
. It's likely the implementation will be done in stages: first
rendering images correctly when dragging windows from monitor to
monitor, and then rendering all web page content correctly.

In sum, while it has taken a while for implementations to finally
start tackling the color management problem, it's happening now.

-Ken


Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Justin Novosad
On Fri, May 16, 2014 at 12:27 PM, Ian Hickson i...@hixie.ch wrote:

 On Fri, 16 May 2014, Justin Novosad wrote
  Blink/WebKit uses output-referred color space, which is bad for some
  inter-op cases, but good for performance. Calling drawImage will produce
  inter-operable behavior as far as visual output is concerned, but
  getImageData will yield values that have the display color profile baked
  in.

 I'm not quite sure what you mean here. If you mean that you can set
 'fillStyle' to a colour, fillRect the canvas, and the get the data for
 that pixel and find that it's a different colour, then that's
 non-conforming. If you mean that you can take a bitmap image without
 colour-profile information and draw it on a canvas and then getImageData()
 will return different results, then again, that's non-conforming.

 If you mean that drawing an image with a color profile will result in
 getImageData() returning different colour pixels on different systems,
 then that's allowed, because the colour space of the canvas (and the rest
 of the Web platform, which must be the same colour space) is not defined.

 Yes the later is what I mean. It is allowed, and it is causing headaches
for many web developers. One possible solution would be to impose that
ImageData be in sRGB color space. Unfortunately, that would imply loss of
precision due to color space conversion rounding errors in a
getImageData/putImageData round trip.  But that is probably a lesser evil.
 I wonder if making this change would break anything on the web...


  Some web developers have worked around this by reverse-engineering the
  client-specific canvas to sRGB colorspace transform by running a test
  pattern through drawImage+getImageData.  It is horrible that we are
  making devs resort to this.

 I'm not really sure what this work around achieves. Can you elaborate?


For example, if a web app wants to apply an image processing algorithm, it
would use getImageData to retrieve the original pixel values, process the
data, and display the results using putImageData.  The color space of the
image data is undefined and it affects the behavior of the image processing
algorithm.  In order to standardize the behavior of the image processing
algorithm, the image data must be converted to a known color space.  The
required color space transformation can not be queried but it can be
determined experimentally by taking an img that is in a known color space
and contains known color values.  You draw that image to a canvas using
drawImage, and read it back using getImageData.  The color values returned
by getImageData and the known corresponding color values of the original
image provide a set of color-space correspondences that can be used to feed
a curve fitting algorithm in order to reverse-engineer the parameters of
the color space conversion that maps the unknown ImageData color space to
the known color space of the test image.


 If you just want to do everything in sRGB, then putting all your images in
 sRGB but without giving color space information (or setting the option to
 'strip', if we add these createImageBitmap() options) would result in what
 you want, no?


Only if the canvas backing store is forced to be in sRGB.


 You'd have to manually (or on the server) convert images that were in
 other colour spaces, though.


  Adding a colorspace option to createImageBitmap is not enough IMHO. I
  think we need a more global color-management approach for canvas.

 If we need colour management, we need it for the Web as a whole, not just
 for canvas. So far, though, implementations have not been implementing the
 features that have been proposed, so...:

http://www.w3.org/TR/css3-color/#dropped


I think think CSS and HTML can survive well without  color management
features, as long as the color behavior is well defined, which seems to be
the case except for canvas
ImageData is problematic because it stores data that in an undefined color
space.



 --
 Ian Hickson   U+1047E)\._.,--,'``.fL
 http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'



Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Rik Cabanier
On Fri, May 16, 2014 at 12:16 PM, Justin Novosad ju...@google.com wrote:




 On Fri, May 16, 2014 at 12:27 PM, Ian Hickson i...@hixie.ch wrote:

 On Fri, 16 May 2014, Justin Novosad wrote
  Blink/WebKit uses output-referred color space, which is bad for some
  inter-op cases, but good for performance. Calling drawImage will produce
  inter-operable behavior as far as visual output is concerned, but
  getImageData will yield values that have the display color profile baked
  in.

 I'm not quite sure what you mean here. If you mean that you can set
 'fillStyle' to a colour, fillRect the canvas, and the get the data for
 that pixel and find that it's a different colour, then that's
 non-conforming. If you mean that you can take a bitmap image without
 colour-profile information and draw it on a canvas and then getImageData()
 will return different results, then again, that's non-conforming.

 If you mean that drawing an image with a color profile will result in
 getImageData() returning different colour pixels on different systems,
 then that's allowed, because the colour space of the canvas (and the rest
 of the Web platform, which must be the same colour space) is not defined.

 Yes the later is what I mean. It is allowed, and it is causing headaches
 for many web developers. One possible solution would be to impose that
 ImageData be in sRGB color space. Unfortunately, that would imply loss of
 precision due to color space conversion rounding errors in a
 getImageData/putImageData round trip.


Can you explain why that is? Presumably, the image data is converted to
sRGB before you use it to composite its pixels.


 But that is probably a lesser evil.  I wonder if making this change would
 break anything on the web...


  Some web developers have worked around this by reverse-engineering the
  client-specific canvas to sRGB colorspace transform by running a test
  pattern through drawImage+getImageData.  It is horrible that we are
  making devs resort to this.

 I'm not really sure what this work around achieves. Can you elaborate?


 For example, if a web app wants to apply an image processing algorithm, it
 would use getImageData to retrieve the original pixel values, process the
 data, and display the results using putImageData.  The color space of the
 image data is undefined and it affects the behavior of the image processing
 algorithm.  In order to standardize the behavior of the image processing
 algorithm, the image data must be converted to a known color space.  The
 required color space transformation can not be queried but it can be
 determined experimentally by taking an img that is in a known color space
 and contains known color values.  You draw that image to a canvas using
 drawImage, and read it back using getImageData.  The color values returned
 by getImageData and the known corresponding color values of the original
 image provide a set of color-space correspondences that can be used to feed
 a curve fitting algorithm in order to reverse-engineer the parameters of
 the color space conversion that maps the unknown ImageData color space to
 the known color space of the test image.


I agree. That is horrible!


 If you just want to do everything in sRGB, then putting all your images in
 sRGB but without giving color space information (or setting the option to
 'strip', if we add these createImageBitmap() options) would result in what
 you want, no?


 Only if the canvas backing store is forced to be in sRGB.


Is the Web page not composited in sRGB? If so, it seems the backing store
should be sRGB too.


 You'd have to manually (or on the server) convert images that were in
 other colour spaces, though.


  Adding a colorspace option to createImageBitmap is not enough IMHO. I
  think we need a more global color-management approach for canvas.

 If we need colour management, we need it for the Web as a whole, not just
 for canvas. So far, though, implementations have not been implementing the
 features that have been proposed, so...:

http://www.w3.org/TR/css3-color/#dropped


 I think think CSS and HTML can survive well without  color management
 features, as long as the color behavior is well defined, which seems to be
 the case except for canvas
 ImageData is problematic because it stores data that in an undefined color
 space.



  --
 Ian Hickson   U+1047E)\._.,--,'``.fL
 http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'





Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Justin Novosad
On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com wrote:


 Is the Web page not composited in sRGB? If so, it seems the backing store
 should be sRGB too.



The web page is not composited in sRGB. It is composited in the output
device's color space, which is often sRGB or close to sRGB, but not always.
A notable significant exception is pre Snow Leopard Macs that use a gamma
1.8 transfer curve.
By the way, sniffing the display color profile through getImageData is a
known fingerprinting technique. This factor alone can be sufficient to
fingerprint a user who has a calibrated monitor.


Re: [whatwg] WebGL and ImageBitmaps

2014-05-16 Thread Rik Cabanier
On Fri, May 16, 2014 at 3:06 PM, Justin Novosad ju...@google.com wrote:

 On Fri, May 16, 2014 at 5:42 PM, Rik Cabanier caban...@gmail.com wrote:


 Is the Web page not composited in sRGB? If so, it seems the backing store
 should be sRGB too.



 The web page is not composited in sRGB. It is composited in the output
 device's color space, which is often sRGB or close to sRGB, but not always.
 A notable significant exception is pre Snow Leopard Macs that use a gamma
 1.8 transfer curve.
 By the way, sniffing the display color profile through getImageData is a
 known fingerprinting technique. This factor alone can be sufficient to
 fingerprint a user who has a calibrated monitor.


I'm unable to reproduce what you're describing. So, if I fill with a color
and repeatedly do a getImageData/putImageData, should I see color shifts?


Re: [whatwg] WebGL and ImageBitmaps

2014-05-15 Thread Kenneth Russell
On Fri, May 9, 2014 at 2:58 PM, Ian Hickson i...@hixie.ch wrote:
 On Fri, 9 May 2014, Ian Hickson wrote:
 On Thu, 18 Jul 2013, Justin Novosad wrote:
 
  To help us iterate further, I've attempted to capture the essence of
  this thread on the whatwg wiki, using the problem solving template. I
  tried to capture the main ideas that we seem to agree on so far and I
  started to think about how to handle special cases.
 
  http://wiki.whatwg.org/wiki/ImageBitmap_Options

 Are the strongly desired options in the above wiki page still the
 options we should be adding?

 On the assumption that they are, I filed some bugs to cover this:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=25642
createImageBitmap() options: image data orientation

https://www.w3.org/Bugs/Public/show_bug.cgi?id=25643
createImageBitmap() options: color space handling

https://www.w3.org/Bugs/Public/show_bug.cgi?id=25644
createImageBitmap() options: alpha channel handling

 As usual with such bugs, if there is interest amongst browser vendors in
 implementing any of these, please comment on the bug.

For the record, I've commented on the above bugs. I'd personally like
to see them all specified and implemented so that image loading for
the purpose of uploading to WebGL textures can be more efficient.

-Ken


Re: [whatwg] WebGL and ImageBitmaps

2014-05-15 Thread Ian Hickson
On Wed, 14 May 2014, Kenneth Russell wrote:
 
  On the assumption that they are, I filed some bugs to cover this:
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25642
 createImageBitmap() options: image data orientation
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25643
 createImageBitmap() options: color space handling
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25644
 createImageBitmap() options: alpha channel handling
 
  As usual with such bugs, if there is interest amongst browser vendors 
  in implementing any of these, please comment on the bug.
 
 For the record, I've commented on the above bugs. I'd personally like to 
 see them all specified and implemented so that image loading for the 
 purpose of uploading to WebGL textures can be more efficient.

I see that your comment was that the WebGL WG would like them specified.

Does your saying that mean that Chrome wants to implement these also? Just 
having a working group want them specified doesn't help get us to multiple 
implementations...

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'


Re: [whatwg] WebGL and ImageBitmaps

2014-05-15 Thread Kenneth Russell
On Thu, May 15, 2014 at 10:59 AM, Ian Hickson i...@hixie.ch wrote:
 On Wed, 14 May 2014, Kenneth Russell wrote:
 
  On the assumption that they are, I filed some bugs to cover this:
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25642
 createImageBitmap() options: image data orientation
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25643
 createImageBitmap() options: color space handling
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=25644
 createImageBitmap() options: alpha channel handling
 
  As usual with such bugs, if there is interest amongst browser vendors
  in implementing any of these, please comment on the bug.

 For the record, I've commented on the above bugs. I'd personally like to
 see them all specified and implemented so that image loading for the
 purpose of uploading to WebGL textures can be more efficient.

 I see that your comment was that the WebGL WG would like them specified.

 Does your saying that mean that Chrome wants to implement these also? Just
 having a working group want them specified doesn't help get us to multiple
 implementations...

To clarify the Chrome team's desire to implement these I added another
comment to each bug, and pointed to the relevant portion of the WebGL
spec in response to the latest comment on
https://www.w3.org/Bugs/Public/show_bug.cgi?id=25644 .

-Ken


Re: [whatwg] WebGL and ImageBitmaps

2014-05-14 Thread Glenn Maynard
On Mon, May 12, 2014 at 3:19 AM, K. Gadd k...@luminance.org wrote:

 On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
  I'm assuming you're referring to the case where if you try to draw a
  subpart of an image and for some reason it has to be sampled (e.g. you're
  drawing it larger than the source), the anti-aliasing is optimised for
  tiling and so you get leakage from the next sprite over.
 
  If so, the solution is just to separate the sprites by a pixel of
  transparent black, no?

 This is the traditional solution for scenarios where you are sampling
 from a filtered texture in 3d. However, it only works if you never
 scale images, which is actually not the case in many game scenarios.


That's only an issue when sampling without premultiplication, right?

I had to refresh my memory on this:

https://zewt.org/~glenn/test-premultiplied-scaling/

The first image is using WebGL to blit unpremultiplied.  The second is
WebGL blitting premultiplied.  The last is 2d canvas.  (We're talking about
canvas here, of course, but WebGL makes it easier to test the different
behavior.)  This blits a red rectangle surrounded by transparent space on
top of a red canvas.  The black square is there so I can tell that it's
actually drawing something.

The first one gives a seam around the transparent area, as the white pixels
(which are completely transparent in the image) are sampled into the
visible part.  I think this is the problem we're talking about.  The second
gives no seam, and the Canvas one gives no seam, indicating that it's a
premultiplied blit.  I don't know if that's specified, but the behavior is
the same in Chrome and FF.


On Tue, May 13, 2014 at 8:59 PM, K. Gadd k...@luminance.org wrote:

 On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
  Can you give an explicit example where browsers are having different
  behavior when using drawImage?

 I thought I was pretty clear about this... colorspace conversion and
 alpha conversion happen here depending on the user's display
 configuration, the color profile of the source image, and what browser
 you're using. I've observed differences between Firefox and Chrome
 here, along with different behavior on OS X (presumably due to their
 different implementation of color profiles).

 In this case 'different' means 'loading  drawing an image to a canvas
 gives different results via getImageData'.


That's a description, not an explicit example.  An example would be a URL
demonstrating the issue.

The effects of color profiles should never be visible to script--they
should be applied when the canvas is drawn to the screen, not when the
image is decoded or the canvas is manipulated.  That seems hard to
implement, though, if you're blitting images to a canvas that all have
different color profiles.  It's probably better to ignore color profiles
for canvas entirely than to expose the user's monitor configuration like
this...

-- 
Glenn Maynard


Re: [whatwg] WebGL and ImageBitmaps

2014-05-14 Thread K. Gadd
Replies inline

On Wed, May 14, 2014 at 4:27 PM, Glenn Maynard gl...@zewt.org wrote:
 On Mon, May 12, 2014 at 3:19 AM, K. Gadd k...@luminance.org wrote:

 This is the traditional solution for scenarios where you are sampling
 from a filtered texture in 3d. However, it only works if you never
 scale images, which is actually not the case in many game scenarios.


 That's only an issue when sampling without premultiplication, right?

 I had to refresh my memory on this:

 https://zewt.org/~glenn/test-premultiplied-scaling/

 The first image is using WebGL to blit unpremultiplied.  The second is WebGL
 blitting premultiplied.  The last is 2d canvas.  (We're talking about canvas
 here, of course, but WebGL makes it easier to test the different behavior.)
 This blits a red rectangle surrounded by transparent space on top of a red
 canvas.  The black square is there so I can tell that it's actually drawing
 something.

 The first one gives a seam around the transparent area, as the white pixels
 (which are completely transparent in the image) are sampled into the visible
 part.  I think this is the problem we're talking about.  The second gives no
 seam, and the Canvas one gives no seam, indicating that it's a premultiplied
 blit.  I don't know if that's specified, but the behavior is the same in
 Chrome and FF.

The reason one pixel isn't sufficient is that if the minification
ratio is below 50% (say, 33%), sampling algorithms other than
non-mipmapped-bilinear will begin sampling more than 4 pixels (or one
quad, in gpu shading terminology), so you now need enough transparent
pixels around all your textures to ensure that sampling never crosses
the boundaries into another image.

http://fgiesen.wordpress.com/2011/07/10/a-trip-through-the-graphics-pipeline-2011-part-8/
explains the concept of quads, along with relevant issues like
centroid interpolation. Anyone talking about correctness or
performance in modern accelerated rendering might benefit from reading
this whole series.

You do make the good point that whether or not the canvas
implementation is using premultiplied textures has an effect on the
result of scaling and filtering (since doing scaling/filtering on
nonpremultiplied rgba produces color bleeding from transparent
pixels). Is that currently specified? I don't think I've seen bleeding
artifacts recently, but I'm not certain whether the spec requires this
explicitly.

This issue is however not color bleeding - color bleeding is a math
'error' that results from not using premultiplication - but that the
filtering algorithm samples pixels outside the actual 'rectangle'
intended to be drawn. (This is an implicit problem with sampling based
on texture coordinates and derivatives instead of pixel offsets)

If you search for 'padding texture atlases' you can see some examples
that show why this is a tricky problem and a single pixel of padding
is not sufficient:
http://wiki.polycount.com/EdgePadding

There are some related problems here for image compression as well,
due to the block-oriented nature of codecs like JPEG and DXTC. Luckily
they aren't something the user agent has to deal with in their canvas
implementation, but that's another example where a single pixel of
padding isn't enough.

 On Tue, May 13, 2014 at 8:59 PM, K. Gadd k...@luminance.org wrote:
 I thought I was pretty clear about this... colorspace conversion and
 alpha conversion happen here depending on the user's display
 configuration, the color profile of the source image, and what browser
 you're using. I've observed differences between Firefox and Chrome
 here, along with different behavior on OS X (presumably due to their
 different implementation of color profiles).

 In this case 'different' means 'loading  drawing an image to a canvas
 gives different results via getImageData'.


 That's a description, not an explicit example.  An example would be a URL
 demonstrating the issue.

http://joedev.net/JSIL/Numbers/ was the first game to report an issue
from this, because his levels are authored as images. He ended up
solving the problem by following my advice to manually strip color
profile information from all his images (though this is not a panacea;
a browser could decide that profile-information-less images are now
officially sRGB, and then profile-convert them to the display profile)

It's been long enough that I don't know if his uploaded build works
anymore or whether it will demonstrate the issue. It's possible he
removed his dependency on images by now.

Here is what I told the developer in an email thread when he first
reported the issue (and by 'reported' I mean 'sent me a very confused
email saying that his game didn't work in Firefox and he had no idea
why'):

 The reason it's not working in Firefox right now is due to a firefox bug, 
 because your PNG files contain what's called a 'sRGB chunk': 
 https://bugzilla.mozilla.org/show_bug.cgi?id=867594
 I don't know if this bug can be fixed on Firefox's side because it's an 

Re: [whatwg] WebGL and ImageBitmaps

2014-05-14 Thread Rik Cabanier
On Tue, May 13, 2014 at 6:59 PM, K. Gadd k...@luminance.org wrote:

 On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
  Can you give an explicit example where browsers are having different
  behavior when using drawImage?

 I thought I was pretty clear about this... colorspace conversion and
 alpha conversion happen here depending on the user's display
 configuration, the color profile of the source image, and what browser
 you're using. I've observed differences between Firefox and Chrome
 here, along with different behavior on OS X (presumably due to their
 different implementation of color profiles).

 In this case 'different' means 'loading  drawing an image to a canvas
 gives different results via getImageData'.

 On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
  Would this be solved with Greg's proposal for flags on ImageBitmap:
 
 http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2013-June/251541.html

 I believe so. I think I was on record when he first posted that I
 consider the alpha and colorspace flags he described as adequate.
 FlipY is considerably less important to me, but I can see how people
 might want it (honestly, reversing the order of scanlines is a very
 cheap operation; you can do it in the sampling stage of your shader,
 and actually *have* to in OpenGL because of their coordinate system
 when you're doing render-to-texture.)

 On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
  Very specifically here, by 'known color space' i just mean that the
  color space of the image is exposed to the end user. I don't think we
  can possibly pick a standard color space to always use; the options
  are 'this machine's current color space' and 'the color space of the
  input bitmap'. In many cases the color space of the input bitmap is
  effectively 'no color space', and game developers feed the raw rgba to
  the GPU. It's important to support that use case without degrading the
  image data.
 
 
  Is that not the case today?

 It is very explicitly not the case, which is why we are discussing it.
 It is not currently possible to do lossless manipulation of PNG images
 in a web browser using canvas. The issues I described where you get
 different results from getImageData are a part of that.

 On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
  Safari never created a temporary image and I recently updated Firefox so
 it
  matches Safari.
  Both Safari, IE and Firefox will now sample outside of the drawImage
 region.
  Chrome does not but they will fix that at some point.

 This is incorrect. A quick google search for 'webkit drawimage source
 rectangle temporary' reveals such, in a post to this list.

 http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-December/080583.html
 My statement to this effect was based on my (imperfect) memory of that
 post. 'CGImage' (to me) says Safari since it's an Apple API, and the
 post mentions Safari.


I made a codepen that showed the issue: http://codepen.io/adobe/pen/jIzbv
Firefox was not matching the behavior on mac because it created a
intermediate image. I fixed that in
https://bugzilla.mozilla.org/show_bug.cgi?id=987292

I agree that the code you linked to exists in WebKit but they add padding
so it samples outside the source again.


Re: [whatwg] WebGL and ImageBitmaps

2014-05-14 Thread Rik Cabanier
On Wed, May 14, 2014 at 7:45 PM, Glenn Maynard gl...@zewt.org wrote:

 On Wed, May 14, 2014 at 6:27 PM, Glenn Maynard gl...@zewt.org wrote:

  That's only an issue when sampling without premultiplication, right?
 
  I had to refresh my memory on this:
 
  https://zewt.org/~glenn/test-premultiplied-scaling/
 
  The first image is using WebGL to blit unpremultiplied.  The second is
  WebGL blitting premultiplied.  The last is 2d canvas.  (We're talking
 about
  canvas here, of course, but WebGL makes it easier to test the different
  behavior.)  This blits a red rectangle surrounded by transparent space on
  top of a red canvas.  The black square is there so I can tell that it's
  actually drawing something.
 
  The first one gives a seam around the transparent area, as the white
  pixels (which are completely transparent in the image) are sampled into
 the
  visible part.  I think this is the problem we're talking about.  The
 second
  gives no seam, and the Canvas one gives no seam, indicating that it's a
  premultiplied blit.  I don't know if that's specified, but the behavior
 is
  the same in Chrome and FF.
 

 It looks right on red, but if the background is green you can still see the
 post-premultiplied black being pulled in.  It's really just GL_REPEAT that
 you want, repeating the outer edge.


 On Wed, May 14, 2014 at 9:21 PM, K. Gadd k...@luminance.org wrote:

  The reason one pixel isn't sufficient is that if the minification
  ratio is below 50% (say, 33%), sampling algorithms other than
  non-mipmapped-bilinear will begin sampling more than 4 pixels (or one
  quad, in gpu shading terminology), so you now need enough transparent
  pixels around all your textures to ensure that sampling never crosses
  the boundaries into another image.
 

 I'm well aware of the issues of sampling sprite sheets; I've dealt with the
 issue at length in the past.  That's unrelated to my last mail, however,
 which was about premultiplication (which is something I've not used as
 much).


  I agree with this, but I'm not going to assume it's actually possible
  for a canvas implementation to work this way. I assume that color
  profile conversions are non-trivial (in fact, I'm nearly certain they
  are non-trivial), so doing the conversion every time you render a
  canvas to the compositor is probably expensive, especially if your GPU
  isn't powerful enough to do it in a shader (mobile devices, perhaps) -
  so I expect that most implementations do the conversion once at load
  time, to prepare an image for rendering. Until it became possible to
  retrieve image pixels with getImageData, this was a good, safe
  optimization.
 

 What I meant is that I think color correction simply shouldn't apply to
 canvas at all.  That may not be ideal, but I'm not sure of anything else
 that won't cause severe interop issues.


Maybe the color correction described here is happening:
https://hsivonen.fi/png-gamma/

If so, the image that's drawn on the canvas should match what the browser
is showing on screen.
Without an example, it's just speculation of course.


 To be clear, colorspace conversion--converting from sRGB to RGB--isn't a
 problem, other than probably needing to be specified more clearly and being
 put behind an option somewhere, so you can avoid a lossy colorspace
 conversion.  The problem is color correction that takes the user's monitor
 configuration into account, since the user's monitor settings shouldn't be
 visible to script.  I don't know enough about color correction to know if
 this can be done efficiently in an interoperable way, so the data scripts
 see isn't affected by the user's configuration.


Yes, color correction from sRGB to your monitor should not affect drawing
on canvas. (What if you had multiple monitors :-))


Re: [whatwg] WebGL and ImageBitmaps

2014-05-13 Thread K. Gadd
On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
 Can you give an explicit example where browsers are having different
 behavior when using drawImage?

I thought I was pretty clear about this... colorspace conversion and
alpha conversion happen here depending on the user's display
configuration, the color profile of the source image, and what browser
you're using. I've observed differences between Firefox and Chrome
here, along with different behavior on OS X (presumably due to their
different implementation of color profiles).

In this case 'different' means 'loading  drawing an image to a canvas
gives different results via getImageData'.

On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
 Would this be solved with Greg's proposal for flags on ImageBitmap:
 http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2013-June/251541.html

I believe so. I think I was on record when he first posted that I
consider the alpha and colorspace flags he described as adequate.
FlipY is considerably less important to me, but I can see how people
might want it (honestly, reversing the order of scanlines is a very
cheap operation; you can do it in the sampling stage of your shader,
and actually *have* to in OpenGL because of their coordinate system
when you're doing render-to-texture.)

On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
 Very specifically here, by 'known color space' i just mean that the
 color space of the image is exposed to the end user. I don't think we
 can possibly pick a standard color space to always use; the options
 are 'this machine's current color space' and 'the color space of the
 input bitmap'. In many cases the color space of the input bitmap is
 effectively 'no color space', and game developers feed the raw rgba to
 the GPU. It's important to support that use case without degrading the
 image data.


 Is that not the case today?

It is very explicitly not the case, which is why we are discussing it.
It is not currently possible to do lossless manipulation of PNG images
in a web browser using canvas. The issues I described where you get
different results from getImageData are a part of that.

On Mon, May 12, 2014 at 4:44 PM, Rik Cabanier caban...@gmail.com wrote:
 Safari never created a temporary image and I recently updated Firefox so it
 matches Safari.
 Both Safari, IE and Firefox will now sample outside of the drawImage region.
 Chrome does not but they will fix that at some point.

This is incorrect. A quick google search for 'webkit drawimage source
rectangle temporary' reveals such, in a post to this list.
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-December/080583.html
My statement to this effect was based on my (imperfect) memory of that
post. 'CGImage' (to me) says Safari since it's an Apple API, and the
post mentions Safari.

-kg


Re: [whatwg] WebGL and ImageBitmaps

2014-05-12 Thread K. Gadd
Gosh, this thread is old. I'm going to try and compose a coherent
response but at this point I've forgotten a lot of the context...

On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
 On Thu, 18 Jul 2013, K. Gadd wrote:

 Ultimately the core here is that without control over colorspace
 conversion, any sort of deterministic image processing in HTML5 is off
 the table, and you have to write your own image decoders, encoders, and
 manipulation routines in JavaScript using raw typed arrays. Maybe that's
 how it has to be, but it would be cool to at least support basic
 variations of these use cases in Canvas since getImageData/putImageData
 already exist and are fairly well-specified (other than this problem,
 and some nits around source rectangles and alpha transparency).

 Given that the user's device could be a very low-power device, or one with
 a very small screen, but the user might still want to be manipulating very
 large images, it might be best to do the master manipulation on the
 server anyway.

This request is not about efficient image manipulation (as you point
out, this is best done on a high power machine) - without control over
colorspace conversion any image processing is nondeterministic. There
are games and apps out there that rely on getting the exact same
pixels out of a given Image on all machines, and that's impossible
right now due to differing behaviors. You see demoscene projects
packing data into bitmaps (yuck), or games using images as the
canonical representation of user-generated content. The latter, I
think, is entirely defensible - maybe even desirable, since it lets
end users interact with the game using photoshop or mspaint.
Supporting these use cases in a cross-browser manner is impossible
right now, yet they work in the desktop versions of these games.

On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
 On Thu, 18 Jul 2013, K. Gadd wrote:
 Out of the features suggested previously in the thread, I would
 immediately be able to make use of control over colorspace conversion
 and an ability to opt into premultiplied alpha. Not getting
 premultiplied alpha, as is the case in virtually every canvas
 implementation I've tried, has visible negative consequences for image
 quality and also reduces the performance of some use cases where bitmap
 manipulation needs to happen, due to the fact that premultiplied alpha
 is the 'preferred' form for certain types of rendering and the math
 works out better. I think the upsides to getting premultiplication are
 the same here as they are in WebGL: faster uploads/downloads, better
 results, etc.

 Can you elaborate on exactly what this would look like in terms of the API
 implications? What changes to the spec did you have in mind?

I don't remember what my exact intent here was, but I'll try to resynthesize it:
The key here is to have a clear understanding of what data you get out
of an ImageBitmap. It is *not* necessary for the end user to be able
to specify it, as long as the spec dictates that all browsers provide
the exact same format to end users.
If we pick one format and lock to it, we want a format that discards
as little source image data as possible (preferably *no* data is
discarded) - which would mean the raw source image data, without any
colorspace or alpha channel conversion applied.

This allows all the procedural image manipulation cases described
above, and makes it a very fast and straightforward path for loading
images you plan to pass off to the GPU as a WebGL texture. There's a
bit more on this below...

On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
 On Thu, 18 Jul 2013, K. Gadd wrote:
 To clearly state what would make ImageBitmap useful for the use cases I
 encounter and my end-users encounter:
 ImageBitmap should be a canonical representation of a 2D bitmap, with a
 known color space, known pixel format, known alpha representation
 (premultiplied/not premultiplied), and ready for immediate rendering or
 pixel data access. It's okay if it's immutable, and it's okay if
 constructing one from an img or a Blob takes time, as long as once I have
 an ImageBitmap I can use it to render and use it to extract pixel data
 without user configuration/hardware producing unpredictable results.

 This seems reasonable, but it's not really detailed enough for me to turn
 it into spec. What colour space? What exactly should we be doing to the
 alpha channel?

Very specifically here, by 'known color space' i just mean that the
color space of the image is exposed to the end user. I don't think we
can possibly pick a standard color space to always use; the options
are 'this machine's current color space' and 'the color space of the
input bitmap'. In many cases the color space of the input bitmap is
effectively 'no color space', and game developers feed the raw rgba to
the GPU. It's important to support that use case without degrading the
image data.

Alpha channel is simpler, but 

Re: [whatwg] WebGL and ImageBitmaps

2014-05-12 Thread Rik Cabanier
On Mon, May 12, 2014 at 1:19 AM, K. Gadd k...@luminance.org wrote:

 Gosh, this thread is old. I'm going to try and compose a coherent
 response but at this point I've forgotten a lot of the context...

 On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
  On Thu, 18 Jul 2013, K. Gadd wrote:
 
  Ultimately the core here is that without control over colorspace
  conversion, any sort of deterministic image processing in HTML5 is off
  the table, and you have to write your own image decoders, encoders, and
  manipulation routines in JavaScript using raw typed arrays. Maybe that's
  how it has to be, but it would be cool to at least support basic
  variations of these use cases in Canvas since getImageData/putImageData
  already exist and are fairly well-specified (other than this problem,
  and some nits around source rectangles and alpha transparency).
 
  Given that the user's device could be a very low-power device, or one
 with
  a very small screen, but the user might still want to be manipulating
 very
  large images, it might be best to do the master manipulation on the
  server anyway.

 This request is not about efficient image manipulation (as you point
 out, this is best done on a high power machine) - without control over
 colorspace conversion any image processing is nondeterministic. There
 are games and apps out there that rely on getting the exact same
 pixels out of a given Image on all machines, and that's impossible
 right now due to differing behaviors. You see demoscene projects
 packing data into bitmaps (yuck), or games using images as the
 canonical representation of user-generated content. The latter, I
 think, is entirely defensible - maybe even desirable, since it lets
 end users interact with the game using photoshop or mspaint.
 Supporting these use cases in a cross-browser manner is impossible
 right now, yet they work in the desktop versions of these games.

 On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
  On Thu, 18 Jul 2013, K. Gadd wrote:
  Out of the features suggested previously in the thread, I would
  immediately be able to make use of control over colorspace conversion
  and an ability to opt into premultiplied alpha. Not getting
  premultiplied alpha, as is the case in virtually every canvas
  implementation I've tried, has visible negative consequences for image
  quality and also reduces the performance of some use cases where bitmap
  manipulation needs to happen, due to the fact that premultiplied alpha
  is the 'preferred' form for certain types of rendering and the math
  works out better. I think the upsides to getting premultiplication are
  the same here as they are in WebGL: faster uploads/downloads, better
  results, etc.
 
  Can you elaborate on exactly what this would look like in terms of the
 API
  implications? What changes to the spec did you have in mind?

 I don't remember what my exact intent here was, but I'll try to
 resynthesize it:
 The key here is to have a clear understanding of what data you get out
 of an ImageBitmap. It is *not* necessary for the end user to be able
 to specify it, as long as the spec dictates that all browsers provide
 the exact same format to end users.
 If we pick one format and lock to it, we want a format that discards
 as little source image data as possible (preferably *no* data is
 discarded) - which would mean the raw source image data, without any
 colorspace or alpha channel conversion applied.


Can you give an explicit example where browsers are having different
behavior when using drawImage?


 This allows all the procedural image manipulation cases described
 above, and makes it a very fast and straightforward path for loading
 images you plan to pass off to the GPU as a WebGL texture. There's a
 bit more on this below...


Would this be solved with Greg's proposal for flags on ImageBitmap:
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2013-June/251541.html


 On Fri, May 9, 2014 at 12:02 PM, Ian Hickson i...@hixie.ch wrote:
  On Thu, 18 Jul 2013, K. Gadd wrote:
  To clearly state what would make ImageBitmap useful for the use cases I
  encounter and my end-users encounter:
  ImageBitmap should be a canonical representation of a 2D bitmap, with a
  known color space, known pixel format, known alpha representation
  (premultiplied/not premultiplied), and ready for immediate rendering or
  pixel data access. It's okay if it's immutable, and it's okay if
  constructing one from an img or a Blob takes time, as long as once I
 have
  an ImageBitmap I can use it to render and use it to extract pixel data
  without user configuration/hardware producing unpredictable results.
 
  This seems reasonable, but it's not really detailed enough for me to turn
  it into spec. What colour space? What exactly should we be doing to the
  alpha channel?

 Very specifically here, by 'known color space' i just mean that the
 color space of the image is exposed to the end user. I don't think 

Re: [whatwg] WebGL and ImageBitmaps

2014-05-09 Thread Glenn Maynard
On Fri, May 9, 2014 at 2:02 PM, Ian Hickson i...@hixie.ch wrote:

 Given that the user's device could be a very low-power device, or one with
 a very small screen, but the user might still want to be manipulating very
 large images, it might be best to do the master manipulation on the
 server anyway.


If I have a photo library with thousands of images, I don't want to upload
each image--possibly megabytes each--to the server in order to manipulate
it.

Also, doing the work on the user's system scales to lots of users more
sensibly than doing manipulations of large images on a server.

I'm assuming you're referring to the case where if you try to draw a
 subpart of an image and for some reason it has to be sampled (e.g. you're
 drawing it larger than the source), the anti-aliasing is optimised for
 tiling and so you get leakage from the next sprite over.

 If so, the solution is just to separate the sprites by a pixel of
 transparent black, no?


If you're downscaling by more than 2:1, you need to put more than one pixel
between the images, which means you have to author sprite sheets
differently depending on how far down you need to zoom.  A drawing flag
makes a lot more sense.

 --
Glenn Maynard


Re: [whatwg] WebGL and ImageBitmaps

2014-05-09 Thread Ian Hickson
On Fri, 9 May 2014, Ian Hickson wrote:
 On Thu, 18 Jul 2013, Justin Novosad wrote:
 
  To help us iterate further, I've attempted to capture the essence of 
  this thread on the whatwg wiki, using the problem solving template. I 
  tried to capture the main ideas that we seem to agree on so far and I 
  started to think about how to handle special cases.
  
  http://wiki.whatwg.org/wiki/ImageBitmap_Options
 
 Are the strongly desired options in the above wiki page still the 
 options we should be adding?

On the assumption that they are, I filed some bugs to cover this:

   https://www.w3.org/Bugs/Public/show_bug.cgi?id=25642
   createImageBitmap() options: image data orientation

   https://www.w3.org/Bugs/Public/show_bug.cgi?id=25643
   createImageBitmap() options: color space handling

   https://www.w3.org/Bugs/Public/show_bug.cgi?id=25644
   createImageBitmap() options: alpha channel handling

As usual with such bugs, if there is interest amongst browser vendors in 
implementing any of these, please comment on the bug.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'