Re: Reading image bytes to a PNG in a typed array
If we have a context we can use in a worker on which we can set the
framebuffer to which we would bind the texture and which we can readPixel
back into bytes, which we would stick into a surfaceless 2D canvas so we
can kick off the encoding to png (or just implement it in JS after
implementing zlib) then yes, it could be done in workers.
On any account, these things are all not yet available. I've meanwhile
implemented the required functionality via the long chain of clutches that
takes 6 4092x4092 textures (albed, normal, specular, specularity,
occlusion, height) and packes them into a tarfile and offers it as a
blob-url to the user. It takes about 30 seconds to pack up, during which
the page is pretty much frozen since it's 100% synchronous. The outcome is
about 50mb in size. But hey, at least it works, after a fashion.
On Sat, Jan 26, 2013 at 7:18 PM, Gregg Tavares g...@google.com wrote:
Could this be solved in workers?
x) Create canvas, set to desired size
x) Create 2D context
x) Create imageData object
x) Create a WebGL framebuffer object
x) Attach texture as color target to framebuffer
x) read back pixels into canvas2d's imageData.data member
x) ctx.putImageData into the canvas
1) Set CanvasProxy (or whatever it's called) to the size you want
2) Draw Texture
3) call CanvasProxy's toDataURL('image/png')
4) Set the CanvasProxy back to the original size
5) snip off the mime/encoding header
6) implement base64 decode in JS and decode to Uint8Array
Less steps and it's now async as well.
On Wed, Jan 16, 2013 at 8:02 AM, Florian Bösch pya...@gmail.com wrote:
Whatever the eventual solution to this problem, it should be the user of
the API driving the decision how to get the data.
On Wed, Jan 16, 2013 at 4:56 PM, Kyle Huey m...@kylehuey.com wrote:
On Wed, Jan 16, 2013 at 7:50 AM, Glenn Maynard gl...@zewt.org wrote:
On Wed, Jan 16, 2013 at 9:40 AM, Florian Bösch pya...@gmail.comwrote:
Perhaps we should think of a better scheme to export data than
toFoo(). Maybe toData('url'), toData('arraybuffer') toData('blob') or
perhaps toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think
that if you're starting to write toA, toB, toC, toX methods on an object,
you've not thought this really trough what's a parameter, and what's a
method.
We should be avoiding the need to return data in a bunch of different
interfaces in the first place. If the data is large, or takes a long or
nondeterministic amount of time to create (eg. something that would be
async in the UI thread), return a Blob; otherwise return an ArrayBuffer.
The user can convert from there as needed.
Well, the problem is that we fundamentally screwed up when we specced
Blob. It has a synchronous size getter which negates many of the
advantages of FileReader extracing data asynchronously. For something like
image encoding (that involves compression), where you have to perform the
operation to know the size, Blob and ArrayBuffer are effectively
interchangeable from the implementation perspective, since both require you
to perform the operation up front.
- Kyle
Re: Reading image bytes to a PNG in a typed array
Could this be solved in workers?
x) Create canvas, set to desired size
x) Create 2D context
x) Create imageData object
x) Create a WebGL framebuffer object
x) Attach texture as color target to framebuffer
x) read back pixels into canvas2d's imageData.data member
x) ctx.putImageData into the canvas
1) Set CanvasProxy (or whatever it's called) to the size you want
2) Draw Texture
3) call CanvasProxy's toDataURL('image/png')
4) Set the CanvasProxy back to the original size
5) snip off the mime/encoding header
6) implement base64 decode in JS and decode to Uint8Array
Less steps and it's now async as well.
On Wed, Jan 16, 2013 at 8:02 AM, Florian Bösch pya...@gmail.com wrote:
Whatever the eventual solution to this problem, it should be the user of
the API driving the decision how to get the data.
On Wed, Jan 16, 2013 at 4:56 PM, Kyle Huey m...@kylehuey.com wrote:
On Wed, Jan 16, 2013 at 7:50 AM, Glenn Maynard gl...@zewt.org wrote:
On Wed, Jan 16, 2013 at 9:40 AM, Florian Bösch pya...@gmail.com wrote:
Perhaps we should think of a better scheme to export data than toFoo().
Maybe toData('url'), toData('arraybuffer') toData('blob') or perhaps
toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think that if
you're starting to write toA, toB, toC, toX methods on an object, you've
not thought this really trough what's a parameter, and what's a method.
We should be avoiding the need to return data in a bunch of different
interfaces in the first place. If the data is large, or takes a long or
nondeterministic amount of time to create (eg. something that would be
async in the UI thread), return a Blob; otherwise return an ArrayBuffer.
The user can convert from there as needed.
Well, the problem is that we fundamentally screwed up when we specced
Blob. It has a synchronous size getter which negates many of the
advantages of FileReader extracing data asynchronously. For something like
image encoding (that involves compression), where you have to perform the
operation to know the size, Blob and ArrayBuffer are effectively
interchangeable from the implementation perspective, since both require you
to perform the operation up front.
- Kyle
Re: Reading image bytes to a PNG in a typed array
On Mon, Jan 14, 2013 at 7:04 AM, Florian Bösch pya...@gmail.com wrote: On Mon, Jan 14, 2013 at 4:00 PM, Glenn Maynard gl...@zewt.org wrote: You want toBlob, not toDataURL. So how would I stick a blob into an arraybuffer? I don't think there's any reason we can't have toArrayBuffer as well. The hard part from an implementation perspective is asynchronously encoding the image, not what we stick it in. - Kyle
Re: Reading image bytes to a PNG in a typed array
On Wed, Jan 16, 2013 at 6:29 AM, Kyle Huey m...@kylehuey.com wrote: I don't think there's any reason we can't have toArrayBuffer as well. The hard part from an implementation perspective is asynchronously encoding the image, not what we stick it in. No technical reason, but I'd avoid adding yet more paths to do the same thing unless there's a real gain. It'd be bad to head down the path of every single Blob-returning operation having an ArrayBuffer duplicate. In this particular case, he wants ArrayBuffers to put the data in a ZIP, which probably means he should be doing this from a worker anyway, to avoid doing a bunch of CRC calculations on the UI thread. In that case, the current API isn't even inconvenient, since once you post the Blob to the worker you can use FileReaderSync and not have an extra async operation to do. (Pass an array of Blobs to a worker and get back a Blob of a ZIP is a generally useful operation, so maybe he wouldn't even have to write this himself.) -- Glenn Maynard
Re: Reading image bytes to a PNG in a typed array
Perhaps we should think of a better scheme to export data than toFoo().
Maybe toData('url'), toData('arraybuffer') toData('blob') or perhaps
toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think that if
you're starting to write toA, toB, toC, toX methods on an object, you've
not thought this really trough what's a parameter, and what's a method.
On Wed, Jan 16, 2013 at 4:26 PM, Glenn Maynard gl...@zewt.org wrote:
On Wed, Jan 16, 2013 at 6:29 AM, Kyle Huey m...@kylehuey.com wrote:
I don't think there's any reason we can't have toArrayBuffer as well.
The hard part from an implementation perspective is asynchronously encoding
the image, not what we stick it in.
No technical reason, but I'd avoid adding yet more paths to do the same
thing unless there's a real gain. It'd be bad to head down the path of
every single Blob-returning operation having an ArrayBuffer duplicate.
In this particular case, he wants ArrayBuffers to put the data in a ZIP,
which probably means he should be doing this from a worker anyway, to avoid
doing a bunch of CRC calculations on the UI thread. In that case, the
current API isn't even inconvenient, since once you post the Blob to the
worker you can use FileReaderSync and not have an extra async operation to
do.
(Pass an array of Blobs to a worker and get back a Blob of a ZIP is a
generally useful operation, so maybe he wouldn't even have to write this
himself.)
--
Glenn Maynard
Re: Reading image bytes to a PNG in a typed array
On Wed, Jan 16, 2013 at 9:40 AM, Florian Bösch pya...@gmail.com wrote:
Perhaps we should think of a better scheme to export data than toFoo().
Maybe toData('url'), toData('arraybuffer') toData('blob') or perhaps
toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think that if
you're starting to write toA, toB, toC, toX methods on an object, you've
not thought this really trough what's a parameter, and what's a method.
We should be avoiding the need to return data in a bunch of different
interfaces in the first place. If the data is large, or takes a long or
nondeterministic amount of time to create (eg. something that would be
async in the UI thread), return a Blob; otherwise return an ArrayBuffer.
The user can convert from there as needed.
--
Glenn Maynard
Re: Reading image bytes to a PNG in a typed array
On Wed, Jan 16, 2013 at 7:50 AM, Glenn Maynard gl...@zewt.org wrote:
On Wed, Jan 16, 2013 at 9:40 AM, Florian Bösch pya...@gmail.com wrote:
Perhaps we should think of a better scheme to export data than toFoo().
Maybe toData('url'), toData('arraybuffer') toData('blob') or perhaps
toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think that if
you're starting to write toA, toB, toC, toX methods on an object, you've
not thought this really trough what's a parameter, and what's a method.
We should be avoiding the need to return data in a bunch of different
interfaces in the first place. If the data is large, or takes a long or
nondeterministic amount of time to create (eg. something that would be
async in the UI thread), return a Blob; otherwise return an ArrayBuffer.
The user can convert from there as needed.
Well, the problem is that we fundamentally screwed up when we specced
Blob. It has a synchronous size getter which negates many of the
advantages of FileReader extracing data asynchronously. For something like
image encoding (that involves compression), where you have to perform the
operation to know the size, Blob and ArrayBuffer are effectively
interchangeable from the implementation perspective, since both require you
to perform the operation up front.
- Kyle
Re: Reading image bytes to a PNG in a typed array
Whatever the eventual solution to this problem, it should be the user of
the API driving the decision how to get the data.
On Wed, Jan 16, 2013 at 4:56 PM, Kyle Huey m...@kylehuey.com wrote:
On Wed, Jan 16, 2013 at 7:50 AM, Glenn Maynard gl...@zewt.org wrote:
On Wed, Jan 16, 2013 at 9:40 AM, Florian Bösch pya...@gmail.com wrote:
Perhaps we should think of a better scheme to export data than toFoo().
Maybe toData('url'), toData('arraybuffer') toData('blob') or perhaps
toData(URL), toData(ArrayBuffer) or toData(Blob). I tend to think that if
you're starting to write toA, toB, toC, toX methods on an object, you've
not thought this really trough what's a parameter, and what's a method.
We should be avoiding the need to return data in a bunch of different
interfaces in the first place. If the data is large, or takes a long or
nondeterministic amount of time to create (eg. something that would be
async in the UI thread), return a Blob; otherwise return an ArrayBuffer.
The user can convert from there as needed.
Well, the problem is that we fundamentally screwed up when we specced
Blob. It has a synchronous size getter which negates many of the
advantages of FileReader extracing data asynchronously. For something like
image encoding (that involves compression), where you have to perform the
operation to know the size, Blob and ArrayBuffer are effectively
interchangeable from the implementation perspective, since both require you
to perform the operation up front.
- Kyle
Re: Reading image bytes to a PNG in a typed array
On Mon, Jan 14, 2013 at 1:20 PM, Florian Bösch pya...@gmail.com wrote: Having a texture in WebGL and wanting to encode it into a typed array as PNG, I have found that the only way to do it is the following convoluted method. Could you list the use cases? http://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F is relevant in particular. -- http://annevankesteren.nl/
Re: Reading image bytes to a PNG in a typed array
The usecase is that I write a productivity application using WebGL which has a need to bundle images residing in computed textures into one archive (tar, zip etc.). - The images belong together (such as normal, height, tangent, specular color, specular power, albedo etc.) - The archive has to land in a typed array so it can efficiently be put into a blob url, put into an XHR for upload to cloud storage, etc. - Which means the images encoded have to land in a typed array - Which means the described process is currently the best on offer by browsers. - Which should be rectified. On Mon, Jan 14, 2013 at 1:46 PM, Anne van Kesteren ann...@annevk.nl wrote: On Mon, Jan 14, 2013 at 1:20 PM, Florian Bösch pya...@gmail.com wrote: Having a texture in WebGL and wanting to encode it into a typed array as PNG, I have found that the only way to do it is the following convoluted method. Could you list the use cases? http://wiki.whatwg.org/wiki/FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F is relevant in particular. -- http://annevankesteren.nl/
Re: Reading image bytes to a PNG in a typed array
On Mon, Jan 14, 2013 at 6:20 AM, Florian Bösch pya...@gmail.com wrote:
1) Create canvas, set to desired size
2) Create 2D context
3) Create imageData object
4) Create a WebGL framebuffer object
5) Attach texture as color target to framebuffer
6) read back pixels into canvas2d's imageData.data member
7) ctx.putImageData into the canvas
You want to be rendering to a WebGL canvas, not a 2D canvas. There's the
limitation of not being able to draw to anything except the displayed
canvas, but I think that's being worked on within WebGL.
8) call canvases toDataURL('image/png')
9) snip off the mime/encoding header
10) implement base64 decode in JS and decode to Uint8Array
You want toBlob, not toDataURL.
--
Glenn Maynard
Re: Reading image bytes to a PNG in a typed array
On Mon, Jan 14, 2013 at 4:00 PM, Glenn Maynard gl...@zewt.org wrote: You want toBlob, not toDataURL. So how would I stick a blob into an arraybuffer?
Re: Reading image bytes to a PNG in a typed array
On Mon, Jan 14, 2013 at 9:04 AM, Florian Bösch pya...@gmail.com wrote: On Mon, Jan 14, 2013 at 4:00 PM, Glenn Maynard gl...@zewt.org wrote: You want toBlob, not toDataURL. So how would I stick a blob into an arraybuffer? http://dev.w3.org/2006/webapi/FileAPI/#dfn-filereader -- Glenn Maynard
