On Thu, Jun 25, 2015 at 9:40 AM, Justin Novosad <ju...@google.com> wrote:
> > > On Thu, Jun 25, 2015 at 5:56 AM, Ashley Gullen <ash...@scirra.com> wrote: > >> I see that OffscreenCanvas also specifies a close() method for >> ImageBitmap. Perhaps browsers could use copy-on-write with the >> toImageData() method? Then you can efficiently either transfer or copy >> depending on if you close the ImageBitmap you transferred to an ImageData. >> > >> Transfer: >> createImageBitmap() -> imageBitmap.toImageData() -> imageBitmap.close() >> No copy made, contents transferred to the imageData >> >> createImageBitmap() -> imageBitmap.toImageData() -> access resulting >> imageData.data >> Copy made on first access >> > > Copy on write is probably not a good idea because it would require > injecting an observer pattern (or some kind of notification mechanism) into > Uint8ClampedArray data assignment, which would realistically degrade > performance of Image processing pixel loops. Having a one step transfer > API (transferToImageData) avoids adding overhead to what is otherwise just > a simple memory access. > Something more: Blink, and probably other implementations as well, stores decoded images in alpha pre-multiplied form. ImageData is not pre-multiplied. For this reason sharing a buffer between ImageBitmap and ImageData would be somewhat inconvenient. >> >> On 24 June 2015 at 21:54, Kenneth Russell <k...@google.com> wrote: >> >>> On Wed, Jun 24, 2015 at 1:28 PM, Ashley Gullen <ash...@scirra.com> >>> wrote: >>> > Sorry for the confusion. Yes, the latest URL is: >>> > >>> https://www.scirra.com/labs/specs/imagebitmap-conversion-extensions.html >>> > I'm new to specs and WebIDL, my intent was to say those are new >>> methods on >>> > ImageBitmap. Is "partial interface ImageBitmap" the correct way to say >>> that? >>> > (I updated the URL to say that instead) >>> > >>> > The wording of "undue latency" in the ImageBitmap spec does not appear >>> to >>> > rule out ImageBitmap storing its encoded image data, and lazily >>> decoding it. >>> > However I'm not sure what the point of that would be, since it seems >>> to be >>> > how drawing HTMLImageElements to a canvas already works. I see a >>> couple of >>> > options: >>> > - allow lazy decoding in ImageBitmap, and therefore converting to >>> ImageData >>> > is an explicit request to decompress >>> > - require ImageBitmap to decompress its contents, and make it available >>> > through a read-only Uint8ClampedArray like ImageData has. Therefore, >>> > converting to ImageData indicates the intent to modify this content, >>> > therefore a copy is warranted. >>> > - provide a way to "neuter" the ImageBitmap when converting it to >>> ImageData, >>> > transferring its contents and avoiding any copy, which is useful if the >>> > ImageBitmap is a temporary object only being created for the purposes >>> of >>> > getting the ImageData. I guess this would be similar to transferrable >>> > objects with workers. >>> >>> Perhaps this could be done by specifying a "transferToImageData" method. >>> >>> This proposal makes ImageBitmap neuterable, and there's intent to >>> implement it in order to allow rendering to canvas contexts from >>> worker threads: https://wiki.whatwg.org/wiki/OffscreenCanvas . >>> >>> -Ken >>> >>> >>> > - add some kind of "load" method on ImageBitmap. It may store its >>> contents >>> > in compressed form, but calling "load" causes it to be decompressed (or >>> > loaded from somewhere else). The ImageBitmap is only guaranteed to be >>> drawn >>> > with "undue latency" after the "load" call. Therefore for the use case >>> of >>> > low-latency rendering "load" can always be called immediately after >>> > creation, but for conversion purposes not calling "load" avoids >>> duplicating >>> > memory. >>> > >>> > Note for devices with discrete GPUs, it's possible that ImageBitmap >>> stores >>> > the decompressed image data in a GPU texture but does not have it in >>> system >>> > RAM. In this case making a copy for the ImageData is also warranted. >>> > >>> > I'm a web developer proposing this because it would be useful for my >>> > purposes, I've no idea which of these would be favoured by >>> implementors or >>> > the spec process. I'm happy to get involved in spec work though so >>> please >>> > let me know if you have any feedback/suggestions on anything I'm doing >>> here. >>> > >>> > Ashley >>> > >>> > >>> > >>> > On 24 June 2015 at 19:12, Boris Zbarsky <bzbar...@mit.edu> wrote: >>> >> >>> >> On 6/19/15 5:43 AM, Ashley Gullen wrote: >>> >>> >>> >>> I've not done this before, so I've no idea if this is the >>> right/useful >>> >>> approach, but I drafted a spec for it here: >>> >>> >>> >>> https://www.scirra.com/labs/specs/imagedata-blob-extensions.html >>> >> >>> >> >>> >> Ashley, >>> >> >>> >> We at Mozilla were just discussing this proposal, and we have a >>> concern. >>> >> >>> >> With this proposal, going from an <img> to an ImageData requires >>> >> conversion of an intermediate ImageBitmap. Unfortunately, an >>> ImageBitmap is >>> >> specified to be paintable "without undue latency", which we interpret >>> to >>> >> mean it needs to have decoded image data, and the ImageData will end >>> up >>> >> needing to copy said data in practice (because it needs an editable >>> copy). >>> >> >>> >> That creates two copies of the decoded image data in memory, which >>> seems >>> >> fairly undesirable on memory-constrained devices. >>> >> >>> >> -Boris >>> >> >>> >> >>> > >>> >> >> >
