The question is what happens when an image that has different "data" and
"display" windows is output to an image file type that does not support that
concept. Does it write the pixels (data) as the whole image and lose the
metadata about the display window, or does it consider the display window
sacrosanct and crop and/or pad the pixel data to conform? And at what level of
abstraction should this logic live?
OK, here is what I hope will be the final word on the matter:
I think that high-level classes and apps that deal with the level of
abstraction of entire images (e.g., ImageBuf, oiiotool) ought to provide simple
API commands and facilities that will crop/pad the data to match the display
window. I can accept that this should perhaps be the default behavior when
outputting to a file format that doesn't support the distinction.
But I think that low-level ImageOutput (individual format writers) should write
precisely the pixels they get from the app; no more, no less. I have two
reasons for this:
1. Philosophical: This level of abstraction is meant to hide the details of
individual format library APIs (TIFFfoo() versus JPEGbar() versus ImfBlah()),
but is not supposed to abstract entire images or their semantics. In other
words, an app hands a pile of pixels to an ImageOutput, and expects those
pixels to be written, without being overly clever.
2. Practical: Consider briefly the logistics of handling this automatically in
an ImageOutput. Let's say the display window is (0,0)-(1023,1023), the data
window is (3,3)-(34,34), and the image is tiled with 16x16 tiles. Now, from the
app's point of view, it needs to call write_tile() four times, once for each
tile, which are by definition always aligned to the data window. Upon the
first write_tile() call, what does the ImageOutput do? Well, if it is to pad
automatically, then the output image's tiles start at 0, not at 3, so it needs
to "re-tile" on the fly -- what it really needs to do is split that "app tile"
appropriately to the 4 "output tiles", 3 of which can't be output then, since
they overlap with other app tiles not yet received, so the ImageOutput would
have to buffer those pending tiles, ugh. Basically, it's a nightmare, would
make hard-to-follow (and probably buggy) code, and have a lot of the tricky
bits replicated across many different ImageOutput subclass
es. No way.
So, to recap, the only practicable solution I see is:
a) ImageOutputs write pixels they are sent, as they currently do (no change
needed).
b) But the ImageOutput::supports() method can be extended to allow the app to
query if the image format supports separate display/data windows. Proposed
syntax: supports("displaywindow")
c) Add a simple, single utility function to handle the auto crop/pad:
ImageBufAlgo::FixToDisplayWindow (ImageBuf &img, const std::string
&format)
If the named format does not support display windows (or unconditionally, if
format == ""),
pad and/or crop the image so that its pixel data window exactly fills the
display window.
Padding is black, unless img's ImageSpec contains metadata
"oiio:bordercolor"
(of type float[nchans]) specifying a different border color to use for
pixels beyond the
data window.
d) Change the default behavior of oiiotool and other image-writing utilities to
call this when writing images.
Comments?
-- lg
On Oct 17, 2011, at 9:50 PM, Will Rosecrans wrote:
> On Mon, Oct 17, 2011 at 8:48 PM, Larry Gritz <[email protected]> wrote:
>> The higher-level tools (iconvert, oiiotool) are another story entirely, and
>> I'm willing to bow to common sense, consensus, or precedent of other apps.
>> The two cases we care about are when the input image (a) has a data window
>> smaller than the display window (a crop window) or (b) has a data window
>> larger than the display window (overscan regions), but the output image
>> format does not support these notions at all (e.g. JPEG). What should
>> happen? What do other programs do?
>> Hugh implies that Shake would pad with some black (or some kind of settable
>> background color?), or crop to the display window. Do you agree?
>> Anybody familiar enough with Nuke to know what it does in these
>> circumstances?
>> Jeremy, do you have an opinion on any of this?
>> -- lg
>>
>
> If rendering something in nuke where the internal "data window" is
> smaller than the "display window" to a format the doesn't have such
> concepts, it will render a result that is the size of the "display
> window." which is basically the data surrounded by black. It's
> actually a bit more complicated than that because nuke wants to repeat
> edge pixels, but a crop node defaults to adding a one pixel black
> border around the cropped area (just outside the display window, so
> the resulting so that you edge-repeat the black border in the
> undefined space.) If rendering to EXR, nuke will correctly preserve
> the windows. I don't think nuke has a ngcolor in the sense that Shake
> did.
>
> In nuke-speak, you use a crop node to set the data window, and a
> reformat node to set the display window. You can also reformat in a
> crop node to do both at once. But, if you have the black border
> enabled when you reformat while you crop, the resulting data and
> display windows are not identical. The data window extends one pixel
> past the display window in each direction for the invisible black
> border. (This works fairly well in practice, despite being oddly
> complicated to explain nuke's behavior, you seldom have to think about
> it very explicitly.)
>
> In Shakespeak, the Crop node set the display window, and a setDOD node
> set the data window, basically. It was simpler in Shake.
>
> When rendering an EXR out of nuke where the display window is smaller
> than the data window, Compressor on Mac OS X will try to interpret the
> EXR in terms of the data window. The rule seems to be based on
> 'whichever is bigger.' The result is a QuickTime movie made from an
> EXR sequence that is physically larger than expected, and has
> unexpected stuff visible in it, and a very confused and angry editor.
> I consider the Compressor behavior to be incorrect, and I have never
> desired it.
>
> Basically, if I am making images where the data window matters, I'll
> keep strictly within the realm of tools and formats that maintain the
> distinction. If I need to use a tool that doesn't understand the data
> window, then I'll change my settings in the originating tool to make
> my images unambiguous. Efforts by intermediate tools to care about
> the data window in formats that don't understand it have never turned
> out useful to me. They should just use the display window, IMHO.
> _______________________________________________
> Oiio-dev mailing list
> [email protected]
> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
