The question is what happens when an image that has different "data" and "display" windows is output to an image file type that does not support that concept. Does it write the pixels (data) as the whole image and lose the metadata about the display window, or does it consider the display window sacrosanct and crop and/or pad the pixel data to conform? And at what level of abstraction should this logic live?
OK, here is what I hope will be the final word on the matter:
I think that high-level classes and apps that deal with the level of abstraction of entire images (e.g., ImageBuf, oiiotool) ought to provide simple API commands and facilities that will crop/pad the data to match the display window. I can accept that this should perhaps be the default behavior when outputting to a file format that doesn't support the distinction.
But I think that low-level ImageOutput (individual format writers) should write precisely the pixels they get from the app; no more, no less. I have two reasons for this:
1. Philosophical: This level of abstraction is meant to hide the details of individual format library APIs (TIFFfoo() versus JPEGbar() versus ImfBlah()), but is not supposed to abstract entire images or their semantics. In other words, an app hands a pile of pixels to an ImageOutput, and expects those pixels to be written, without being overly clever.
2. Practical: Consider briefly the logistics of handling this automatically in an ImageOutput. Let's say the display window is (0,0)-(1023,1023), the data window is (3,3)-(34,34), and the image is tiled with 16x16 tiles. Now, from the app's point of view, it needs to call write_tile() four times, once for each tile, which are by definition always aligned to the data window. Upon the first write_tile() call, what does the ImageOutput do? Well, if it is to pad automatically, then the output image's tiles start at 0, not at 3, so it needs to "re-tile" on the fly -- what it really needs to do is split that "app tile" appropriately to the 4 "output tiles", 3 of which can't be output then, since they overlap with other app tiles not yet received, so the ImageOutput would have to buffer those pending tiles, ugh. Basically, it's a nightmare, would make hard-to-follow (and probably buggy) code, and have a lot of the tricky bits replicated across many different ImageOutput subclass
es. No way.
So, to recap, the only practicable solution I see is:
a) ImageOutputs write pixels they are sent, as they currently do (no change needed).
b) But the ImageOutput::supports() method can be extended to allow the app to query if the image format supports separate display/data windows. Proposed syntax: supports("displaywindow")
c) Add a simple, single utility function to handle the auto crop/pad:
ImageBufAlgo::FixToDisplayWindow (ImageBuf &img, const std::string &format)
If the named format does not support display windows (or unconditionally, if format == ""),
pad and/or crop the image so that its pixel data window exactly fills the display window.
Padding is black, unless img's ImageSpec contains metadata "oiio:bordercolor"
(of type float[nchans]) specifying a different border color to use for pixels beyond the
data window.
d) Change the default behavior of oiiotool and other image-writing utilities to call this when writing images.
Comments?
-- lg
On Oct 17, 2011, at 9:50 PM, Will Rosecrans wrote:
On Mon, Oct 17, 2011 at 8:48 PM, Larry Gritz <[email protected]> wrote:
The higher-level tools (iconvert, oiiotool) are another story entirely, and
I'm willing to bow to common sense, consensus, or precedent of other apps.
The two cases we care about are when the input image (a) has a data window
smaller than the display window (a crop window) or (b) has a data window
larger than the display window (overscan regions), but the output image
format does not support these notions at all (e.g. JPEG). What should
happen? What do other programs do?
Hugh implies that Shake would pad with some black (or some kind of settable
background color?), or crop to the display window. Do you agree?
Anybody familiar enough with Nuke to know what it does in these
circumstances?
Jeremy, do you have an opinion on any of this?
-- lg
If rendering something in nuke where the internal "data window" is
smaller than the "display window" to a format the doesn't have such
concepts, it will render a result that is the size of the "display
window." which is basically the data surrounded by black. It's
actually a bit more complicated than that because nuke wants to repeat
edge pixels, but a crop node defaults to adding a one pixel black
border around the cropped area (just outside the display window, so
the resulting so that you edge-repeat the black border in the
undefined space.) If rendering to EXR, nuke will correctly preserve
the windows. I don't think nuke has a ngcolor in the sense that Shake
did.
In nuke-speak, you use a crop node to set the data window, and a
reformat node to set the display window. You can also reformat in a
crop node to do both at once. But, if you have the black border
enabled when you reformat while you crop, the resulting data and
display windows are not identical. The data window extends one pixel
past the display window in each direction for the invisible black
border. (This works fairly well in practice, despite being oddly
complicated to explain nuke's behavior, you seldom have to think about
it very explicitly.)
In Shakespeak, the Crop node set the display window, and a setDOD node
set the data window, basically. It was simpler in Shake.
When rendering an EXR out of nuke where the display window is smaller
than the data window, Compressor on Mac OS X will try to interpret the
EXR in terms of the data window. The rule seems to be based on
'whichever is bigger.' The result is a QuickTime movie made from an
EXR sequence that is physically larger than expected, and has
unexpected stuff visible in it, and a very confused and angry editor.
I consider the Compressor behavior to be incorrect, and I have never
desired it.
Basically, if I am making images where the data window matters, I'll
keep strictly within the realm of tools and formats that maintain the
distinction. If I need to use a tool that doesn't understand the data
window, then I'll change my settings in the originating tool to make
my images unambiguous. Efforts by intermediate tools to care about
the data window in formats that don't understand it have never turned
out useful to me. They should just use the display window, IMHO.
_______________________________________________
Oiio-dev mailing list
[email protected]
http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
--
Larry Gritz
[email protected]_______________________________________________
Oiio-dev mailing list
[email protected]http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
