Thanks, Nathan, that's all correct and I couldn't have said it better myself. Though I will expand just a bit to spell out some details:
A great illustration of this can be found the implementation for ImageBufAlgo::pow(), here: https://github.com/OpenImageIO/oiio/blob/v2.5.2.0-dev/src/libOpenImageIO/imagebufalgo_pixelmath.cpp#L397 The first thing you'll see is pow_impl, which is the "inner" function that applies the power function on one input ImageBuf and writes the results to a second (for this function, the two images may be the same ImageBuf if you want to operate in place) and the power value. This function uses ImageBuf::Iterator (and ConstIterator) to walk the pixels of the ROI region specified, and also it is templated on the data types of the images (because the iterators themselves are templated on the underlying data type, automatically translating to and from float values for you as you access pixel data). The "outer" layer, which follows that, is ImageBufAlgo::pow() itself (actually, two versions: one that takes a reference to the output image, and a second that instead returns an output image). This function, notably, uses the OIIO_DISPATCH_COMMON_TYPES2 macro, which does all the heavy lifting in terms of (a) subdividing the full size of the image into subsections to farm out to the different threads, and (b) handles all the different pixel data types for you, by turning it underneath into specialized calls to the different template instantiations (by data type) of pow_impl. Look inside imagebufalgo_util.h to see the different DISPATCH macros you have to choose from, as well as other helpful functions such as IBAprep(). This is typical of how IBA functions are implemented, and the total for all the above is about 35 lines of code. The iterators do a lot of work for you: they walk the ROI you care about, they know how to handle "wrap modes" when they walk over the edge of the image, and they know how to translate to and from the buffer's data type (i.e., it[c] gives you the float value for channel c of the current pixel, even if the buffer is some other data type). They also completely handle the case of an ImageBuf that is not entirely in memory and is using an underlying ImageCache to automatically page in portions as they are needed. On Thu, Jun 8, 2023 at 12:07 PM Nathan Rusch <nathanru...@gmail.com> wrote: > Hi Vlad, > > The `ImageBuf` iterators are not inherently multithreaded. In other words, > simply using an iterator to apply some operation will not transparently > distribute the operation over multiple threads. > > Most of the `ImageBufAlgo` operations are implemented using the > `ImageBufAlgo::parallel_image` helper, which is an easy way to > automatically map a function/lambda to sub-regions of an `ImageBuf` (in > parallel, if running with multithreading enabled). You can see a lot of > examples of this in the `ImageBufAlgo` source code, where a general > function implementation is written to process an ROI (using `ImageBuf` > iterators), and then `parallel_image` is used to call that function/lambda > on an `ImageBuf`, which transparently dices it into smaller `ROI`s based on > the runtime thread count. There are other helpers in `parallel.h` that > might be interesting to you as well. > > I *believe* individual `ImageBuf` iterators are thread-safe as well, since > they are more or less shims over the `ImageBuf`'s internal implementation, > but I'm not 100% confident about this. In my experience, it's generally so > easy to implement something in terms of ROIs and then let OIIO parallelize > it for you that I've never had to experiment with that question. > > Hope this helps. > -Nathan > > On 6/8/2023 3:42 AM, Vladlen Erium wrote: > > > One more question about Iterators. Do they are multithreaded or not? > > Thanx Larry! > > I definitely will check ImageBuf::Iterator. And do my best. > Common or not, but if you'll check some of 3D format parsers, you can find > that they are by default normalize vertices normals. And even if most 3D > DCC apps do the same with normal maps inputs, and most normal maps > generator output normalized data, it still a good to have this function in > openimageio, well, just in case. :D > > Hi, Vlad. > > That's probably the simplest approach with the existing set of IBA > functions, yes. Maybe there is some optimizing you can do around the edges > -- like, `mul(img,img)` may be faster than `pow(img,2.0)`, I'm not sure, > and definitely you want to use the variety of IBA functions that takes a > destination image rather than returning an ImageBuf, in order to minimize > needless buffer copying. But those obvious tricks will only get you so far. > > If you're doing this a lot and it's performance critical, a better way > would be to write it as a single function that uses ImageBuf::Iterator to > traverse the image and do all the operations at once for each pixel, with > no extra buffer copies. Looking at the source code to any of the usual IBA > functions that take one input image and produce one output image will > provide you with a good example to copy and change the guts to make your > new function. > > I don't recall anybody asking for this particular thing before, but if you > think it is a commonly needed operation, then by all means propose a PR to > add this new function after you've implemented it. > > > > On Sun, Jun 4, 2023 at 6:28 PM Vladlen Erium <v...@hdri.xyz> wrote: > >> What should be most efficient way to implement normalizing vector data >> images (normals) using ImageBuffAlgo? >> >> For this moment I only see the way to do this in four steps. >> ImageBuffAlgo::madd for [0.0,1.0] -> [-1.0,1.0] >> ImageBuffAlgo::pow for power of 2 >> ImageBuffAlgo::sum_channels for vector magnitude >> ImageBuffAlgo::div (sec and magnitude) to normalize vector length >> ImageBuffAlgo::madd for normalize to [0.0, 1.0] range. >> >> This not only required to make so many steps but also required a lot of >> temporary buffers, that for huge textures can required lot of memory. When >> all steps can be done per pixel and perfectly parallelized (shaders, cuda). >> >> Maybe I missed some OIIO functions? 🤔 >> >> Best regards: >> Vlad >> >> PS: btw, looks like Google completely filter out all Larry messages if >> Gmail used for subscription to this mailing list. They even not in spam >> folder, where mail list messages quite often can be moved 😒 >> _______________________________________________ >> Oiio-dev mailing list >> Oiio-dev@lists.openimageio.org >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> > > > -- > Larry Gritz > l...@imageworks.com > _______________________________________________ > Oiio-dev mailing list > Oiio-dev@lists.openimageio.org > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > _______________________________________________ > Oiio-dev mailing list > Oiio-dev@lists.openimageio.org > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > > > _______________________________________________ > Oiio-dev mailing > listOiio-dev@lists.openimageio.orghttp://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > > > _______________________________________________ > Oiio-dev mailing list > Oiio-dev@lists.openimageio.org > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > -- Larry Gritz l...@imageworks.com
_______________________________________________ Oiio-dev mailing list Oiio-dev@lists.openimageio.org http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
