I'm sorry, I did misstate a couple details in my last email: 1. The DISPATCH macros *only* handle the general preparation and the instantiation of different "impl" templates based on the data types involved. Those macros do not handle the parallelization.
2. The pow_impl itself handles the parallelization, as you can see from the fact that it calls ImageBufAlgo::parallel_image. Sorry for any confusion. On Thu, Jun 8, 2023 at 12:59 PM Larry Gritz <l...@imageworks.com> wrote: > Thanks, Nathan, that's all correct and I couldn't have said it better > myself. Though I will expand just a bit to spell out some details: > > A great illustration of this can be found the implementation for > ImageBufAlgo::pow(), here: > > https://github.com/OpenImageIO/oiio/blob/v2.5.2.0-dev/src/libOpenImageIO/imagebufalgo_pixelmath.cpp#L397 > > The first thing you'll see is pow_impl, which is the "inner" function that > applies the power function on one input ImageBuf and writes the results to > a second (for this function, the two images may be the same ImageBuf if you > want to operate in place) and the power value. This function uses > ImageBuf::Iterator (and ConstIterator) to walk the pixels of the ROI region > specified, and also it is templated on the data types of the images > (because the iterators themselves are templated on the underlying data > type, automatically translating to and from float values for you as you > access pixel data). > > The "outer" layer, which follows that, is ImageBufAlgo::pow() itself > (actually, two versions: one that takes a reference to the output image, > and a second that instead returns an output image). This function, notably, > uses the OIIO_DISPATCH_COMMON_TYPES2 macro, which does all the heavy > lifting in terms of (a) subdividing the full size of the image into > subsections to farm out to the different threads, and (b) handles all the > different pixel data types for you, by turning it underneath into > specialized calls to the different template instantiations (by data type) > of pow_impl. Look inside imagebufalgo_util.h to see the different DISPATCH > macros you have to choose from, as well as other helpful functions such > as IBAprep(). > > This is typical of how IBA functions are implemented, and the total for > all the above is about 35 lines of code. > > The iterators do a lot of work for you: they walk the ROI you care about, > they know how to handle "wrap modes" when they walk over the edge of the > image, and they know how to translate to and from the buffer's data type > (i.e., it[c] gives you the float value for channel c of the current pixel, > even if the buffer is some other data type). They also completely handle > the case of an ImageBuf that is not entirely in memory and is using an > underlying ImageCache to automatically page in portions as they are needed. > > > > > On Thu, Jun 8, 2023 at 12:07 PM Nathan Rusch <nathanru...@gmail.com> > wrote: > >> Hi Vlad, >> >> The `ImageBuf` iterators are not inherently multithreaded. In other >> words, simply using an iterator to apply some operation will not >> transparently distribute the operation over multiple threads. >> >> Most of the `ImageBufAlgo` operations are implemented using the >> `ImageBufAlgo::parallel_image` helper, which is an easy way to >> automatically map a function/lambda to sub-regions of an `ImageBuf` (in >> parallel, if running with multithreading enabled). You can see a lot of >> examples of this in the `ImageBufAlgo` source code, where a general >> function implementation is written to process an ROI (using `ImageBuf` >> iterators), and then `parallel_image` is used to call that function/lambda >> on an `ImageBuf`, which transparently dices it into smaller `ROI`s based on >> the runtime thread count. There are other helpers in `parallel.h` that >> might be interesting to you as well. >> >> I *believe* individual `ImageBuf` iterators are thread-safe as well, >> since they are more or less shims over the `ImageBuf`'s internal >> implementation, but I'm not 100% confident about this. In my experience, >> it's generally so easy to implement something in terms of ROIs and then let >> OIIO parallelize it for you that I've never had to experiment with that >> question. >> >> Hope this helps. >> -Nathan >> >> On 6/8/2023 3:42 AM, Vladlen Erium wrote: >> >> >> One more question about Iterators. Do they are multithreaded or not? >> >> Thanx Larry! >> >> I definitely will check ImageBuf::Iterator. And do my best. >> Common or not, but if you'll check some of 3D format parsers, you can >> find that they are by default normalize vertices normals. And even if most >> 3D DCC apps do the same with normal maps inputs, and most normal maps >> generator output normalized data, it still a good to have this function in >> openimageio, well, just in case. :D >> >> Hi, Vlad. >> >> That's probably the simplest approach with the existing set of IBA >> functions, yes. Maybe there is some optimizing you can do around the edges >> -- like, `mul(img,img)` may be faster than `pow(img,2.0)`, I'm not sure, >> and definitely you want to use the variety of IBA functions that takes a >> destination image rather than returning an ImageBuf, in order to minimize >> needless buffer copying. But those obvious tricks will only get you so far. >> >> If you're doing this a lot and it's performance critical, a better way >> would be to write it as a single function that uses ImageBuf::Iterator to >> traverse the image and do all the operations at once for each pixel, with >> no extra buffer copies. Looking at the source code to any of the usual IBA >> functions that take one input image and produce one output image will >> provide you with a good example to copy and change the guts to make your >> new function. >> >> I don't recall anybody asking for this particular thing before, but if >> you think it is a commonly needed operation, then by all means propose a PR >> to add this new function after you've implemented it. >> >> >> >> On Sun, Jun 4, 2023 at 6:28 PM Vladlen Erium <v...@hdri.xyz> wrote: >> >>> What should be most efficient way to implement normalizing vector data >>> images (normals) using ImageBuffAlgo? >>> >>> For this moment I only see the way to do this in four steps. >>> ImageBuffAlgo::madd for [0.0,1.0] -> [-1.0,1.0] >>> ImageBuffAlgo::pow for power of 2 >>> ImageBuffAlgo::sum_channels for vector magnitude >>> ImageBuffAlgo::div (sec and magnitude) to normalize vector length >>> ImageBuffAlgo::madd for normalize to [0.0, 1.0] range. >>> >>> This not only required to make so many steps but also required a lot of >>> temporary buffers, that for huge textures can required lot of memory. When >>> all steps can be done per pixel and perfectly parallelized (shaders, cuda). >>> >>> Maybe I missed some OIIO functions? 🤔 >>> >>> Best regards: >>> Vlad >>> >>> PS: btw, looks like Google completely filter out all Larry messages if >>> Gmail used for subscription to this mailing list. They even not in spam >>> folder, where mail list messages quite often can be moved 😒 >>> _______________________________________________ >>> Oiio-dev mailing list >>> Oiio-dev@lists.openimageio.org >>> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >>> >> >> >> -- >> Larry Gritz >> l...@imageworks.com >> _______________________________________________ >> Oiio-dev mailing list >> Oiio-dev@lists.openimageio.org >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> >> _______________________________________________ >> Oiio-dev mailing list >> Oiio-dev@lists.openimageio.org >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> >> >> >> _______________________________________________ >> Oiio-dev mailing >> listOiio-dev@lists.openimageio.orghttp://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> >> >> _______________________________________________ >> Oiio-dev mailing list >> Oiio-dev@lists.openimageio.org >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> > > > -- > Larry Gritz > l...@imageworks.com > -- Larry Gritz l...@imageworks.com
_______________________________________________ Oiio-dev mailing list Oiio-dev@lists.openimageio.org http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
