For anybody who was following this on the mail list but not through the GH issues and PRs...
This turned out to be a bug in the JPEG reader, not really related to or specific to IOProxy. Here is the patch that provided a fix: https://github.com/OpenImageIO/oiio/pull/2549 <https://github.com/OpenImageIO/oiio/pull/2549> > On Apr 7, 2020, at 7:25 AM, Etienne Sandré-Chardonnal > <etienne.san...@m4x.org> wrote: > > Hi again, > > After looking at OIIO source code, I don't think libJpeg reentrancy is in > cause, but simply the way we use IOProxies. As an IOProxy has an internal > state (such as current offset position in the memory buffer), there is no way > we can use it in multiple threads. This would be the same by passing an > IOFile, it would share the FILE* handle between threads, and things would > probably explode. > > The only case it works with concurrent threads, is when no IOProxy is passed, > so JpegInput creates an IOFile itself, which will only be used in the current > thread. > > Instead of attaching an IOProxy to a texture, we should attach an > IOProxyFactory which creates IOProxies on demand. This requires some redesign > in the API, though. It seems the current API is simply not able to handle > IOProxies added to an ImageCache, when there are multiple lookup threads. > > Cheers, > > Etienne > > > > > Le mar. 7 avr. 2020 à 15:50, Etienne Sandré-Chardonnal > <etienne.san...@m4x.org <mailto:etienne.san...@m4x.org>> a écrit : > Hi Larry, > > I have a simple code that reproduces the bug, posted on GitHub. > https://github.com/OpenImageIO/oiio/issues/2544 > <https://github.com/OpenImageIO/oiio/issues/2544> > > The errors message returned by textureSystem->geterror() are less informative > than what I get with OSL when an error handler is passed to RendererServices, > is there a way to pass an error handler to OIIO? > > Cheers > > Le mar. 7 avr. 2020 à 10:49, Etienne Sandré-Chardonnal > <etienne.san...@m4x.org <mailto:etienne.san...@m4x.org>> a écrit : > Hi, in order to save you some time, I will try to reproduce with a modified > testshade. > As far as I know, libjpeg is thread safe if you don’t share > jpeg_decompress_struct handles between threads. Do you share JpegInput > objects between threads? > I have written support for generic IOProxy reading in JpegInput, how can I > share it? Is a diff OK? > > Regards, > > Étienne > > > > Le mar. 7 avr. 2020 à 00:56, Larry Gritz <l...@larrygritz.com > <mailto:l...@larrygritz.com>> a écrit : > Not a known issue. I'll have to look into it. Sounds like maybe there is a > missing lock somewhere to protect against libjpeg being not thread safe. > > > >> On Apr 6, 2020, at 3:40 PM, Etienne Sandré-Chardonnal >> <etienne.san...@m4x.org <mailto:etienne.san...@m4x.org>> wrote: >> > > >> Hi, >> >> I am facing a strange bug while trying to make threaded concurrent accesses >> to a JPEG texture read from a memory buffer. >> >> The code for adding the texture to the image cache is: >> oiioProxy.reset(new OIIO::Filesystem::IOMemReader(data.data(), data.size())); >> OIIO::Filesystem::IOProxy * oiioProxyPtr = oiioProxy.get(); >> OIIO::ImageSpec config; >> config.attribute("oiio:ioproxy", OIIO::TypeDesc::PTR, &oiioProxyPtr); >> prepareData.textureSystem()->imagecache()->add_file(OIIO::ustring(_oslPath), >> nullptr, &config, true); >> >> This code works well with a PNG image. With a JPEG, I have multiple error >> messages, and the texture is corrupted: >> >> Error:[RendererServices::texture] JPEG error: Improper call to JPEG library >> in state 0 ("A6283DE8-F912-B0C8-04AC-D34B8265F2A9.jpg") >> JPEG error: Improper call to JPEG library in state 0 >> ("A6283DE8-F912-B0C8-04AC-D34B8265F2A9.jpg") >> JPEG error: Improper call to JPEG library in state 0 >> ("A6283DE8-F912-B0C8-04AC-D34B8265F2A9.jpg") >> Error:[RendererServices::texture] Could not open file >> "A6283DE8-F912-B0C8-04AC-D34B8265F2A9.jpg" >> Could not open file "A6283DE8-F912-B0C8-04AC-D34B8265F2A9.jpg" >> >> If I try rendering with a single thread, it works. The problem randomly >> appears with increasing number of threads, and is almost always there with >> max threads. >> Removing the IOProxy and passing the same texture as a jpeg file on disk >> works. >> >> I am still using OIIO 2.1.10. Is this a known issue? I have struggled all >> day trying to debug a hacked OIIO build, until finding the issue was still >> there with an original build... >> >> Cheers, >> >> Etienne > >> _______________________________________________ >> Oiio-dev mailing list >> Oiio-dev@lists.openimageio.org <mailto:Oiio-dev@lists.openimageio.org> >> http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org >> <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > > -- > Larry Gritz > l...@larrygritz.com <mailto:l...@larrygritz.com> > > > > > _______________________________________________ > Oiio-dev mailing list > Oiio-dev@lists.openimageio.org <mailto:Oiio-dev@lists.openimageio.org> > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org > <http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org> > _______________________________________________ > Oiio-dev mailing list > Oiio-dev@lists.openimageio.org > http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org -- Larry Gritz l...@larrygritz.com
_______________________________________________ Oiio-dev mailing list Oiio-dev@lists.openimageio.org http://lists.openimageio.org/listinfo.cgi/oiio-dev-openimageio.org
