Hi Jason, Thanks for the further background details. I can't think of another way of solving this one elegantly - ideally we'd have a OS level method doing this iterative mkdir and not need to worry about multiple processes/threads all trying to do the same thing at the same time, but without this "proper" fix I think the method you have gone for is good enough for me.
Fix now merged and submitted to svn/trunk and OSG-3.2 branch. Robert. On 28 April 2014 14:48, Jason Beverage <[email protected]> wrote: > Hi Robert, > > Initially in osgEarth we did protect against this with a global mutex but as > you said, that's only good for working within a single process against > multiple threads. When we started dealing with multiple processes the issue > appeared again and I had to dig in and figure out what the real issue was. > It's simply that the current code treats calling mkdir on an existing > directory as an error when in reality the function has achieved it's goal of > making sure the directory exists so it shouldn't bail out early. > > The fix was inspired by a workaround for a similar race condition in the > TileCache code: > http://svn.osgeo.org/tilecache/trunk/tilecache/TileCache/Caches/Disk.py > > Jason > > > On Mon, Apr 28, 2014 at 8:05 AM, Robert Osfield <[email protected]> > wrote: >> >> Hi Jason, >> >> I have just review the change but it doesn't feel like an elegant >> solution. I haven't workout a better solution yet though... >> >> A mutex to prevent makeDirectory being called by multiple threads at >> once might be one way, but this wouldn't protect against a totally >> separate process/application s for creating the directory, >> >> Do you consider/discount other solutions? >> >> Robert. >> >> >> >> On 25 April 2014 22:31, Jason Beverage <[email protected]> wrote: >> > Hi Robert, >> > >> > Here is a fix for a small race condition in osgDB::makeDirectory. It >> > attempts to create all the directories in the given path and stops >> > attempting to make directories when one of them fails. I've added a >> > check >> > to see if the failure occurred b/c the directory was created by another >> > thread or process. >> > >> > We were running into issues occasionally in osgEarth where multiple >> > threads >> > were writing out files like /1/2/3.jpg and /1/3/4.jpg. Both threads >> > would >> > try to create the /1 directory and only one of them would succeed. So >> > the >> > first thread would write out the full /1/2/3.jpg while the second thread >> > wouldn't create the /1/3 directory b/c /1 was already created and the >> > writing of /1/3/4.jpg would fail. >> > >> > Thanks! >> > >> > Jason >> > >> > _______________________________________________ >> > osg-submissions mailing list >> > [email protected] >> > >> > http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org >> > >> _______________________________________________ >> osg-submissions mailing list >> [email protected] >> >> http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org > > > > _______________________________________________ > osg-submissions mailing list > [email protected] > http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org > _______________________________________________ osg-submissions mailing list [email protected] http://lists.openscenegraph.org/listinfo.cgi/osg-submissions-openscenegraph.org
