On Fri, Oct 24, 2014 at 03:35:06PM +0100, Robert Osfield wrote: > Hi Pjotr, > > On 24 October 2014 14:54, Pjotr Svetachov <pjotrsvetac...@gmail.com> wrote: > > > We too have some trouble with the database pager. It all started after we > > used DatabasePager::setUpThreads to use more threads. We get memory > > corruption but only on linux. Tried to use valgrind but then the problem > > does not occur. > > > > Curiously I have found the a similar behaviour, I get crashes but when I > run in valgrind I don't get any and the only issues picked up are ones that > occur during cleanup and look to benign.
Running valgrind with memcheck is deficient when it comes to debugging multithreaded race conditions this is because while it supports running multithreaded programs and valgrind runs each application thread in a real thread, the valgrind data structures can't support multiple threads and so only one thread is allowed to run at a time. So any program that runs under valgrind has little concurrency as it turns into one thread running for a while followed by another, so these kinds of race conditions don't show up. valgrind --tool=helgrind The helgrind tool is designed to find potential problems accessing data without a lock. It sill will only run one thread at a time, but it is designed to look for these kinds of problems. This was very useful for me to track down what I posted. >From my earlier post adding a sleep, in StateToCompile::apply causes the one thread to wait and allows the other thread to get in and mess things up, then it will crash very frequently and valgrind memcheck will see the access after delete. OpenThreads::Thread::microSleep(100000); > The crashes I get look likely to be memory corruption. This happens from > my new shader based terrain technique, I suspect even though the code is > different there are similarities in that they create subgraphs that contain > geometry and state that need compiling, culling and rendering. > > > > > We are still in the process of pinpointing it down, some theories I have > > and it might help you so here are my two cents: > > - we have some osgb files that reference the same images. We also have > > some files that use proxy nodes to reference other files because we have > > some objects that can be build out of subobjects. > > > > There are some specific .osgb threading issues related to initialization of > the wrappers. There is chance that you are hitting up against this. For > my own work I'm not currently testing against .osgb and get problems so I > believe the .osgb issues can be dealt with separately. > > > > > - What could happen is that the database pager tries to load this > > referenced image or object and find it in the registry cache. Then it could > > occur that more database pager threads run the incremental compilation over > > the same objects. > > > > The IncrementalCompileOperator only compiles objects from the > GraphicsThreads assoicated with teh GraphicsContext and this ensures that > for a single context all the GL objects are compiled in series, and > duplicates if they exist will be checked to see if they compiling and > they've already compiled will be skipped. Given this I don't think there > is any danger of multiple entries. > > Today I added a mutex around the access to the subgraph managed by the > custom TerrainTechnique I have, this looks to be fixed the crashes, but as > I yet I can't spot the time that the subgraph is contended. The > GeometryTechnique has a similar local subgraph so may also be hitting up to > the same issue. > > The odd thing is that the design of the DatabasePager should be avoided > this contention issue, something I haven't yet got to the bottom of. > > Robert. > _______________________________________________ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org -- David Fries <da...@fries.net> PGP pub CB1EE8F0 http://fries.net/~david/ _______________________________________________ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
