Re: [Flightgear-devel] Segfault with OSG multi threading
Good morning, On Monday, May 30, 2011 19:54:31 ThorstenB wrote: On 30.05.2011 19:25, Stefan Seifert wrote: The segfault happens when the main loop thread tries to access GL information. I know next to nothing about openGL programming but I seem to recall that it's not allowed to access the same GL context in different threads. So how is this supposed to work? Well, wouldn't surprise me to find more related multi-threading issues. Not sure how long OSG itself supports it, but I guess it wasn't a major priority at the time when FG was adapted to OSG. And I'm sure more fixes will be necessary to safely support it. Well, there are several threading issues left in flightgear. Openscenegraph supports threading *very* well since ll tha time I know this project. But flightgear was written initially and has still left plenty of single threading assumptions that are not so easy to remove. I always tried to move into this direction, but never got this into a state where I knew that everything *should* work. I know that some of us regularily play with the threading stuff, And I thought that in the time I have not followed fightgear too close Tim had make that work. But when looking into this, I realize that there is still plenty of stuff to do in this area. Also this explaines various crashes on the LinuxTag booth. If you use multithreading it works in the current state just by accident ... The particular source lines also have an //OSGFIXME comment - another hint that work was left behind. Yep... Try replacing the calls to SGIsOpenGLExtensionSupported - they provide constant information anyway. It'd be enough to check version/capabilities once during init and then use constant data at run-time. Try if that helps. Patches welcome :). Well, this is even more complicated: What happens is that in a multithreaded environment we do *not* have *any* OpenGL context active in the main thread. This means whatever code calls either some of the osg extension query function with a second argument of true or calling the above function can result in undefined behaviour - which is what you experience. Only the draw thread has a opengl context guaranteed. Which makes it immediately clear that the only place where we can reliably call these functions is duning the draw traversal. The osg extension functions take two arguments the first one is the context to ask for, which we currently just hope that all context support the same set of features - which is not true in certain configurations. And the second argument is to really ask for the extensions or use the cached result by just testing against a bool value. On a singlethreaded environment we just have the current thread bound to the current context and everything works magically. But on multiple threaded osg this fails. And depending on the opengl driver it may fail or may not fail ... Also, the mentioned call is relatively slow as it searches a string for a substring where it is in the end mostly a boolean operation to decide if a combination of opengl features is availalbel or not. So if you are in a runtime sensitive code path, don't use SGIsOpenGLExtensionSupported directly. So, what to do: The real solution is hard. Tim effecs code *probably* shows how to solve this. I have never double checked if this is done in a waterproof way, but I would guess this is ... A solution that might at least work for most configurations could be like this: Make sure the splash screen draw already asks for a bunch of extensions in the osg way. Then make sure we do not need the decision before the spash screen is there. Tehn finally use the osg extension query functions with an appropriate second argument to make sure the cached result is taken for this mentioned static initialization time usage in the sky code. Hope this helps a little ... Greetings Mathias -- Simplify data backup and recovery for your virtual environment with vRanger. Installation's a snap, and flexible recovery options mean your data is safe, secure and there when you need it. Data protection magic? Nope - It's vRanger. Get your free trial download today. http://p.sf.net/sfu/quest-sfdev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Segfault with OSG multi threading
On Tuesday 31 May 2011 14:17:23 Mathias Fröhlich wrote: So, what to do: The real solution is hard. Tim effecs code *probably* shows how to solve this. I have never double checked if this is done in a waterproof way, but I would guess this is ... A solution that might at least work for most configurations could be like this: Make sure the splash screen draw already asks for a bunch of extensions in the osg way. Then make sure we do not need the decision before the spash screen is there. Tehn finally use the osg extension query functions with an appropriate second argument to make sure the cached result is taken for this mentioned static initialization time usage in the sky code. Hope this helps a little ... First: thanks to you, Thorsten and Anders for your help. I tried just commenting out the extension checks and indeed, FG starts and is using two threads which for the first time since I switched to using the free radeon driver gave me somewhat usable framerates up to 15fps :) Next I will try your suggestion using OSGs caching. Seems like the least intrusive way to get this working with enabled checks. Stefan -- Simplify data backup and recovery for your virtual environment with vRanger. Installation's a snap, and flexible recovery options mean your data is safe, secure and there when you need it. Data protection magic? Nope - It's vRanger. Get your free trial download today. http://p.sf.net/sfu/quest-sfdev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Segfault with OSG multi threading
On Friday 13 May 2011 11:03:24 Anders Gidenstam wrote: Presumably it will also increase the risk of triggering any race condition and/or unsynchronized data access bugs that may be lurking in the code. There are some known ones (e.g. during the creation of a particle system) but there could easily be more. In this case you should investigate the arguments and locals in the different stack frames to see if you can see something looking bad. Stack looks quite fine and it does not feel like a race condition to me. It's 100% reproducable and always in the same place and adding a sleep here and there does not change anything. Well everything looks fine except for the big picture. There are two threads running. One running OSG rendering stuff and the other FG's main loop. The segfault happens when the main loop thread tries to access GL information. I know next to nothing about openGL programming but I seem to recall that it's not allowed to access the same GL context in different threads. So how is this supposed to work? (gdb) thread apply all bt Thread 2 (Thread 0x7fffec712700 (LWP 15449)): #0 0x77bcd38c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x73a4021d in OpenThreads::Condition::wait(OpenThreads::Mutex*) () from /usr/local/lib64/libOpenThreads.so.12 #2 0x74dc2d3b in osgViewer::Renderer::ThreadSafeQueue::takeFront() () from /usr/local/lib64/libosgViewer.so.76 #3 0x74dc3506 in osgViewer::Renderer::draw() () from /usr/local/lib64/libosgViewer.so.76 #4 0x73de291e in osg::GraphicsContext::runOperations() () from /usr/local/lib64/libosg.so.76 #5 0x73e3693a in osg::OperationThread::run() () from /usr/local/lib64/libosg.so.76 #6 0x73de4a88 in osg::GraphicsThread::run() () from /usr/local/lib64/libosg.so.76 #7 0x73a3fd90 in OpenThreads::ThreadPrivateActions::StartThread(void*) () from /usr/local/lib64/libOpenThreads.so.12 #8 0x77bc8a3f in start_thread () from /lib64/libpthread.so.0 #9 0x7328267d in clone () from /lib64/libc.so.6 #10 0x in ?? () Thread 1 (Thread 0x77fa7760 (LWP 15446)): #0 glGetString () at glapi_x86-64.S:9877 #1 0x00abe70a in SGIsOpenGLExtensionSupported (extName=0xbe7cda GL_ARB_texture_env_combine) at extensions.cxx:64 #2 0x009687cc in SGCloudLayer::rebuild (this=0x7fffe31785b0) at cloud.cxx:389 #3 0x00967d0b in SGCloudLayer::SGCloudLayer (this=0x7fffe31785b0, tex_path=...) at cloud.cxx:210 #4 0x004357c4 in fgIdleFunction () at main.cxx:424 #5 0x004a1d8e in fgOSMainLoop () at fg_os_osgviewer.cxx:284 #6 0x00436903 in fgMainInit (argc=1, argv=0x7fffdbd8) at main.cxx:659 #7 0x00433899 in main (argc=1, argv=0x7fffdbd8) at bootstrap.cxx:243 Cheers, Stefan -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Segfault with OSG multi threading
On 30.05.2011 19:25, Stefan Seifert wrote: The segfault happens when the main loop thread tries to access GL information. I know next to nothing about openGL programming but I seem to recall that it's not allowed to access the same GL context in different threads. So how is this supposed to work? Well, wouldn't surprise me to find more related multi-threading issues. Not sure how long OSG itself supports it, but I guess it wasn't a major priority at the time when FG was adapted to OSG. And I'm sure more fixes will be necessary to safely support it. The particular source lines also have an //OSGFIXME comment - another hint that work was left behind. Try replacing the calls to SGIsOpenGLExtensionSupported - they provide constant information anyway. It'd be enough to check version/capabilities once during init and then use constant data at run-time. Try if that helps. Patches welcome :). cheers, Thorsten -- vRanger cuts backup time in half-while increasing security. With the market-leading solution for virtual backup and recovery, you get blazing-fast, flexible, and affordable data protection. Download your free trial now. http://p.sf.net/sfu/quest-d2dcopy1 ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] Segfault with OSG multi threading
Hi! While trying to get some double digit frame rates out of FG next, I tried to enable OSG multi threading in preferences.xml, but that leads to a segfault: #0 0x772af357 in glGetString () from /usr/lib64/libGL.so.1 #1 0x00ab9b3a in SGIsOpenGLExtensionSupported (extName=0xbe075a GL_ARB_texture_env_combine) at extensions.cxx:64 #2 0x0096501c in SGCloudLayer::rebuild (this=0xcabfa90) at cloud.cxx:389 #3 0x0096455b in SGCloudLayer::SGCloudLayer (this=0xcabfa90, tex_path=...) at cloud.cxx:210 #4 0x00435430 in fgIdleFunction () at main.cxx:429 #5 0x004a16f6 in fgOSMainLoop () at fg_os_osgviewer.cxx:284 #6 0x00436535 in fgMainInit (argc=1, argv=0x7fffdbc8) at main.cxx:662 #7 0x00433519 in main (argc=1, argv=0x7fffdbc8) at bootstrap.cxx:243 multithreading-mode is disabled by default with a note, that it breaks screenshots. Is this the only reason or is it known not to be stable? Thanks, Stefan -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Segfault with OSG multi threading
On Fri, 13 May 2011, Stefan Seifert wrote: multithreading-mode is disabled by default with a note, that it breaks screenshots. Is this the only reason or is it known not to be stable? Presumably it will also increase the risk of triggering any race condition and/or unsynchronized data access bugs that may be lurking in the code. There are some known ones (e.g. during the creation of a particle system) but there could easily be more. In this case you should investigate the arguments and locals in the different stack frames to see if you can see something looking bad. Cheers, Anders -- --- Anders Gidenstam WWW: http://www.gidenstam.org/FlightGear/ -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
