Hi!
Thanks for your quick reply and sorry for the delayed reply on your
reply. We have been busy removing a few other bugs from our application.
The problem with the doubled memory consumption remains though. We made
a short test program to provoke the behavior, based on the multithread
example in the Tutorial. The program is attached to this mail.
Display lists is turned off, as you suggested, setting the prototype
before creating the torus.
The behavior of the test program is similar to our application. What
happens is:
* Thread 1 (main) - osgInit ... etc
* T1 - create Thread 2
* T1 - glut calls display and stops on barrier
* T2 - turn off display lists and create a torus that takes about
40 MB (very smooth :-)
* T2 - enter animation loop
* T2 - enter barrier
* T1 - calls applyAndClear for the first time, consuming another 40 MB
The major difference from the tutorial example should be that the scene
isn't created in the rendering thread. We must do this in our
application because scene loading and manipulating is handled by the
non-rendering GUI-thread.
The environment:
* Windows XP with latest updates.
* Binary distribution OpenSG 1.4 with MS STL
* Visual Studio .Net 2003
The doubled memory consumption is not supposed to take place if
I understand your reply correctly. Is this due to a bug in OpenSG or in
our usage of the API?
Best regards,
Johan Grafström, Fredrik Andersson
Dirk Reiners wrote:
Hi Johan,
On Mon, 2005-05-02 at 18:54 +0200, Johan Grafstrom wrote:
We are using OpenSG to visualize a large model on Windows. The model
takes about 1 GB of memory. The model may take a while to render
(depending of how much of it is currently visible). To keep the user
interface responsive, rendering is therefor done in a separate thread.
The GUI-thread reads and writes the scenegraph, so we need the two
aspects. However since there is a lot of data in the scenegraph, we
cannot simply double its size. When the second thread starts (and the
first changeList->applyAndClear() is called) the program locks for a
while and the memory usage doubles, usually crashing due to memory (or
rather, address space) overflow.
I don't think that's related to the aspects, see below.
What happens is probably:
* The nodes and cores are created with aspects when scenegraph is read
from file. The data fields is initially empty.
* During reading, the geometry data (vertices etc) is assigned to the
current aspect data fields of the the nodes. The fields is added to
the changelist.
* When applyAndClear is called the first time all geometry data is
copied to fill the aspect of the second thread, thus consuming lots
of memory.
Unless there is a bug that shouldn't happen. The applyAndClear() for
MultiFields only creates a new reference to the same data, so no new
memory should be allocated.
Some possible solutions to the problem:
A Using only one aspect and make sure only one thread at a time can
access the scenegraph. We will need to be able to interrupt the
rendering in order to keep the application responsive (eg by calling
renderAction->setTravMask(0))
Well, you can run mutiple threads and a single aspect, as long as you're
careful about changing things synchronized.
B Change OpenSG so that number of aspects may be set per
FieldContainer or FieldContainerType. Ensuring that the geometry
nodes (and their data) data (which is static in our application)
has only one aspect would solve our problem.
That would be a pretty serious issue. Right now the ChangeList manager
can just copy the data around, but what is it supposed to do, if there
is no copy of the node in the aspect it's supposed to copy to?
Alternative B would be the "nicest" solution of these, if it is'nt too
much work.
I don't see that happening, it's a very big change with lots of
undefined problems in my mind right now.
Questions:
* Are there some other, better, solutions?
As I said, unless there is a bug this is not really the problem.
My guess is that the memory is consumed during the first rendering
(could you confirm that by putting a pause after the sync and before the
first render?). By default OpenSG uses Display Lists to speed up
rendering, and those take about the same amount of memory as the
geometry (somewhat more, actually). So unless you've switched them off,
that's going to be an issue. You can switch them off individually using
geocore->setDlistCache(false); or globally by calling
FieldContainerPtr pProto = Geometry::getClassType().getPrototype();
GeometryPtr pGeoProto = GeometryPtr::dcast(pProto);
if(pGeoProto != NullFC)
{
pGeoProto->setDlistCache(false);
}
before you start loading.
Please try that, it should at least relieve the memory pressure.
* Otherwise, how difficult would it be to fix B above, e g by making
getNumAspects() a method of FieldContainer or FieldContainerType? If
we did, would you accept it into the OpenSG project?
I'd be hard pressed to do that, as it will involve serious changes
throughout the whole system to make this work reliably. I really hope
the abovementioned changes fix your problem. ;)
Sorry if the mail got uneccessarily lengthy...
No problem, I prefer longer and detailed mails to the "My program
doesn't work. How do I fix it?" ones.
Yours
Dirk
<?xml version="1.0" encoding="Windows-1252"?>
<VisualStudioProject
ProjectType="Visual C++"
Version="7.10"
Name="Viewer"
ProjectGUID="{3D356E7A-0629-488C-B9E3-2C8C8D02A84F}"
RootNamespace="Viewer"
Keyword="ManagedCProj">
<Platforms>
<Platform
Name="Win32"/>
</Platforms>
<Configurations>
<Configuration
Name="Debug|Win32"
OutputDirectory="$(SolutionDir)$(ConfigurationName)"
IntermediateDirectory="$(ConfigurationName)"
ConfigurationType="1"
CharacterSet="2"
ManagedExtensions="TRUE">
<Tool
Name="VCCLCompilerTool"
Optimization="0"
AdditionalIncludeDirectories="include;"$(OSG_HOME)\include""
PreprocessorDefinitions="WIN32;_DEBUG;WINVER=0x0400;_WIN32_WINDOWS=0x0410;_WIN32_WINNT=0x0400;OSG_BUILD_DLL;_OSG_HAVE_CONFIGURED_H_;QT_DLL;QT_PLUGIN;QT_THREAD_SUPPORT;OSG_WITH_GLUT;_WINDOWS;UNICODE;_GNU_SOURCE;QT_CLEAN_NAMESPACE;QT_ACCESSIBILITY_SUPPORT;QT_TABLET_SUPPORT;QT_NO_STL;_WINDLL"
MinimalRebuild="FALSE"
BasicRuntimeChecks="0"
RuntimeLibrary="1"
WarningLevel="3"
DebugInformationFormat="3"/>
<Tool
Name="VCCustomBuildTool"/>
<Tool
Name="VCLinkerTool"
AdditionalDependencies="OSGBaseD.lib
OSGSystemD.lib OSGWindowGlutD.lib"
OutputFile="$(OutDir)\$(ProjectName).exe"
LinkIncremental="2"
AdditionalLibraryDirectories="$(OSG_HOME)\lib"
GenerateDebugInformation="TRUE"
AssemblyDebug="1"/>
<Tool
Name="VCMIDLTool"/>
<Tool
Name="VCPostBuildEventTool"/>
<Tool
Name="VCPreBuildEventTool"/>
<Tool
Name="VCPreLinkEventTool"/>
<Tool
Name="VCResourceCompilerTool"/>
<Tool
Name="VCWebServiceProxyGeneratorTool"/>
<Tool
Name="VCXMLDataGeneratorTool"/>
<Tool
Name="VCWebDeploymentTool"/>
<Tool
Name="VCManagedWrapperGeneratorTool"/>
<Tool
Name="VCAuxiliaryManagedWrapperGeneratorTool"/>
</Configuration>
<Configuration
Name="Release|Win32"
OutputDirectory="$(SolutionDir)$(ConfigurationName)"
IntermediateDirectory="$(ConfigurationName)"
ConfigurationType="1"
CharacterSet="2"
ManagedExtensions="TRUE">
<Tool
Name="VCCLCompilerTool"
PreprocessorDefinitions="WIN32;NDEBUG"
MinimalRebuild="FALSE"
RuntimeLibrary="0"
WarningLevel="3"
DebugInformationFormat="3"/>
<Tool
Name="VCCustomBuildTool"/>
<Tool
Name="VCLinkerTool"
OutputFile="$(OutDir)\$(ProjectName).exe"
LinkIncremental="1"
GenerateDebugInformation="TRUE"/>
<Tool
Name="VCMIDLTool"/>
<Tool
Name="VCPostBuildEventTool"/>
<Tool
Name="VCPreBuildEventTool"/>
<Tool
Name="VCPreLinkEventTool"/>
<Tool
Name="VCResourceCompilerTool"/>
<Tool
Name="VCWebServiceProxyGeneratorTool"/>
<Tool
Name="VCXMLDataGeneratorTool"/>
<Tool
Name="VCWebDeploymentTool"/>
<Tool
Name="VCManagedWrapperGeneratorTool"/>
<Tool
Name="VCAuxiliaryManagedWrapperGeneratorTool"/>
</Configuration>
</Configurations>
<References>
</References>
<Files>
<Filter
Name="Source Files"
Filter="cpp;c;cxx;def;odl;idl;hpj;bat;asm;asmx"
UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}">
<File
RelativePath=".\13multithreading2.cpp">
</File>
</Filter>
<Filter
Name="Header Files"
Filter="h;hpp;hxx;hm;inl;inc;xsd"
UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}">
</Filter>
<Filter
Name="Resource Files"
Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx"
UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}">
</Filter>
</Files>
<Globals>
</Globals>
</VisualStudioProject>
// all needed include files
#include <OpenSG/OSGGLUT.h>
#include <OpenSG/OSGConfig.h>
#include <OpenSG/OSGSimpleGeometry.h>
#include <OpenSG/OSGGLUTWindow.h>
#include <OpenSG/OSGSimpleSceneManager.h>
#include <OpenSG/OSGThreadManager.h>
OSG_USING_NAMESPACE
using namespace std;
SimpleSceneManager *mgr;
NodePtr scene;
//we will store the transformation globally - this
//is not necessary, but comfortable
TransformPtr trans;
Thread* animationThread;
Barrier *syncBarrier;
int setupGLUT( int *argc, char *argv[] );
NodePtr createScenegraph(){
// the scene must be created here
NodePtr n = makeTorus(.5,2,1024,1024);
//add a simple Transformation
trans = Transform::create();
beginEditCP(trans);
Matrix m;
m.setIdentity();
trans->setMatrix(m);
endEditCP(trans);
NodePtr transNode = Node::create();
beginEditCP(transNode);
transNode->setCore(trans);
transNode->addChild(n);
endEditCP(transNode);
return transNode;
}
//this function will run in a thread and simply will
//rotate the cube by setting a new transformation matrix
void rotate(void *args) {
// we won't stop calculating new matrices....
// NEW STUFF
// Disable displaylist on new geometry
FieldContainerPtr pProto = Geometry::getClassType().getPrototype();
GeometryPtr pGeoProto = GeometryPtr::dcast(pProto);
if(pGeoProto != NullFC) {
pGeoProto->setDlistCache(false);
}
// this consumes about 40 MB
scene = createScenegraph();
mgr->setRoot(scene);
mgr->showAll();
// END NEW STUFF
while(true) {
Real32 time = glutGet(GLUT_ELAPSED_TIME);
Matrix m;
m.setIdentity();
m.setRotate(Quaternion(Vec3f(0,1,0), time/1000));
beginEditCP(trans);
trans->setMatrix(m);
endEditCP(trans);
// nothing unusual until here
//well that's new...
//wait until two threads are cought in the
//same barrier
syncBarrier->enter(2);
//just the same again
syncBarrier->enter(2);
}
}
int main(int argc, char **argv)
{
ChangeList::setReadWriteDefault();
osgInit(argc,argv);
int winid = setupGLUT(&argc, argv);
GLUTWindowPtr gwin = GLUTWindow::create();
gwin->setId(winid);
gwin->init();
// in the tutorial example, the scengraph was created in the
// main/render thread.
// scene = createScenegraph();
syncBarrier = Barrier::get("Barrier");
mgr = new SimpleSceneManager;
mgr->setWindow(gwin );
//create the thread that will run generation of new matrices
animationThread = dynamic_cast<Thread *>(ThreadManager::the()
->getThread("anim"));
//do it...
animationThread->runFunction(rotate, 1, NULL);
glutMainLoop();
return 0;
}
void reshape(int w, int h)
{
mgr->resize(w, h);
glutPostRedisplay();
}
void display(void)
{
// we wait here until the animation thread enters
//the first barrier
syncBarrier->enter(2);
// now we sync data
// First time this is run, it consumes another 40 MB !!
animationThread->getChangeList()->applyAndClear();
// and again
syncBarrier->enter(2);
// now render...
mgr->redraw();
}
void mouse(int button, int state, int x, int y)
{
if (state)
mgr->mouseButtonRelease(button, x, y);
else
mgr->mouseButtonPress(button, x, y);
glutPostRedisplay();
}
void motion(int x, int y)
{
mgr->mouseMove(x, y);
glutPostRedisplay();
}
int setupGLUT(int *argc, char *argv[])
{
glutInit(argc, argv);
glutInitDisplayMode(GLUT_RGB | GLUT_DEPTH | GLUT_DOUBLE);
int winid = glutCreateWindow("OpenSG First Application");
glutDisplayFunc(display);
glutMouseFunc(mouse);
glutMotionFunc(motion);
glutReshapeFunc(reshape);
glutIdleFunc(display);
return winid;
}
