Re: [osg-users] Bug in OverlayNode.cpp

[Chris Long]

Hi,

I'd like to request that someone patch OverlayNode with this one-line fix. We 
are still having to work around it in our OSG application.

Thank you!

Cheers,
Chris

--
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=47268#47268





___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

Re: [osg-users] Bug in OverlayNode.cpp

[Jason Daly]

On 04/24/2012 03:21 PM, Chris Long wrote:

Hi,

I'd like to request that someone patch OverlayNode with this one-line fix. We 
are still having to work around it in our OSG application.


You'll probably get quicker results if you patch the file yourself and 
post it (the whole modified file) to the Submissions forum.


--J

___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[osg-users] Bug in OverlayNode.cpp

[Chris Long]

Hi,

I have found a bug at OverlayNode.cpp:335:


Code:
for(unsigned int i=0; i  baseVertices.size()-1; ++i)




The problem is when baseVertices is empty, baseVertices.size() is 0 and (here's 
the surprising part) the compiler treats baseVertices.size()-1 as an unsigned 
int, which turns it into some huge positive number (18446744073709551615 on 
Linux on amd64). So the loop actually executes and seg faults.

I have this case occurring in my application, I think because of an OverlayNode 
that has a valid overlay subgraph node with no children of its own.

I'm sure there are other ways to fix this, but I would suggest fixing it by 
changing the above line to:


Code:
  for(unsigned int i=0; i+1  baseVertices.size(); ++i)




Thank you!

Cheers,
Chris

--
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=38233#38233





___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org