Re: [osg-users] Refactoring DatabasePager NeedToRemovestringflagging technique

2009-11-23 Thread Wojciech Lewandowski

Hi Robert,

My apologies for SARCASM. I wanted to be provocative because former posts 
went unnoticed. Maybe I have gone too far...


But I would strongly defend merits of arguments in the post. You say we did 
wrong, but whats you recommendation on mixing many intersections with 
rendering ?  Its very common scenario. How  should we tackle it, if current 
approach is wrong ?


Second important issue for me is usage of _name in scene graph. I always 
expected that _name is reserved for users and its a normal rule in all Scene 
Graph implementations that libraries do not change it.  Names are ususlly 
used to identify certain portions of models and hook up the code properly. 
Thats something that provide standard linking mechanisms between artists and 
programmers works.


Please also note that we use cache becuse we otherwise were loading PageLOD 
files twice. Is it reasonable ?.  Even if we skip intersections,  other 
situations are also possible. For example we may have few highly detailed 
special PageLOD tiles with ariports which we want to preload and keep in 
memory for whole application runtime. So we modify readFileCallback to work 
for such cache and return these preloaded models each time thery are 
requested. Their names will be changed as well and NeedToRemove scheme 
stops working anyway.


With all due Respect ;-)
Wojtek Lewandowski

- Original Message - 
From: Robert Osfield robert.osfi...@gmail.com

To: OpenSceneGraph Users osg-users@lists.openscenegraph.org
Sent: Monday, November 23, 2009 2:59 PM
Subject: Re: [osg-users] Refactoring DatabasePager 
NeedToRemovestringflagging technique



Hi Wojtek,

On Mon, Nov 23, 2009 at 11:53 AM, Wojciech Lewandowski
lewandow...@ai.com.pl wrote:

3. I don't know if this was deliberate or not, but IntersectionVisitor
in USE_HGHEST_LEVEL_OF_DETAIL mode does load highest level tiles 
temporary.

It does intersections and free them. It does not hook them up to their
parents.


This is deliberate - if you start modifying the scene graph during the
Intersection traversal then you can't run it multi-threaded anymore.

There is also the issue of having to maintain a PageLOD with it's
children loaded from lowest res to highest res - you can't skip
intermediate children and just attach the highest rest one, keeping
the loaded subgraph local.

It's also worth mentioning that IntersectionVsitor itself does do any
loaded of data, it's callback that you attach to it that do it so
usual usage of IntersectionVisitor doesn't load highest level of
detail - it doesn't load anything.  Hence the
DatabaseCacheReadCallback...


4. Above has this horrible effect that if some node was not already loaded
by DatabasePager it will be constantly loaded and removed by
IntersectionVisitor. SARCASM#1: But don't loose faith there is a solution.
See next point.


Well if you want the IntersectionVisitor to happen asynchronously from
the DatabasePager loaded then you have to have keep the two separate,
or manage the integration very careful by using a
IntersectionReadCallback that is design to integrate the two.

Please note DatabaseCacheReadCallback is not designed to integrate
DatabasePager and IntersectionVisitor, it's a specific solution for
IntersectionVisition and is wholly parallel and independent to
DatabasePager and is meant to be this way.  IntersectionVisitor is
synchronous - you have to get the subgraph immediately, so the reading
of external tiles has to be done in the same thread.


5. We found out that we can use osgSim::DatabaseCacheReadCallback to
mitigate former problem. This is an IntersectVisitor read callback that
keeps internal cache to avoid repetitive loading and freeing. Well...this
works but only for IntesectVisitor. DatabasePager does not know anything
about this cache so when DatabasePager finally decides to load a tile it
does it again, although IntersectionVisitor have already loaded it.
SARCASM#2: But don't loose hope yet, because we found a solution to this 
as

well
6. How to make sure DatabasePager sees the tile IntersectVisitor already
loaded ? Its simple: We could use osgDB cache. So we started to load tiles
with CACHE_NODES option and everything seemed to be fine
7 But after some time we started to observe crashes and memory leaks. Long
story about them is in my friend Pawel Ksiezopolski post Re: [osg-users]
PagedLOD experts? from November 5th. Short story is that caching tiles
does not free renamed NeedToRemove nodes, but keep them in memory so 
with

time some if them land int the scene again. When this happens, logic that
was invented to remove not used nodes removes wrong ones leaving those 
that
should be removed. Hence we get PageLOD thrashing and memory leaks. Cool 
HUH

? (Yes its SARCASM#3)


Well if you do start mixing stuff that wasn't intended to be mixed in
the way you are mixing it well perhaps we shouldn't be too surprised
that problems eventually do ensue.

Also the SCARCASM stuff really isn't helpful and just lowers

Re: [osg-users] Refactoring DatabasePager NeedToRemovestringflagging technique

2009-11-23 Thread Glenn Waldron
On Mon, Nov 23, 2009 at 11:10 AM, Robert Osfield
robert.osfi...@gmail.comwrote:

 Hi Wojtek,

 On Mon, Nov 23, 2009 at 2:34 PM, Wojciech Lewandowski
 lewandow...@ai.com.pl wrote:
  But I would strongly defend merits of arguments in the post. You say we
 did
  wrong, but whats you recommendation on mixing many intersections with
  rendering ?  Its very common scenario. How  should we tackle it, if
 current
  approach is wrong ?

 The right approach is a difficult one.  Getting a paged scene graph to
 work with intersections at highest resolutions and at the same time
 manage things for rendering with requires just the appropriate LOD
 child for the needs of visuals is awkward.   I know often vis-sim apps
 don't even try to mix the two, and have a separate process entirely
 for dealing intersections as for doing the visuals.  Some sims even
 run the visuals and intersection testing on entirely different
 machines.  Other sims use entirely separate databases for intersection
 testing and visuals.  Then there are others that use a height field
 for height above terrain testing...


Wojtek,

The keep it separate approach is what we use in osgEarth. The idea is to
fetch terrain tiles directly, based on your target sampling resolution,
instead of traversing the whole LOD hierarchy. Take a took at the
ElevationManager utility. Perhaps it can provide some inspiration:

http://wush.net/trac/osgearth/browser/trunk/src/osgEarthUtil/ElevationManager

Glenn
___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org