Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-16 Thread Serge Lages
pager header file. >> >> -- >> Roland >> >> > -Original Message- >> > From: [EMAIL PROTECTED] >> > [mailto:[EMAIL PROTECTED] On Behalf >> > Of Robert Osfield >> > Sent: zondag 15 juni 2008 18:03 >> > To: OpenSce

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-16 Thread Robert Osfield
On Sun, Jun 15, 2008 at 6:58 PM, Smeenk, R.J.M. (Roland) <[EMAIL PROTECTED]> wrote: > Hello Robbert, > > The SVN version is missing the changes (ReadQueue definition) to the > Databasepager header file. How many times do will I have to type this same message now fixed:) ___

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-16 Thread Serge Lages
zondag 15 juni 2008 18:03 > > To: OpenSceneGraph Users > > Subject: Re: [osg-users] New Improved DatabasePager,now with > > even more threading! Please take a taste today :-) > > > > Hi Guys, > > > > While away on training last week, we came across the Windo

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-15 Thread Smeenk, R.J.M. (Roland)
To: OpenSceneGraph Users > Subject: Re: [osg-users] New Improved DatabasePager,now with > even more threading! Please take a taste today :-) > > Hi Guys, > > While away on training last week, we came across the Windows > debug build reporting problem in std::sort and find call

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-15 Thread Robert Osfield
Hi Guys, While away on training last week, we came across the Windows debug build reporting problem in std::sort and find calls, and investigations into the issue suggested that it was down to the cull thread modifying the DatabaseRequest fields at the same time that those same fields were being u

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-13 Thread Serge Lages
Hi James, I tried to look to this problem but can't find anything, moreover it seems to work without problems in release (I'll try to make some stress tests soon to be sure). On Fri, Jun 13, 2008 at 1:02 AM, Lynch, James <[EMAIL PROTECTED]> wrote: > > I'm having a similar issue with DatabasePage

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-12 Thread Lynch, James
I'm having a similar issue with DatabasePager::RequestQueue::takeFirst() (osg trunk, Debug build, windows XP VS2005). I'm getting some assertion errors at _requestList.sort ("sequence not ordered"). Even after adding the updateBlock(). I'm pretty new to this code section, so I'm probably off ba

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-03 Thread Csaba Halász
On Tue, Jun 3, 2008 at 4:48 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > > Could you do an svn update on the OSG and let know if things are no > work fine at your end, It works, thanks. -- Csaba ___ osg-users mailing list osg-users@lists.openscenegr

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-03 Thread Robert Osfield
Hi Csaba, Thanks for spotting the missing updateBlock(). I've now added this and checked it in. I put the updateBlock() inside the if () {} code block, but other than that is identical to your own change. Must admit I hadn't spotted the problem, the downside of having a four core machine... Co

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-06-03 Thread Csaba Halász
On Tue, May 27, 2008 at 12:08 PM, Tim Moore <[EMAIL PROTECTED]> wrote: > > Indeed. I've made the necessary changes to FlightGear, and it seems to > be working fine. Except for being a cpu hog. Apparently, the pager thread is not blocked properly and continuously spins. The fact that the queue is

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-30 Thread Robert Osfield
Hi Serge, On Fri, May 30, 2008 at 2:42 PM, Serge Lages <[EMAIL PROTECTED]> wrote: > Have you already updated the pager with these changes ? I checked in the changes about 20 mins after my previous post, so yes they are checked in now. > I was looking to the debug crash, but I'll wait for these m

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-30 Thread Serge Lages
Hi Robert, Have you already updated the pager with these changes ? I was looking to the debug crash, but I'll wait for these modifications to see if it changes something. Cheers, On Fri, May 30, 2008 at 12:16 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > Hi Pagees, > > I've been looking into

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-30 Thread Robert Osfield
Hi Pagees, I've been looking into the problem of memory growth in the new rev of the DatabasePager and I've tracked the problem down to the new mechanism of attached DatabaseRequest's via a ref_ptr<> to PagedLOD/ProxyNode nodes. The DatabaseRequest themselves had a ref_ptr<> which creates a circu

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Robert Osfield
Hi Serge, On Fri, May 23, 2008 at 4:17 PM, Serge Lages <[EMAIL PROTECTED]> wrote: > There's no hight memory usage for me. I tried to investigate further on this > problem but can't find anything. If I create only one thread in the pager it > also crash. The memory issue I only see if I run the ap

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Serge Lages
On Fri, May 23, 2008 at 10:22 AM, Robert Osfield <[EMAIL PROTECTED]> wrote: > > One problem that I have observed steady growth in memory usage on a > very large database I have. I don't know yet if this is a leak or a > load balancing problem. I wouldn't expect this issue to be related to > the

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Rahul Jain
Hi All, Looking at the enthusiasm all around new improved DatabasePager I would also like to try my hands on it by reading data from http, but have not much idea about how to do that. Will my node look like this for using http database paging. PagedLOD { DataVariance UNSPECIFIED nodeM

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread J.P. Delport
Hi Robert, Yes, I also wish there was an easy wrapper around OpenThreads to help one debug the multi-threaded execution. What do you have in mind w.r.t an "easy wrapper around OpenThreads"? I was hoping for something like GLIntercept (that sits between the app and OpenGL and can trace all c

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Robert Osfield
Hi J.P, On Fri, May 23, 2008 at 9:53 AM, J.P. Delport <[EMAIL PROTECTED]> wrote: > Yes, you are right for the case where a lock is only released by the same > thread that acquired it. This is the normal use case that e.g. scopedlock > enforces. Ahhh this is a different issue, as you say ScopedLoc

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread J.P. Delport
Hi, Robert Osfield wrote: Hi J.P, On Fri, May 23, 2008 at 9:16 AM, J.P. Delport <[EMAIL PROTECTED]> wrote: I've reviewed all the places that read and write the RequestQueue::_requestList and they all look like they acquire the _requestMutex before they use the list. are all the acquires from

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Robert Osfield
Hi J.P, On Fri, May 23, 2008 at 9:16 AM, J.P. Delport <[EMAIL PROTECTED]> wrote: >> I've reviewed all the places that read and write the >> RequestQueue::_requestList and they >> all look like they acquire the _requestMutex before they use the list. > > are all the acquires from different threads?

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Robert Osfield
Hi Lamoliatte and Stephane, > Thanks for the confirmation, I'll try to investigate further today. It seems > that even in release mode it's not stable, I got some weird crashes > yesterday. Thanks for the testing. Sorry to hear about the instabilities. Alas at my end everything is rock solid s

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread J.P. Delport
Hi, Robert Osfield wrote: Hi Serge, On Thu, May 22, 2008 at 3:30 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: So there must be somewhere else that isn't playing ball properly and hasn't acquired the lock as it should. I've reviewed all the places that read and write the RequestQueue::_reque

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-23 Thread Serge Lages
On Fri, May 23, 2008 at 8:08 AM, Stephane Lamoliatte < [EMAIL PROTECTED]> wrote: > Hi Serge, > > I test the new DatabasePager with a regular pagedLOD database. Like you : > all seems to work fine in release mode but I have a crash in debug mode. > I work with Visual studio 2005 under Windows 2000.

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Stephane Lamoliatte
Hi Serge, I test the new DatabasePager with a regular pagedLOD database. Like you : all seems to work fine in release mode but I have a crash in debug mode. I work with Visual studio 2005 under Windows 2000. Serge Lages a écrit : On Thu, May 22, 2008 at 5:22 PM, Robert Osfield <[EMAIL PROTE

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Serge Lages
On Thu, May 22, 2008 at 5:22 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > > I've reviewed all the places that read and write the > RequestQueue::_requestList and they > all look like they acquire the _requestMutex before they use the list. > > This leaves me a bit empty handed w.r.t what might

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi Serge, On Thu, May 22, 2008 at 3:30 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > So there must be somewhere else that isn't playing ball properly and > hasn't acquired the lock as it should. I've reviewed all the places that read and write the RequestQueue::_requestList and they all look li

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi Serge, Curious. The particular bit of code does actually acquire the mutex lock - the whole method is: void DatabasePager::RequestQueue::takeFirst(osg::ref_ptr& databaseRequest) { OpenThreads::ScopedLock lock(_requestMutex); if (!_requestList.empty()) { _requestList.sort(

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Serge Lages
Forget what I've said, I found a workaround. My custom PagedLOD works juste fine with the new DatabasePager, but only in release mode. In debug, Visual Studio spot an error in : DatabasePager.cpp line 272 : if (!_requestList.empty()) { _requestList.sort(SortFileRequestFunctor());

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi Serge, On Thu, May 22, 2008 at 2:11 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: >> I customize the data storage. :) I now wonder if the PerRangeData shouldn't be a base class that you extend, if you derive from this then the various PagedLOD methods would still be available, so no need to go

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi Serge, On Thu, May 22, 2008 at 1:57 PM, Serge Lages <[EMAIL PROTECTED]> wrote: > I customize the data storage. :) > My PagedLOD is really particular because it doesn't store directly nodes, > but layers (we have the texture, the terrain and the data on top of the > terrain in different layers)

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Serge Lages
On Thu, May 22, 2008 at 2:47 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > Hi Serge, > > On Thu, May 22, 2008 at 1:42 PM, Serge Lages <[EMAIL PROTECTED]> > wrote: > > Hum... Should it be possible to make the PagedLOD getDatabaseRequest > method > > virtual ? :) > > This would possible. It is ho

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi Serge, On Thu, May 22, 2008 at 1:42 PM, Serge Lages <[EMAIL PROTECTED]> wrote: > Hum... Should it be possible to make the PagedLOD getDatabaseRequest method > virtual ? :) This would possible. It is however, something that is meant to be maintained by the DatabasePager directly, and all it ne

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Serge Lages
On Thu, May 22, 2008 at 12:09 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > > Another change to the DatabasePager was to store the DatabaseRequest > objects that are used internally by the pager directly on the > ProxyNode/PagedLOD nodes as well as internally in the various queues > in the Datab

Re: [osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Serge Lages
Hi Robert, All these things are good news ! :) I am currently building and I'll let you know how it works here (I have a custom PagedLOD which calls requestNodeFile). On Thu, May 22, 2008 at 12:09 PM, Robert Osfield <[EMAIL PROTECTED]> wrote: > Hi All, > > Over the last week I've been working on

[osg-users] New Improved DatabasePager, now with even more threading! Please take a taste today :-)

2008-05-22 Thread Robert Osfield
Hi All, Over the last week I've been working on improving the performance of the DatabasePager when paging data over http, and storing the tiles in a local file cache, I checked in the first cut of the work last night. Previous rev of the DatabasePager had only a single thread for reading tiles,