Re: [osg-users] DatabasePager question ...
Thanks for to take my indent to an review task. may i can help to work out a more general framework for the database pager. the current works nice and fast, but is still not as general as i can be (design) the way to introduce configuration will be really good. and any application can be better work with the osg core. .adrian 2008/11/13 Robert Osfield <[EMAIL PROTECTED]> > Hi Adrian, > > On Thu, Nov 13, 2008 at 9:06 AM, Adrian Egli OpenSceneGraph (3D) > <[EMAIL PROTECTED]> wrote: > > Ok, i solved it > > with > > osg::DisplaySettings::instance()->setNumOfDatabaseThreadsHint(5); > > osg::DisplaySettings::instance()->setNumOfHttpDatabaseThreadsHint(0); > > i am not sure this is the right way to implement our behavoiur but ... > > This effectively switches off the shunting of http requests off to a > separate thread. If this is appropriate for your app then it's the > right way to do it. > > I'm not familiar with your app and how the paging/terrain generation > is intended to behave so whether it's the right way I cannot say. > Ideally we should have a page that doesn't require end users to > subclass from it common tasks, but to provide configuration to meet > the range of needs. For really unusually paging needs subclassing > might still be required, but the more we can avoid subclassing the > less issues we are likely to have when internal implementations change > over time. > > Robert > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > -- Adrian Egli ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi Adrian, On Thu, Nov 13, 2008 at 9:06 AM, Adrian Egli OpenSceneGraph (3D) <[EMAIL PROTECTED]> wrote: > Ok, i solved it > with > osg::DisplaySettings::instance()->setNumOfDatabaseThreadsHint(5); > osg::DisplaySettings::instance()->setNumOfHttpDatabaseThreadsHint(0); > i am not sure this is the right way to implement our behavoiur but ... This effectively switches off the shunting of http requests off to a separate thread. If this is appropriate for your app then it's the right way to do it. I'm not familiar with your app and how the paging/terrain generation is intended to behave so whether it's the right way I cannot say. Ideally we should have a page that doesn't require end users to subclass from it common tasks, but to provide configuration to meet the range of needs. For really unusually paging needs subclassing might still be required, but the more we can avoid subclassing the less issues we are likely to have when internal implementations change over time. Robert ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi Adrian,
I'm afraid I had a real hard time following your post. I think what
it boils down to is that you want to be able to choose which
DatabaseThreads accept or pass on requests as the http file vs local
file requests are appropriate for your tileid based filename handles.
I will need to think about this topic further to see what tweaks can
be made to make it more customisable.
Robert.
On Thu, Nov 13, 2008 at 8:01 AM, Adrian Egli OpenSceneGraph (3D)
<[EMAIL PROTECTED]> wrote:
> Hi Robert,
>
> I did a review of the HTTP and NON HTTP pager file read. I still use the
> readNodeFile callback mechansim and this work. But for our
> work, it's not only there a good idea to pass the custom code, because the
> generation of the custom database (on the flow) doesn't
> work as fast as the file read is. so if we are moving around over the
> terrain, and once we come back to a position we still were, and
> the database pager should read all files still generated and there is one
> missing and unfortunately the missing one has the highest
> prio it will take that one, then the other will not paged in as long as the
> missing one is generating, but it should be loaded all others,
> and once the missing once is generated it will immedently paged in. like the
> http request, if it's in the cache then we can load it without
> downloading, directly read in.
>
> so may we should change the mechansime, its not really general implemented:
>
> you assume that we have only HTTP or NON HTTP, non http is no problem there
> is only files. also cached ones and so.
> but if we don't have http:// starting file names, you assume that it's a
> file. but may it's a database request code for the tile gerneration
> instead of a real existing file name, if we have such information, then we
> should be able to check the "filename"/ "request" not only
> against http:// starting string may we can append a user request qeueu and a
> user thread to pregenerate / preread the files. we should only
> have a method
>
> itr : Iterate over all QUEUEandThread (non http, http, user ) {
>
> if ( itr->requestForThisQueue(databaseRequest->_fileName)) {
>
> itr->queue
>
> }
>
> }
>
> Class QUEUEandThread {
>
> DatabaseThread _thread;
> RequestQueue _queue;
>
> bool requestForThisQueue(std::string reqStr) {
> // if HTTP queue is
>return osgDB::containsServerAddress(reqStr)
> }
> }
>
>
> so we would be able to append user requests queues very fast and the code
> will be more general for future task.
>
> /adrian
>
> 2008/11/12 Robert Osfield <[EMAIL PROTECTED]>
>>
>> Hi Adrian,
>>
>> On Wed, Nov 12, 2008 at 6:39 PM, Adrian Egli OpenSceneGraph (3D)
>> <[EMAIL PROTECTED]> wrote:
>> > Ok i see, so you would append a custum reading thread to the osg
>> > database to
>> > avoid subclassing.
>>
>> There shouldn't be any need for a custom reading thread, just use the
>> existing threads to call you code via the osgDB::readNodeFile
>> mechanism - just put your code in a plugin and let the OSG call it.
>>
>> Robert.
>> ___
>> osg-users mailing list
>> osg-users@lists.openscenegraph.org
>> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>
>
>
> --
>
> Adrian Egli
>
> ___
> osg-users mailing list
> osg-users@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>
>
___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Ok, i solved it
with
osg::DisplaySettings::instance()->setNumOfDatabaseThreadsHint(5);
osg::DisplaySettings::instance()->setNumOfHttpDatabaseThreadsHint(0);
i am not sure this is the right way to implement our behavoiur but ...
adrian
2008/11/13 Adrian Egli OpenSceneGraph (3D) <[EMAIL PROTECTED]>
> Hi Robert,
>
> I did a review of the HTTP and NON HTTP pager file read. I still use the
> readNodeFile callback mechansim and this work. But for our
> work, it's not only there a good idea to pass the custom code, because the
> generation of the custom database (on the flow) doesn't
> work as fast as the file read is. so if we are moving around over the
> terrain, and once we come back to a position we still were, and
> the database pager should read all files still generated and there is one
> missing and unfortunately the missing one has the highest
> prio it will take that one, then the other will not paged in as long as the
> missing one is generating, but it should be loaded all others,
> and once the missing once is generated it will immedently paged in. like
> the http request, if it's in the cache then we can load it without
> downloading, directly read in.
>
> so may we should change the mechansime, its not really general implemented:
>
>
> you assume that we have only HTTP or NON HTTP, non http is no problem there
> is only files. also cached ones and so.
> but if we don't have http:// starting file names, you assume that it's a
> file. but may it's a database request code for the tile gerneration
> instead of a real existing file name, if we have such information, then we
> should be able to check the "filename"/ "request" not only
> against http:// starting string may we can append a user request qeueu and
> a user thread to pregenerate / preread the files. we should only
> have a method
>
> itr : Iterate over all QUEUEandThread (non http, http, user ) {
>
> if ( itr->requestForThisQueue(databaseRequest->_fileName)) {
>
> itr->queue
>
> }
>
> }
>
> Class QUEUEandThread {
>
> DatabaseThread _thread;
> RequestQueue _queue;
>
> bool requestForThisQueue(std::string reqStr) {
> // if HTTP queue is
>return osgDB::containsServerAddress(reqStr)
> }
> }
>
>
> so we would be able to append user requests queues very fast and the code
> will be more general for future task.
>
> /adrian
>
> 2008/11/12 Robert Osfield <[EMAIL PROTECTED]>
>
>> Hi Adrian,
>>
>>
>> On Wed, Nov 12, 2008 at 6:39 PM, Adrian Egli OpenSceneGraph (3D)
>> <[EMAIL PROTECTED]> wrote:
>> > Ok i see, so you would append a custum reading thread to the osg
>> database to
>> > avoid subclassing.
>>
>> There shouldn't be any need for a custom reading thread, just use the
>> existing threads to call you code via the osgDB::readNodeFile
>> mechanism - just put your code in a plugin and let the OSG call it.
>>
>> Robert.
>> ___
>> osg-users mailing list
>> osg-users@lists.openscenegraph.org
>> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>>
>
>
>
> --
>
> Adrian Egli
>
--
Adrian Egli
___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi Robert,
I did a review of the HTTP and NON HTTP pager file read. I still use the
readNodeFile callback mechansim and this work. But for our
work, it's not only there a good idea to pass the custom code, because the
generation of the custom database (on the flow) doesn't
work as fast as the file read is. so if we are moving around over the
terrain, and once we come back to a position we still were, and
the database pager should read all files still generated and there is one
missing and unfortunately the missing one has the highest
prio it will take that one, then the other will not paged in as long as the
missing one is generating, but it should be loaded all others,
and once the missing once is generated it will immedently paged in. like the
http request, if it's in the cache then we can load it without
downloading, directly read in.
so may we should change the mechansime, its not really general implemented:
you assume that we have only HTTP or NON HTTP, non http is no problem there
is only files. also cached ones and so.
but if we don't have http:// starting file names, you assume that it's a
file. but may it's a database request code for the tile gerneration
instead of a real existing file name, if we have such information, then we
should be able to check the "filename"/ "request" not only
against http:// starting string may we can append a user request qeueu and a
user thread to pregenerate / preread the files. we should only
have a method
itr : Iterate over all QUEUEandThread (non http, http, user ) {
if ( itr->requestForThisQueue(databaseRequest->_fileName)) {
itr->queue
}
}
Class QUEUEandThread {
DatabaseThread _thread;
RequestQueue _queue;
bool requestForThisQueue(std::string reqStr) {
// if HTTP queue is
return osgDB::containsServerAddress(reqStr)
}
}
so we would be able to append user requests queues very fast and the code
will be more general for future task.
/adrian
2008/11/12 Robert Osfield <[EMAIL PROTECTED]>
> Hi Adrian,
>
> On Wed, Nov 12, 2008 at 6:39 PM, Adrian Egli OpenSceneGraph (3D)
> <[EMAIL PROTECTED]> wrote:
> > Ok i see, so you would append a custum reading thread to the osg database
> to
> > avoid subclassing.
>
> There shouldn't be any need for a custom reading thread, just use the
> existing threads to call you code via the osgDB::readNodeFile
> mechanism - just put your code in a plugin and let the OSG call it.
>
> Robert.
> ___
> osg-users mailing list
> osg-users@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>
--
Adrian Egli
___
osg-users mailing list
osg-users@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi Adrian, On Wed, Nov 12, 2008 at 6:39 PM, Adrian Egli OpenSceneGraph (3D) <[EMAIL PROTECTED]> wrote: > Ok i see, so you would append a custum reading thread to the osg database to > avoid subclassing. There shouldn't be any need for a custom reading thread, just use the existing threads to call you code via the osgDB::readNodeFile mechanism - just put your code in a plugin and let the OSG call it. Robert. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Ok i see, so you would append a custum reading thread to the osg database to avoid subclassing. /adrian 2008/11/12 Robert Osfield <[EMAIL PROTECTED]> > Hi Adrian, > > Personally I'd write the code to retrieve the tiles as an OSG plugin > or using a Registry::ReadFileCallback. > > The pager now support multiple database reading threads to the latency > between request and when it's loaded needn't be long. This would > simplify your code significantly and would avoid the need for > subclassing of the DatabasePager. > > Robert. > > On Wed, Nov 12, 2008 at 4:06 PM, Adrian Egli OpenSceneGraph (3D) > <[EMAIL PROTECTED]> wrote: > > Hi all , > > > > One of our application is working with a derrived class of database > pager. > > Our class has just over written > > ::requestNodeFile(const std::string& inFileName,osg::Group* group, float > > priority, const osg::FrameStamp* framestamp,osg::ref_ptr > > &databaseRequest) > > method. There we get a request form osgDB core (osgUtil::cull visitor) > and > > we generate a event to generate the right tile from a database. the event > > gets handled in a second thread, once he has finish > > generating the right tile, we insert the information into a list. once a > > requested tile is in a liste we can make use of the common database pager > > requestnodefile method. so we just call > > > DatabasePager::requestNodeFile(inFileName,group,priority,framestamp,databaseRequest); > > so it seams to work, but unfortunatelly there are > > in some cases missing requestnodefile calls, means the database pager > should > > still have such missing tiles, but i don't get right informed, or the > > database pager doesn't page in the tiles, may there is > > something wrong with the framestamp. or is there something implement to > give > > a maximum of time to page in a tile. ? after such a time slot there will > no > > longer paged in the requested tile. ? > > > > adrian > > > > -- > > > > Adrian Egli > > > > ___ > > osg-users mailing list > > osg-users@lists.openscenegraph.org > > > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > > > > > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > -- Adrian Egli ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi Adrian, Personally I'd write the code to retrieve the tiles as an OSG plugin or using a Registry::ReadFileCallback. The pager now support multiple database reading threads to the latency between request and when it's loaded needn't be long. This would simplify your code significantly and would avoid the need for subclassing of the DatabasePager. Robert. On Wed, Nov 12, 2008 at 4:06 PM, Adrian Egli OpenSceneGraph (3D) <[EMAIL PROTECTED]> wrote: > Hi all , > > One of our application is working with a derrived class of database pager. > Our class has just over written > ::requestNodeFile(const std::string& inFileName,osg::Group* group, float > priority, const osg::FrameStamp* framestamp,osg::ref_ptr > &databaseRequest) > method. There we get a request form osgDB core (osgUtil::cull visitor) and > we generate a event to generate the right tile from a database. the event > gets handled in a second thread, once he has finish > generating the right tile, we insert the information into a list. once a > requested tile is in a liste we can make use of the common database pager > requestnodefile method. so we just call > DatabasePager::requestNodeFile(inFileName,group,priority,framestamp,databaseRequest); > so it seams to work, but unfortunatelly there are > in some cases missing requestnodefile calls, means the database pager should > still have such missing tiles, but i don't get right informed, or the > database pager doesn't page in the tiles, may there is > something wrong with the framestamp. or is there something implement to give > a maximum of time to page in a tile. ? after such a time slot there will no > longer paged in the requested tile. ? > > adrian > > -- > > Adrian Egli > > ___ > osg-users mailing list > osg-users@lists.openscenegraph.org > http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org > > ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] DatabasePager question ...
Hi, the osgDB::DatabasePager discards a database request if its frameNumber is earlier than the current frameNumber. Your requestNodeFile method has to continuously call osgDB::requestNodeFile even if the tile has already been requested. Adrian Egli OpenSceneGraph (3D) wrote: Hi all , One of our application is working with a derrived class of database pager. Our class has just over written ::requestNodeFile(const std::string& inFileName,osg::Group* group, float priority, const osg::FrameStamp* framestamp,osg::ref_ptr &databaseRequest) method. There we get a request form osgDB core (osgUtil::cull visitor) and we generate a event to generate the right tile from a database. the event gets handled in a second thread, once he has finish generating the right tile, we insert the information into a list. once a requested tile is in a liste we can make use of the common database pager requestnodefile method. so we just call DatabasePager::requestNodeFile(inFileName,group,priority,framestamp,databaseRequest); so it seams to work, but unfortunatelly there are in some cases missing requestnodefile calls, means the database pager should still have such missing tiles, but i don't get right informed, or the database pager doesn't page in the tiles, may there is something wrong with the framestamp. or is there something implement to give a maximum of time to page in a tile. ? after such a time slot there will no longer paged in the requested tile. ? adrian -- Adrian Egli ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
[osg-users] DatabasePager question ...
Hi all , One of our application is working with a derrived class of database pager. Our class has just over written ::requestNodeFile(const std::string& inFileName,osg::Group* group, float priority, const osg::FrameStamp* framestamp,osg::ref_ptr &databaseRequest) method. There we get a request form osgDB core (osgUtil::cull visitor) and we generate a event to generate the right tile from a database. the event gets handled in a second thread, once he has finish generating the right tile, we insert the information into a list. once a requested tile is in a liste we can make use of the common database pager requestnodefile method. so we just call DatabasePager::requestNodeFile(inFileName,group,priority,framestamp,databaseRequest); so it seams to work, but unfortunatelly there are in some cases missing requestnodefile calls, means the database pager should still have such missing tiles, but i don't get right informed, or the database pager doesn't page in the tiles, may there is something wrong with the framestamp. or is there something implement to give a maximum of time to page in a tile. ? after such a time slot there will no longer paged in the requested tile. ? adrian -- Adrian Egli ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
