Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-12-03 Thread Robert Osfield
Hi Torben, I don't know the cause of the failure. Properly set up system(s) shouldn't fail. One thing you have to be careful of that you're file system will need to be able to handle the async writes robustly. Robert. On Wed, Dec 2, 2009 at 9:50 PM, Torben Dannhauer z...@saguaro-fight-club.de

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-12-03 Thread Torben Dannhauer
Hi Robert, Hmm difficult to determine the responsible part which causes the failure. My last try failed becuase KDE plasma-desktop crashed. But the system was under heavy load an got slower and slower. Because all SATA Ports are busy by source disks, I used an external USB drive to store the

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-12-03 Thread Robert Osfield
Hi Torben, On Thu, Dec 3, 2009 at 12:41 PM, Torben Dannhauer z...@saguaro-fight-club.de wrote: Hi Robert, Hmm difficult to determine the responsible part which causes the failure. My last try failed becuase KDE plasma-desktop crashed. But the  system was under heavy load an got slower and

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-12-02 Thread Torben Dannhauer
Hi, Well, 64 bit is besser, it fails after 31000 tasks - but it fails.. But fortunately resuming in VPB is now possible. I Posted it to the VPB wiki unser usage exampes http://www.openscenegraph.org/projects/VirtualPlanetBuilder/wiki/Resume @Mod: Please close this Thread.. Thank you!

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-12-02 Thread J.P. Delport
Hi Torben, thanks for taking the time to put a page on the wiki. rgds jp Torben Dannhauer wrote: Hi, Well, 64 bit is besser, it fails after 31000 tasks - but it fails.. But fortunately resuming in VPB is now possible. I Posted it to the VPB wiki unser usage exampes

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-11-26 Thread Torben Dannhauer
Hi, Now an update to my compiling Problem: I have new hardware ( Nehalem architecture ) I installed a fresh system: 64 Bit Kubuntu 9.10 Karmic Koala with OSG 2.8.2 and the corresponding VPB. In constrast to my last setup, local rendering with all source data does not fail. Last night I

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-06 Thread Torben Dannhauer
Hi Robert, at the moment I dont Use a Cluster, I would be happy if i could compile the large dateset one with the local 8 cores (I tried tu run it localle with 2 up to 8 cores) For smaller Datebases I used SSHFS as lightweight cluster filesystem, it worked great. Because running the osgdem

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-06 Thread Robert Osfield
Hi Torben, On Tue, Oct 6, 2009 at 1:31 PM, Torben Dannhauer z...@saguaro-fight-club.de wrote: Some last important questions: Which Kubuntu version do you use? Is it 32 or 64 bit? I use Kubuntu 9.04 64bit right now. I've used 32bit and 64bit version of Kubuntu for the last two year during the

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-06 Thread Pierre Bourdin (gmail)
Hi all, Le mardi 06 octobre 2009 à 12:31 +, Torben Dannhauer a écrit : Hi Robert, at the moment I dont Use a Cluster, I would be happy if i could compile the large dateset one with the local 8 cores (I tried tu run it localle with 2 up to 8 cores) For smaller Datebases I used

[osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Torben Dannhauer
Hi, I try to generate a big (750 GB) database with VPB 0.9.10 OSG 2.8.2 under kubuntu 8.04 (32 bit). Unfortunately it seems to be too big, even Linux crashes (windows crashed even in quite small databases, therefor I use only Linux for Database generation). The crash it self is a failure of

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Robert Osfield
Hi Torben, To work out what is causing the failures have a look at the logs/*.log and the task/*.task files for the build, look for warnings and error messages. The status line in the .task file should say completed when the task has been successfully completed, and pending if it's still in the

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Torben Dannhauer
Hi Robert, checking 7 log files and 7 task files manually is a lot of work ;) my syslog told my that osgdem crashed multiple, sometimes caused bei libxbc and sometimes by gdal. I'll try got get any more detailed error message. what was your biggest successfvul rendered database?

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Robert Osfield
Hi Torben, On Mon, Oct 5, 2009 at 3:27 PM, Torben Dannhauer z...@saguaro-fight-club.de wrote: checking 7 log files and 7 task files manually is a lot of work ;) my syslog told my that osgdem crashed multiple, sometimes caused bei libxbc and sometimes by gdal. I'll try got get any more

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Torben Dannhauer
Hi Robert, i've restartet the build process to get the error messages. For example: The logfile of the task contains: Code: 0.011: Adding terrainTile 0.339: DataSet::_run() 0 0 3.161: started DataSet::createDestination(13) 3.236: Time for after_reproject

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Robert Osfield
Hi Torben, If you can isolate a specifc task that fails you should be able to re-run that osgdem task on it own using the command line options list in the task file. If running osgdem crashes then the next step would be to build the OSG + VPB as a debug and then run osgdemd in gdb or another

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Torben Dannhauer
Hi Robert, executing the taskfiles manually works without crash. If I would write a script which analyses all taskfiles regarding their pending status, and execute that taskfile if it's status is pending - would it result in a usable database? Because osgdem itselfs works properly I assume

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Robert Osfield
Hi Torben, Have a look at how much memory each individual task is requiring as it runs, it's memory use will grow through to when it builds the last level into the quad tree/pyramid that it's handling. It could be that one task takes up lots of memory, but runs fine, but run multiple osgdem in a

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Torben Dannhauer
Hi Robert, my diskdrive is mounted with default settings, which includes async. vpbmaster calls up to 250 Mb, seems to be OK. The usage of an osgdem call is not pssible, because ALL osgdem calls fails, so no one lives long enought to get his memory consumption. re-running vpbmaster with the

Re: [osg-users] [vpb] lots of failed tasks: Resuming vpbmaster run

2009-10-05 Thread Robert Osfield
Hi Torben, On Mon, Oct 5, 2009 at 6:18 PM, Torben Dannhauer z...@saguaro-fight-club.de wrote: my diskdrive is mounted with default settings, which includes async. I can't recall the exact settings off the top of my head (it's a year since I last looked into the specific topic of NFS), so you'll