Re: [Flightgear-devel] OSG and multicore processor

[Mathias Fröhlich]

On Tuesday 07 November 2006 14:07, Didier Fabert wrote:
> hy,
>
> a reason to use OSG is a perf improvement with SMP. but i see nothing about
> it here.
> my processor (amd 4200 double core) occupation is always 50-55%. perhaps is
> it not done yet?
We do not yet make use of that. osg is a techonlogy enabler for multithreading 
but until we can make use of that, many things need to be cleaned up.
Past that we can make use of that.
... will announce that.

   Greetings

   Mathias

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Erik Hofman]

Curtis Olson wrote:
> On 11/7/06, *Ampere K. Hardraade* <[EMAIL PROTECTED] 
> > wrote:
> 
> Shouldn't there be three?  Or is FlightGear not getting inputs from
> the user when it hangs?
> 
> Well by my count: 1. main thread, 2. scenery paging thread, 3. weather 
> fetching thread.

4. OpenAL audio thread.

Erik

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Ampere K. Hardraade]

On Tuesday 07 November 2006 23:20, Curtis Olson wrote:
> Really!  We should go out of our way to find a workable non-threaded
> solution before we add new threads to the code.
One could always break FlightGear into sub-applications and let them 
communicate via UDP ports.

Ampere

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Curtis Olson]

On 11/7/06, Ampere K. Hardraade <[EMAIL PROTECTED]> wrote:
Shouldn't there be three?  Or is FlightGear not getting inputs from the userwhen it hangs?Well by my count: 1. main thread, 2. scenery paging thread, 3. weather fetching thread. 
My opinions on:Subtle bugs -- these get hidden anyway without the "help" of multiple threads.Seriously, and believe me when I say this.  Threads open the door to a whole new level of even more subtle, difficult, and hard to trace bugs than you would have otherwise.  I suspect you won't buy that, but it's true.  Threading bugs are nasty because subtle timing issues can produce code that works fine for hours or perhaps only bomb once every couple days.  This can be incredibly difficult to debug because it is nearly impossible to reproduce and the debug/edit/compile/run cycle can be hours or days for one iteration.  The tendency is often that if it works most of the time, and it's hard to find the problem, let's not worry about it and move on.  Pretty soon you have an application that has trouble running more than an hour or two without crashing due to a number of different bugs that have crept in.  And if you get to that point, you need the resources of, oh, let's say microsoft, to dig yourself out of that hole.  Not that I would pick on microsoft in this context for any particular reason.
Seriously, threads are really cool in a simple text book example, but in a complex real world application, danger, danger, danger! 
Consider one end, you have a code that takes up 0.5% of the processor timeupdating the property tree, and on the other hand, you have an externalapplication that is feeding off these properties to drive a hydraulic motion
platform.  [snip]#1 is completely out of the question, as someone could be killed.  So, #2 isthe only viable option.  Unfortunately, single thread and #2 don't mix,because alarms would be going up all the time.
Clearly there are cases where you can't proceed with out threads.  Threaded scenery paging is one, threaded metar fetching is another.  Neither consume a huge amount of CPU time over the course of the application, but each has specific reasons why it makes sense to put those in a thread.
But threading even these "simple" things has been extremely painful.  I've been intimately involved in that pain several times for each of these threads.Really!  We should go out of our way to find a workable non-threaded solution before we add new threads to the code.
If there is an abolute requirement from an architecture standpoint, then ok, let's talk about it.But if we just want to make use of our fancy multi-core CPU's, well then, hold on a second, what chunk of cpu intensive stuff are we planning to move over and run on another thread on another core?
My whole point is not that we should never do threads because we already do in a couple special cases.  But we just need to be really smart, really thoughtful, and really actually require a thread before we put ourself through the pain and a new order of magnitude of subtle bugs again.
Curt.-- Curtis Olson - University of Minnesota - FlightGear Projecthttp://baron.flightgear.org/~curt/  
http://www.humanfirst.umn.edu/  http://www.flightgear.orgUnique text: 2f585eeea02e2c79d7b1d8c4963bae2d
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Ampere K. Hardraade]

On Tuesday 07 November 2006 14:50, Durk Talsma wrote:
> With the advent of multicore CPU's I have actually been considering the
> possibilities of moving parts of the AI code to a separate thread. The
> problem with AI isn't so much that it uses huge amounts of CPU time, but
> that it needs it (at least in the current implementation) in spikes.
> Currently this results in quite considerable pauses (which can actually
> last up to a few seconds, or more). The two main offenders are:
> - AI model loading, due to disk access.
> - The AI network route tracing algorithm, which just needs to do a lot of
> searching.
>
> It would possibly be possible to keep the trace algorithm in a single
> thread. By limiting the number of computations per frame,  but I don't
> really see a clean way to implement that right away (especially not for the
> trace algorithm.
>
> Cheers,
> Durk
Why not make it runs as a stand-alone application on a remote machine, and 
distribute the AI information to every user who is logged on?  This would 
allow everyone on the multiplayer to see the same AI instances; and for those 
who are running FG on multiple computers and monitors, the same instance of 
clouds across all their screens?

Ampere

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Ampere K. Hardraade]

On Tuesday 07 November 2006 10:50, Curtis Olson wrote:
> I'm not saying we should not ever do threading in flightgear, indeed we
> have two sub threads along with the main program already.
Shouldn't there be three?  Or is FlightGear not getting inputs from the user 
when it hangs?

> My only point is we shouldn't thread something that takes 0.5% of the
> processor time of the main thread just because we think threads are cool.
I disagree.  Performance isn't everything.  More on this later.

> Threads impose a huge penalty in terms of complicating the code, hiding
> really subtle bugs, and maintaining threaded code over time just
> complicates this matter because new people come in and make changes when
> they don't fully understand all the subtle interrelationships (timing,
> functionality, etc) of the code.
My opinions on:
Subtle bugs -- these get hidden anyway without the "help" of multiple threads.

New people coming in who don't know what they are doing -- we get these with 
single thread application too.  People just "dive into the code" without any 
aid of documentation -- the documentation like those used by the mechanic who 
fixes your car -- the maintenance manuals.  And what do people who don't know 
what they are doing do?  They introduce subtle bugs.  And other people who 
don't know what they are doing create the so call "work-arounds" and hide 
these subtle bugs.  And the cycle goes on...

So, if the complexity of multithreading causes things to break constantly and 
whenever someone screws up, it isn't exactly a bad thing.

> So sure, we can add more threads, but there needs to be some really
> substantial justification for doing it, and if there isn't a significant
> performance based justification as part of that, then I will be very
> sceptical.
Some simulation require the stability of their update frequency.  With these, 
you can't have a process that interrupts and ocassionaly lengthens the length 
of one iteration.

Consider one end, you have a code that takes up 0.5% of the processor time 
updating the property tree, and on the other hand, you have an external 
application that is feeding off these properties to drive a hydraulic motion 
platform.  Now, suppose the user is in some maneuver that brings into view a 
very sophisicated scene, and due to the single thread, causes a momentary 
lockup of the renderer, and hence the lockup of the simulator.  The external 
application was used to seeing the property tree getting update once per 0.05 
second, but now, it sees ONE update of 1 seconds, and assumed that the 
acceleration as 100g instead of 5g.  So:

1) Should the hydraulic jacks try to simulate that 100g?
2) Should the safety system kicks in and shutdown the sim?

#1 is completely out of the question, as someone could be killed.  So, #2 is 
the only viable option.  Unfortunately, single thread and #2 don't mix, 
because alarms would be going up all the time.

As you can see, although putting that 0.5% in its own thread would make no 
performance gain, it would make a huge difference in terms of the realism of 
the simulation, and its impression on those engineers who are evaluating 
FlightGear.
>
> Regards,
>
> Curt.

Ampere

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[leee]

On Tuesday 07 November 2006 20:11, Curtis Olson wrote:
> On 11/7/06, leee <[EMAIL PROTECTED]> wrote:
> > I suspect that the graphics will prove to be the trickiest problem - it's
> > by
> > far the greatest resource user -
>
> Right, in all this we need to remember that in all our past benchmarking,
> the graphics themselves account for 90-95% of the workload.  OSG provides
> some opportunities to thread the graphics rendering itself, but beyond
> that, the remaining 5-10% of computational effort is pretty small in
> comparison. Further dividing that up into additional threads may not be
> such a huge win, unless as Durk says there are tasks that are "bursty" like
> metar fetching which has to block on network IO ... that makes a lot of
> sense to put into a separate thread, but does anyone realize the pain we
> wen through to debug that simple task?  And then does anyone recall that a
> subsequent patch broke it all again and we had to wind all the way through
> the process and figure it all out again.  Lot's of pain was endured by at
> least a couple developers to make that work.
>
> So yes, threading does indeed make sense in certain specific contexts, but
> we need to be really careful about what we do, exactly why we do, and what
> specific peformance benefits we *know* we will get by threading some new
> task.
>
> Curt.

One thing to remember when saying that the graphics accounts for 90-95% of the 
utilisation is that it doesn't necessarily follow that all the other 
subsystems get the resources that they require.  For example, I'd like to be 
able to run the autopilot pid controllers at much higher and more consistent 
rates than is possible now and even though the FDM rates are supposed to be 
guaranteed, in practice, it doesn't always seem to be the case.

Personally, I'm not a fan of 'threading' as a parallel-processing solution, 
not only because of the problems you mention, but also because it doesn't 
scale.

LeeE


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Douglas Campos]

i've found here (pentium D 805) :

okay, we lost 3d clouds and some shiny transparencies, but...

the frame rate, although lower, gave me a smoother play. no bigger
drops, like with the plib branch.

in general, it left the impression of a more robust fgfs

just my 3cents

(sorry for the bad english)

On 11/7/06, Lou Sanchez-Chopitea <[EMAIL PROTECTED]> wrote:
> Hi,
> Curtis Olson wrote:
>
> > On 11/7/06, *leee* <[EMAIL PROTECTED]
> > > wrote:
> >
> > I suspect that the graphics will prove to be the trickiest problem
> > - it's by
> > far the greatest resource user -
> >
> >
> >
> > Right, in all this we need to remember that in all our past
> > benchmarking, the graphics themselves account for 90-95% of the
> > workload.  OSG provides some opportunities to thread the graphics
> > rendering itself, but beyond that, the remaining 5-10% of
> > computational effort is pretty small in comparison.  Further dividing
> > that up into additional threads may not be such a huge win, unless as
> > Durk says there are tasks that are "bursty" like metar fetching which
> > has to block on network IO ... that makes a lot of sense to put into a
> > separate thread, but does anyone realize the pain we wen through to
> > debug that simple task?  And then does anyone recall that a subsequent
> > patch broke it all again and we had to wind all the way through the
> > process and figure it all out again.  Lot's of pain was endured by at
> > least a couple developers to make that work.
>
> That's OK, I foresee (and am currently feeling) more pain, since FG
> won't run on my system (dual Athlon system running Core 5). Hasn't ever
> worked even in Plib days. Working on the FG threading is something that
> interests me since I am looking at such things at work right now.
>
> > So yes, threading does indeed make sense in certain specific contexts,
> > but we need to be really careful about what we do, exactly why we do,
> > and what specific peformance benefits we *know* we will get by
> > threading some new task.
> >
> > Curt.
> > --
> > Curtis Olson - University of Minnesota - FlightGear Project
> > http://baron.flightgear.org/~curt/
> > 
> > http://www.humanfirst.umn.edu/  http://www.flightgear.org
> > Unique text: 2f585eeea02e2c79d7b1d8c4963bae2d
> >
> >
> >
> >-
> >Using Tomcat but need to do more? Need to support web services, security?
> >Get stuff done quickly with pre-integrated technology to make your job easier
> >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >
> >
> >
> >___
> >Flightgear-devel mailing list
> >Flightgear-devel@lists.sourceforge.net
> >https://lists.sourceforge.net/lists/listinfo/flightgear-devel
> >
> >
> Cheers
>
> Lou
>
>
>
> -
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ___
> Flightgear-devel mailing list
> Flightgear-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Lou Sanchez-Chopitea]

Hi,
Curtis Olson wrote:

> On 11/7/06, *leee* <[EMAIL PROTECTED] 
> > wrote:
>
> I suspect that the graphics will prove to be the trickiest problem
> - it's by
> far the greatest resource user - 
>
>
>
> Right, in all this we need to remember that in all our past 
> benchmarking, the graphics themselves account for 90-95% of the 
> workload.  OSG provides some opportunities to thread the graphics 
> rendering itself, but beyond that, the remaining 5-10% of 
> computational effort is pretty small in comparison.  Further dividing 
> that up into additional threads may not be such a huge win, unless as 
> Durk says there are tasks that are "bursty" like metar fetching which 
> has to block on network IO ... that makes a lot of sense to put into a 
> separate thread, but does anyone realize the pain we wen through to 
> debug that simple task?  And then does anyone recall that a subsequent 
> patch broke it all again and we had to wind all the way through the 
> process and figure it all out again.  Lot's of pain was endured by at 
> least a couple developers to make that work.

That's OK, I foresee (and am currently feeling) more pain, since FG 
won't run on my system (dual Athlon system running Core 5). Hasn't ever 
worked even in Plib days. Working on the FG threading is something that 
interests me since I am looking at such things at work right now.

> So yes, threading does indeed make sense in certain specific contexts, 
> but we need to be really careful about what we do, exactly why we do, 
> and what specific peformance benefits we *know* we will get by 
> threading some new task.
>
> Curt.
> -- 
> Curtis Olson - University of Minnesota - FlightGear Project
> http://baron.flightgear.org/~curt/ 
>    
> http://www.humanfirst.umn.edu/  http://www.flightgear.org
> Unique text: 2f585eeea02e2c79d7b1d8c4963bae2d
>
>
>
>-
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
>
>
>___
>Flightgear-devel mailing list
>Flightgear-devel@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>  
>
Cheers

Lou



-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Olaf Flebbe]

Hi,

> How are you profiling ?

I had a Intel VTune eval license in the past.

Olaf

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Frederic Bouvier]

Selon Olaf Flebbe :

> Norman Vine schrieb:
> > Curtis Olson writes:
> >> Right, in all this we need to remember that in all our past
> >> benchmarking, the graphics themselves account for 90-95% of
> >> the workload.
> >
> > I haven't profiled FGFS in a long time  however I guess that
> > is true if you consider ground intersection and database
> > paging part of the GFX code
>
> I just did a profile. OpenGL related stuff is about 20%, groundcache
> about 60%.

How are you profiling ?

-Fred

--
Frédéric Bouvier
http://frfoto.free.fr  Photo gallery - album photo
http://www.fotolia.fr/p/2278/partner/2278  Other photo gallery
http://fgsd.sourceforge.net/   FlightGear Scenery Designer

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Olaf Flebbe]

Hi,

> 
> a reason to use OSG is a perf improvement with SMP. but i see nothing about 
> it 
> here.
> my processor (amd 4200 double core) occupation is always 50-55%. perhaps is 
> it 
> not done yet? 

No, it is not possible at this time.

Olaf

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Olaf Flebbe]

Norman Vine schrieb:
> Curtis Olson writes:
>> Right, in all this we need to remember that in all our past 
>> benchmarking, the graphics themselves account for 90-95% of 
>> the workload.  
> 
> I haven't profiled FGFS in a long time  however I guess that 
> is true if you consider ground intersection and database 
> paging part of the GFX code

I just did a profile. OpenGL related stuff is about 20%, groundcache 
about 60%.

Olaf

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Norman Vine]

Curtis Olson writes:
> 
> Right, in all this we need to remember that in all our past 
> benchmarking, the graphics themselves account for 90-95% of 
> the workload.  

I haven't profiled FGFS in a long time  however I guess that 
is true if you consider ground intersection and database 
paging part of the GFX code

Cheers

Norman


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Curtis Olson]

On 11/7/06, leee <[EMAIL PROTECTED]> wrote:
I suspect that the graphics will prove to be the trickiest problem - it's byfar the greatest resource user - Right, in all this we need to remember that in all our past benchmarking, the graphics themselves account for 90-95% of the workload.  OSG provides some opportunities to thread the graphics rendering itself, but beyond that, the remaining 5-10% of computational effort is pretty small in comparison.  Further dividing that up into additional threads may not be such a huge win, unless as Durk says there are tasks that are "bursty" like metar fetching which has to block on network IO ... that makes a lot of sense to put into a separate thread, but does anyone realize the pain we wen through to debug that simple task?  And then does anyone recall that a subsequent patch broke it all again and we had to wind all the way through the process and figure it all out again.  Lot's of pain was endured by at least a couple developers to make that work.
So yes, threading does indeed make sense in certain specific contexts, but we need to be really careful about what we do, exactly why we do, and what specific peformance benefits we *know* we will get by threading some new task.
Curt.-- Curtis Olson - University of Minnesota - FlightGear Projecthttp://baron.flightgear.org/~curt/  
http://www.humanfirst.umn.edu/  http://www.flightgear.orgUnique text: 2f585eeea02e2c79d7b1d8c4963bae2d
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[leee]

On Tuesday 07 November 2006 15:39, Frederic Bouvier wrote:
> Quoting Peter Gervais :
> > I've used the ACE programming framework to implement threaded
> > applications in the past. There are considerable advantages to using such
> > as a framework to do threading. As well, it also supports message queues
> > which work very well with threads. It has all the paradigm required to
> > support threading, locking etc ...
>
> Beware : ACE is bigger than FGFS. OpenSceneGraph comes already with its
> threading library : OpenThreads, and FG already use threads with a simpler
> one
>
> : pthreads. I don't think we need to jump on another tool. We should first
>
> think about the architecture.
>
> -Fred
>
> --
> Frédéric Bouvier
> http://frfoto.free.fr  Photo gallery - album photo
> http://www.fotolia.fr/p/2278/partner/2278  Other photo gallery
> http://fgsd.sourceforge.net/   FlightGear Scenery Designer

In the long term I believe that FG will need to become capable of exploiting 
MPP systems and in this respect I am not just thinking in terms of a few 
multiple cores on simple smp systems but also of loosely coupled clusters, or 
even just networks of hetrogenous systems.

Simply making FG more 'threaded' wouldn't adress this and instead FG itself 
would need to be broken down into a number of discrete inter-communicating 
applications that could be run either on a single host system or across many.

As Fred says, we need to think about the architecture but at the same time we 
also need to consider how much data we need to shift about and how much 
bandwidth is likely to be available to ensure that the architecture is 
viable.

It sounds like an impossible task but in some respects we already have some 
parts of FG working this way e.g. terrasynch and Atlas.  I'd guess that some 
parts of FG could probably split out with very little trouble but there will 
be other functions that would need a lot of thought and work to be able to 
operate asynchronously.

I suspect that the graphics will prove to be the trickiest problem - it's by 
far the greatest resource user - but I wonder if we haven't already solved 
some of the problems in the multi-screen stuff, which is similar in some 
respects to box-rendering.  I dunno how much time is required for assembling 
and ordering the objects to be rendered compared with the final rendering of 
the entire scene - this would be pretty crucial.

...just stuff I've been thinking about for quite a while.

LeeE


-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Durk Talsma]

On Tuesday 07 November 2006 16:50, Curtis Olson wrote:
> Threads impose a huge penalty in terms of complicating the code, hiding
> really subtle bugs, and maintaining threaded code over time just
> complicates this matter because new people come in and make changes when
> they don't fully understand all the subtle interrelationships (timing,
> functionality, etc) of the code.
>
> So sure, we can add more threads, but there needs to be some really
> substantial justification for doing it, and if there isn't a significant
> performance based justification as part of that, then I will be very
> sceptical.
>

With the advent of multicore CPU's I have actually been considering the 
possibilities of moving parts of the AI code to a separate thread. The 
problem with AI isn't so much that it uses huge amounts of CPU time, but that 
it needs it (at least in the current implementation) in spikes. Currently 
this results in quite considerable pauses (which can actually last up to a 
few seconds, or more). The two main offenders are:
- AI model loading, due to disk access.
- The AI network route tracing algorithm, which just needs to do a lot of 
searching. 

It would possibly be possible to keep the trace algorithm in a single thread. 
By limiting the number of computations per frame,  but I don't really see a 
clean way to implement that right away (especially not for the trace 
algorithm.

Cheers,
Durk

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Didier Fabert]

Le mardi 7 novembre 2006 16:39, Frederic Bouvier a écrit :
> Beware : ACE is bigger than FGFS. OpenSceneGraph comes already with its
> threading library : OpenThreads, and FG already use threads with a simpler
> one
>
> : pthreads. I don't think we need to jump on another tool. We should first
>
> think about the architecture.
>
> -Fred

don't forget:
- two years ago, we didn't sure that 64bits will be alive today.
- today we're not sure that multi-core are the future

the processor manufacturers explore many ways, it's not sure the multi-core is 
the future. perhaps, it exist just because of technology problem which can be 
solved after.

then we can thinking about but just thinking.
-- 
Didier Fabert
[EMAIL PROTECTED]
KFreeFlight project : A FlightGear GUI-Frontend designed for KDE users
http://kfreeflight.sourceforge.net

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Curtis Olson]

On 11/7/06, Frederic Bouvier <[EMAIL PROTECTED]> wrote:
Ok, but you have to acknowledge that the trend is to multiply the number ofcores that are possibly less porwerful on their own. Look the frequencies : 2years ago, you had one core clocked over 3Ghz, now 2 cores are clocked at about
2.4 / 2.6 Ghz, and both Intel and AMD are announcing/releasing Quad core thesedays.So, if we don't want FG not using more than 50% CPU, and perhaps not more than25% further in time, we will have to go to that direction.
I'm not saying we should not ever do threading in flightgear, indeed we have two sub threads along with the main program already.My only point is we shouldn't thread something that takes 
0.5% of the processor time of the main thread just because we think threads are cool.Threads impose a huge penalty in terms of complicating the code, hiding really subtle bugs, and maintaining threaded code over time just complicates this matter because new people come in and make changes when they don't fully understand all the subtle interrelationships (timing, functionality, etc) of the code.
So sure, we can add more threads, but there needs to be some really substantial justification for doing it, and if there isn't a significant performance based justification as part of that, then I will be very sceptical.
Regards,Curt.-- Curtis Olson - University of Minnesota - FlightGear Projecthttp://baron.flightgear.org/~curt/  
http://www.humanfirst.umn.edu/  http://www.flightgear.orgUnique text: 2f585eeea02e2c79d7b1d8c4963bae2d
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Frederic Bouvier]

Quoting Peter Gervais :

> I've used the ACE programming framework to implement threaded applications in
> the past. There are considerable advantages to using such as a framework to
> do threading. As well, it also supports message queues which work very well
> with threads. It has all the paradigm required to support threading, locking
> etc ...

Beware : ACE is bigger than FGFS. OpenSceneGraph comes already with its
threading library : OpenThreads, and FG already use threads with a simpler one
: pthreads. I don't think we need to jump on another tool. We should first
think about the architecture.

-Fred

--
Frédéric Bouvier
http://frfoto.free.fr  Photo gallery - album photo
http://www.fotolia.fr/p/2278/partner/2278  Other photo gallery
http://fgsd.sourceforge.net/   FlightGear Scenery Designer

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Peter Gervais]

I've used the ACE programming framework to 
implement threaded applications in the past. There are considerable advantages 
to using such as a framework to do threading. As well, it also supports message 
queues which work very well with threads. It has all the paradigm required to 
support threading, locking etc ... 
 

  - Original Message - 
  From: 
  Curtis 
  Olson 
  To: FlightGear developers 
  discussions 
  Sent: Tuesday, November 07, 2006 9:01 
  AM
  Subject: Re: [Flightgear-devel] OSG and 
  multicore processor
  On 11/7/06, Frederic Bouvier 
  <[EMAIL PROTECTED]> wrote:
  
  FGFS 
doesn't use threading much at that time. Only for the scenery loader 
andmetar. One can thing of putting the FDM or AI in other threads, or 
maybe evendivide the viewport into small areas that could be rendered in 
parallel. I don't know if its doable, but for now, you will only see an 
improvement runningmultiple programs in parallel. As you already see, 
Atlas or Terrasync can run ontheir own core and not disturb 
  FG.
  In FlightGear we intentionally minimize the use of threading to 
  places where there is a huge need.Adding a thread significantly 
  increases the complexity of the code and can hide a large variety of execution 
  order and timing problems which can be incredibly difficult to debug and fix. 
  If we are ever add additional threads, it will be only 
  because we are forced to by clear and overwhelming performance considerations 
  (and not for design or preference reasons.) 
  Regards,Curt.-- Curtis Olson - University of Minnesota 
  - FlightGear Projecthttp://baron.flightgear.org/~curt/  http://www.humanfirst.umn.edu/ 
    http://www.flightgear.orgUnique text: 
  2f585eeea02e2c79d7b1d8c4963bae2d 
  
  

  -Using 
  Tomcat but need to do more? Need to support web services, security?Get 
  stuff done quickly with pre-integrated technology to make your job 
  easierDownload IBM WebSphere Application Server v.1.0.1 based on Apache 
  Geronimohttp://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
  
  

  ___Flightgear-devel 
  mailing 
  listFlightgear-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/flightgear-devel
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Didier Fabert]

Le mardi 7 novembre 2006 15:24, Frederic Bouvier a écrit :
> Ok, but you have to acknowledge that the trend is to multiply the number of
> cores that are possibly less porwerful on their own. Look the frequencies :
> 2 years ago, you had one core clocked over 3Ghz, now 2 cores are clocked at
> about 2.4 / 2.6 Ghz, and both Intel and AMD are announcing/releasing Quad
> core these days.
>
> So, if we don't want FG not using more than 50% CPU, and perhaps not more
> than 25% further in time, we will have to go to that direction.
>
> -Fred

it's brand new that two-core processor are cheap. and quad-core is not for 
tomorrow (even they exists, they will be too expensive when they will be 
available). then it's not urgent. but it's the future.

i saw a pentium4 which have two years old, running at 3 GHz, but it was 
nitrogen liquid cooling :-)
-- 
Didier Fabert
[EMAIL PROTECTED]
KFreeFlight project : A FlightGear GUI-Frontend designed for KDE users
http://kfreeflight.sourceforge.net

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Frederic Bouvier]

Quoting Curtis Olson:

> On 11/7/06, Frederic Bouvier <[EMAIL PROTECTED]> wrote:
>
> > FGFS doesn't use threading much at that time. Only for the scenery loader
> > and
> > metar. One can thing of putting the FDM or AI in other threads, or maybe
> > even
> > divide the viewport into small areas that could be rendered in parallel. I
> > don't know if its doable, but for now, you will only see an improvement
> > running
> > multiple programs in parallel. As you already see, Atlas or Terrasync can
> > run on
> > their own core and not disturb FG.
>
>
>
> In FlightGear we intentionally minimize the use of threading to places where
> there is a huge need.
>
> Adding a thread significantly increases the complexity of the code and can
> hide a large variety of execution order and timing problems which can be
> incredibly difficult to debug and fix.
>
> If we are ever add additional threads, it will be only because we are forced
> to by clear and overwhelming performance considerations (and not for design
> or preference reasons.)

Ok, but you have to acknowledge that the trend is to multiply the number of
cores that are possibly less porwerful on their own. Look the frequencies : 2
years ago, you had one core clocked over 3Ghz, now 2 cores are clocked at about
2.4 / 2.6 Ghz, and both Intel and AMD are announcing/releasing Quad core these
days.

So, if we don't want FG not using more than 50% CPU, and perhaps not more than
25% further in time, we will have to go to that direction.

-Fred

--
Frédéric Bouvier
http://frfoto.free.fr  Photo gallery - album photo
http://www.fotolia.fr/p/2278/partner/2278  Other photo gallery
http://fgsd.sourceforge.net/   FlightGear Scenery Designer

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Douglas Campos]

IMHO, full threaded fgfs would be nice at 2.0beta release ;)


On 11/7/06, Curtis Olson <[EMAIL PROTECTED]> wrote:
> On 11/7/06, Frederic Bouvier <[EMAIL PROTECTED]> wrote:
>
> > FGFS doesn't use threading much at that time. Only for the scenery loader
> and
> > metar. One can thing of putting the FDM or AI in other threads, or maybe
> even
> > divide the viewport into small areas that could be rendered in parallel. I
> > don't know if its doable, but for now, you will only see an improvement
> running
> > multiple programs in parallel. As you already see, Atlas or Terrasync can
> run on
> > their own core and not disturb FG.
>
>
> In FlightGear we intentionally minimize the use of threading to places where
> there is a huge need.
>
> Adding a thread significantly increases the complexity of the code and can
> hide a large variety of execution order and timing problems which can be
> incredibly difficult to debug and fix.
>
> If we are ever add additional threads, it will be only because we are forced
> to by clear and overwhelming performance considerations (and not for design
> or preference reasons.)
>
> Regards,
>
>  Curt.
> --
> Curtis Olson - University of Minnesota - FlightGear Project
> http://baron.flightgear.org/~curt/
> http://www.humanfirst.umn.edu/   http://www.flightgear.org
> Unique text: 2f585eeea02e2c79d7b1d8c4963bae2d
> -
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> ___
> Flightgear-devel mailing list
> Flightgear-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>
>
>

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Curtis Olson]

On 11/7/06, Frederic Bouvier <[EMAIL PROTECTED]> wrote:
FGFS doesn't use threading much at that time. Only for the scenery loader andmetar. One can thing of putting the FDM or AI in other threads, or maybe evendivide the viewport into small areas that could be rendered in parallel. I
don't know if its doable, but for now, you will only see an improvement runningmultiple programs in parallel. As you already see, Atlas or Terrasync can run ontheir own core and not disturb FG.
In FlightGear we intentionally minimize the use of threading to places where there is a huge need.Adding a thread significantly increases the complexity of the code and can hide a large variety of execution order and timing problems which can be incredibly difficult to debug and fix.
If we are ever add additional threads, it will be only because we are forced to by clear and overwhelming performance considerations (and not for design or preference reasons.) Regards,
Curt.-- Curtis Olson - University of Minnesota - FlightGear Projecthttp://baron.flightgear.org/~curt/  http://www.humanfirst.umn.edu/
  http://www.flightgear.orgUnique text: 2f585eeea02e2c79d7b1d8c4963bae2d
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel

Re: [Flightgear-devel] OSG and multicore processor

[Frederic Bouvier]

Quoting Didier Fabert :

> hy,
>
> a reason to use OSG is a perf improvement with SMP. but i see nothing about
> it
> here.
> my processor (amd 4200 double core) occupation is always 50-55%. perhaps is
> it
> not done yet?
>
> on the other hand, my processor let me use Atlas without reduce fgfs perf.
> ps: fgfs is compiled with the "--with-threads" option

FGFS doesn't use threading much at that time. Only for the scenery loader and
metar. One can thing of putting the FDM or AI in other threads, or maybe even
divide the viewport into small areas that could be rendered in parallel. I
don't know if its doable, but for now, you will only see an improvement running
multiple programs in parallel. As you already see, Atlas or Terrasync can run on
their own core and not disturb FG.

-Fred

--
Frédéric Bouvier
http://frfoto.free.fr  Photo gallery - album photo
http://www.fotolia.fr/p/2278/partner/2278  Other photo gallery
http://fgsd.sourceforge.net/   FlightGear Scenery Designer

-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel