On Tuesday 04 Aug 2009, Curtis Olson wrote: > On Tue, Aug 4, 2009 at 11:05 AM, leee <l...@spatial.plus.com> wrote: > > That's interesting. Could you elaborate on that a little more > > i.e. did you split a single scene into 'render boxes' or were > > you, in effect, running four discrete but 'collaborative' > > instances, each just looking in a different direction? > > This was a single instance of FlightGear rendering to 8 separate > windows.
Ok - ta. > > What proportion of the drawing task is occupied with assembling > > the scene objects, compared with passing the assembled data to > > the video card for display? (I'm just wondering about the > > scope for box-rendering here) > > I'm not sure I fully understand your question, but a lot of this > is dependent on how much of the opengl pipeline is implemented in > hardware versus libraries/drivers on a particular video card and > operating system. (Quality and efficiency of the actual > implementation also factors in here.) There can be huge > variability here across systems. Fair enough. To be honest, the question was at the limits of my understanding. What inspired it though is that when I'm rendering any of my 3D stuff the rendering process is distributed across several systems - the single scene is split into many separate boxes, and because the performance of the different systems in my render farm varies widely, I typically split the scene up into 20x10 boxes. Now this is all ray-traced software rendering, not hardware rendering, and there is an additional overhead because the scene needs to be split up and the subsequent results combined, but the bottom line is that this technique can allow much more processing power to be used and certainly enough to compensate for the overhead. I've honestly no idea though, how far, or even if this technique can be applied to h/w rendering. >> One of the big problems I had with FG was its pseudo asynchronous > > operation, which still meant that the rates at which you could > > run things like the FDM, autopilot and Nasal were effectively > > limited by the frame rate and which could lead to an aircraft > > being stable on one system but unstable on another. I would > > really like to see these subsystems running on their own cores, > > or even more preferably, on their own networked hardware, so > > that I could run them at much higher and guaranteed rates. > > > > I think the bottom line is that FG is simply going to have to > > face up to this issue at some point. > > What you are referring to here is less of a threaded system and > more of a real time system. You want guarantees that a function > will execute at a specific fixed rate. Threads don't > automatically give you that, in fact they don't give you that at > all. You need a real time system with some sort of external > timmer trigger and an interrupt service routine to get really > tight timings. > > Personally I've seen several threaded applications that *thought* > they were getting their routines to run at a particular rate > until they dug in and actually timed what was going on and > throught a bit more about how their code was structured and what > the thread primatives they were using actually offered. I'm not really thinking in terms of 'threading' at all, which I think is a very limited and half-house sort of technique. But neither though do I think it needs to be thought of as a pure real time system. Rather, I'm thinking in terms of the external FDM mechanism already present in FG. Running the FDM on it's own hardware system doesn't need to be any more real time than the FDM running within FG on the same system but because it's not going to be limited by the frame rate it could safely be run much faster and with proportionately more consistency than with FG. If you're running it at say 100Hz within FG I would expect to be able to run it several times faster, if not tens of times faster if the system it was running on wasn't spending most of its time rendering. You'll still get a variation in the rate that the FDM runs at but I suspect that the variation would be about the same in absolute terms. Let's say that if we get a variation of +/- 10 iteration difference per second running within FG it would probably be about the same running on its own system, but as we're running at a higher rate the difference is proprotionally smaller, perhaps down from 10% to 1%. > > A few people here have been > > bringing this topic up for a few years and now that multicore > > processors are the norm it's clear that the issue isn't going > > to go away. Like it or not, and I mean no offence or criticism > > by this, the current FG architecture is now obsolete. > > The only point I would make here is that threading and timing is > an extremely snakey area to deal with. Most people I've talked > to don't understand the nuances of what's really going on nearly > as well as they imagine they do. I certainly admit to gaps and > weaknesses in my own understanding. And I certainly don't mean > to imply that anyone participating in this discussion falls under > that same description. > > Here's my question to you: > > Once you've split the flight dynamics (for example) off into a > separate thread/process running on it's own dedicated core, what > mechanism do you propose to use to force it to run at a specific > fixed and guaranteed rate? And this is an honest question, > bucause the generic cross-platform timing mechanisms I'm aware > have few guarantees. You might have the best luck with a > busy/wait type architecture, but then you are needlessly burning > CPU most of the time, since the flight dynamics will consume > maybe 0.0001% of the dedicated CPU. Or do you somehow setup a > counter and a compare register and use that to trigger an > interrupt at a fixed interval and run an iteration of the flight > dynamics out of the interrupt service routine? But can you do > that in a cross-platform independent way? Can you even get > access to these low level mechanisms from most modern operating > systems? Like I say, I don't think we need to achieve strict real time processing here, but we could achieve both higher rates and proportionally smaller variations in those rates using the existing timing techniques in FG. >> While single core and single processor systems were the norm it >> was fine - the software design was well fitted to the systems on >> which it ran - but parallel processing has always been the way >> things were going to go. It has been inevitable ever since >> super-computer designers realised that the only way that ever >> increasing performance could be achieved was by parallelism and >> now it's well and truly on the desktop. > > And the big challenge with modern super computer clusters is to > find the parallelism in your task (as much that exists) or to > redesign the task to improve the parallelism. (Well that and how > to keep 2000 8-core machines running at the same time in a closet > from reaching temperatures so hot they melt their way through to > the earth's core.) But you are faced with really fast processors > and in comparison, very slow interconnects between nodes ... even > with fast/low latency networks like myrinet and infiniband. I absolutely agree with you - it's finding the parallelism that's the hardest part but things like the FDM, autopilot and Nasal do seem like obvious candidates. Even if we can't precisely balance the load, we could still improve the performance of parts of FG. Sure, processors are getting faster, if not by increases in clock speed, which has largely reached or are rapidly approaching a plateau, then by improving performance per clock and VLIW, but that too can only be taken so far. Barring quantum computing, which is still questionable, parallelism is really the only way to go. >> Of course, it's not going to be easy, but denying it won't make >> the issue go away. > > What I have always hoped to communicate is that we need solid > justification before adding more complex threading. There is a > huge downside in terms of complexity and extremely hard to find > bugs so there needs to be an equally strong upside to justify the > change. There's nothing wrong with nebulous observations about > the direction that computer technology is moving. But what I > want to hear is some specifics. What specific change is proposed > and how specifically will that actually work, what specifically > will be improved, and how will that improvement manifest itself? > > I've seen a couple misguided thread based applications in my life > with a large gap between actual reality and programmer wished-for > reality. My only purpose here is to make sure whatever is > proposed does indeed achieve the actual goals (and pin people > down on what those goals really are). Threading again ;-) best avoided in my opinion, pretty much for the reasons you give. Instead of thinking of FG as a single threaded application, it needs to be a collection of standalone programs that run collaboratively - go back to thinking in terms of the external FDM option. >> Tied in with this, I think, is that FG is still very much a >> developer-led project, so the developers, by and large, only > > work on things that interest them. However, the FG user base > > is constantly growing, and will continue to grow, and at some > > point FG will have to become user-led if it is to become more > > than just a stage for its developers to perform upon. FG will, > > if it has not already, reach the point where it is bigger than > > its developers and the project should dictate what the > > developers do, rather than the other way around. > > > > I think its a sad fact of reality but I think that FG has > > reached the point where FG development needs to be actively > > managed and directed, and not just left to the whims and > > desires of its developers. > > I don't disagree with any of this, but as soon as we start asking > people to no longer act as willing volunteers, we need to start > developing a financial model of compensation and that implies > income and sales (donations?) in enough quantity to properly > compensate the non-volunteers. There are some nibbles in this > area, but progress is slow. When you start considering the > amounts of money required to compensate a few people for some > non-trivial amounts of time ... people don't part with those > kinds of resources without a *lot* of thought, a *lot* of > investigation, a *lot* of soul searching ... or we could get a > gov't grant ... these days that money gets scattered around in a > pretty hapahazard way (at least by external observation from > someone who doesn't know all the details of who knows who...) > Anyone have a relative that's well placed? :-) Well, I don't see it as asking people to no longer act as willing volunteers but rather asking people to volunteer to work on specific problems or issues. Sure, some people will only be interested in implementing new 'cool' features, but others will know that there is a degree of responsibility attached to being allowed to perform on the FG stage. It would be a very sad thing if an FG developer abandoned FG in a sulk because they couldn't do exactly what they wanted and nothing else - that's just take with no give - and in any case, the FG developers don't seem to be so selfish and small minded. If there is a real need, and I believe that there is, I'd like to think the the FG developers are mature enough to accept it. Of course, money and sponsorship could help, but it doesn't mean that it's impossible any other way. > > > Like I said, I mean no criticism of the developers by this; > > they have achieved an immense amount by getting FG to this > > point, but in getting FG this far they've made FG into a bigger > > thing that needs handling differently if it isn't going to > > stagnate and be left behind. > > I definitely appreciate the comments and perspective. > > Curt. I've got an awful lot of fun and satisfaction from FG but I do think it has some problems that it's having trouble facing up to. I want to see FG getting better and better, both in terms of features and quality but I can't see it happening without facing up to those problems. LeeE ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
