Re: [Flightgear-devel] Sea color
> With your scientific background you should know better than > cherry-picking that statistic. Those on the forum are a self-selected > minority of our users. (...) > No I'm most definitely not. What I'm saying is that by "optimising" for a > subset of users, you run the risk of sub-optimising for the rest. *sigh* Did you read what I tried to tell you in my last mail? Without optimization, the code would run in single digit framerates. So it has to be optimized, it's not a question of if, just of how. The only system for which I can do this is my own, because I don't have data for any other system. If I get good solid data from your side, I can include that into the optimization. But please remember - we're talking optimizing the last 5% or so here. In brute force mode, it would never ever run fast enough. Generating clouds on a one per frame basis for instance is not an option - it's way too slow. Doing all administrative tasks every frame is not an option - this generates way too much load. > I think Advanced Weather is good. I'd like the opportunity to exploit my > (fairly - it's getting a bit long in the tooth now) high end system to > enjoy it - not have to put up with an experience that is not as good overall > as > basic weather. And I'd like to support that. So what I need is data how your system responds to certain things, then I can understand what causes the problems, then I can hopefully fix them. What I don't need at this point is your theory what is wrong unless you understand sufficiently well why things are coded the way they are. I have a certain advantage here - I know very well what the individual loops do and how and where they burn performance. > Unfortunately, Advanced Weather can and does crash Fg here. I haven't > looked into it properly - but there's no obvious reason atm. Never happened to me. Also, I found it very difficult to crash Flightgear from Nasal in general - in all my time coding in Nasal, I have just discovered a single way of causing a segfault. So, I have to leave it to you to figure out the why, I can't reproduce it, and I have logged hundreds of flight hours with Advanced Weather on. > For a start we can try to make the Nasal better, I think that might help > a > bit. I spent an hour or so picking over your code. So far I've found: > > 88 declared but unused variables > 47 declarations of the same or similar variables > 427 instances of "else if" instead of "elsif" > 100 instances of I = I + 1 instead of i+=1 > Numerous examples of variables declared inside for loops, and some of > those > are inside other for loops > Variables declared inside condition statements. Nasal isn't C++ and variable declaration isn't the same thing for instance. If you want to speed up code, you have to go where performance is burnt. And that happens wherever you call add-cloud() or geodinfo() or setprop() or getprop(), i.e. the hard-coded stuff - that needs to be carefully distributed across frames for a smooth experience. What you do Nasal-internally is subleading and usually I couldn't ever measure a performance impact. As I said, I am currently going over the code and throwing out stuff which the transition to Stuart's cloud system has made obsolete and which are currently just de-activated by if-statements. That allows to streamline some other processes, because some situation can't occur any more, and that should take some load off the main loops and definitely reduce load for the GC if this is anywhere sensitive to code size. I would expect much more from that than from semantic details (I don't even know whether else if or elseif would be faster in Nasal - has that been tested? ). In the mean time, can we do the following: There are two issues: 1) framerate impact while a weather tile is built (that happens a lot initially as up to 9 tiles are built, and later depends on the speed of your aircraft - the Concorde triggers a new tile every 16 seconds, the C-172 takes a few minutes. 2) framerate impact and smoothness while no tile is built For the time being, I'd like to focus on 2) (because optimizing tile building is *really* tough and I don't even have an idea for a better algorithm which doesn't break in some situations) so you may have to live with stutter/framerate drop caused by tile-building for a while longer. Without tile-building, I'd like to aim at a worst frame duration of <33 ms, so we can have a smooth 30 fps at least (currently we can apparently have 50 ms). If more comes out, the better. To gauge the impact of various measures, I'd like to see framerate and worst frame duration readings without tile building going on. So you have to wait after starting Advanced Weather for about a minute till 9 tiles are built and the visibility has reached its target value and all terrain is loaded, then take the readings. If the log option in the GUI is on, the tile building info is written to the console, so you
Re: [Flightgear-devel] Sea color
Thorsten > > Conclusion: don't try to optimise, particularly for a poor system - > > you might make it better for that system, but more likely you will > > make it worse for everyone else. > > Judging by framerate comparisons with people in the forum, my system is > still somewhere in the upper third - many people have to live with less. > Judging by user requests and comments, more than 90% want higher > framerate out of the system by any means, you represent a minority of users > who would be willing to sacrifice framerate for smoothness. With your scientific background you should know better than cherry-picking that statistic. Those on the forum are a self-selected minority of our users. We have no idea what our users out there are using, neither hardware nor OS. We don't know if they have tried Advanced Weather and abandoned it, or put up with stuttering, or never tried it. Anecdotally, there are plenty of complaints about stuttering, and AFAIKS these are really only apparent with Advanced Weather or Concorde. > Most of us are happy to see Rembrandt or lightfields with anything > resembling 20 fps. So, just who is that 'everyone else'? And for whom do we > optimize? > > You're basically saying I should optimize things for you - but that would make > it worse for everyone else not running a high-end system. No I'm most definitely not. What I'm saying is that by "optimising" for a subset of users, you run the risk of sub-optimising for the rest. > > Right now, with Advanced Weather we have a weather simulator with a > > FlightSim attached. We're spending 10 (yes 10!) times as long in the > > Events Sub-module with Advanced Weather than in Basic, and 5 times as > > long as we spend in Flight. > > That'd probably be because Advanced Weather does ~10 times more > complex calculations than Basic weather... And I'm not surprised that it takes > more than flight either - flight is comparatively cheap, that ran with decent > accuracy 10 years ago. To compute a non-local environment (i.e. that knows > that conditions 'here' are different from conditions 'there') is quite a bit more > expensive and we could not do that 10 years ago. As for your comment, yes, > that's quite true - that's just how the problem is. Teaching thousands of > clouds where convection is, or how to flow over terrain obstacles is > expensive, even if you do it schematically. That's what's needed to give you > semi-realistic distributions of clouds. If you're happy with just clouds in the > sky, that's cheaper, but not what Advanced Weather is for. I think Advanced Weather is good. I'd like the opportunity to exploit my (fairly - it's getting a bit long in the tooth now) high end system to enjoy it - not have to put up with an experience that is not as good overall as basic weather. > > Writing data to the Property Tree is bad. This one is not evidence > > based > > That's evidence based (I have done some testing a while ago just how long it > takes) - it's currently minimized in my code, but the tree is the interface > between menu, C++, Shaders and Nasal, so ultimately some stuff has to be > written or read. > > > To be honest, I think there are design difficulties with Advanced > > Weather. > > There should only be one loop - the "update loop" - running at main > > frame rate which activates/deactivates the various sub-modules as > > required. I think I have done enough work here to demonstrate that > > this is a viable way forward. > > The current design with multiple loops > > * has robustness (a problem in one submodule doesn't crash the whole > system) > * has good framerates (which is an issue for the majority of us) > * is easy to debug (as it doesn't hand over too many parameters between > iterations) Unfortunately, Advanced Weather can and does crash Fg here. I haven't looked into it properly - but there's no obvious reason atm. > If running the individual loops at main framerate solves your smoothness > problems, then we can start for instance making the loop timers user- > configurable. Sacrificing framerate for smoothness is a compromise you > might be willing to make, but I am not, I need the framerate. Yup - with "bare" fg I can see over 200fps (not at KSFO of course). I can spare a few for smoothness. > I agree that for your problem, your solution is the correct way forward - but > my problems are different from yours. So what do we do about that? For a start we can try to make the Nasal better, I think that might help a bit. I spent an hour or so picking over your code. So far I've found: 88 declared but unused variables 47 declarations of the same or similar variables 427 instances of "else if" instead of "elsif" 100 instances of I = I + 1 instead of i+=1 Numerous examples of variables declared inside for loops, and some of those are inside other for loops Variables declared inside condition statements. Nasal is tolerant of those kind of things where other languages/compilers would at least warn y
Re: [Flightgear-devel] Sea color
-Original Message- From: Ron Jensen Sent: Saturday, May 19, 2012 2:54 AM To: FlightGear developers discussions Subject: Re: [Flightgear-devel] Sea color On Friday 18 May 2012 10:59:22 Alan Teeder wrote: > On the subject of frames rates I have a couple of questions. > 2. Is there a mechanism for making the core - fdm, afcs, equations of > motion etc. run at a higher priority than the rest of the simulation? The JSBSim FDM and systems coded in JSBSim run at or near a stable 120Hz by running multiple times per graphics frame. Ron Thanks for that. Alan -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
On Friday 18 May 2012 10:59:22 Alan Teeder wrote: > On the subject of frames rates I have a couple of questions. > 2. Is there a mechanism for making the core - fdm, afcs, equations of > motion etc. run at a higher priority than the rest of the simulation? The JSBSim FDM and systems coded in JSBSim run at or near a stable 120Hz by running multiple times per graphics frame. Ron -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Thorsten, > It is for Vivian. We seem to agree that a stable 20 fps is possible (I get > > it and Vivian gets it if he throttles down), but he aims at more. Which is > > okay with me. I understood it wasn't stable. That's why he was happy with stable 24fps. I probably misunderstood here. > Well, I'm not seeing anything like that, and neither is Vivian - we were > > talking about frames delayed to 50-60 ms which are a noticeable stutter > > when you can run with 40 fps, not about second long delays. I agree that > > what you have way more serious and needs to be understood and fixed asap > if > that is possible. I tried the 777-200LR at CustomScenery "Alaska" with "Cold Sector" and visibility at default at the first 10min I had the mentioned stutters. But suddenly framerates stabilized and I noticed the less delays. So the scenery tile loading was the cause for stutters, so finally I could see your work in its all glory and got some nice screenshots. Without using the system monitor (which has an big influence on Framerates here) I have: With "Cold sector", cloud density of 0.25 I had more or less stable 37fps not throttled. Delay had been about 60-84ms. With the Cub at Terrasync TNCM I get similar framerates and delays after tile loading, no loops disabled and settimer kept. With visibility set to 50km to keep tile loading time smaller, with all settimer set to zero and with disabling the loops I get 29-30fps throttled down and a delay of 34ms. Unthrottled I get 60fps and a delay of 30-40ms at TNCM! Then again Enabling buffer loop delay increase to average 50ms and gets less stable (20-98ms) Enabling now dynamics loop delay gets worse at the first second- then stabilization. (20-41ms) Enabling effect loop delay keeps between 20-41ms The same with timing loop. Again, all this with all settimer set to 0.0. Framerates changes its values according to the number of clouds. With the mentioned scenario I got always about 50-60fps at ground. Tropical weather is unusable for me. But similar weather scenarios with Basic weather aren't better. I did not yet tried other weather scenarios. > So if you, or Vivian, or anyone else can come up with code or an idea > that actually fixes a problem, you'll see me say 'Thank you'. Sorry, but this sounds even cheaper ;-) If we already had the solution, Vivian or me had already brought it up. But you are adressed here, as you are the developer of this piece of artwork. See it like that: If I have a problem with any product and don't find the solution, you will always try to ask the producer if he might have an idea as he probably does know his product better than any other! Nethertheless, we all don't know what you already had considered. And that's why all those "cheap" suggestions coming up. Like on the Lightfield shader issue it is better when you suggest a list of ideas which can be the issue. That might be many paths of possible issues, and any of those paths has to be tried to find out which is the right one. Vivian already suggested some paths, if you think this isn't the path to the solution, it would be good to propose some other paths. > (As a side remark, I remember specifically asking here what people would > > like to see in a combined weather - so far no answers. ) Hmm... probably because when they want to have a combined weather probably an Advanced Weather which runs per default (METAR, real life weather fetch and all those nice scenarios we have in both) without explicit enabling and giving similar framerates like basic weather. Heiko still in work: http://www.hoerbird.net/galerie.html But already done: http://www.hoerbird.net/reisen.html -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Alan > > On the subject of frames rates I have a couple of questions. > > 1. Is there a mechanism for odd and even frames (or even > frame.1,frame.2,frame3...frame.n for a once in every n frames task) to be > run separately, or is everything that is scheduled at a specific rate executed > one go? > 2. Is there a mechanism for making the core - fdm, afcs, equations of motion > etc. run at a higher priority than the rest of the simulation? > If you go to the menu item Debug > Monitor System Performance you will see which sub-modules run at main frame rate and those which run at 2 x main frame rate. Items can be made to run at greater than main frame rate, but not at less, easily, and I guess more sub-modules could be added to the 2x list with more effort. Vivian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
On the subject of frames rates I have a couple of questions. 1. Is there a mechanism for odd and even frames (or even frame.1,frame.2,frame3...frame.n for a once in every n frames task) to be run separately, or is everything that is scheduled at a specific rate executed one go? 2. Is there a mechanism for making the core - fdm, afcs, equations of motion etc. run at a higher priority than the rest of the simulation? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
> The facts are: > > -it isn't that much important if you have 20fps or 60fps. But it is more > important that the framerates are stable !!! It is for Vivian. We seem to agree that a stable 20 fps is possible (I get it and Vivian gets it if he throttles down), but he aims at more. Which is okay with me. > In the moment I can see stutters which interrupts the simulation > every <1-2 seconds for even about between one, two second. In this time > I always loose control about an aircraft and autopilot begins to play > crazy. Well, I'm not seeing anything like that, and neither is Vivian - we were talking about frames delayed to 50-60 ms which are a noticeable stutter when you can run with 40 fps, not about second long delays. I agree that what you have way more serious and needs to be understood and fixed asap if that is possible. So can you run the same diagnostics I suggested to Vivian, i.e. selectively disabling the list of loop flags and see if that improves anything? If I am not much mistaken, no major loop runs at a 1-2 second interval. For reference: * effect volumes run per frame * interpolation runs at 0.2 sec * thermal and wave lift are usually of because not needed, but they run per frame * housekeeping runs per frame * tile management runs at 4.0 sec (but has very weak computational load) * dynamics runs per frame * quadtree runs per frame * convective loop runs at 30 seconds, but isn't currently on Two points worth noting: 1) most loops already run per frame, so we're really talking about if interpolation should be clocked faster or not and if running tile management at 4 seconds is a good idea and 2) it's rather unlikely that coincidence of not-per-frame loops creates what you experience because the timing doesn't work out. Thus, starting from Vivian's theory that this is the problem gets us precisely nowhere I can see in your case (it may explain Vivian's findings though). > Maybe I'm able to create a diagram showing Latency/Framerates in > Advanced Weather. Though my computer is not the newest one, I would > expect a much better behavior. Me too. > -Advanced weather is indeed a high fidelity weather simulation. Together > with the shaders, and especially lightfield shader, we get a very good > simulation of the atmosphere already, which is needed for a more > realistic simulation of flight. So due to this it is clear that it > needs a bit more perfomance than other stuff in FGFS. But this doesn't > explain those stutters. No, it doesn't. Look, believe me or not, I am really trying to find out what the issues are. That's difficult, if the discussion boils down to 'I found a for-loop inside the main loop, that's probably what causes problems'. I put the for loop there for a reason having to do with thinking a few months about a difficult problem, I'm trying to explain these reasons so that we can get the biases out of the way and actually get down to identifying what needs to be addressed and what is red herring. I'm also trying to be realistic - most people seem to be able to get a stable 20 fps, i.e. worst frame delay of 50 ms (your case needs to be understood though), I might be able to improve the code to give a stable 30 fps, but I don't see it coming that tasks can be chopped and distributed such that you can expect a smooth 50 or 60 fps. > Here are people who does have enough experience and knowledge about > Project FlightGear and its codes and especially nasal which should be > taken seriously. If somebody posts a piece of code replacing something I have written and demonstrates that this runs faster, I take this very seriously. If somebody here suggests a faster algorithm to solve the problem, I take this very seriously (Stuart has outclassed my way of moving clouds by more than an order of magnitude - so we do it his way and deal with the secondary problems caused by the approximation. TorstenD has both advised me how to write loop structures across frames and done his version of the terrain sampling which is better than mine, so we use it now. There's a history of me adopting other people's solutions if they are demonstrated to be superior). On the other hand, 'This is bad design' or 'This should run faster' is cheap to say. Claiming that 'If you do it like this it would run faster' is also cheap - I've tested a few dozend strategies which fail already, and I know for a fact that what looks promising doesn't work automatically. I'll listen to such suggestions, but I reserve the right to dismiss them based on my experience with the problem at hand unless it is demonstrated to me that this actually works. So if you, or Vivian, or anyone else can come up with code or an idea that actually fixes a problem, you'll see me say 'Thank you'. > But I do see a wish by many developers and users to have both weathers > combined, and even more important working without any stutters. And this > means that s
Re: [Flightgear-devel] Sea color
Hello Thorsten, Somehow I don't like the way this discussion is going. > You had a worst of 27 in your list > http://dl.dropbox.com/u/57645542/stagger-data.htm > running everything and were unhappy. Now a stable worst of 24 makes you > happy? The facts are: -it isn't that much important if you have 20fps or 60fps. But it is more important that the framerates are stable !!! In the moment I can see stutters which interrupts the simulation every <1-2 seconds for even about between one, two second. In this time I always loose control about an aircraft and autopilot begins to play crazy. Maybe I'm able to create a diagram showing Latency/Framerates in Advanced Weather. Though my computer is not the newest one, I would expect a much better behavior. -Advanced weather is indeed a high fidelity weather simulation. Together with the shaders, and especially lightfield shader, we get a very good simulation of the atmosphere already, which is needed for a more realistic simulation of flight. So due to this it is clear that it needs a bit more perfomance than other stuff in FGFS. But this doesn't explain those stutters. -Everyone here is impressed by the outcome. Everyone would like to use it. Compared with X-Plane or MSFS we can really be proud to have such thing! And say thank you for this contribution! So your hard work you did is highly appreciated by everyone. Here are people who does have enough experience and knowledge about Project FlightGear and its codes and especially nasal which should be taken seriously. That does not mean, that your opinions are not taken serious. It is quite the opposite. But I do see a wish by many developers and users to have both weathers combined, and even more important working without any stutters. And this means that some parts of your code maybe needs some overhaul. Even we do know that you have worked on the code in the best manner to have it fast as possible. Vivian wrote: >> "Optimising" code is bad. You might make it better for your system, >> but make it worse for everyone else. OSG/OpenGL do a pretty good job >> all >> by themselves vs Thorsten wrote: > Optimizing code is badly needed. OSG/OpenGL don't do a pretty good job > all by themselves for sufficiently complex tasks and that's a fact - > been there, seen it - Advanced Weather without any optimization would > drive you to single digit framerates no matter what system you run > easily. It's all nice that you get enough framerate out of the water > shader and can affort to compute cloud coverage in the shader per frame > per > pixel, but those of us not running high end machines would like to > run it as well, and I am > glad I figured out a way to make it 50% > faster for me. You said it already: Optimizing code is badly needed. And optimizing can be done in many ways. The pure nasal code can just be optimized, or the algorithm can be hardcoded somewhere, or etc No idea if OSG/OpenGL is really the limiting factor. What Vivian wanted to say is, that something that works on your computer does not automatically must on other computers. As much as I know he has now a pretty good and strong computer. And I don't know what you do understand on "high end machines". Today we have to seperate between computers with many powerful cores, but weak inboard graphiccards; computers with single or just duo cores with low rates, but strong, higher graphic cards; or such machines where you have the best of all. You can see it yourself: Advanced Weather is running fine on your system, but you have big troubles with rembrandt. On my system it is quite the opposite: advanced weathers gives me unusuable stutters, but rembrandt is really fine doing. Maybe we should handle this more like in our bug tracker: Unusable stutters with Advanced Weather What steps will reproduce the problem? 1. take any aircraft and start from any airport 2. select advanced weather 3. watch Latency and framerates What is the expected output? What do you see instead? Expected is a stable framerate, maybe with lower framerates than without advanced weather. Instead we get stutters which occurs every 1-2 seconds or even less, with pauses about 1-2 seconds. It is undependant of direction of view (cloudy parts vs non cloudy parts) and gets worse the more clouds are beeing generated. ... At least Optimizing code to try to prevent such issues feels for me like healing the symptoms, but not healing the real cause, whatever this might be. Cheers Heiko still in work: http://www.hoerbird.net/galerie.html But already done: http://www.hoerbird.net/reisen.html -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.
Re: [Flightgear-devel] Sea color
After sleeping it over... > Now set > all Advanced Weather settime() to 0.0 and retest with METAR. Wow!!! > Improvement, not as good as Basic Weather , but much better. Worst fps is > stable at 24, av. is still unstable. You had a worst of 27 in your list http://dl.dropbox.com/u/57645542/stagger-data.htm running everything and were unhappy. Now a stable worst of 24 makes you happy? It'd be good to test this not with METAR but with high pressure core. Basically, if you clock all loops at 0.0, as far as Nasal is concerned you make the average frame duration come down to the worst frame duration. Which would indeed trade framerate for smoothness, except... on my system, this runs into the garbage problem, so I get a really bad worst frame delay when I run all stuff at all times. So the lesson I learned is as long as garbage collection is a problem, avoid running stuff on a per-frame basis whenever you can. This looks to be very system specific, though. > Right now, with Advanced Weather we have a weather simulator with a > FlightSim attached. We're spending 10 (yes 10!) times as long in the Events > Sub-module with Advanced Weather than in Basic, and 5 times as long as we > spend in Flight. > For loops are bad - in C++ as much as in Nasal. For loops hold up > the execution of the main loop, even when they do very little. I'm not quite sure you actually appreciate the task being done, so let me expand a bit. First of all, in terms of raw floating point operations, the water shader beats flight (and weather) probably by a few orders of magnitude. So what we really have is a water reflection simulator with a few percent of the remaining performance dedicated to environment and flight. What saves our ass in performance lists is just that water reflection can run on the GPU and hence doesn't even show up in CPU load comparison - it slows you down nevertheless in practice, dramatically so. Flight has the task of solving equations of motion for a single object, based on a series of coefficients (to be determine by functions or interpolation tables). At the core is a set of differential equations, we know how to solve them by discretizing, replacing derivatives by differences applying some error corrections and cleverness and integrating forward in time. I know the problem reasonably well, I could write code doing it if needed. Advanced Weather has to solve equations of motion for all clouds in the scene (on a nice summer afternoon, say we have about 5000 Cumulus clouds, 1000 of these have a significant thermal underneath). If the equations of motion would be as complex as for an airplane, we'd have a framerate (if we start out having 60 fps without clouds) of 60/6000 = 0.01. That wouldn't be good. So we use very simple equations of motion which give plausible behaviour. Say these are a factor 100 faster, which gives us a framerate of 1 fps. Still not good. This is where the brute-force approach ends. Next, we realize that not all objects need priority attention. Nearby objects like the thermal we're in are more relevant than distant objects (like the thermal we're not in). Visible objects like clouds need more attention than invisible objects like thermals - it's more acceptable for a thermal to have a 'jumpy' motion than for a cloud nearby. Thus, we do not need to solve every equation of motion per graphical frame. It's sufficient to solve them at a slower rate, we can visit every object in the scene only once in a while. But it shouldn't be too jumpy - a 10 m discontinuous motion can't be felt in a glider, but a 100 m jump can (I tested these and compared with my real-life experience). Clouds in the scene should best not move discontinuously at all (you'd be the first to point this out...). Thermals should not be decorrelated from cap clouds over time, rain should not leak out underneath clouds,... that limits the way we can distribute tasks. In other words, the weather system needs to know where the clouds are even if the clouds are actually moved in the scene by the shader. The weather system still needs to correlate thermals and do the vertical motion of the clouds in response to terrain obstacles, and for that it doesn't just need the information where the cloud is but also what the terrain underneath is like. Say in doing all that cleverly, we can get the framerate from 1 to 30 fps. It's clear that assigning priorities will cause some fluctuations in the framerate, because the system has to make guesses what is important at the precise moment, and sometimes it guesses wrong and the task it decided to do is easier than expected. At the same time, we may also move through the scene. We have supersonic aircraft, the Concorde at cruise altitude for instance goes through 3 weather tiles every 50 seconds, that gives you a whopping 16 seconds to get to know the terrain, build 5000 clouds dependent on what you fly over, display them, and remo
Re: [Flightgear-devel] Sea color
> Conclusion: don't try to optimise, particularly for a poor system - you > might make it better for that system, but more likely you will make it > worse for everyone else. Judging by framerate comparisons with people in the forum, my system is still somewhere in the upper third - many people have to live with less. Judging by user requests and comments, more than 90% want higher framerate out of the system by any means, you represent a minority of users who would be willing to sacrifice framerate for smoothness. Most of us are happy to see Rembrandt or lightfields with anything resembling 20 fps. So, just who is that 'everyone else'? And for whom do we optimize? You're basically saying I should optimize things for you - but that would make it worse for everyone else not running a high-end system. > Right now, with Advanced Weather we have a weather simulator with a > FlightSim attached. We're spending 10 (yes 10!) times as long in the > Events Sub-module with Advanced Weather than in Basic, and 5 times as long as > we > spend in Flight. That'd probably be because Advanced Weather does ~10 times more complex calculations than Basic weather... And I'm not surprised that it takes more than flight either - flight is comparatively cheap, that ran with decent accuracy 10 years ago. To compute a non-local environment (i.e. that knows that conditions 'here' are different from conditions 'there') is quite a bit more expensive and we could not do that 10 years ago. As for your comment, yes, that's quite true - that's just how the problem is. Teaching thousands of clouds where convection is, or how to flow over terrain obstacles is expensive, even if you do it schematically. That's what's needed to give you semi-realistic distributions of clouds. If you're happy with just clouds in the sky, that's cheaper, but not what Advanced Weather is for. > Writing data to the Property Tree is bad. This one is not evidence > based That's evidence based (I have done some testing a while ago just how long it takes) - it's currently minimized in my code, but the tree is the interface between menu, C++, Shaders and Nasal, so ultimately some stuff has to be written or read. > To be honest, I think there are design difficulties with Advanced > Weather. > There should only be one loop - the "update loop" - running at main frame > rate which activates/deactivates the various sub-modules as required. I > think I have done enough work here to demonstrate that this is a viable > way forward. The current design with multiple loops * has robustness (a problem in one submodule doesn't crash the whole system) * has good framerates (which is an issue for the majority of us) * is easy to debug (as it doesn't hand over too many parameters between iterations) If running the individual loops at main framerate solves your smoothness problems, then we can start for instance making the loop timers user-configurable. Sacrificing framerate for smoothness is a compromise you might be willing to make, but I am not, I need the framerate. I agree that for your problem, your solution is the correct way forward - but my problems are different from yours. So what do we do about that? * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Thorsten Testing continues > > Concorde has of the order of 6000 lines of active code, and yes, it > > displays exactly the same discontinuities as Advanced Weather (approx. > > 10,000 lines of code). So far, I have not found any other examples. > Just to idly continue my list - the A380 has in excess of 8000 lines of Nasal, the > 777 has about 3500 lines, ... I don't know what you usually fly, but the > modern, more complex aircraft tend to accumulate a lot of Nasal when > aircraft makers start to care about systems. A380 - broken Mig 15bis - broken B777 - works - framerate is adversely affected and stutters a bit, but acceptable. Notably, it increases the time spent in the Events sub-module, but only by approx. 10ms, or 2 times. > I have just tried to start Flightgear without the Advanced Weather module > loaded, disabling all weather, using the ufo to minimize any Nasal impact. The > bottom line is, no matter what I do, as soon as I actually move around I > always get spikes in the frame delay raging from 30 to 45 ms, i.e. I never have > a smooth framerate even if I throw out basically everything. The ufo is far from a Nasal-free zone. Compare it with the Seahawk? It has no pooh traps. > So I can't ever get to the smooth state you apparently have on your system, > which means I can't meaningfully test what makes a difference. I can only > suggest that you try disabling the loops I suggested and play with the loop > timings. If you think letting loops run per frame helps, by all means try - with > the possible exception of the tile management, I'm fairly certain the rest can > run per frame without bad side effects - the times are chosen to give the > best performance on my system, but this may not work the same way for > you. Nil weather is very smooth in basic weather with shaders on but veg, obj, and bldgs off. Tried METAR. Basic weather now shows some stutter. Now set all Advanced Weather settime() to 0.0 and retest with METAR. Wow!!! Improvement, not as good as Basic Weather , but much better. Worst fps is stable at 24, av. is still unstable. That's the good news - the bad news is that Emilian does not see an improvement. Conclusion: don't try to optimise, particularly for a poor system - you might make it better for that system, but more likely you will make it worse for everyone else. > If you can find a working combination, I'll make it an option in the GUI and > see if it helps others. > > > Yes, I was around when Nasal was introduced to Flightgear. Up to now > > it's been a few 100s of lines, not many 1000s. We have long been aware > > that GC could cause problems, but haven't fixed it because the effects > > weren't too bad while Nasal was small and framerates were low, and we > > had no obvious solution. > > You know what really bugs me? The direction fingers are pointing. Well, as Gene Buckle pointed out: if you point a finger, then 3 are pointing right back at you. > We have an implementation of Nasal which dumps all the GC into a single > frame and is apparently sensitive to the total amount of code, regardless if > the code is actually run or not. This fact has historically not been widely > advertized or explained. That turns out to be a problem. We were aware that "GC was a problem". We also avoided dumping nasal into the Nasal folder because "everything in there runs". Too much stuff is either buried in obscure Wiki pages or are simply part of the hive memory; there are fewer worker bees than there were and the memory grows weaker. > The way this usually comes across is 'Advanced Weather causes stutter'. But > it actually doesn't really (or at least that remains to be shown) - what causes > stutter is mainly the GC, and Advanced Weather just happens to trigger this. > The range of suggested solutions in the past included almost everything, > from avoiding Nasal to porting code to Nasal to hacking around the problem > to loading things on-demand - except fixing the actual cause of the problems. To be honest, I think there are design difficulties with Advanced Weather. There should only be one loop - the "update loop" - running at main frame rate which activates/deactivates the various sub-modules as required. I think I have done enough work here to demonstrate that this is a viable way forward. > I don't honestly know how complex code to collect garbage across many > frames is, but somehow I doubt that in terms of man-hours the effort beats > porting the existing large-scale Nasal codes to C++. Just my 2 cents in any > case. > Sorry, this really had to get out. Now, back to being constructive. As I said, I > try to do what I can in terms of getting old code out and streamlining the rest > - hopefully we get some improvements. Right now, with Advanced Weather we have a weather simulator with a FlightSim attached. We're spending 10 (yes 10!) times as long in the Events Sub-module with Advanced Weather than in Basic, and 5 times as long as we spend in Fli
Re: [Flightgear-devel] Sea color
> Concorde has of the order of 6000 lines of active code, and yes, it > displays exactly the same discontinuities as Advanced Weather (approx. 10,000 > > lines of code). So far, I have not found any other examples. Just to idly continue my list - the A380 has in excess of 8000 lines of Nasal, the 777 has about 3500 lines, ... I don't know what you usually fly, but the modern, more complex aircraft tend to accumulate a lot of Nasal when aircraft makers start to care about systems. I have just tried to start Flightgear without the Advanced Weather module loaded, disabling all weather, using the ufo to minimize any Nasal impact. The bottom line is, no matter what I do, as soon as I actually move around I always get spikes in the frame delay raging from 30 to 45 ms, i.e. I never have a smooth framerate even if I throw out basically everything. So I can't ever get to the smooth state you apparently have on your system, which means I can't meaningfully test what makes a difference. I can only suggest that you try disabling the loops I suggested and play with the loop timings. If you think letting loops run per frame helps, by all means try - with the possible exception of the tile management, I'm fairly certain the rest can run per frame without bad side effects - the times are chosen to give the best performance on my system, but this may not work the same way for you. If you can find a working combination, I'll make it an option in the GUI and see if it helps others. > Yes, I was around when Nasal was introduced to Flightgear. Up to now it's > been a few 100s of lines, not many 1000s. We have long been aware that GC > could cause problems, but haven't fixed it because the effects weren't > too bad while Nasal was small and framerates were low, and we had no obvious > solution. You know what really bugs me? The direction fingers are pointing. We have an implementation of Nasal which dumps all the GC into a single frame and is apparently sensitive to the total amount of code, regardless if the code is actually run or not. This fact has historically not been widely advertized or explained. That turns out to be a problem. The way this usually comes across is 'Advanced Weather causes stutter'. But it actually doesn't really (or at least that remains to be shown) - what causes stutter is mainly the GC, and Advanced Weather just happens to trigger this. The range of suggested solutions in the past included almost everything, from avoiding Nasal to porting code to Nasal to hacking around the problem to loading things on-demand - except fixing the actual cause of the problems. I don't honestly know how complex code to collect garbage across many frames is, but somehow I doubt that in terms of man-hours the effort beats porting the existing large-scale Nasal codes to C++. Just my 2 cents in any case. Sorry, this really had to get out. Now, back to being constructive. As I said, I try to do what I can in terms of getting old code out and streamlining the rest - hopefully we get some improvements. * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
> Yes. You have produced a very nice piece of work which gives variable
> results in different systems. We need to at least understand the cause
> and try to fix it
GC according to what I can test.
> Pity you didn't use standard scenery. I don't think custom scenery and
> the ufo will produce representative results.
The ufo minimizes other interfering Nasal and gives me the highest framerate,
so it's a good test case to see problems which show up for high framerate.
Custom scenery is because I fly that most of the time, so just bad habit.
> Loading and unloading scenery tiles every few seconds? Any evidence for
> that?
With high visibility and the ufo at 500 kt? You bet there's tiles loading every
few seconds...
> Yes it probably does. And we should be trying to eliminating these
> wherever feasible. And don't forget that there's Nasal everywhere which is
> doing
> its Garbage Collection.
GC is not relevant here, since there are no visible framerate delayes when I
remain static, but they are there when I move - Nasal or GC shouldn't depend on
my state of motion.
> No that is not what I'm talking about. Altering the view changes the
> frame
> rate here, shaders or not, just try looking at KSFO from the air and then
> away from it.
> The issue I was on about is uneven framerates with big differences
> between
> the worst and average framerate. Are you seeing shaders causing that?
As I explained before, yes, I'm seeing precisely that in turns and my
interpretation is that I'm seeing stuff being loaded into the scene and
unloaded from the scene. Shaders have a substantial impact there - turning into
ocean-dominated scenery causes uneven framerate here till I end my turn.
> As I said - here it never gets back to normal, but ThorstenB explained
> that we are not unloading object when disabling, and that a restart is
> needed.
As far as I can measure it gets back to normal here. Might have to do with the
lack of Nasal in the ufo.
> They are inherently problematic, the only question is how much of a
> problem,
> but sometimes they're unavoidable.
>
> What's with this one?
>
> setsize(terrain_n,40);
> for(var j=0;j<40;j=j+1){terrain_n[j]=0;}
>
> can't this be:
>
> setsize(terrain_n=[],40);
This runs only once at startup, not inside any of the continuous loops. But it
could be changed.
> Do we need setsize() at all? I don't see it much used outside Advanced
> Weather.
It allows to cut elements out of a vector, which is very neat. No idea if there
is another way of doing that in Nasal.
> Concorde has of the order of 6000 lines of active code, and yes, it
> displays
> exactly the same discontinuities as Advanced Weather (approx. 10,000
> lines
> of code). So far, I have not found any other examples.
Vostok seems to have around 5000, MiG-15bis has 7000+ lines, the Garmin GPS
comes with ~4000 lines of Nasal; there's the bombable air combat package (since
as far as I know air combat features wouldn't get committed, anyone without the
infrastructure to generate binaries for various system is practically forced to
use Nasal since the number of users who would use air-combat feature patches
and then compile is very limited, whereas distributing Nasal files one can
reach at least some users),...
...as I said, it's here to stay.
> What an odd metaphor - you very definitely do the first , while waiting
> for the second. If the fuse box cannot be fixed, then you are stuck with the
> first, and that's where we are right now. Let's hope there is a solution.
Then why don't you simply try running Advanced Weather 'lightl? After all, it's
a modular system all along.
Testing for 5 minutes and 500 km, it seems to run fine here with
dynamics-loop-flag=0
timing-loop-flag=0
effect-loop-flag=0
buffer-loop-flag=0
effect-loop-flag=0
which leaves you all the visuals and takes out the harder-to-spot details (2d
clouds no longer move, no evolution of convective clouds, no working thermals,
weather doesn't drift with the wind, only clouds do, so weather doesn't change
if you wait at a location, no rain (isn't working anyway right now...),...
According to your table, that takes care of a fair share of problems. Edit
'set_tile()' in Nasal/local_weather/local_weather.nas to start the loops you'd
like to have. We might even have a checkbox in the menu for the purpose.
But as I said, I think really the GC needs to be addressed. There's only so
much hacking around the actual problem one can do.
I am at the moment throwing unused code out - no idea if it will help much, but
it will reduce the line count somewhat.
I see that it'd be possible to port weather effects to C++ if anyone would want
to work on that - this is stable code from my side, I haven't touched it for
ages, it would be useful in more applications (think of localized turbulence
behind starting AI aircraft for instance) and it is very modular - so I'm all
for doing that.
> Yes, I was arou
Re: [Flightgear-devel] Sea color
Thorsten > I did a bit of testing of my own yesterday, and I have made a few other > observations as well. > > User feedback: > > I've largely come to ignore that (with few exceptions such as Vivian's > performance table), because trying to make sense of it is a path into > madness. Just a few examples: On my own system, Advanced Weather > tends to lead to better framerates than Basic Weather for similar cloud cover. > In fair weather, I'm getting 10% more, in overcast conditions sometimes even > a factor 2. I have some user feedback (4 cases by now) confirming that. Heiko > recently said that he previously got better framerates using Advanced, but > not any more. My problem - I didn't really change anything except parameter > values and such things between the two occasions. Logical conclusion: The > performance of Advanced Weather can change uncorrelated with changes in > the code, and it affects different users in a different way? I too saw improved framerates in Advanced Weather a short while back, but they seems to have disappeared. It is quite hard to compare like-with-like between Basic and Advanced Weather. > A while ago, the user sgofferj experienced 'stutter' when running Flightgear. > Vivian was kind enough to immediately blame Advanced Weather on the > occasion, however the screenshots clearly showed (and sgofferj later > confirmed) that he had never loaded Advanced Weather (read the story > here http://www.flightgear.org/forums/viewtopic.php?f=68&t=15235). > Logical conclusion: Advanced Weather is able to cause stutter even if the > module is never loaded? > > So, for some it's faster and they gain a lot of performance, for some it's > slower, for some it causes stutter when loaded, for others the mere > presence on the harddisk causes stutter, for some it works fine... should any > of this really cause action from my side? Largely, this is anecdotal evidence. Yes. You have produced a very nice piece of work which gives variable results in different systems. We need to at least understand the cause and try to fix it > 'Bare Flightgear' > > My testcase was France Custom Scenery with the ufo, visibility probably 50 > km or so, no shaders. Removing *all* weather and just remaining stationary, > I get 70 fps with a worst frame delay of pretty constant 14 ms, since 14*70 = > 980 ~ 1000 this means I get very evenly spaced frames. However, just flying > in a circle, my worst frame delay starts to get jumpy every 2-3 seconds, it > goes to 35 - 45 ms, a bit dependent on the scenery to sky ratio. My > interpretation is that I'm seeing the scene being shuffled in and out of the > GPU memory - as I turn, the scene in the visual field changes, and the GPU > needs to compute the changed elements, causing delayed frames. > Pity you didn't use standard scenery. I don't think custom scenery and the ufo will produce representative results. > Just flying straight, the worst frame delay also changes and jumps to 45-55 > ms every few seconds. After a bit of experimenting, that's probably the > effect of loading and unloading of scenery tiles, objects and such like. Loading and unloading scenery tiles every few seconds? Any evidence for that? > That is to say, the bare structure of Flightgear has discontinuous workloads as > well, and they cause frame durations of up to 55 ms on my systems. There > are also genuine (but rare) pathologies like the first look back in the F-16 (a > few seconds of frame delay...) - I guess that's particularly large texture > sheets loading. Yes it probably does. And we should be trying to eliminating these wherever feasible. And don't forget that there's Nasal everywhere which is doing its Garbage Collection. > 'Full Flightgear' > > In my favourite setting, I have weather and lightfield shaders on and a > decent visibility. On the occasion, that delivers me a framerate of ~20 fps > with a constant worst frame delay of 50-55 ms. Since 20 *50 = 1000, the > frames come nice and like a clockwork whatever I do - scenery loading causes > upward spikes to 55 ms from 50 ms, but that's all, i.e. now the GPU is slow > enough to allow all CPU-related stuff to finish nicely. I can reproduce that here. If I throttle the framerate to 20, less than the worst framerate, then I get a steady framerate, but then I get very noticeable but even stutter on the display. I'm aware that there is a school of thought which suggests that regular flickering can trigger epilepsy, but I'm not at all sure if what we are producing falls into this category. Increasing the throttling framerate above the worst (23 fps) makes things gradually worse, reintroducing the spikes in framerate at 30hz. I think we need better than 50 hz for good visual appearance. 40Hz can be acceptable. > 'Shader impacts' > > Vivian always goes talking about smooth framerate, but for instance the > water shader is one of the worst offenders in this department, because on > my system it makes the framerate dramatically depen
Re: [Flightgear-devel] Sea color
I did a bit of testing of my own yesterday, and I have made a few other observations as well. User feedback: I've largely come to ignore that (with few exceptions such as Vivian's performance table), because trying to make sense of it is a path into madness. Just a few examples: On my own system, Advanced Weather tends to lead to better framerates than Basic Weather for similar cloud cover. In fair weather, I'm getting 10% more, in overcast conditions sometimes even a factor 2. I have some user feedback (4 cases by now) confirming that. Heiko recently said that he previously got better framerates using Advanced, but not any more. My problem - I didn't really change anything except parameter values and such things between the two occasions. Logical conclusion: The performance of Advanced Weather can change uncorrelated with changes in the code, and it affects different users in a different way? A while ago, the user sgofferj experienced 'stutter' when running Flightgear. Vivian was kind enough to immediately blame Advanced Weather on the occasion, however the screenshots clearly showed (and sgofferj later confirmed) that he had never loaded Advanced Weather (read the story here http://www.flightgear.org/forums/viewtopic.php?f=68&t=15235). Logical conclusion: Advanced Weather is able to cause stutter even if the module is never loaded? So, for some it's faster and they gain a lot of performance, for some it's slower, for some it causes stutter when loaded, for others the mere presence on the harddisk causes stutter, for some it works fine... should any of this really cause action from my side? Largely, this is anecdotal evidence. 'Bare Flightgear' My testcase was France Custom Scenery with the ufo, visibility probably 50 km or so, no shaders. Removing *all* weather and just remaining stationary, I get 70 fps with a worst frame delay of pretty constant 14 ms, since 14*70 = 980 ~ 1000 this means I get very evenly spaced frames. However, just flying in a circle, my worst frame delay starts to get jumpy every 2-3 seconds, it goes to 35 - 45 ms, a bit dependent on the scenery to sky ratio. My interpretation is that I'm seeing the scene being shuffled in and out of the GPU memory - as I turn, the scene in the visual field changes, and the GPU needs to compute the changed elements, causing delayed frames. Just flying straight, the worst frame delay also changes and jumps to 45-55 ms every few seconds. After a bit of experimenting, that's probably the effect of loading and unloading of scenery tiles, objects and such like. That is to say, the bare structure of Flightgear has discontinuous workloads as well, and they cause frame durations of up to 55 ms on my systems. There are also genuine (but rare) pathologies like the first look back in the F-16 (a few seconds of frame delay...) - I guess that's particularly large texture sheets loading. 'Full Flightgear' In my favourite setting, I have weather and lightfield shaders on and a decent visibility. On the occasion, that delivers me a framerate of ~20 fps with a constant worst frame delay of 50-55 ms. Since 20 *50 = 1000, the frames come nice and like a clockwork whatever I do - scenery loading causes upward spikes to 55 ms from 50 ms, but that's all, i.e. now the GPU is slow enough to allow all CPU-related stuff to finish nicely. 'Shader impacts' Vivian always goes talking about smooth framerate, but for instance the water shader is one of the worst offenders in this department, because on my system it makes the framerate dramatically dependent on the view angle. Looking up into the sky and looking down into the sea makes a difference of a factor 4 (80 fps vs. 20 fps without lightfields, half of that with). To me, this is quite natural, as the workload of the water shader per fragment is way higher as the workload of any of the skydome-coloring codes. Baseline - you don't need Nasal to get uneven framerates, GLSL (and C++) can deliver that just fine. 'Advanced Weather and a diagnosis' Starting from the 'bare case' and running just advanced weather without lightfields, I get a framerate of 60 fps and a pretty constant worst frame delay of 50 ms. Since that is 3000 ms, the numbers don't match up and there are indeed pathologically delayed frames occurring on a regular basis. De-activating the heaviest per-frame loop (dynamics-loop-flag=0) gets me almost back to the normal case, i.e. worst frame delay goes to a pretty constant 22 (60*22 = 1320, not precisely smooth but much better) with occasional spikes to 55 (note that they've been present without weather as well - that's terrain loading). Note that this is somewhat supported by Vivian's data (although not by his words) - deactivating mentioned loop brings his standard-deviation of the frame duration down by 1/3, de-activating all per-frame loops brings the standard deviation almost down to normal. The implication is that what is causing the p
Re: [Flightgear-devel] Sea color
ThorstenB > On 11.05.2012 10:21, Renk Thorsten wrote: > > The problem with that consequence is: As you switched all loops off, > > the Nasal part of Advanced Weather ceased to run completely and all > > that's left is the same cloud-generating functionality which is > > responsible for Basic Weather (C++ and shader work). If killing all > > Nasal did not solve the problems, then they can't be caused by Nasal, > > and porting to C++ will not fix them. > > Not quite true. A significant part of Nasal-related frame rate impact is caused > by garbage collection. Its delay/jitter only depends on the number of Nasal > objects and their references which need to be searched. > Increasing the number of Nasal objects (code+data) existing in memory also > increases the delay. > > The amount of Nasal code which is actually being executed only influences > the g/c frequency, i.e. whether the effect is visible every few seconds vs > several times per second. > > This is why we are loading large Nasal modules, like advanced weather, on > demand only (= only when feature enabled). We're not unloading objects > when disabling a feature though - this usually requires restarting fgfs. This > should be remembered when comparing such features (i.e. restart fgfs after > disabling advanced weather etc). > Thanks for that explanation. In the light of this I did some more testing. Switching from Basic to Advanced Weather increases the time spent in the events submodule from a stable 44 ms to over 1000 ms unstable. Setting all flags to 0 reduces this to 90 ms. Switching back to Basic, the time drops to 75. Adding more nasal loops into the equation by lowering the flaps and gear on the Seahawk, I think I can see the effect as a slight increase in the time spent in the events submodule. I'm not clear how deleterious having one submodule taking significantly more time than any other might be. If that is a problem, then large nasal script are not a good idea even if we only enable them on demand. The instability is clearly not a good thing, but is this a GC problem? If it is, then again, large nasal scripts are not good. Perhaps James can fix the GC issue, and this issue will disappear. I hope so. Vivian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
On 11.05.2012 10:21, Renk Thorsten wrote: > The problem with that consequence is: As you switched all loops off, > the Nasal part of Advanced Weather ceased to run completely and all > that's left is the same cloud-generating functionality which is > responsible for Basic Weather (C++ and shader work). If killing all > Nasal did not solve the problems, then they can't be caused by Nasal, > and porting to C++ will not fix them. Not quite true. A significant part of Nasal-related frame rate impact is caused by garbage collection. Its delay/jitter only depends on the number of Nasal objects and their references which need to be searched. Increasing the number of Nasal objects (code+data) existing in memory also increases the delay. The amount of Nasal code which is actually being executed only influences the g/c frequency, i.e. whether the effect is visible every few seconds vs several times per second. This is why we are loading large Nasal modules, like advanced weather, on demand only (= only when feature enabled). We're not unloading objects when disabling a feature though - this usually requires restarting fgfs. This should be remembered when comparing such features (i.e. restart fgfs after disabling advanced weather etc). cheers, Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
> Tested extensively today, and the short answer is "no". There is some > improvement as the flags are successively set to 0, but Advanced Weather > never becomes as smooth as Basic Weather (and that's not without > staggers of > its own). (...) > I suppose the bottom line is that this just might beyond what Nasal was > designed for, and we need to migrate to C++. The problem with that consequence is: As you switched all loops off, the Nasal part of Advanced Weather ceased to run completely and all that's left is the same cloud-generating functionality which is responsible for Basic Weather (C++ and shader work). If killing all Nasal did not solve the problems, then they can't be caused by Nasal, and porting to C++ will not fix them. * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Thorsten > > Does the problem go away if you set local-weather/dynamics-loop-flag=1 > > from the property browser *after* Advanced Weather is started? > > Does the problem go away if you set local-weather/timing-loop-flag=1 > > from the property browser *after* Advanced Weather is started? > > Does the problem go away if you set local-weather/effect-loop-flag=1 > > from the property browser *after* Advanced Weather is started? > > Does the problem go away if you set > > local-weather/interpolation-loop-flag=1 from the property browser > > *after* Advanced Weather is started? > > Does the problem go away if you set > > local-weather/housekeeping-loop-flag=1 from the property browser > > *after* Advanced Weather is started? > > Does the problem go away if you set local-weather/tile-loop-flag=1 > > from the property browser *after* Advanced Weather is started? > Tested extensively today, and the short answer is "no". There is some improvement as the flags are successively set to 0, but Advanced Weather never becomes as smooth as Basic Weather (and that's not without staggers of its own). The longer answer is that the symptoms are readily seen in the System Performance Monitor. There are large differences between the "worst" and "average" frame rates, exceeding 50%. In contrast, the Basic Weather difference is approximately 10%, with some occasional excursions beyond this. Switching from Basic Weather to Advanced, I see that the submodule "events" changes dramatically. With Basic Weather, the numbers in this submodule are consistent with other submodules. With Advanced, events stands out from the rest: not only are the numbers much larger than any of the others but they fluctuate wildly from sample to sample. The total time spent in event during a sample are 3 times that spent in the next largest,"flight" which is the FDM. It is my understanding that submodule "events" is where much of the work for nasal is done: listeners, loop timers and the like. I have placed todays test results here: http://dl.dropbox.com/u/57645542/stagger-data.htm What can be done? I cannot speak authoritatively here - perhaps some expert can jump in. Personally I would try trading in some framerate for smoothness, and let all loops run at maximum frame rate. I think at least some of the problem is load variation as various loop cut in. Loops with diverse time intervals must coincide from time to time; perhaps we are seeing that. I did try starting loops at random intervals in AI Models at one time, but that seemed to make things worse. We know that "for" loops do not help, perhaps those loops can be advanced on index per frame. We know we have problems with Garbage Collection. I suppose the bottom line is that this just might beyond what Nasal was designed for, and we need to migrate to C++. HTH Vivian -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Does the problem go away if you set local-weather/dynamics-loop-flag=1 > from the property browser *after* Advanced Weather is started? > Does the problem go away if you set local-weather/timing-loop-flag=1 > from the property browser *after* Advanced Weather is started? > Does the problem go away if you set local-weather/effect-loop-flag=1 > from the property browser *after* Advanced Weather is started? > Does the problem go away if you set > local-weather/interpolation-loop-flag=1 from the property browser > *after* Advanced Weather is started? > Does the problem go away if you set > local-weather/housekeeping-loop-flag=1 from the property browser *after* > Advanced Weather is started? > Does the problem go away if you set local-weather/tile-loop-flag=1 from > the property browser *after* Advanced Weather is started? Silly me - these flags need to be set to zero of course to disable the loops successively. * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
> It is the largest expanse of intertidal mudflats and sand in the United > Kingdom! Neither of these locations could be remotely described as deep > ocean, and yes, we should certainly be investigating how to best model > such areas of sea. Okay, if you're specifically after deep ocean (i.e. 'sufficiently far' away from the coast, sufficiently deep), then I would agree that it looks pretty much the same everywhere apart from seasonal phenomena. I never really fly across ocean, and so most of the modelling issues I'm interested in are in some sense close to the coast. > I do feel free to disagree - the Caribbean just isn't turquoise, as you > say. There was a time when we prided ourselves on FG being the most realistic > > sim around. Making the ocean the wrong colour to maintain an illusion just > doesn't seem to me to sit right with that. You're talking as if there is an alternative 1a) we make a realistic model of the ocean color 1b) we make the ocean a location dependent plausible but ultimately wrong color But the real alternative is 2a) we make the ocean a constant wrong color everywhere 2b) we make the ocean a location dependent plausible but ultimately wrong color If I had the choice between 1a) and 1b), I would choose 1a) - but I don't see that it can be done. However, given the choice 2), I go for the more realistic alternative there. So - what are you actually arguing or disagreeing with? That a constant sea color everywhere is superior to a location dependent color because the latter is still not 'the real thing'? That we should create a model for sediment transport and algae bloom? That we should feed sea depth information to the shader (but where does it come from, and how is it stored)? > Yup - getting to grips with all that is difficult. Not too dissimilar to > getting reasonable representations of towns and countryside in various > regions in the world, I suppose Getting the town is way easier, because it doesn't need to blend smoothly into anything. So local information is sufficient - all we need to tell the shader is 'If the location is in this (lat, lon) range, use this texture, this set of random buildings, this normal map, this placement mask,... Getting the sea color requires either all depth info tied to the vertex (elevation of the water bottom and elevation of the surface to get lakes) which isn't really forseen in our terrain generating or storing toolchain, or then to somehow do non-local magic in the shader (= how far is this vertex from the coast, then interpolating it's probably that deep). In addition you need (rgb) extinction parameters how the light intensity changes as it passes through water, these are in general location-dependent even inside the scene, since it was mentioned you would need abnormal position dependent reflection for oil films, ... I can see how to get city, town and countryside reasonable for various parts of the world within our current infrastructure. I can't see how to do a realistic sea color model without major modifications of how we store and manage scenery data and how information flows to the shaders. > Anyway, on a more prosaic note - your sea colour script appears to be > bugged here. Hm, you're right. I wasn't really factoring in someone would want to re-start Advanced Weather a few times... and I've found a scheme to optimize it a bit more, so there is potentially a lot of improve. > I wish I could run Advanced Weather here - the frame rates are not at all > bad, but the stagger in the frame rate is too much, and seems to give me > a headache. Okay, maybe we can just try to find out what the problem is and restructure/reconfigure if needed. Single core or multi-core machine with multithreading on (my Nasal runs way smoother if a second core is available)? Does the problem go away if you set local-weather/dynamics-loop-flag=1 from the property browser *after* Advanced Weather is started? Does the problem go away if you set local-weather/timing-loop-flag=1 from the property browser *after* Advanced Weather is started? Does the problem go away if you set local-weather/effect-loop-flag=1 from the property browser *after* Advanced Weather is started? Does the problem go away if you set local-weather/interpolation-loop-flag=1 from the property browser *after* Advanced Weather is started? Does the problem go away if you set local-weather/housekeeping-loop-flag=1 from the property browser *after* Advanced Weather is started? Does the problem go away if you set local-weather/tile-loop-flag=1 from the property browser *after* Advanced Weather is started? (Please test from top to bottom, since this starts with the 'make it more realistic even where most users won't notice' functions and ends with the things which really will break functionality). * Thorsten -- Live Security Virtual Conference Exclusive li
Re: [Flightgear-devel] Sea color
Renk Thorsten writes:
>> There are good sources for sea colour out there - here is one:
>>
>> http://oceancolor.gsfc.nasa.gov/FEATURE/IMAGES/A2008129125500.Scotland.png
>
>> The Northern North Sea, away from the turbidity and major river outfalls
>> of the Southern North Sea, is indistinguishable from the Atlantic, the other
>> side of Scotland.
>
> 4 West, 54 North I get an rgb value of (93, 113, 121)
> 2 East, 54 North I get (31, 71, 83)
>
> There's pretty significant variation everywhere, but I don't see that in that
> picture that the two would even be similar.
>
>> There are many examples in this archive, and if there
>> are differences in the deep ocean colour in any of the oceans they are darned
>> hard to spot. Similarly, if there is a difference between the Atlantic
>> and the Mediterranean, it's very hard to see.
>
> We might not be after the same thing here...
>
> In reality, water color depends on a lot of factors:
>
> * light reflection at the surface, i.e. light diffuseness, intensity, wave
> patterns, sky color, ...
> -> we have a decent way to account for that in the shader
>
> * sediment and algae in the water -> water being a flowing substance,
> all these are variable phenomena, rivers carry a lot of mud in spring
> when the snow thaws, algae bloom seasonally,... we can't model this
> realistically in any case
>
> * near the coast, depth and nature of the bottom -> white underwater
> sand looks quite differently from overgrown rocky bottom, deep water
> looks different from shallow water... we simply don't have this
> information - we might get depth somehow, pass true depth to the
> shader, use it to determine color, then let the shader move the vertex
> up to the water surface, but it's a bit tricky.
>
> Given that we can't do so much realistically anyway for lack of data
> (and lack of framerate - I could probably write a detailed water color
> computing code, but that'd really drive framerate down), my idea is
> more to re-create iconic pictures.
>
> When approaching Nice in sunshine, I want to see one of these postcard
> Cote d'Azur pictures. When landing on a drilling rig in the Northern
> Sea, I don't expect to see this deep blue. Probably in reality the
> differences are driven by differences in lighting, average weather and
> some sediment/algae component - but when I can't do the realistic
> thing, I might as well do the iconic thing.
>
> Many people (including myself) seem to feel that the sea should look
> different in different places. I'm entirely willing to tweak physics
> here a bit to create a better illusion that one is in the place by
> fulfilling the expectation. True, the actual Caribbean deep ocean is
> not turquoise. Then again, the actual Caribbean city doesn't look the
> Flightgear way either. But making sea color a lighter turquoise in the
> Caribbean helps me to maintain the illusion that I am in the Caribbean
> rather than the Northern Sea.
>
> Feel free to disagree, but for me creating the visual environment has
> much more to do with credible illusions than with getting the physics
> of the scene right (disclaimer: my position is diffferent for the
> fidelity of the FDM where we can actually get the physics really right
> since we have often the required amount of data).
There are also nice patterns drawn by wind and by oil.
When the sea is flat, over distances on the order of hundreds of meters
you have changes in the wavelets that give quite different patterns from
a distance.
And furthermore, there some oil floating in nice patterns seeing at
different scales, generally in spiral.
They're stricking when saw from space, but they are also visible from
the beaches.
https://docs.google.com/viewer?a=v&q=cache:P6Iom6_AjrMJ:armi.ucsd.edu/Spirals.pdf+&hl=en&pid=bl&srcid=ADGEEShQ-kOpl7JfwiDDG2SHOGs7fkAkhVrVHwuUs1tJZO9pLKhS9ks-VrdHB2k68QgP1jJ2pKH7HwSKqnzn1tCJ0GLa88JxHVR_4xbQifElWm4FlScwpk0lZ8hfTZWjJYSld-d_ryw7&sig=AHIEtbQ9vmtNjZC9WylNau0sW17FgGEe9A&pli=1
--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Sea color
Thorsten > > There are good sources for sea colour out there - here is one: > > > > > http://oceancolor.gsfc.nasa.gov/FEATURE/IMAGES/A2008129125500.Scotlan > d > > .png > > > The Northern North Sea, away from the turbidity and major river > > outfalls of the Southern North Sea, is indistinguishable from the > > Atlantic, the other side of Scotland. > > 4 West, 54 North I get an rgb value of (93, 113, 121) > 2 East, 54 North I get (31, 71, 83) 2 E 54N? that's the Southern N Sea off the Humber estuary. That is one of the major sources of silt flowing into the North Sea . The depth is probably less than 150m there. 4 W 54N? That's just off Morecambe Bay, which forms part of the Irish Sea. It is the largest expanse of intertidal mudflats and sand in the United Kingdom! Neither of these locations could be remotely described as deep ocean, and yes, we should certainly be investigating how to best model such areas of sea. > There's pretty significant variation everywhere, but I don't see that in that > picture that the two would even be similar. > > > There are many examples in this archive, and if there are differences > > in the deep ocean colour in any of the oceans they are darned hard to > > spot. Similarly, if there is a difference between the Atlantic and the > > Mediterranean, it's very hard to see. > > We might not be after the same thing here... > > In reality, water color depends on a lot of factors: > > * light reflection at the surface, i.e. light diffuseness, intensity, wave > patterns, sky color, ... > -> we have a decent way to account for that in the shader > > * sediment and algae in the water > -> water being a flowing substance, all these are variable phenomena, > -> rivers carry a lot of mud in spring when the snow thaws, algae bloom > -> seasonally,... we can't model this realistically in any case > > * near the coast, depth and nature of the bottom > -> white underwater sand looks quite differently from overgrown rocky > bottom, deep water looks different from shallow water... we simply don't > have this information - we might get depth somehow, pass true depth to the > shader, use it to determine color, then let the shader move the vertex up to > the water surface, but it's a bit tricky. Yup - getting to grips with all that is difficult. Not too dissimilar to getting reasonable representations of towns and countryside in various regions in the world, I suppose > Given that we can't do so much realistically anyway for lack of data (and lack > of framerate - I could probably write a detailed water color computing code, > but that'd really drive framerate down), my idea is more to re-create iconic > pictures. > When approaching Nice in sunshine, I want to see one of these postcard > Cote d'Azur pictures. When landing on a drilling rig in the Northern Sea, I > don't expect to see this deep blue. Probably in reality the differences are > driven by differences in lighting, average weather and some sediment/algae > component - but when I can't do the realistic thing, I might as well do the > iconic thing. > > Many people (including myself) seem to feel that the sea should look > different in different places. I'm entirely willing to tweak physics here a bit to > create a better illusion that one is in the place by fulfilling the expectation. > True, the actual Caribbean deep ocean is not turquoise. Then again, the > actual Caribbean city doesn't look the Flightgear way either. But making sea > color a lighter turquoise in the Caribbean helps me to maintain the illusion > that I am in the Caribbean rather than the Northern Sea. > > Feel free to disagree, but for me creating the visual environment has much > more to do with credible illusions than with getting the physics of the scene > right (disclaimer: my position is diffferent for the fidelity of the FDM where > we can actually get the physics really right since we have often the required > amount of data). I do feel free to disagree - the Caribbean just isn't turquoise, as you say. There was a time when we prided ourselves on FG being the most realistic sim around. Making the ocean the wrong colour to maintain an illusion just doesn't seem to me to sit right with that. But I would also agree that we need to sort out some of the more obvious problems of the shallow seas, if that is possible. Anyway, on a more prosaic note - your sea colour script appears to be bugged here. After I got it to run (my bad, I broke it) I discovered that it iterates at 1 sec not 5 as stated. If I restart Advanced Weather the sampling locations are re-added to the interpolation_vector which grows in size: I have reached 32 so far, but sometimes FG crashes before that point. In any case running Advanced Weather FG crashes here at some point, running out of memory. I would be surprised if this were related to sea-colour, and might indeed be a general problem which Advanced Weather runs into sooner than Basic Weather. You re-instantiate "var ppos" in
Re: [Flightgear-devel] Sea color
> There are good sources for sea colour out there - here is one: > > http://oceancolor.gsfc.nasa.gov/FEATURE/IMAGES/A2008129125500.Scotland.png > The Northern North Sea, away from the turbidity and major river outfalls > of the Southern North Sea, is indistinguishable from the Atlantic, the other > side of Scotland. 4 West, 54 North I get an rgb value of (93, 113, 121) 2 East, 54 North I get (31, 71, 83) There's pretty significant variation everywhere, but I don't see that in that picture that the two would even be similar. > There are many examples in this archive, and if there > are differences in the deep ocean colour in any of the oceans they are darned > hard to spot. Similarly, if there is a difference between the Atlantic > and the Mediterranean, it's very hard to see. We might not be after the same thing here... In reality, water color depends on a lot of factors: * light reflection at the surface, i.e. light diffuseness, intensity, wave patterns, sky color, ... -> we have a decent way to account for that in the shader * sediment and algae in the water -> water being a flowing substance, all these are variable phenomena, rivers carry a lot of mud in spring when the snow thaws, algae bloom seasonally,... we can't model this realistically in any case * near the coast, depth and nature of the bottom -> white underwater sand looks quite differently from overgrown rocky bottom, deep water looks different from shallow water... we simply don't have this information - we might get depth somehow, pass true depth to the shader, use it to determine color, then let the shader move the vertex up to the water surface, but it's a bit tricky. Given that we can't do so much realistically anyway for lack of data (and lack of framerate - I could probably write a detailed water color computing code, but that'd really drive framerate down), my idea is more to re-create iconic pictures. When approaching Nice in sunshine, I want to see one of these postcard Cote d'Azur pictures. When landing on a drilling rig in the Northern Sea, I don't expect to see this deep blue. Probably in reality the differences are driven by differences in lighting, average weather and some sediment/algae component - but when I can't do the realistic thing, I might as well do the iconic thing. Many people (including myself) seem to feel that the sea should look different in different places. I'm entirely willing to tweak physics here a bit to create a better illusion that one is in the place by fulfilling the expectation. True, the actual Caribbean deep ocean is not turquoise. Then again, the actual Caribbean city doesn't look the Flightgear way either. But making sea color a lighter turquoise in the Caribbean helps me to maintain the illusion that I am in the Caribbean rather than the Northern Sea. Feel free to disagree, but for me creating the visual environment has much more to do with credible illusions than with getting the physics of the scene right (disclaimer: my position is diffferent for the fidelity of the FDM where we can actually get the physics really right since we have often the required amount of data). * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] Sea color (was: Re: Regional textures merge request)
> Hmm I've got something wrong here then - if I understand that right I > select > Advanced Weather, and skydome and the sea colour stuff should run? I get > this: > > http://dl.dropbox.com/u/57645542/fgfs-screen-190.png > > It doesn't seem to have any interpolation. The color seems about right to me for what the interpolation should give at the location. There is no interpolation in the scene, because the color change is assumed to happen on a 1000 km scale, so it changes smoothly for the whole scene as a function of position - take the ufo, fly 1000 km away from TNCM, compare sea color and it should go back to the darker blue-green default. Admittedly I haven't really tried flying 1000 km, I've just tested it on startup in different locations. Try starting at Nice (LFMN), that should have a dark blue. > I was thinking of this for loop > for (var i = 0; i < ivector_size; i = i + 1) > I take the view that any for loop is not good, but if ivector_size is > small, I guess it won't matter. It depends on what is executed in the loop. If it's a distance computation or property tree access, then there shouldn't be much more than 20 operations per frame. But I've done benchmark tests with basic math - say the sine function. Here, I could execute about 1.000 operations per frame without creating a measurable slowdown. In this particular loop, it's just adding numbers - that's probably a factor 100 cheaper than computing a sine, so if my benchmark extrapolates correctly, you can run it ~100.000 times per frame before running into trouble. In practice, I think it's unlikely that we'll use more much than 100 points. I have been coding some performance-eating stuff (and even some badly designed performance eating stuff), but that's not one of these cases. Cheers, * Thorsten -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
