Re: [Flightgear-devel] Nav-cache
I wonder how we deal with this with the next release- I'm sure a whole lot more users will complain about the stuck while launching FGFS. By printing a message like 'Building database during first startup' on the screen? By including the binary nav data with the release? Doesn't look like a show-stopper to me... Let me just say that I'm _extremely_ grateful for this feature - I so frequently just start Flightgear, have a look how a parameter change looks in practice and end it again, and reducing the startup time by half makes a lot of difference. * Thorsten -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
On 4 Oct 2012, at 07:27, Renk Thorsten wrote: By printing a message like 'Building database during first startup' on the screen? That's already done. By including the binary nav data with the release? This is possible but there's something weird going on with some people's rebuilds. Touching the files will cause a rebuild, so rebuilds need to take a 'reasonable' amount of time. Shipping the binary data improves first-launch perceptions, but many applications do additional work on their first launch after an install, so it's not my top concern. As I said in the bug, my concern is why the rebuild times are so bad for some people - it's purely a CPU / disk-bound operation, and for me and all people I've asked, it's around the ~60sec for debug, ~30sec for release times for a complete rebuild. (That time will be dramatically reduced if we get the taxiway data out of apt.dat in the future, since taxiways account for 80% of the lines in the file). Regards, James -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Hello, I thought today it would be nice again to have a flight in FGFS, since I found some spare time. Updated FGData, download the latest Jenkins build #726 for win32 and launched it. Since a whole while now, a bit more than 30min, it is stuck at Loading navigation datas. And I'm still waiting, and waiting... I guess it has something to do with the changes announced here, but it is just a guess. No relevant console output (just: Warning: GraphicsWindowWin32::grabFocus() - Failed grabbing the focus) win32 Jenkins Build #726, updated FGData Dualcore 2,6 Ghz, 4GB Ram, Nvidia GeForce GTX460 1GB VRAM Bug Report: https://code.google.com/p/flightgear-bugs/issues/detail?id=894 Cheers Heiko still in work: http://www.hoerbird.net/galerie.html But already done: http://www.hoerbird.net/reisen.html -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/03/2012 07:30 PM, Heiko Schulz wrote: But already done: http://www.hoerbird.net/reisen.html Gives me a 404 error. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlBsfpkACgkQty+BhcbHvXioQwCfe9QrHjxs04IURsZeww6V/tDK e5oAn0XRapdpkWVC8YrdYoinT6PWgL88 =19st -END PGP SIGNATURE- -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
On 10/03/2012 07:30 PM, Heiko Schulz wrote: But already done: http://www.hoerbird.net/reisen.html Gives me a 404 error. me too ;-) Changed the signature any other comments on my problem? After a while it loaded now the nav datas, but whenever I start FGFS again, it takes the same time again. Not sure if this behavior is intended... -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Hi Heiko, De: Heiko Schulz any other comments on my problem? After a while it loaded now the nav datas, but whenever I start FGFS again, it takes the same time again. Not sure if this behavior is intended... I know some people erase the content of the folder having autosave.xml. This folder is now used to host a navigation data cache. The first time, a SQL database is built to speedup future start. If this cache is erased every time, it defeats the purpose of the cache and make the start much longer every time. Just guessing... Regards, -Fred -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Hi Fred, I know some people erase the content of the folder having autosave.xml. This folder is now used to host a navigation data cache. The first time, a SQL database is built to speedup future start. If this cache is erased every time, it defeats the purpose of the cache and make the start much longer every time. Just guessing... Regards, -Fred Yes, that might be the case. In the meanwhile I found out myself that it hosts the navigation cache in the autosave-folder. The reason why it had to built it up again was simply, that I didn't quit FGFS on the usual normal way but just hitting the Close-button of the FGFS-window. I wonder how we deal with this with the next release- I'm sure a whole lot more users will complain about the stuck while launching FGFS. Regards Heiko -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
On 19 Sep 2012, at 19:18, Martin Spott wrote: Yes, I've seen matches of the octree_node's with data from navaids for example, but as far as I remember there's no logical link, just a geographical. Right ? Depends what you mean - I'm parsing the name of the marker beacon / ILS to find the matching runway. This isn't a new feature - I added that logic last year or more - so I could have a FGRunway* runway() accessor on FGNavRecord. It's the sqlite rowID of the airport record - both in the positioned table, but also the 'airports' table. Ah, indeed, the rowid - that's neat. The rowid wasn't that obvious from the SQL dump I was staring at (I simply didn't care counting the rows myself ;-) Right, by default sqlite doesn't include the rowid column in SELECT results - you need to ask for it explicitly. James -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
would any of these changes have affected the Equipment/map performance ? I get a lot of disc activity now while panning the map ... and it takes 10 -15 seconds now before it refreshes , and thats with every mouse drag.This only used to happen when zooming out , and the refresh time was much shorter... Syd -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Syd wrote: -Original Message- From: syd adams [mailto:adams@gmail.com] Sent: 20 September 2012 16:22 To: FlightGear developers discussions Subject: Re: [Flightgear-devel] Nav-cache would any of these changes have affected the Equipment/map performance ? I get a lot of disc activity now while panning the map ... and it takes 10 -15 seconds now before it refreshes , and thats with every mouse drag.This only used to happen when zooming out , and the refresh time was much shorter... First time I used it, it took 10 mins to zoom out to the full extent, but once the cache was established it refreshed very quickly. Vivian -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Yes, the map performance is 'different'. However I'm about to port the map to use the canvas, so please forgive the current performance issues for a little while. James On 20 Sep 2012, at 16:22, syd adams adams@gmail.com wrote: would any of these changes have affected the Equipment/map performance ? I get a lot of disc activity now while panning the map ... and it takes 10 -15 seconds now before it refreshes , and thats with every mouse drag.This only used to happen when zooming out , and the refresh time was much shorter... Syd -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
[Flightgear-devel] Nav-cache
I've just pushed a large change to next, which adds a binary cache of most of the navigation data. The cache is stored in FG_HOME/navdata.cache, and rebuilt if the timestamps on any of the data files change (apt.dat, nav.data, fix.dat and so on). When the cache needs to be rebuilt, startup will take a bit longer than before, but when the cache is valid, startup is /much/ faster, especially for debug builds. Memory consumption is also lower, since we don't keep airports / fixes / taxiways / runways in memory until they're needed. This is a fairly large change, so please look out for any bugs related to navaids / startup position / GPS / route-manager searches and lookups either getting different results or failing. I've tested here locally and things seem sane, and have had positive feedback from various testers on IRC and email, but I still expect to find some edge cases which need further work. There's future work to move even more data into the cache - eg parking positions - which will further help performance for FMS / map systems since we won't need to parse lots of XML data repeatedly. I'm going to let this change settle down before adding more cached data, however. Regards, James -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
Hi James, nice feature - I like storing this sort of stuff in structured databases :-) There's one item looking a little bit strange to me: Apparently the positioned table has a numeric identifier airport to refer runways and taxiways to their respective airport. This seems to be a simple sequence - but the records containing the airport name and ident (type = 1) are always having airport = 0. The entire schema is probably going to work as expected as long as the ordering in the positioned table remains unchanged (this looks to me being mostly a verbatim adaption of the apt.dat plus a few custom additions), but I don't understand how you manage to group runways and taxiways together with their airport if the ordering gets confused. Cheers, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
On 19 Sep 2012, at 17:47, Martin Spott wrote: Hi James, nice feature - I like storing this sort of stuff in structured databases :-) There's one item looking a little bit strange to me: Apparently the positioned table has a numeric identifier airport to refer runways and taxiways to their respective airport. This seems to be a simple sequence - but the records containing the airport name and ident (type = 1) are always having airport = 0. Not just runways or taxiways - also marker beacons, ILSs, towers, comm frequencies and anything else that might be located at the airport. It's the sqlite rowID of the airport record - both in the positioned table, but also the 'airports' table. For all the secondary tables which extend 'positioned', the rowids are the join key. Sqlite rowIds are 64 bit unsigned values, and that's *also* the PositionedID values you will see in the code in several places now. The entire schema is probably going to work as expected as long as the ordering in the positioned table remains unchanged (this looks to me being mostly a verbatim adaption of the apt.dat plus a few custom additions), but I don't understand how you manage to group runways and taxiways together with their airport if the ordering gets confused. There's no ordering dependency - when I see the first airport line, I add the row to positioned, and that generates the new row ID. That's then stored by the airport-loader, and passed when inserting rows for all other things located at the airport. The grouping is then something like 'select from positioned where type=blah and airport=airportID'. You could do a positioned-positioned join to select directly by ICAO, but I'm trying (and have managed!) to avoid any JOINs apart from when creating the cache, i.e very rarely. All the runtime queries should be a single table, and on an indexed column. James -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
Re: [Flightgear-devel] Nav-cache
James Turner wrote: On 19 Sep 2012, at 17:47, Martin Spott wrote: There's one item looking a little bit strange to me: Apparently the positioned table has a numeric identifier airport to refer runways and taxiways to their respective airport. This seems to be a simple sequence - but the records containing the airport name and ident (type = 1) are always having airport = 0. Not just runways or taxiways - also marker beacons, ILSs, towers, comm frequencies and anything else that might be located at the airport. Yes, I've seen matches of the octree_node's with data from navaids for example, but as far as I remember there's no logical link, just a geographical. Right ? It's the sqlite rowID of the airport record - both in the positioned table, but also the 'airports' table. Ah, indeed, the rowid - that's neat. The rowid wasn't that obvious from the SQL dump I was staring at (I simply didn't care counting the rows myself ;-) Thanks, Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Flightgear-devel mailing list Flightgear-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/flightgear-devel
