Are we talking about validating the build process and checking that FG runs,
or about checking the validity of the simulation?
For the former the suggested "buildbot" , or similar, approach, perhaps with
a very simple autopilot guided flight, would be adequate.
Simulation validity checking is another issue, and given the (I hope I do
not offend anyone) rather basic flight models used for most aircraft models
in the FG library as well as the limited availability of accurate predicted
response data is probably not attainable by a project such as Flightgear.
As a now retired flight simulator professional most of my time was devoted
to checking and validating my simulations before they would be accepted for
training (by CAA/FAA) or for flight handling research (by my company's
aerodynamics department). Each aircraft model required tests tailored to the
use that the simulation was designed for.
_____
From: Curtis Olson [mailto:curtol...@gmail.com]
Sent: 04 August 2009 12:37
To: FlightGear developers discussions
Subject: Re: [Flightgear-devel] Automated builds & tests
On Tue, Aug 4, 2009 at 2:00 AM, Tom P <zomm...@gmail.com> wrote:
Hi everybody
I'd like to hear thoughts from the FG community about setting up a system to
perform builds & execute a suite of tests on FlightGear, all automatically.
Right now I've experimented a bit with buildbot, a neat "continuous
integration" tool used by Mozilla and other projects, and I have a system
that can:
* check-out from various repositories
* build all FlightGear components
* perform rudimentary tests on the FG simulator just built, like verifing
the output on the command line and starting the simulator.
Now the next step would be to go airborne!
And there are two issues to resolve before take-off:
1) how to drive the input of the simulator
2) how to read its state
For the second one, I've seen examples of reading the property tree from an
external process, so we should be set, but the solution to driving the sim's
input is still not clear.
Specifically, I'd want to drive it as similarly as possible as when it's
controlled from a keyboard, not go through the property tree to force FGFS
into certain conditions.
By the way, the current setup works on Ubuntu x86-64, but buildbot is easily
extensible and supports Windows and MacOS platforms, so this could become a
cross-platform testing tool for the project.
Hi Tom,
Because, of variations in flight dynamics models, possible variations in
weather conditions, possible variations in frame rates, etc., simply
replaying a series of keyboard commands is probably not going to lead to
repeatable results. I suspect the replayed flight could diverge quite
quickly and quite substantially from the original flight. If you want to
test the simulator during flight, I really think you will have the most luck
under some sort of scripted autopilot control.
You should think about exactly what you are trying to measure and validate.
As soon as you fire up the sim and start the aircraft moving, you've
suddenly moved into the world of flight dynamics and you are looking at the
physics/mathematics model of the aircraft. That's a good thing to look at
though.
One idea to consider is to setup a series of scripted flight tests that
parallel the FAA simulator certification tests. I've gone through the Level
3 FTD set of tests and automated them for work (so I can't share the
resulting scripts unfortunately) but it was an interesting process.
For instance, configure some specific weather conditions, start the aircraft
out at some particular altitude and speed. Setup the aircraft with a
specific weight and CG. Configure the throttle for some particular RPM,
keep the wings straight and level. Now measure the rate of climb (descent)
you observe once the phugoid settles out.
Another test involved setting up straight and level flight at certain known
conditions, then commanding full rudder input while maintaining the same
heading and altitude (steady state side slip.) Measure the resulting bank
angle, amount of required aileron input, and side slip angle.
This is all very interesting stuff, but it tends to focus you more on the
validity of the aircraft model and less on the validity of the simulator
code. The complexity of this in combined with the complexity of everything
else you could be testing and evaluating is quite staggering. That said,
having a few key spot or sanity checks can't hurt either.
Regards,
Curt.
--
Curtis Olson: http://baron.flightgear.org/~curt/
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
