Hi Alan
The initial objective is to validate the build process and perform basic
checks on a running FG.
Checking the accuracy of the simulation, especially from the point of
view of flight models, is beyond the scope at this point, but it's not
impossible once the test infrastructure evolves.
Tom
Alan Teeder wrote:
Are we talking about validating the build process and checking that FG
runs, or about checking the validity of the simulation?
For the former the suggested "buildbot" , or similar, approach,
perhaps with a very simple autopilot guided flight, would be adequate.
Simulation validity checking is another issue, and given the (I hope I
do not offend anyone) rather basic flight models used for most
aircraft models in the FG library as well as the limited availability
of accurate predicted response data is probably not attainable by a
project such as Flightgear.
As a now retired flight simulator professional most of my time was
devoted to checking and validating my simulations before they would be
accepted for training (by CAA/FAA) or for flight handling research (by
my company's aerodynamics department). Each aircraft model required
tests tailored to the use that the simulation was designed for.
------------------------------------------------------------------------
*From:* Curtis Olson [mailto:curtol...@gmail.com]
*Sent:* 04 August 2009 12:37
*To:* FlightGear developers discussions
*Subject:* Re: [Flightgear-devel] Automated builds & tests
On Tue, Aug 4, 2009 at 2:00 AM, Tom P <zomm...@gmail.com
<mailto:zomm...@gmail.com>> wrote:
Hi everybody
I'd like to hear thoughts from the FG community about setting up a
system to perform builds & execute a suite of tests on FlightGear,
all automatically.
Right now I've experimented a bit with buildbot, a neat
"continuous integration" tool used by Mozilla and other projects,
and I have a system that can:
* check-out from various repositories
* build all FlightGear components
* perform rudimentary tests on the FG simulator just built, like
verifing the output on the command line and starting the simulator.
Now the next step would be to go airborne!
And there are two issues to resolve before take-off:
1) how to drive the input of the simulator
2) how to read its state
For the second one, I've seen examples of reading the property
tree from an external process, so we should be set, but the
solution to driving the sim's input is still not clear.
Specifically, I'd want to drive it as similarly as possible as
when it's controlled from a keyboard, not go through the property
tree to force FGFS into certain conditions.
By the way, the current setup works on Ubuntu x86-64, but buildbot
is easily extensible and supports Windows and MacOS platforms, so
this could become a cross-platform testing tool for the project.
Hi Tom,
Because, of variations in flight dynamics models, possible variations
in weather conditions, possible variations in frame rates, etc.,
simply replaying a series of keyboard commands is probably not going
to lead to repeatable results. I suspect the replayed flight could
diverge quite quickly and quite substantially from the original
flight. If you want to test the simulator during flight, I really
think you will have the most luck under some sort of scripted
autopilot control.
You should think about exactly what you are trying to measure and
validate. As soon as you fire up the sim and start the aircraft
moving, you've suddenly moved into the world of flight dynamics and
you are looking at the physics/mathematics model of the aircraft.
That's a good thing to look at though.
One idea to consider is to setup a series of scripted flight tests
that parallel the FAA simulator certification tests. I've gone
through the Level 3 FTD set of tests and automated them for work (so I
can't share the resulting scripts unfortunately) but it was an
interesting process.
For instance, configure some specific weather conditions, start the
aircraft out at some particular altitude and speed. Setup the
aircraft with a specific weight and CG. Configure the throttle for
some particular RPM, keep the wings straight and level. Now measure
the rate of climb (descent) you observe once the phugoid settles out.
Another test involved setting up straight and level flight at certain
known conditions, then commanding full rudder input while maintaining
the same heading and altitude (steady state side slip.) Measure the
resulting bank angle, amount of required aileron input, and side slip
angle.
This is all very interesting stuff, but it tends to focus you more on
the validity of the aircraft model and less on the validity of the
simulator code. The complexity of this in combined with the
complexity of everything else you could be testing and evaluating is
quite staggering. That said, having a few key spot or sanity checks
can't hurt either.
Regards,
Curt.
--
Curtis Olson: http://baron.flightgear.org/~curt/
<http://baron.flightgear.org/%7Ecurt/>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
------------------------------------------------------------------------
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
