Hi Curt
That's very advanced stuff about FAA simulator certification tests and
testing all aspects of flight dynamics.
As a starter, I was thinking about something simpler than validating the
accuracy of the flight models.
And yes, it cannot be just a script replaying a sequence of commands in
a dumb way, we need a feedback loop between driving the input and
reading the internal state of the simulator.
Once it's clear how to drive the input, these are some of the tests that
come to mind:
- start the simulator
- it starts in X seconds, and the test fails if the time is above a
specific threshold (say 50 secs)
- take a profile of the start sequence (I'm thinking about using
oprofile on Linux)
- shows that we spend X % of the startup time in some routine, loading
scenery, and Y % starting subsystems
- with the airplane on the ground
- check that input is working, moving surfaces and reading back the
specific property
- start engine(s)
- oh, yeah, it would be ideal to have a unified way to start engines
on all planes, in addition to the realistic way, of course!
- check parking brakes
- it would be really nice to always start with brakes engaged, it's
an item on my wishlist
- rev engines up and down
- take off
- use an extremely simple feedback loop to
- maintain direction on the runway (we know the angle)
- and attitude during take off
- exit loop once we are above a certain altitude
- take screen snapshot (it could be the final step of each test)
- once airborne,
- gear in
- insert autopilot
- test wing leveler
- test heading control
- test altitude control
- ... you get the picture
The nice thing is that if we make this generic enough, we could use it
to test the simulator with most of the planes released with FlightGear
(or in CVS for that matter).
So, to get back to my question, is there a way to drive the simulator
input from an external process?
Thanks
Tom "Tom-cat"
Curtis Olson wrote:
On Tue, Aug 4, 2009 at 2:00 AM, Tom P <zomm...@gmail.com
<mailto:zomm...@gmail.com>> wrote:
Hi everybody
I'd like to hear thoughts from the FG community about setting up a
system to perform builds & execute a suite of tests on FlightGear,
all automatically.
Right now I've experimented a bit with buildbot, a neat
"continuous integration" tool used by Mozilla and other projects,
and I have a system that can:
* check-out from various repositories
* build all FlightGear components
* perform rudimentary tests on the FG simulator just built, like
verifing the output on the command line and starting the simulator.
Now the next step would be to go airborne!
And there are two issues to resolve before take-off:
1) how to drive the input of the simulator
2) how to read its state
For the second one, I've seen examples of reading the property
tree from an external process, so we should be set, but the
solution to driving the sim's input is still not clear.
Specifically, I'd want to drive it as similarly as possible as
when it's controlled from a keyboard, not go through the property
tree to force FGFS into certain conditions.
By the way, the current setup works on Ubuntu x86-64, but buildbot
is easily extensible and supports Windows and MacOS platforms, so
this could become a cross-platform testing tool for the project.
Hi Tom,
Because, of variations in flight dynamics models, possible variations
in weather conditions, possible variations in frame rates, etc.,
simply replaying a series of keyboard commands is probably not going
to lead to repeatable results. I suspect the replayed flight could
diverge quite quickly and quite substantially from the original
flight. If you want to test the simulator during flight, I really
think you will have the most luck under some sort of scripted
autopilot control.
You should think about exactly what you are trying to measure and
validate. As soon as you fire up the sim and start the aircraft
moving, you've suddenly moved into the world of flight dynamics and
you are looking at the physics/mathematics model of the aircraft.
That's a good thing to look at though.
One idea to consider is to setup a series of scripted flight tests
that parallel the FAA simulator certification tests. I've gone
through the Level 3 FTD set of tests and automated them for work (so I
can't share the resulting scripts unfortunately) but it was an
interesting process.
For instance, configure some specific weather conditions, start the
aircraft out at some particular altitude and speed. Setup the
aircraft with a specific weight and CG. Configure the throttle for
some particular RPM, keep the wings straight and level. Now measure
the rate of climb (descent) you observe once the phugoid settles out.
Another test involved setting up straight and level flight at certain
known conditions, then commanding full rudder input while maintaining
the same heading and altitude (steady state side slip.) Measure the
resulting bank angle, amount of required aileron input, and side slip
angle.
This is all very interesting stuff, but it tends to focus you more on
the validity of the aircraft model and less on the validity of the
simulator code. The complexity of this in combined with the
complexity of everything else you could be testing and evaluating is
quite staggering. That said, having a few key spot or sanity checks
can't hurt either.
Regards,
Curt.
--
Curtis Olson: http://baron.flightgear.org/~curt/
<http://baron.flightgear.org/%7Ecurt/>
------------------------------------------------------------------------
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
------------------------------------------------------------------------
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
