Hi,
While working on performance issues reported by cppcheck, I wanted to
compare performance to a known baseline and came up with this simple test
bench.
It's:
- written in Python
- uses the telnet interface to FG
- doesn't have other dependencies and
- runs on Linux and Mac (tested) and run on Windows in its first incarnation
The script is called framerate.py and it replays a pre-recorded flight
(test2.out.gz described here:
http://wiki.flightgear.org/Suggested_Prerecorded_Flights) while collecting
numbers on frame rate and maximum latency.
The repository for this is at :
https://github.com/flighten/test
but if there's an option to host this on Gitorious (fg/test ?), I would be
glad to
Tom
On Sat, Jun 22, 2013 at 12:34 AM, Renk Thorsten <thorsten.i.r...@jyu.fi>wrote:
> > It would need to take in all of the, by now well known, variables -
> > making it by no means a simple beast to manage.
> >
> > If this could be automated in some way it would be much easier to
> > capture, and then submit, consistent data.
>
> Let's think this from the end. What would we like to know?
>
> I'm usually interested in very specific before/after questions. For
> instance, I can push some shader code into a conditional clause and
> benchmark this to run faster on my system. I'd like to know - does it
> generalize? I've learned that optimization seems to generalize across
> nvidia hardware, but I'd like to get feedback in a before/after situation
> from a Radeon user.
>
> Or, system dependent optimizations. Stuart has introduced a cloud LOD
> system and has some framerate gain from it in overcast layers. I've been
> playing with it and couldn't get much clear difference in performance, so I
> just switched it off completely. What I'd be interested in is - for what
> hardware do we see framerate gain, and what LOD distances would people
> typically select in order to get a good balance between visuals and
> framerate. Or would they prefer to vary cloud density, or cloud visibility
> radius? If we would know what most people select if given the choice, we
> could set reasonable defaults and structure the GUI accordingly.
>
> In the present case - take my suspicion that I'm running something wrong
> because I have the gut feeling Rembrandt should be running faster - what I
> really would like to know is if I am indeed doing something wrong (James,
> please let me know if you find anything...). It's difficult to see how a
> performance benchmark test would answer the question unless you code in all
> possible causes of low performance already.
>
> For these questions, a standardized benchmark isn't so terribly useful,
> because I'm at least partially after how users would adjust a balance if
> they start on it, or I have a very specific question evaluating a specific
> change. A standardized benchmark would be, if we get enough data, be more
> of a general warning system - suppose we regularly monitor performance on
> 50 different systems, and after some commit we see 20% performance drop on
> 35 of them - that's indicate that the commit might be in some way
> problematic. But for this, we would require a regular time history -
> basically the monitoring script should run and report after every update of
> either FG or the drivers.
>
> Well, that's what I would like to know - what kind of information would
> others like to have?
>
> * Thorsten
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Flightgear-devel mailing list
> Flightgear-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/flightgear-devel
>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Flightgear-devel mailing list
Flightgear-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/flightgear-devel
