Am 18.04.2010 um 04:57 schrieb Dan Kegel: > At the moment, my plan is to simply dedicate a machine > and script a daily run of a couple benchmarks with +fps, > and upload the resulting framerates to a web page > so we can notice if somebody checks in something that > slows them down. > - Dan I still have my cxtest scripts and php hackerish that runs a bunch of benchmarks, and I'm still running them. Unfortunately this stuff is a pain to maintain, for various reasons.
I'm testing 3dmark2k, 3dmark2k1, ut2004, halflife2 and team fortress 2. I am also running the Wine tests. The big requirement for the benchmark games is that I have a way to automatically read the results from the games. Running with +fps and collecting the output is imprecise IMO. The main problems are: *) Maintainance. The tf2 timedemo format changes every now and then, and similar issues *) The way I wrote the code. Very hacky. *) The test machines. I could use a server farm, but I don't have any. The boxes are standing in my living room. *) System updates(ubuntu) need to be installed and can affect performance. *) Unreliable apps. The steam login fails every now and then, requiring manual intervention.
