>> And as I said in an earlier post, the measurements taken here show the >> _real_ time it takes for the CLI to perform some database query.
> Sorry, your test tool does not quite do what you say... it does not > measure _the_ time it takes, rather, it measures only _a_ single run > which by itself *plus* all the perturbing affects of an uncontrolled > system, is terribly inaccurate. The numbers reported early demonstrate > and support this. A single run of the tool is competing with the rest > of the processes running on the system - there are dozens or hundreds > of threads also running and competing. > > Your test tool does not isolate the various affects that perturb > measurements, so its the benchmarker's job to defeat such affects. > With the earlier results reported being 2-3x out of agreement, it is > clear that what is being benchmarked, is not in fact what you believe > is being measured. Therefore, drawing any conclusions is not very > meaningful, or useful. > > Numerous background processes, virus scanners, network activity, disk > spinup time, low-power to max-power CPU speedup time, swapping, disk > cache, hardware interrupts, are factors which need to be eliminated and > reduced before conclusions can be drawn. All true, but please bear in mind what this tool actually tries to do. There are quite a few complaints about performance of the server. Performance in this context is something that is perceived, it is not a measurement of "top speed". When people complain, they probably just tried to use their SB. During that test, there were all sorts of other processes running, just as you explained. That very experience of performance makes them act and send out a call for help. This tool tries to do exactly that. It is _intended_ to run on a system that is polluted by all sorts of junk. The measurement would not be realistic if there wasn't any real life interference by whatever tries to slow the server down. All we have now is some vague indication of performance. If someone complains "the server stalls when I navigate to that menu, then click right, and then press play", it could be very handy to have the queries to the database that correspond to his actions, and have his server (running all the junk that is messing up the machine) to spit out a more tangible value than "it is sooo slow". I don't expect the tool to be very accurate in the light of all that is said, but the bottom line is that if someone wants their toy to play a piece of music, and it takes say 1 minute to start the play whereas a "normal" server should be able to start in about a second, this tool could give a more accurate indication of what the user experiences. If the stats are very poor, maybe people could do some digging into what is making the server so slow. Turn off whatever service they suspect, run a couple more tests (using the same tool with the same queries on the tuned server), and if these new tests show a significant and consistent drop in response time (say, a factor of two or three), then I guess they are on to something. These are just ballpark figures (and very probably a huge ballpark at that), but still the tool could be used to quantify what people see on their messed-up server. It is no different than the server stats that the nightlies can spit out. They too have to be scrutinized with care, and cannot be readily compared to other installs. Niek. _______________________________________________ Discuss mailing list [email protected] http://lists.slimdevices.com/lists/listinfo/discuss
