"Greg Smith" <[EMAIL PROTECTED]> writes: > On Sun, 26 Aug 2007, Kevin Grittner wrote: > >> I also think we need to somehow develop a set of tests which report maximum >> response time on (what should be) fast queries while the database is under >> different loads, so that those of us for whom reliable response time is more >> important than maximum overall throughput are protected from performance >> regressions. > > My guess is that the DBT2 tests that Heikki has been running are a more > complicated than you think they are; there are response time guarantee > requirements in there as well as the throughput numbers. The tests that I run > (which I haven't been publishing yet but will be with the final patch soon) > also report worst-case and 90-th percentile latency numbers as well as TPS. A > "regression" that improved TPS at the expense of those two would not be > considered an improvement by anyone involved here.
TPCC requires that the 90th percentile response time be under 5s for most transactions. It also requires that the average be less than the 90th percentile which helps rule out circumstances where the longest 10% response times are *much* longer than 5s. However in practice neither of those requirements really rule out some pretty bad behaviour as long as it's rare enough. Before the distributed checkpoint patch went in we were finding 60s of zero activity at every checkpoint. But there were so few transactions affected that in the big picture it didn't impact the 90th percentile. It didn't even affect the 95th percentile. I think you had to look at the 99th percentile before it even began to impact the results. I can't really imagine a web site operator being happy if he was told that only 1% of user's clicks resulted in a browser timeout... -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---------------------------(end of broadcast)--------------------------- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match