On Tue, Oct 01, 2013 at 02:50:29PM +0200, Fabien COELHO wrote:
> I do not think that there is a clean and simple way to take the
> start/stop period into account when computing global performances of
> a run. The TPC-C benchmark tells to ignore the warmup/closure
> period, whatever they are, and only perform measures within the
> steady state. However the full graph must be provided when the bench
> is provided.
That makes sense to me. "pgbench --progress" and "pgbench --log
--aggregate-interval" are good tools for excluding non-steady-state periods.
> About better measures: If I could rely on having threads, I would
> simply synchronise the threads at the beginning so that they
> actually start after they are all created, and one thread would
> decide when to stop and set a shared volatile variable to stop all
> transactions more or less at once. In this case, the thread start
> time would be taken just after the synchronization, and maybe only
> by thread 0 would be enough.
>
> Note that this is pretty standard stuff with threads, ISTM that it
> would solve most of the issues, *but* this is not possible with the
> "thread fork emulation" implemented by pgbench, which really means
> no threads at all.
You could do those same things in the fork emulation mode using anonymous
shared memory, like we do in the server. That would permit removing the
current "#ifdef PTHREAD_FORK_EMULATION" wart, too.
For the time being, I propose the attached comment patch.
--
Noah Misch
EnterpriseDB http://www.enterprisedb.com
diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c
index 926246e..816400f 100644
--- a/contrib/pgbench/pgbench.c
+++ b/contrib/pgbench/pgbench.c
@@ -2896,7 +2896,16 @@ main(int argc, char **argv)
}
disconnect_all(state, nclients);
- /* get end time */
+ /*
+ * XXX We compute results as though every client of every thread started
+ * and finished at the same time. That model can diverge noticeably
from
+ * reality for a short benchmark run involving relatively many threads.
+ * The first thread may process notably many transactions before the
last
+ * thread begins. Improving the model alone would bring limited
benefit,
+ * because performance during those periods of partial thread count can
+ * easily exceed steady state performance. This is one of the many ways
+ * short runs convey deceptive performance figures.
+ */
INSTR_TIME_SET_CURRENT(total_time);
INSTR_TIME_SUBTRACT(total_time, start_time);
printResults(ttype, total_xacts, nclients, threads, nthreads,
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
