Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-25 Thread Bort, Paul
Jim Nasby wrote: > > Why is it being hard-coded? I think it makes a lot more sense to allow > pg_bench options to be specified in the buildfarm config. Even better > yet would be specifying them on the command line, which would allow > members to run a more rigorous test once a day/week (I'm think

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-25 Thread Jim C. Nasby
On Sun, Jul 23, 2006 at 11:52:14PM -0400, Bort, Paul wrote: > -hackers, > > With help from Andrew Dunstan, I'm adding the ability to do a pgbench > run after all of the other tests during a buildfarm run. > > Andrew said I should solicit opinions as to what parameters to use. A > cursory search

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Andrew Dunstan
Bort, Paul wrote: Given the concerns about running this on machines that don't have a lot of CPU and disk to spare, should it ship disabled? Yes, certainly. Andrew, what do you think of pgbench reports shipping separately? I have no idea how the server end is set up, so I don't know how mu

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Bort, Paul
Andrew Dunstan wrote: > > We are really not going to go in this direction. If you want ideal > performance tests then a heterogenous distributed collection of > autonomous systems like buildfarm is not what you want. > > You are going to have to live with the fatc that there will be > occasio

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Andrew Dunstan
Tom Lane wrote: Mark Kirkwood <[EMAIL PROTECTED]> writes: Scale factor 10 produces an accounts table of about 130 Mb. Given that most HW these days has at least 1G of ram, this probably means not much retrieval IO is tested (only checkpoint and wal fsync). Do we want to try 100 or even 200?

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Andrew Dunstan
Gavin Sherry wrote: Not all machines stay the same over time. A machine may by upgraded, a machine may be getting backed up or may in some other way be utilised during a performance test. This would skew the stats for that machine. It may confuse people more than help them... At the very least,

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Andrew Dunstan
Dave Page wrote: With help from Andrew Dunstan, I'm adding the ability to do a pgbench run after all of the other tests during a buildfarm run. Please ensure the run is optional. The machine hosting Snake and Bandicoot is currently running 16 builds a day, and I'd prefer not to signif

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Stefan Kaltenbrunner
Mark Kirkwood wrote: > Tom Lane wrote: >> "Bort, Paul" <[EMAIL PROTECTED]> writes: >>> Andrew said I should solicit opinions as to what parameters to use. A >>> cursory search through the archives led me to pick a scaling factor of >>> 10, 5 users, and 100 transactions. >> >> 100 transactions seems

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-24 Thread Dave Page
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Bort, Paul > Sent: 24 July 2006 04:52 > To: pgsql-hackers@postgresql.org > Subject: [HACKERS] Adding a pgbench run to buildfarm > > -hackers, > > With help from

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Gavin Sherry
On Mon, 24 Jul 2006, Mark Kirkwood wrote: > Tom Lane wrote: > > Mark Kirkwood <[EMAIL PROTECTED]> writes: > >> Scale factor 10 produces an accounts table of about 130 Mb. Given that > >> most HW these days has at least 1G of ram, this probably means not much > >> retrieval IO is tested (only check

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Mark Kirkwood
Tom Lane wrote: Mark Kirkwood <[EMAIL PROTECTED]> writes: Scale factor 10 produces an accounts table of about 130 Mb. Given that most HW these days has at least 1G of ram, this probably means not much retrieval IO is tested (only checkpoint and wal fsync). Do we want to try 100 or even 200? (o

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Tom Lane
Mark Kirkwood <[EMAIL PROTECTED]> writes: > Scale factor 10 produces an accounts table of about 130 Mb. Given that > most HW these days has at least 1G of ram, this probably means not much > retrieval IO is tested (only checkpoint and wal fsync). Do we want to > try 100 or even 200? (or recommen

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Bort, Paul
> 100 transactions seems barely enough to get through startup > transients. > Maybe 1000 would be good. OK. > > I think the hard part of this is the reporting process. How > do we track how performance varies over time? It doesn't > seem very useful to compare different buildfarm members, b

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Mark Kirkwood
Tom Lane wrote: "Bort, Paul" <[EMAIL PROTECTED]> writes: Andrew said I should solicit opinions as to what parameters to use. A cursory search through the archives led me to pick a scaling factor of 10, 5 users, and 100 transactions. 100 transactions seems barely enough to get through startup t

Re: [HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Tom Lane
"Bort, Paul" <[EMAIL PROTECTED]> writes: > Andrew said I should solicit opinions as to what parameters to use. A > cursory search through the archives led me to pick a scaling factor of > 10, 5 users, and 100 transactions. 100 transactions seems barely enough to get through startup transients. May

[HACKERS] Adding a pgbench run to buildfarm

2006-07-23 Thread Bort, Paul
-hackers, With help from Andrew Dunstan, I'm adding the ability to do a pgbench run after all of the other tests during a buildfarm run. Andrew said I should solicit opinions as to what parameters to use. A cursory search through the archives led me to pick a scaling factor of 10, 5 users, and 1