Re: [HACKERS] distributed performance testing
On Sat, Aug 13, 2005 at 06:29:58PM -0400, Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Incidentally, use of a different SCM system might well make constructing test sets more simple. Imagine, say, in SVN, you would create a branch called test-set--mm-dd or some such, make your changes there, add a test script under some well known name, and commit the branch. This seems a pretty unconvincing argument for SVN ... we could perfectly well do that in CVS, no? In any case, I think it probably makes more sense to specify tests as 'pull from CVS as of this date/tag and then (optionally) apply these patches'. It doesn't seem to make sense to clutter up CVS just to be able to run performance tests. In any case, I agree. I've been wondering if it makes sense to setup a result repository for dbt* where people running dbt tests could submit results (along with machine config, etc). ISTM that having that would be beneficial on it's own, and we could then build an additional framework for pushing desired tests out to a set of machines. Of course we could use pgbench for this instead of dbt*, but ISTM that dbt is a better choice since it's useful for a broader set of people. The downside is it requires dbt, but that doesn't seem to be a major issue. Also, using dbt means we can test different use cases (dbt2 ~= TPC-C, dbt3 ~= TPC-H, etc), while pgbench is just a single benchmark. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Softwarehttp://pervasive.com512-569-9461 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] distributed performance testing
On Monday 22 August 2005 13:13, Jim C. Nasby wrote: On Sat, Aug 13, 2005 at 06:29:58PM -0400, Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Incidentally, use of a different SCM system might well make constructing test sets more simple. Imagine, say, in SVN, you would create a branch called test-set--mm-dd or some such, make your changes there, add a test script under some well known name, and commit the branch. This seems a pretty unconvincing argument for SVN ... we could perfectly well do that in CVS, no? In any case, I think it probably makes more sense to specify tests as 'pull from CVS as of this date/tag and then (optionally) apply these patches'. It doesn't seem to make sense to clutter up CVS just to be able to run performance tests. In any case, I agree. I've been wondering if it makes sense to setup a result repository for dbt* where people running dbt tests could submit results (along with machine config, etc). ISTM that having that would be beneficial on it's own, and we could then build an additional framework for pushing desired tests out to a set of machines. Of course we could use pgbench for this instead of dbt*, but ISTM that dbt is a better choice since it's useful for a broader set of people. The downside is it requires dbt, but that doesn't seem to be a major issue. Also, using dbt means we can test different use cases (dbt2 ~= TPC-C, dbt3 ~= TPC-H, etc), while pgbench is just a single benchmark. And there is always http://pgfoundry.org/projects/tpc-w-php/ for a ~= TPC-W workload. -- Darcy Buskermolen Wavefire Technologies Corp. http://www.wavefire.com ph: 250.717.0200 fx: 250.763.1759 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] distributed performance testing
On Mon, Aug 22, 2005 at 02:28:54PM -0700, Darcy Buskermolen wrote: On Monday 22 August 2005 13:13, Jim C. Nasby wrote: Of course we could use pgbench for this instead of dbt*, but ISTM that dbt is a better choice since it's useful for a broader set of people. The downside is it requires dbt, but that doesn't seem to be a major issue. Also, using dbt means we can test different use cases (dbt2 ~= TPC-C, dbt3 ~= TPC-H, etc), while pgbench is just a single benchmark. And there is always http://pgfoundry.org/projects/tpc-w-php/ for a ~= TPC-W workload. True, but then you don't get TPC-C, and dbt1 is ~= TPC-W. So with a package of the full dbt suite (doesn't exist yet, but I suspect it wouldn't be hard to change that), you get W, C, H, and eventually TPC-App. Plus, much of what needs to be developed for our use-case would benefit all dbt users, whereas pgbench is only of use to us internally. dbt is also more flexable, since you can vary workload ratios. For example, you can run dbt2 in a 90% read environment if you wanted. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Softwarehttp://pervasive.com512-569-9461 ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] distributed performance testing
Andrew Dunstan [EMAIL PROTECTED] writes: Incidentally, use of a different SCM system might well make constructing test sets more simple. Imagine, say, in SVN, you would create a branch called test-set--mm-dd or some such, make your changes there, add a test script under some well known name, and commit the branch. This seems a pretty unconvincing argument for SVN ... we could perfectly well do that in CVS, no? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] distributed performance testing
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Incidentally, use of a different SCM system might well make constructing test sets more simple. Imagine, say, in SVN, you would create a branch called test-set--mm-dd or some such, make your changes there, add a test script under some well known name, and commit the branch. This seems a pretty unconvincing argument for SVN ... we could perfectly well do that in CVS, no? You could well be right. And I should know better than to engage in speculation like that and possibly cause us to be inundated by proponents of SCMfoo, most of whom will never write a line of code for postgres let alone ever have to commit anything ;-) Pardon my weakness. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
