Re: [HACKERS] Simulating Clog Contention

2012-02-17 Thread Robert Haas
On Mon, Jan 30, 2012 at 12:26 PM, Robert Haas wrote: >> I was thinking the opposite.  That -i should only print progress >> indication when -d is given.  Or at least knock an order of magnitude >> or two off of how often it does so. > > I'd be in all in favor of having -i emit progress reports 10x

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:48 AM, Jeff Janes wrote: > On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas wrote: >> On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: I think that even in normal (non-initialization) usage, this message should be suppressed when the provided scale factor

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:53 AM, Jeff Janes wrote: > On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen wrote: >> At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: >>> >>> The following patch adds a pgbench option -I to load data using >>> INSERTs >> >> This is just to confirm that the

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen wrote: > At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: >> >> The following patch adds a pgbench option -I to load data using >> INSERTs > > This is just to confirm that the patch applies and builds and works > fine (though of course i

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas wrote: > On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: >>> I think that even in normal (non-initialization) usage, this message >>> should be suppressed when the provided scale factor >>> is equal to the pgbench_branches table count. >> >> The att

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: >> I think that even in normal (non-initialization) usage, this message >> should be suppressed when the provided scale factor >> is equal to the pgbench_branches table count. > > The attached patch does just that.  There is probably no reason to

Re: [HACKERS] Simulating Clog Contention

2012-01-28 Thread Jeff Janes
On Fri, Jan 27, 2012 at 1:45 PM, Jeff Janes wrote: > On Thu, Jan 12, 2012 at 4:31 AM, Simon Riggs wrote: > >> The following patch adds a pgbench option -I to load data using >> INSERTs, so that we can begin benchmark testing with rows that have >> large numbers of distinct un-hinted transaction i

Re: [HACKERS] Simulating Clog Contention

2012-01-27 Thread Jeff Janes
On Thu, Jan 12, 2012 at 4:31 AM, Simon Riggs wrote: > The following patch adds a pgbench option -I to load data using > INSERTs, so that we can begin benchmark testing with rows that have > large numbers of distinct un-hinted transaction ids. With a database > pre-created using this we will be be

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 10:59 AM, Robert Haas wrote: > On Thu, Jan 26, 2012 at 11:41 AM, Merlin Moncure wrote: >> On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen wrote: >>> This is just to confirm that the patch applies and builds and works >>> fine (though of course it does take a long time…

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Robert Haas
On Thu, Jan 26, 2012 at 11:41 AM, Merlin Moncure wrote: > On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen wrote: >> This is just to confirm that the patch applies and builds and works >> fine (though of course it does take a long time… pity there doesn't >> seem to be any easy way to add progr

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen wrote: > This is just to confirm that the patch applies and builds and works > fine (though of course it does take a long time… pity there doesn't > seem to be any easy way to add progress indication like -i has). On any non server grade hardware

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Abhijit Menon-Sen
At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: > > The following patch adds a pgbench option -I to load data using > INSERTs This is just to confirm that the patch applies and builds and works fine (though of course it does take a long time… pity there doesn't seem to be any easy way t

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 1:02 PM, Simon Riggs wrote: > On Thu, Jan 19, 2012 at 5:47 PM, Robert Haas wrote: >> I feel I've adequate explained why it makes sense to me to separate >> those options.  If you want, I'll do the work myself; it will take >> less time than arguing about it. > > If you hav

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 5:47 PM, Robert Haas wrote: > I feel I've adequate explained why it makes sense to me to separate > those options.  If you want, I'll do the work myself; it will take > less time than arguing about it. If you have time to contribute, please use the patch as stands to test

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 11:46 AM, Simon Riggs wrote: > On Thu, Jan 19, 2012 at 4:12 PM, Robert Haas wrote: >> On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs wrote: Also, I don't think the behavior described here should be joined at the hip to --inserts: +        * We do this a

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Marti Raudsepp
On Thu, Jan 19, 2012 at 18:12, Robert Haas wrote: > Right, but the point is that to address Heikki's objection that this > is a special-purpose hack, we should try to make it general, so that > it can be used by other people for other things. Personally I would like to see support for more flexib

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 4:12 PM, Robert Haas wrote: > On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs wrote: >>> Also, I don't think the behavior described here should be joined at >>> the hip to --inserts: >>> >>> +        * We do this after a load by COPY, but before a load via INSERT >>> +      

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs wrote: >> Also, I don't think the behavior described here should be joined at >> the hip to --inserts: >> >> +        * We do this after a load by COPY, but before a load via INSERT >> +        * >> +        * This is done deliberately to ensure that n

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 3:41 PM, Robert Haas wrote: > I agree: I think this is useful. > > However, I think we should follow the precedent of some of the other > somewhat-obscure options we've added recently and have only a long > form option for this: --inserts. Yep, no problem. > Also, I don'

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:18 AM, Simon Riggs wrote: > On Thu, Jan 19, 2012 at 2:36 PM, Heikki Linnakangas > wrote: >> On 12.01.2012 14:31, Simon Riggs wrote: >>> >>> In order to simulate real-world clog contention, we need to use >>> benchmarks that deal with real world situations. >>> >>> Curre

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 2:36 PM, Heikki Linnakangas wrote: > On 12.01.2012 14:31, Simon Riggs wrote: >> >> In order to simulate real-world clog contention, we need to use >> benchmarks that deal with real world situations. >> >> Currently, pgbench pre-loads data using COPY and executes a VACUUM so

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Peter Geoghegan
On 19 January 2012 14:36, Heikki Linnakangas wrote: > No doubt this is handy for testing this particular area, but overall I feel > this is too much of a one-trick pony to include in pgbench. I don't think that being conservative in accepting pgbench options is the right way to go. It's already s

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Heikki Linnakangas
On 12.01.2012 14:31, Simon Riggs wrote: In order to simulate real-world clog contention, we need to use benchmarks that deal with real world situations. Currently, pgbench pre-loads data using COPY and executes a VACUUM so that all hint bits are set on every row of every page of every table. Thu

Re: [HACKERS] Simulating Clog Contention

2012-01-18 Thread Cédric Villemain
> $ pgbench --help > pgbench is a benchmarking tool for PostgreSQL. > > Usage: >  pgbench [OPTIONS]... [DBNAME] > > Initialization options: >  -i           invokes initialization mode using COPY >  -I           invokes initialization mode using INSERTs sounds usefull. what about a long extra opti

[HACKERS] Simulating Clog Contention

2012-01-12 Thread Simon Riggs
In order to simulate real-world clog contention, we need to use benchmarks that deal with real world situations. Currently, pgbench pre-loads data using COPY and executes a VACUUM so that all hint bits are set on every row of every page of every table. Thus, as pgbench runs it sees zero clog acces