Re: [HACKERS] Simulating Clog Contention

2012-02-17 Thread Robert Haas
On Mon, Jan 30, 2012 at 12:26 PM, Robert Haas robertmh...@gmail.com wrote: I was thinking the opposite.  That -i should only print progress indication when -d is given.  Or at least knock an order of magnitude or two off of how often it does so. I'd be in all in favor of having -i emit

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes jeff.ja...@gmail.com wrote: I think that even in normal (non-initialization) usage, this message should be suppressed when the provided scale factor is equal to the pgbench_branches table count. The attached patch does just that.  There is probably

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes jeff.ja...@gmail.com wrote: I think that even in normal (non-initialization) usage, this message should be suppressed when the provided scale factor is equal to the

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: The following patch adds a pgbench option -I to load data using INSERTs This is just to confirm that the patch applies and builds and works fine (though of

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:53 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: The following patch adds a pgbench option -I to load data using INSERTs This is just

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:48 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes jeff.ja...@gmail.com wrote: I think that even in normal (non-initialization) usage, this message should

Re: [HACKERS] Simulating Clog Contention

2012-01-28 Thread Jeff Janes
On Fri, Jan 27, 2012 at 1:45 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Thu, Jan 12, 2012 at 4:31 AM, Simon Riggs si...@2ndquadrant.com wrote: The following patch adds a pgbench option -I to load data using INSERTs, so that we can begin benchmark testing with rows that have large numbers

Re: [HACKERS] Simulating Clog Contention

2012-01-27 Thread Jeff Janes
On Thu, Jan 12, 2012 at 4:31 AM, Simon Riggs si...@2ndquadrant.com wrote: The following patch adds a pgbench option -I to load data using INSERTs, so that we can begin benchmark testing with rows that have large numbers of distinct un-hinted transaction ids. With a database pre-created using

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Abhijit Menon-Sen
At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: The following patch adds a pgbench option -I to load data using INSERTs This is just to confirm that the patch applies and builds and works fine (though of course it does take a long time… pity there doesn't seem to be any easy way to

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: This is just to confirm that the patch applies and builds and works fine (though of course it does take a long time… pity there doesn't seem to be any easy way to add progress indication like -i has). On any non server

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Robert Haas
On Thu, Jan 26, 2012 at 11:41 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: This is just to confirm that the patch applies and builds and works fine (though of course it does take a long time… pity there doesn't seem to

Re: [HACKERS] Simulating Clog Contention

2012-01-26 Thread Merlin Moncure
On Thu, Jan 26, 2012 at 10:59 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 26, 2012 at 11:41 AM, Merlin Moncure mmonc...@gmail.com wrote: On Thu, Jan 26, 2012 at 8:18 AM, Abhijit Menon-Sen a...@toroid.org wrote: This is just to confirm that the patch applies and builds and works

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Heikki Linnakangas
On 12.01.2012 14:31, Simon Riggs wrote: In order to simulate real-world clog contention, we need to use benchmarks that deal with real world situations. Currently, pgbench pre-loads data using COPY and executes a VACUUM so that all hint bits are set on every row of every page of every table.

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Peter Geoghegan
On 19 January 2012 14:36, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: No doubt this is handy for testing this particular area, but overall I feel this is too much of a one-trick pony to include in pgbench. I don't think that being conservative in accepting pgbench options is

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 2:36 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 12.01.2012 14:31, Simon Riggs wrote: In order to simulate real-world clog contention, we need to use benchmarks that deal with real world situations. Currently, pgbench pre-loads data using COPY

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:18 AM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Jan 19, 2012 at 2:36 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 12.01.2012 14:31, Simon Riggs wrote: In order to simulate real-world clog contention, we need to use benchmarks that

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 3:41 PM, Robert Haas robertmh...@gmail.com wrote: I agree: I think this is useful. However, I think we should follow the precedent of some of the other somewhat-obscure options we've added recently and have only a long form option for this: --inserts. Yep, no

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs si...@2ndquadrant.com wrote: Also, I don't think the behavior described here should be joined at the hip to --inserts: +        * We do this after a load by COPY, but before a load via INSERT +        * +        * This is done deliberately to

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 4:12 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs si...@2ndquadrant.com wrote: Also, I don't think the behavior described here should be joined at the hip to --inserts: +        * We do this after a load by COPY, but before

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Marti Raudsepp
On Thu, Jan 19, 2012 at 18:12, Robert Haas robertmh...@gmail.com wrote: Right, but the point is that to address Heikki's objection that this is a special-purpose hack, we should try to make it general, so that it can be used by other people for other things. Personally I would like to see

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 11:46 AM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Jan 19, 2012 at 4:12 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 19, 2012 at 10:55 AM, Simon Riggs si...@2ndquadrant.com wrote: Also, I don't think the behavior described here should be joined at

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Simon Riggs
On Thu, Jan 19, 2012 at 5:47 PM, Robert Haas robertmh...@gmail.com wrote: I feel I've adequate explained why it makes sense to me to separate those options.  If you want, I'll do the work myself; it will take less time than arguing about it. If you have time to contribute, please use the

Re: [HACKERS] Simulating Clog Contention

2012-01-19 Thread Robert Haas
On Thu, Jan 19, 2012 at 1:02 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Jan 19, 2012 at 5:47 PM, Robert Haas robertmh...@gmail.com wrote: I feel I've adequate explained why it makes sense to me to separate those options.  If you want, I'll do the work myself; it will take less time

Re: [HACKERS] Simulating Clog Contention

2012-01-18 Thread Cédric Villemain
$ pgbench --help pgbench is a benchmarking tool for PostgreSQL. Usage:  pgbench [OPTIONS]... [DBNAME] Initialization options:  -i           invokes initialization mode using COPY  -I           invokes initialization mode using INSERTs sounds usefull. what about a long extra option:

[HACKERS] Simulating Clog Contention

2012-01-12 Thread Simon Riggs
In order to simulate real-world clog contention, we need to use benchmarks that deal with real world situations. Currently, pgbench pre-loads data using COPY and executes a VACUUM so that all hint bits are set on every row of every page of every table. Thus, as pgbench runs it sees zero clog