Re: [HACKERS] Reducing stats collection overhead

2007-07-31 Thread Arjen van der Meijden
On 31-7-2007 5:07 Alvaro Herrera wrote: Arjen van der Meijden wrote: Afaik Tom hadn't finished his patch when I was testing things, so I don't know. But we're in the process of benchmarking a new system (dual quad-core Xeon) and we'll have a look at how it performs in the postgres 8.2dev we

Re: [HACKERS] Reducing stats collection overhead

2007-07-30 Thread Alvaro Herrera
Arjen van der Meijden wrote: Afaik Tom hadn't finished his patch when I was testing things, so I don't know. But we're in the process of benchmarking a new system (dual quad-core Xeon) and we'll have a look at how it performs in the postgres 8.2dev we used before, the stable 8.2.4 and a

Re: [HACKERS] Reducing stats collection overhead

2007-05-19 Thread Arjen van der Meijden
Afaik Tom hadn't finished his patch when I was testing things, so I don't know. But we're in the process of benchmarking a new system (dual quad-core Xeon) and we'll have a look at how it performs in the postgres 8.2dev we used before, the stable 8.2.4 and a fresh HEAD-checkout (which we'll

Re: [HACKERS] Reducing stats collection overhead

2007-05-18 Thread Alvaro Herrera
Tom Lane wrote: Arjen van der Meijden told me that according to the tweakers.net benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed here that for small SELECT queries issued as separate transactions, there's a significant difference. I think much of the difference stems

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian
Yes, it seems we will have to do something for 8.3. I assume the method below would reduce frequent updates of the stats_command_string too. --- Tom Lane wrote: Arjen van der Meijden told me that according to the

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Yes, it seems we will have to do something for 8.3. Yeah, we've kind of ignored any possible overhead of the stats mechanism for awhile, but I think we've got to face up to this if we're gonna have it on by default. I assume the method below would

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Yes, it seems we will have to do something for 8.3. Yeah, we've kind of ignored any possible overhead of the stats mechanism for awhile, but I think we've got to face up to this if we're gonna have it on by default. I assume the

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Simon Riggs
On Sun, 2007-04-29 at 00:44 -0400, Tom Lane wrote: The first design that comes to mind is that at transaction end (pgstat_report_tabstat() time) we send a stats message only if at least X milliseconds have elapsed since we last sent one, where X is PGSTAT_STAT_INTERVAL or closely related to

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: So I think that complicating the design with, say, a timeout counter to force out the stats after a sleep interval is not necessary. Doing so would add a couple of kernel calls to every client interaction so I'd really rather avoid that. Any thoughts,

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Lukas Kahwe Smith
Tom Lane wrote: The first design that comes to mind is that at transaction end (pgstat_report_tabstat() time) we send a stats message only if at least X milliseconds have elapsed since we last sent one, where X is PGSTAT_STAT_INTERVAL or closely related to it. We also make sure to flush stats

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Alvaro Herrera
Tom Lane wrote: The first design that comes to mind is that at transaction end (pgstat_report_tabstat() time) we send a stats message only if at least X milliseconds have elapsed since we last sent one, where X is PGSTAT_STAT_INTERVAL or closely related to it. We also make sure to flush

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: The first design that comes to mind is that at transaction end (pgstat_report_tabstat() time) we send a stats message only if at least X milliseconds have elapsed since we last sent one, where X is PGSTAT_STAT_INTERVAL or closely

Re: [HACKERS] Reducing stats collection overhead

2007-04-29 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: If we want to have an idle_in_statement_timeout then we'll need to introduce a select loop instead of just directly blocking on recv anyways. Does that mean we may as well bite the bullet now? If we wanted such a timeout (which I personally don't) we

[HACKERS] Reducing stats collection overhead

2007-04-28 Thread Tom Lane
Arjen van der Meijden told me that according to the tweakers.net benchmark, HEAD is noticeably slower than 8.2.4, and I soon confirmed here that for small SELECT queries issued as separate transactions, there's a significant difference. I think much of the difference stems from the fact that we