Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-12 Thread Vivek Khera
On May 12, 2004, at 3:22 PM, [EMAIL PROTECTED] wrote: I don't see that. But I also set checkpoint segments to about 50 on my big server. But wouldn't that affect checkpoint frequency, not checkpoint cost Seems reasonable. I suppose checkpointing doesn't cost as much disk I/O as vacuum does. My

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-12 Thread jao
Quoting Vivek Khera <[EMAIL PROTECTED]>: > > "TL" == Tom Lane <[EMAIL PROTECTED]> writes: > > TL> Jack Orenstein <[EMAIL PROTECTED]> writes: > >> I'm looking at one case in which two successive transactions, each > >> updating a handful of records, take 26 and 18 *seconds* (not msec) to > >>

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-12 Thread Vivek Khera
> "TL" == Tom Lane <[EMAIL PROTECTED]> writes: TL> Jack Orenstein <[EMAIL PROTECTED]> writes: >> I'm looking at one case in which two successive transactions, each >> updating a handful of records, take 26 and 18 *seconds* (not msec) to >> complete. These transactions normally complete in unde

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Rob Sell wrote: > > If you don't mind slumming on ebay :-) keep an eye out for PERC III cards, > they are dell branded LSI cards. Perc = Power Edge Raid Controller. There > are models on there dual channel u320 and dell usually sells them with > battery backed cache. That's

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Paul Tuckfield wrote: > Love that froogle. > > It looks like a nice card. One thing I didn't get straight is if > the cache is writethru or write back. > > If the original posters problem is truly a burst write problem (and not > linux caching or virtual memory overcom

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Rob Sell
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of scott.marlowe Sent: Tuesday, May 11, 2004 2:23 PM To: Paul Tuckfield Cc: [EMAIL PROTECTED]; Matthew Nuzum; [EMAIL PROTECTED]; Rob Fielding Subject: Re: [PERFORM] Configuring PostgreSQL to minimize impact of

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Paul Tuckfield
Love that froogle. It looks like a nice card. One thing I didn't get straight is if the cache is writethru or write back. If the original posters problem is truly a burst write problem (and not linux caching or virtual memory overcommitment) then writeback is key. On Tue, 11 May 2004, P

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Paul Tuckfield wrote: > If you are having a "write storm" or bursty writes that's burying > performance, a scsi raid controler with writeback cache will greatly > improve the situation, but I do believe they run around $1-2k. If > it's write specific problem, the cache ma

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Paul Tuckfield
The king of statistics in these cases, is probably vmstat. one can drill down on specific things from there, but first you should send some vmstat output. Reducing cache -> reducing IO suggests to me the OS might be paging out shared buffers. This is indicated by activity in the "si" and "so

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004 [EMAIL PROTECTED] wrote: > Quoting Rob Fielding <[EMAIL PROTECTED]>: > > > Assuming you're running with optimal schema and index design (ie you're > > not doing extra work unnecessarily), and your backend has > > better-then-default config options set-up (plenty of tips arou

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread jao
Quoting Rob Fielding <[EMAIL PROTECTED]>: > Assuming you're running with optimal schema and index design (ie you're > not doing extra work unnecessarily), and your backend has > better-then-default config options set-up (plenty of tips around here), > then disk arrangement is critical to smooth

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Rob Fielding
Matthew Nuzum wrote: Jack Orenstein <[EMAIL PROTECTED]> writes: I'm looking at one case in which two successive transactions, each updating a handful of records, take 26 and 18 *seconds* (not msec) to complete. These transactions normally complete in under 30 msec. None of this is necessarily goi

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Tom Lane
Jan Wieck <[EMAIL PROTECTED]> writes: > If we would combine the background writer and the checkpointer, ... which in fact is on my agenda of things to do ... > then a > "checkpoint flush" could actually be implemented as a temporary change > in that activity that basically is done by not reeval

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread Matthew Nuzum
> > Jack Orenstein <[EMAIL PROTECTED]> writes: > > I'm looking at one case in which two successive transactions, each > > updating a handful of records, take 26 and 18 *seconds* (not msec) to > > complete. These transactions normally complete in under 30 msec. ... > None of this is necessarily go

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-10 Thread Tom Lane
Jack Orenstein <[EMAIL PROTECTED]> writes: > I'm looking at one case in which two successive transactions, each > updating a handful of records, take 26 and 18 *seconds* (not msec) to > complete. These transactions normally complete in under 30 msec. I've seen installations in which it seemed that

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-10 Thread Bruce Momjian
Jack Orenstein wrote: > Bruce Momjian wrote: > > [EMAIL PROTECTED] wrote: > > > >>4. I understand that a "background writer" is being contemplated for > >>7.5. Will that replace or augment the checkpoint process? Any > >>comments on how that work will apply to my problem would be > >>appreciated

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-10 Thread Jack Orenstein
Bruce Momjian wrote: [EMAIL PROTECTED] wrote: 4. I understand that a "background writer" is being contemplated for 7.5. Will that replace or augment the checkpoint process? Any comments on how that work will apply to my problem would be appreciated. I wouldn't mind seeing the average performance

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-10 Thread Bruce Momjian
[EMAIL PROTECTED] wrote: > 4. I understand that a "background writer" is being contemplated for > 7.5. Will that replace or augment the checkpoint process? Any > comments on how that work will apply to my problem would be > appreciated. I wouldn't mind seeing the average performance, > (without t

[PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-10 Thread jao
My company is developing a PostgreSQL application. We're using 7.3.4 but will soon upgrade to 7.4.x. Our OS is RedHat 9. Our production machines have 512 MB RAM and IDE disks. So far we've been using default configuration settings, but I have started to examine performance and to modify these sett