Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-28 Thread Kenneth Marshall
On Wed, Aug 22, 2007 at 07:33:35PM +0400, Dmitry Potapov wrote: Hello! We run a large (~66Gb) web-backend database on Postgresql 8.2.4 on Linux. The hardware is Dual Xeon 5130 with 16Gb ram, LSI Megaraid U320-2x scsi controller w/512Mb writeback cache and a BBU. Storage

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-28 Thread Erik Jones
On Aug 22, 2007, at 10:57 AM, Kenneth Marshall wrote: On Wed, Aug 22, 2007 at 07:33:35PM +0400, Dmitry Potapov wrote: Hello! We run a large (~66Gb) web-backend database on Postgresql 8.2.4 on Linux. The hardware is Dual Xeon 5130 with 16Gb ram, LSI Megaraid U320-2x scsi

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-23 Thread Greg Smith
On Thu, 23 Aug 2007, Dmitry Potapov wrote: I'm planning to do so, but before I need to take a look at postgresql source and dev documentation to find how exactly IO is done, to be able to explain the issue to linux kernel people. I can speed that up for you.

[PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Dmitry Potapov
Hello! We run a large (~66Gb) web-backend database on Postgresql 8.2.4 on Linux. The hardware is Dual Xeon 5130 with 16Gb ram, LSI Megaraid U320-2x scsi controller w/512Mb writeback cache and a BBU. Storage setup contains 3 raid10 arrays (data, xlog, indexes, each on different

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Mark Mielke
Are you able to show that the dirty pages are all coming from postgres? Cheers, mark -- Mark Mielke [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Dmitry Potapov
2007/8/22, Joshua D. Drake [EMAIL PROTECTED]: We've run into an issue of IO storms on checkpoints. Once in 20min (which is checkpoint_interval) the database becomes unresponsive for about 4-8 seconds. Query processing is suspended, server does nothing but writing What are your

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Dmitry Potapov
2007/8/22, Mark Mielke [EMAIL PROTECTED]: Are you able to show that the dirty pages are all coming from postgres? I don't know how to prove that, but I suspect that nothing else except postgres writes to disk on that system, because it runs nothing except postgresql and syslog (which I

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Dmitry Potapov
2007/8/22, Kenneth Marshall [EMAIL PROTECTED]: You are working at the correct level. The bgwriter performs the I/O smoothing function at the database level. Obviously, the OS level smoothing function needed to be tuned and you have done that within the parameters of the OS. You may want to

Re: [PERFORM] io storm on checkpoints, postgresql 8.2.4, linux

2007-08-22 Thread Greg Smith
On Wed, 22 Aug 2007, Dmitry Potapov wrote: I found this http://www.westnet.com/~gsmith/content/linux-pdflush.htm If you do end up following up with this via the Linux kernel mailing list, please pass that link along. I've been meaning to submit it to them and wait for the flood of