Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-19 Thread Jim Nasby
On Jun 17, 2007, at 4:39 AM, Simon Riggs wrote: pg_start_backup() should be a normal checkpoint I think. No need for backup to be an intrusive process. Good point. A spread out checkpoint can take a long time to finish, though. Is there risk for running into a timeout or something if it takes

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-19 Thread Heikki Linnakangas
Jim Nasby wrote: On Jun 17, 2007, at 4:39 AM, Simon Riggs wrote: pg_start_backup() should be a normal checkpoint I think. No need for backup to be an intrusive process. Good point. A spread out checkpoint can take a long time to finish, though. Is there risk for running into a timeout or

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2007-06-15 at 11:34 +0100, Heikki Linnakangas wrote: - What units should we use for the new GUC variables? From implementation point of view, it would be simplest if checkpoint_write_rate is given as pages/bgwriter_delay, similarly to bgwriter_*_maxpages. I never

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Simon Riggs
On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: We don't really care about units because the way you use it is to nudge it up a little and see if that works etc.. Not necessarily. If it's given in KB/s, you might very well have an idea of how much I/O your hardware is

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: Do we need to know it so exactly that we look at WALInsertLock? Maybe use info_lck to request the latest page, since that is less heavily contended and we need never wait across I/O. Is there such a value available,

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Simon Riggs
On Sun, 2007-06-17 at 12:00 +0100, Heikki Linnakangas wrote: Simon Riggs wrote: On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: Do we need to know it so exactly that we look at WALInsertLock? Maybe use info_lck to request the latest page, since that is less heavily contended

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: XLogCtl-LogwrtRqst.Write is updated every time we insert an xlog record that advances to a new page. It isn't exactly up to date, but it lags behind by no more than a page. Oh, ok. That would work just fine then. -- Heikki Linnakangas EnterpriseDB

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Fri, 2007-06-15 at 11:34 +0100, Heikki Linnakangas wrote: - What units should we use for the new GUC variables? From implementation point of view, it would be simplest if checkpoint_write_rate is given as pages/bgwriter_delay, similarly to bgwriter_*_maxpages. I never liked those

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Fri, 2007-06-15 at 13:11 +0200, Michael Paesold wrote: Heikki Linnakangas wrote: Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: checkpoint_maxpages would seem like a better name; we've already had those _maxpages settings for 3 releases, so changing that is not really an option (at so late a stage). Sure it is. We've never promised stability of obscure tuning settings. For

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Sat, 2007-06-16 at 11:02 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: checkpoint_maxpages would seem like a better name; we've already had those _maxpages settings for 3 releases, so changing that is not really an option (at so late a stage). Sure it is. Maybe, but

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Michael Paesold
Heikki Linnakangas wrote: Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the category of further development and should be pushed to 8.4. Why

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: - The signaling between RequestCheckpoint and bgwriter is a bit tricky. Bgwriter now needs to deal immediate checkpoint requests, like those coming from explicit CHECKPOINT or CREATE DATABASE commands, differently from those triggered by checkpoint_segments. I'm

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Heikki Linnakangas
Michael Paesold wrote: Heikki Linnakangas wrote: Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the category of further development and should

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Heikki Linnakangas
Alvaro Herrera wrote: Heikki Linnakangas wrote: - The signaling between RequestCheckpoint and bgwriter is a bit tricky. Bgwriter now needs to deal immediate checkpoint requests, like those coming from explicit CHECKPOINT or CREATE DATABASE commands, differently from those triggered by

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: Alvaro Herrera wrote: if (BgWriterShmem-ckpt_time_warn elapsed_secs CheckPointWarning) ereport(LOG, (errmsg(checkpoints are occurring too frequently (%d seconds apart),