Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-19 Thread Heikki Linnakangas
Jim Nasby wrote: On Jun 17, 2007, at 4:39 AM, Simon Riggs wrote: pg_start_backup() should be a normal checkpoint I think. No need for backup to be an intrusive process. Good point. A spread out checkpoint can take a long time to finish, though. Is there risk for running into a timeout or somet

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-19 Thread Jim Nasby
On Jun 17, 2007, at 4:39 AM, Simon Riggs wrote: pg_start_backup() should be a normal checkpoint I think. No need for backup to be an intrusive process. Good point. A spread out checkpoint can take a long time to finish, though. Is there risk for running into a timeout or something if it takes s

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: XLogCtl->LogwrtRqst.Write is updated every time we insert an xlog record that advances to a new page. It isn't exactly up to date, but it lags behind by no more than a page. Oh, ok. That would work just fine then. -- Heikki Linnakangas EnterpriseDB http://www.enterpris

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Simon Riggs
On Sun, 2007-06-17 at 12:00 +0100, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: > >>> Do we need to know it so exactly that we look > >>> at WALInsertLock? Maybe use info_lck to request the latest page, since > >>> that is less heav

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: Do we need to know it so exactly that we look at WALInsertLock? Maybe use info_lck to request the latest page, since that is less heavily contended and we need never wait across I/O. Is there such a value available,

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Simon Riggs
On Sun, 2007-06-17 at 08:51 +0100, Heikki Linnakangas wrote: > > We don't really care about units because > > the way you use it is to nudge it up a little and see if that works > > etc.. > > Not necessarily. If it's given in KB/s, you might very well have an idea > of how much I/O your hardware

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-17 Thread Heikki Linnakangas
Simon Riggs wrote: On Fri, 2007-06-15 at 11:34 +0100, Heikki Linnakangas wrote: - What units should we use for the new GUC variables? From implementation point of view, it would be simplest if checkpoint_write_rate is given as pages/bgwriter_delay, similarly to bgwriter_*_maxpages. I never li

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Sat, 2007-06-16 at 11:02 -0400, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > checkpoint_maxpages would seem like a better name; we've already had > > those _maxpages settings for 3 releases, so changing that is not really > > an option (at so late a stage). > > Sure it is.

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > checkpoint_maxpages would seem like a better name; we've already had > those _maxpages settings for 3 releases, so changing that is not really > an option (at so late a stage). Sure it is. We've never promised stability of obscure tuning settings. For s

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Fri, 2007-06-15 at 13:11 +0200, Michael Paesold wrote: > Heikki Linnakangas wrote: > > Here's an updated WIP version of the LDC patch. I just spreads the > > writes, that achieves the goal of smoothing the checkpoint I/O spikes. I > > think sorting the writes etc. is interesting but falls in t

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-16 Thread Simon Riggs
On Fri, 2007-06-15 at 11:34 +0100, Heikki Linnakangas wrote: > - What units should we use for the new GUC variables? From > implementation point of view, it would be simplest if > checkpoint_write_rate is given as pages/bgwriter_delay, similarly to > bgwriter_*_maxpages. I never liked those *_m

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Alvaro Herrera wrote: > > if (BgWriterShmem->ckpt_time_warn && elapsed_secs < > > CheckPointWarning) > > ereport(LOG, > > (errmsg("checkpoints are occurring too > > frequently (%d seconds apar

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Heikki Linnakangas
Alvaro Herrera wrote: Heikki Linnakangas wrote: - The signaling between RequestCheckpoint and bgwriter is a bit tricky. Bgwriter now needs to deal immediate checkpoint requests, like those coming from explicit CHECKPOINT or CREATE DATABASE commands, differently from those triggered by checkpo

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Heikki Linnakangas
Michael Paesold wrote: Heikki Linnakangas wrote: Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the category of further development and should

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Alvaro Herrera
Heikki Linnakangas wrote: > - The signaling between RequestCheckpoint and bgwriter is a bit tricky. > Bgwriter now needs to deal immediate checkpoint requests, like those > coming from explicit CHECKPOINT or CREATE DATABASE commands, differently > from those triggered by checkpoint_segments. I'

Re: [PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Michael Paesold
Heikki Linnakangas wrote: Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the category of further development and should be pushed to 8.4. Why

[PATCHES] Load Distributed Checkpoints, revised patch

2007-06-15 Thread Heikki Linnakangas
Here's an updated WIP version of the LDC patch. I just spreads the writes, that achieves the goal of smoothing the checkpoint I/O spikes. I think sorting the writes etc. is interesting but falls in the category of further development and should be pushed to 8.4. The documentation changes are n