Re: [PATCHES] Load distributed checkpoint V3

2007-04-06 Thread Greg Smith
On Fri, 6 Apr 2007, Takayuki Tsunakawa wrote: could anyone evaluate O_SYNC approach again that commercial databases use and tell me if and why PostgreSQL's fsync() approach is better than theirs? I noticed a big improvement switching the WAL to use O_SYNC (+O_DIRECT) instead of fsync on my

Re: [PATCHES] Load distributed checkpoint V3

2007-04-06 Thread Takayuki Tsunakawa
From: Greg Smith [EMAIL PROTECTED] If you compare how Oracle handles their writes and checkpoints to the Postgres code, it's obvious they have a different architecture that enables them to support sync writing usefully. I'd recommend the Database Writer Process section of

Re: [PATCHES] Load distributed checkpoint V3

2007-04-06 Thread Simon Riggs
On Fri, 2007-04-06 at 02:53 -0400, Greg Smith wrote: If you compare how Oracle handles their writes and checkpoints to the Postgres code, it's obvious they have a different architecture that enables them to support sync writing usefully. I'd recommend the Database Writer Process section

Re: [PATCHES] Load distributed checkpoint V3

2007-04-06 Thread Greg Smith
On Fri, 6 Apr 2007, Takayuki Tsunakawa wrote: Hmm... what makes you think that sync writes is useful for Oracle and not for PostgreSQL? They do more to push checkpoint-time work in advance, batch writes up more efficiently, and never let clients do the writing. All of which make for a

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Heikki Linnakangas
ITAGAKI Takahiro wrote: Here is the latest version of Load distributed checkpoint patch. Unfortunately because of the recent instrumentation and CheckpointStartLock patches this patch doesn't apply cleanly to CVS HEAD anymore. Could you fix the bitrot and send an updated patch, please? --

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
On Thu, 5 Apr 2007, Heikki Linnakangas wrote: Unfortunately because of the recent instrumentation and CheckpointStartLock patches this patch doesn't apply cleanly to CVS HEAD anymore. Could you fix the bitrot and send an updated patch, please? The Logging checkpoints and other slowdown

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Heikki Linnakangas
Greg Smith wrote: On Thu, 5 Apr 2007, Heikki Linnakangas wrote: Bgwriter has two goals: 1. keep enough buffers clean that normal backends never need to do a write 2. smooth checkpoints by writing buffers ahead of time Load distributed checkpoints will do 2. in a much better way than the

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: The number of buffers evicted by normal backends in a bgwriter_delay period is simple to keep track of, just increase a counter in StrategyGetBuffer and reset it when bgwriter wakes up. We can use that as an estimate of X with some safety margin.

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Heikki Linnakangas
Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: The number of buffers evicted by normal backends in a bgwriter_delay period is simple to keep track of, just increase a counter in StrategyGetBuffer and reset it when bgwriter wakes up. We can use that as an estimate of X with some

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Takayuki Tsunakawa
Hello, long time no see. I'm sorry to interrupt your discussion. I'm afraid the code is getting more complicated to continue to use fsync(). Though I don't intend to say the current approach is wrong, could anyone evaluate O_SYNC approach again that commercial databases use and tell me if and why

Re: [PATCHES] Load distributed checkpoint V3

2007-04-05 Thread Greg Smith
On Thu, 5 Apr 2007, Heikki Linnakangas wrote: The purpose of the bgwriter_all_* settings is to shorten the duration of the eventual checkpoint. The reason to shorten the checkpoint duration is to limit the damage to other I/O activity it causes. My thinking is that assuming the LDC patch is

Re: [PATCHES] Load distributed checkpoint V3

2007-03-25 Thread Greg Smith
On Fri, 23 Mar 2007, ITAGAKI Takahiro wrote: Here is the latest version of Load distributed checkpoint patch. Couple of questions for you: -Is it still possible to get the original behavior by adjusting your tunables? It would be nice to do a before/after without having to recompile, and

Re: [PATCHES] Load distributed checkpoint V3

2007-03-25 Thread ITAGAKI Takahiro
Greg Smith [EMAIL PROTECTED] wrote: Here is the latest version of Load distributed checkpoint patch. Couple of questions for you: -Is it still possible to get the original behavior by adjusting your tunables? It would be nice to do a before/after without having to recompile, and I

Re: [PATCHES] Load distributed checkpoint V3

2007-03-25 Thread Greg Smith
On Mon, 26 Mar 2007, ITAGAKI Takahiro wrote: I'm assuming checkpoints are triggered by timeout in normal use -- and it's my recommended configuration whether the patch is installed or not. I'm curious what other people running fairly serious hardware do in this area for write-heavy loads,

Re: [PATCHES] Load distributed checkpoint V3

2007-03-24 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches It will be applied as soon as one of the PostgreSQL committers reviews and approves it. ---