Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-05-16 Thread Kyotaro HORIGUCHI
Hello, On Sun, May 13, 2012 at 10:38 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I've overlooked that startup process of the standby reads archives first, and then WAL. But the current patch enables progress governing based on checkpoint_segments during archive recovery on

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-05-13 Thread Kyotaro HORIGUCHI
Hello, I've returned from my overseas trip for just over one week. # and I'll leave Japan again after this...     restorePtr = replayPtr = receivePtr But XLByteLT(recievePtr, replayPtr) this should not return true under the condition above.. Something wrong in my assumption? When

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-27 Thread Kyotaro HORIGUCHI
Hello, At Wed, 25 Apr 2012 02:31:24 +0900, Fujii Masao masao.fu...@gmail.com wrote in CAHGQGwE1OvB=HLcahLeL5oP66sxsfsGMgwU3MqAAwZ_Vr=x...@mail.gmail.com If we are allowed to be tolerant of the temporary lack of coherence in shared memory there, the spinlock could be removed. But the

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-27 Thread Kyotaro HORIGUCHI
Thank you for sugestion. This still makes catching up in standby mode slower, as you get many more restartpoints. The reason for ignoring checkpoint_segments during recovery was to avoid that. I may have a misunderstanding around there, or your intention. I understand that standby creates no

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-27 Thread Kyotaro HORIGUCHI
Sorry for broken message. From I said that the former (spinlock) could be dropped, but the latter (read as volatile) should be needed. I said that the former (spinlock) could be dropped from the view of functionarity, but the latter (read as volatile) should be needed. From the view of

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Kyotaro HORIGUCHI
Hello,  - xlog.c: Make StandbyMode shared.  - checkpointer.c: Use IsStandbyMode() to check if postmaster is   under standby mode. IsStandbyMode() looks overkill to me. The standby mode flag is forcibly turned off at the end of recovery, but its change doesn't need to be shared to the

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Heikki Linnakangas
On 23.04.2012 02:59, Fujii Masao wrote: On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, this is new version of standby checkpoint_segments patch. Thanks for the patch! This still makes catching up in standby mode slower, as you get many

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Fujii Masao
On Tue, Apr 24, 2012 at 3:53 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 23.04.2012 02:59, Fujii Masao wrote: On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp  wrote: Hello, this is new version of standby checkpoint_segments patch.

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Fujii Masao
On Tue, Apr 24, 2012 at 3:47 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello,  - xlog.c: Make StandbyMode shared.  - checkpointer.c: Use IsStandbyMode() to check if postmaster is   under standby mode. IsStandbyMode() looks overkill to me. The standby mode flag is

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-22 Thread Fujii Masao
On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, this is new version of standby checkpoint_segments patch. Thanks for the patch!  - xlog.c: Make StandbyMode shared.  - checkpointer.c: Use IsStandbyMode() to check if postmaster is   under

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Fujii Masao
On Wed, Apr 18, 2012 at 10:22 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I tried that at first. But I suppose the requirement here is 'if reading segments comes via replication stream, enable throttling by checkpoint_segments.' and WalRcvInProgress() seems fit to check that.

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Kyotaro HORIGUCHI
I convinced that current patch has a problem, and will come up with the new patch later. I tried that at first. But I suppose the requirement here is 'if reading segments comes via replication stream, enable throttling by checkpoint_segments.' and WalRcvInProgress() seems fit to

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Kyotaro HORIGUCHI
Hello, this is new version of standby checkpoint_segments patch. - xlog.c: Make StandbyMode shared. - checkpointer.c: Use IsStandbyMode() to check if postmaster is under standby mode. regards, -- Kyotaro Horiguchi NTT Open Source Software Center == My e-mail address has been changed

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The first one is for 9.2dev, and the second is 9.1.3 backported version. === By the way, I took a close look around there, I agree with it

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 09:50, Kyotaro HORIGUCHI wrote: This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows down the recovery unnecessarily if you're restoring from a backup and you replay, The variable StandbyMode is false on archive recovery, so no checkpoint triggerred during then. xlog.c:10026

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Sorry, I've wrote something wrong. The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows down the recovery unnecessarily if you're restoring from a backup and you replay, The variable StandbyMode is false on archive recovery, so no checkpoint

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, this message is attached with the patch which did not tested. That is for show the way. On Tue, Apr 17, 2012 at 9:38 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: But I think referring checkpoint_segment on such case should be inhibited, and I suppose it is possible using

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 3:50 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: These seems quite reasonable. These conditions make following conditional expression.     restorePtr = replayPtr = receivePtr But XLByteLT(recievePtr, replayPtr) this should not return true under the

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 11:50 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hmm. StandbyMode is a local variable which cannot be accessed in checkpointer. But WalRcvInProgress() which shows if wal receiver is running seems to be usable to ENABLE governing progress by

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hmm. StandbyMode is a local variable which cannot be accessed in checkpointer. But WalRcvInProgress() which shows if wal receiver is running seems to be usable to ENABLE governing progress by checkpoint_segments. Even when walreceiver is not running and WAL files are read from the

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Simon Riggs
On Mon, Apr 16, 2012 at 1:05 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, this is bug report and a patch for it. The first patch in the attachments is for 9.2dev and next one is for 9.1.3. On the current 9.2dev, IsCheckpointOnSchedule@checkpointer.c does not check

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Fujii Masao
On Mon, Apr 16, 2012 at 9:05 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: In the backported version to 9.1.3, bgwriter.c is modified instead of checkpointer.c in 9.2. And GetWalRcvWriteRecPtr() is used as the equivalent of GetStandbyFlushRecPtr() in 9.2. In 9,2,

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Kyotaro HORIGUCHI
Hello, thank you for comment. In the backported version to 9.1.3, bgwriter.c is modified instead of checkpointer.c in 9.2. And GetWalRcvWriteRecPtr() is used as the equivalent of GetStandbyFlushRecPtr() in 9.2. In 9,2, GetXLogReplayRecPtr() should be used instead of