Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-17 Thread Kyotaro HORIGUCHI
Thank you for committing. On 02/14/2014 10:38 AM, Kyotaro HORIGUCHI wrote: Finally, the patch you will find attached is fixed only in styling mentioned above from your last patch. This patch applies current HEAD and I confirmed that it fixes this issue but I have not checked the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Kyotaro HORIGUCHI
Hello, Before taking up the topic.. At Thu, 13 Feb 2014 19:45:38 +0200, Heikki Linnakangas wrote On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-14 Thread Heikki Linnakangas
On 02/14/2014 10:38 AM, Kyotaro HORIGUCHI wrote: Finally, the patch you will find attached is fixed only in styling mentioned above from your last patch. This patch applies current HEAD and I confirmed that it fixes this issue but I have not checked the lastSourceFailed section. Simple file

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a new segment after writing the end-of-recovery checkpoint, so that the segment on the new timeline is archived sooner? It would be better to zero-fill and switch

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/12/2014 01:24 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-01-13 52d3caff.3010...@vmware.com Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Kyotaro HORIGUCHI
Hello, I might have been misunderstood your words. At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas hlinnakan...@vmware.com wrote in 52fc7e2a.9060...@vmware.com On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 01:37 PM, Kyotaro HORIGUCHI wrote: At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas hlinnakan...@vmware.com wrote in 52fc7e2a.9060...@vmware.com On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 52fc9468.4050...@vmware.com With 9.1, it works, but 9.2 and 9.3 don't archive anything until I remove the test ! -f part. (An alternative fix would be to declare the behavior ok and adjust that example in the config.) Hmm, the behavior is the same in 9.1 and

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 02:42 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 52fc9468.4050...@vmware.com With 9.1, it works, but 9.2 and 9.3 don't archive anything until I remove the test ! -f part. (An alternative fix would be to declare the behavior ok and adjust that example in the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 52fcca40.3060...@vmware.com I was testing this with streaming replication; 9.1 and 9.2 behave the same in that scenario. But they differ when doing archive recovery. Is this an argument for back-patching the don't archive last segment from old timeline

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 03:53 PM, Christoph Berg wrote: Is removing the test ! -f part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a moment, after the partial file was copied, it will have created

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Christoph Berg
Re: Heikki Linnakangas 2014-02-13 52fcd02c.3060...@vmware.com Is removing the test ! -f part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a moment, after the partial file was copied, it

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 04:07 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 52fcd02c.3060...@vmware.com Is removing the test ! -f part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the problem your test script demonstrates can be fixed

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-13 Thread Heikki Linnakangas
On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Tomonari Katsumata
Hi Heikki, I need PostgreSQL9.3 which fixed this problem. It didn't happen in PostgreSQL9.2, so I agree with your proposal which changes are done against 93_STABLE and master. Can you fix this in next release(9.3.3)? Tomonari Katsumata (2014/01/13 20:16), Heikki Linnakangas

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Christoph Berg
Re: Heikki Linnakangas 2014-01-13 52d3caff.3010...@vmware.com Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is still up and running, and

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-02-12 Thread Kyotaro HORIGUCHI
Hello, I need PostgreSQL9.3 which fixed this problem. It didn't happen in PostgreSQL9.2, so I agree with your proposal which changes are done against 93_STABLE and master. Can you fix this in next release(9.3.3)? I was going to push to move this a bit, but... Wouldn't it be better to

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-13 Thread Heikki Linnakangas
On 01/09/2014 10:55 PM, Josh Berkus wrote: On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tomonari Katsumata
Hi, Somebody is reading this thread? This problem seems still remaining on REL9_3_STABLE. Many users would face this problem, so we should resolve this in next release. I think his patch is reasonable to fix this problem. Please check this again. regards, -- Tomonari

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Amit Langote
On Thu, Dec 12, 2013 at 11:00 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for that of the same segment id but for newer timeline.

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 08:18 PM, Heikki Linnakangas wrote: On 12/12/2013 04:00 AM, Kyotaro HORIGUCHI wrote: Hello, we happened to see server crash on archive recovery under some condition. After TLI was incremented, there should be the case that the WAL file for older timeline is archived but not for

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: Actually, why is the partially-filled 00010002 file archived in the first place? ... So, the rationale is that otherwise it would take a long time until that segment is archived. To be precise, I don't think the segment with

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:16 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: Actually, why is the partially-filled 00010002 file archived in the first place? ... So, the rationale is that otherwise it would take a long time until that segment is archived. To be

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/09/2014 10:16 PM, Tom Lane wrote: Don't we want to archive both? If you want to recover to the end of the old timeline, you're going to need that file too, no? Hmm. It should be the responsibility of the original server to archive

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Heikki Linnakangas
On 01/09/2014 10:36 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/09/2014 10:16 PM, Tom Lane wrote: Don't we want to archive both? If you want to recover to the end of the old timeline, you're going to need that file too, no? Hmm. It should be the

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

2014-01-09 Thread Josh Berkus
On 01/09/2014 12:05 PM, Heikki Linnakangas wrote: Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the original server is still up and running, and writing more

Re: [HACKERS] bug of recovery?

2011-12-02 Thread Heikki Linnakangas
On 04.10.2011 09:43, Fujii Masao wrote: On Mon, Oct 3, 2011 at 4:32 PM, Simon Riggssi...@2ndquadrant.com wrote: I don't think this should use the rm_safe_restartpoint machinery. As you said, it's not tied to any specific resource manager. And I've actually been thinking that we will get rid of

Re: [HACKERS] bug of recovery?

2011-11-24 Thread Simon Riggs
On Tue, Oct 4, 2011 at 7:43 AM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Oct 3, 2011 at 4:32 PM, Simon Riggs si...@2ndquadrant.com wrote: I don't think this should use the rm_safe_restartpoint machinery. As you said, it's not tied to any specific resource manager. And I've actually been

Re: [HACKERS] bug of recovery?

2011-10-04 Thread Fujii Masao
On Mon, Oct 3, 2011 at 4:32 PM, Simon Riggs si...@2ndquadrant.com wrote: I don't think this should use the rm_safe_restartpoint machinery. As you said, it's not tied to any specific resource manager. And I've actually been thinking that we will get rid of rm_safe_restartpoint altogether in the

Re: [HACKERS] bug of recovery?

2011-10-03 Thread Heikki Linnakangas
On 29.09.2011 14:31, Fujii Masao wrote: On Tue, Sep 27, 2011 at 8:06 PM, Florian Pflugf...@phlo.org wrote: Actually, why don't we use that machinery to implement this? There's currently no rm_safe_restartpoint callback for RM_XLOG_ID, so we'd just need to create one that checks whether

Re: [HACKERS] bug of recovery?

2011-10-03 Thread Simon Riggs
On Mon, Oct 3, 2011 at 8:21 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 29.09.2011 14:31, Fujii Masao wrote: On Tue, Sep 27, 2011 at 8:06 PM, Florian Pflugf...@phlo.org  wrote: Actually, why don't we use that machinery to implement this? There's currently no

Re: [HACKERS] bug of recovery?

2011-10-03 Thread Simon Riggs
On Mon, Oct 3, 2011 at 6:23 AM, Fujii Masao masao.fu...@gmail.com wrote: So I think that the idea should be implemented separately from the patch I've posted. Agreed. I'll do a final review and commit today. --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL

Re: [HACKERS] bug of recovery?

2011-10-02 Thread Fujii Masao
On Fri, Sep 30, 2011 at 3:57 PM, Simon Riggs si...@2ndquadrant.com wrote: I think we should issue PANIC if the source is a critical rmgr, or just WARNING if from a non-critical rmgr, such as indexes. Ideally, I think we should have a mechanism to allow indexes to be marked corrupt. For

Re: [HACKERS] bug of recovery?

2011-09-30 Thread Simon Riggs
On Fri, Sep 30, 2011 at 2:09 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Sep 29, 2011 at 11:12 PM, Florian Pflug f...@phlo.org wrote: On Sep29, 2011, at 13:49 , Simon Riggs wrote: This worries me slightly now though because the patch makes us PANIC in a place we didn't used to and

Re: [HACKERS] bug of recovery?

2011-09-29 Thread Fujii Masao
On Tue, Sep 27, 2011 at 8:06 PM, Florian Pflug f...@phlo.org wrote: On Sep27, 2011, at 07:59 , Heikki Linnakangas wrote: On 27.09.2011 00:28, Florian Pflug wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have

Re: [HACKERS] bug of recovery?

2011-09-29 Thread Simon Riggs
On Thu, Sep 29, 2011 at 12:31 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Sep 27, 2011 at 8:06 PM, Florian Pflug f...@phlo.org wrote: On Sep27, 2011, at 07:59 , Heikki Linnakangas wrote: On 27.09.2011 00:28, Florian Pflug wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be

Re: [HACKERS] bug of recovery?

2011-09-29 Thread Florian Pflug
On Sep29, 2011, at 13:49 , Simon Riggs wrote: This worries me slightly now though because the patch makes us PANIC in a place we didn't used to and once we do that we cannot restart the server at all. Are we sure we want that? It's certainly a great way to shake down errors in other code...

Re: [HACKERS] bug of recovery?

2011-09-29 Thread Fujii Masao
On Thu, Sep 29, 2011 at 11:12 PM, Florian Pflug f...@phlo.org wrote: On Sep29, 2011, at 13:49 , Simon Riggs wrote: This worries me slightly now though because the patch makes us PANIC in a place we didn't used to and once we do that we cannot restart the server at all. Are we sure we want

Re: [HACKERS] bug of recovery?

2011-09-27 Thread Heikki Linnakangas
On 26.09.2011 21:06, Simon Riggs wrote: On Mon, Sep 26, 2011 at 10:50 AM, Fujii Masaomasao.fu...@gmail.com wrote: Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its

Re: [HACKERS] bug of recovery?

2011-09-27 Thread Heikki Linnakangas
On 27.09.2011 00:28, Florian Pflug wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we exit recovery mode. I believe we also need to prevent

Re: [HACKERS] bug of recovery?

2011-09-27 Thread Simon Riggs
On Tue, Sep 27, 2011 at 6:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I think you're mixing this up with the multi-page page split operations in b-tree. This is different from that. What the invalid_page_tab is for is the situation where you for example, insert to a

Re: [HACKERS] bug of recovery?

2011-09-27 Thread Florian Pflug
On Sep27, 2011, at 07:59 , Heikki Linnakangas wrote: On 27.09.2011 00:28, Florian Pflug wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Florian Pflug
On Sep26, 2011, at 11:50 , Fujii Masao wrote: Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its information is removed from the hash table. If there is unresolved

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Simon Riggs
On Mon, Sep 26, 2011 at 10:50 AM, Fujii Masao masao.fu...@gmail.com wrote: Currently, if a reference to an invalid page is found during recovery, its information is saved in hash table invalid_page_tab. Then, if such a reference is resolved, its information is removed from the hash table. If

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Mon, Sep 26, 2011 at 10:50 AM, Fujii Masao masao.fu...@gmail.com wrote: To prevent the above problem, we should write the contents of the hash table to the disk for every restartpoints, I think. Then, when the server starts recovery, it should

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Florian Pflug
On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we exit recovery mode. I believe we also need to prevent the creation of restart points before

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Fujii Masao
On Tue, Sep 27, 2011 at 7:28 AM, Florian Pflug f...@phlo.org wrote: On Sep26, 2011, at 22:39 , Tom Lane wrote: It might be worthwhile to invoke XLogCheckInvalidPages() as soon as we (think we) have reached consistency, rather than leaving it to be done only when we exit recovery mode. I

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: ISTM that writing an invalid-page table to the disk for every restartpoints is better approach. I still say that's uncalled-for overkill. The invalid-page table is not necessary for recovery, it's only a debugging cross-check. You're more likely to

Re: [HACKERS] bug of recovery?

2011-09-26 Thread Fujii Masao
On Tue, Sep 27, 2011 at 1:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: ISTM that writing an invalid-page table to the disk for every restartpoints is better approach. I still say that's uncalled-for overkill.  The invalid-page table is not necessary