Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2015-01-02 Thread Dennis Kögel
Am 31.12.2014 um 11:40 schrieb Michael Paquier michael.paqu...@gmail.com: As long as master is fixed, I don't actually care. But I agree with Dennis that it's hard to see what's been commited with all the different issues found, and if any commits were done, in which branch. I'd like to be able

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-31 Thread Michael Paquier
On Wed, Dec 31, 2014 at 4:44 PM, Guillaume Lelarge guilla...@lelarge.info wrote: 2014-12-12 14:58 GMT+01:00 Heikki Linnakangas hlinnakan...@vmware.com: Now, what do we do with the back-branches? I'm not sure. Changing the behaviour in back-branches could cause nasty surprises. Perhaps it's best

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-30 Thread Guillaume Lelarge
2014-12-12 14:58 GMT+01:00 Heikki Linnakangas hlinnakan...@vmware.com: On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais j...@dalibo.com: Since few months, we occasionally see .ready files appearing on some slave instances from

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-12 Thread Heikki Linnakangas
On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais j...@dalibo.com: Since few months, we occasionally see .ready files appearing on some slave instances from various context. The two I have in mind are under 9.2.x. […] So it seems for

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-10 Thread Dennis Kögel
Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais j...@dalibo.com: Since few months, we occasionally see .ready files appearing on some slave instances from various context. The two I have in mind are under 9.2.x. […] So it seems for some reasons, these old WALs were forgotten by

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-28 Thread Fujii Masao
On Tue, Oct 28, 2014 at 1:12 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/27/2014 02:12 PM, Fujii Masao wrote: On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-28 Thread Heikki Linnakangas
On 10/27/2014 06:12 PM, Heikki Linnakangas wrote: On 10/27/2014 02:12 PM, Fujii Masao wrote: On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Fujii Masao
On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same server. I.e. we should

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 02:12 PM, Fujii Masao wrote: On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Heikki Linnakangas
On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same server. I.e. we should never try to archive WAL files belonging to another timeline. I just remembered that

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Michael Paquier
On Fri, Oct 24, 2014 at 3:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same server. I.e. we should

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Robert Haas
On Fri, Oct 24, 2014 at 9:05 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So, this is what I came up with for master. Does anyone see a problem with it? In the proposed commit message, you mis-spelled significant as signicant. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: In this case, the patch seems to make the

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Heikki Linnakangas
On 10/23/2014 08:59 AM, Fujii Masao wrote: On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: In this

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: Sounds reasonable, for back-branches. Although I'm still worried we might miss some corner-case unless we go with a more wholesale solution. Don't really want to be

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Heikki Linnakangas
On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: Sounds reasonable, for back-branches. Although I'm still worried we might miss some corner-case unless we go with a

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 5:09 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier michael.paqu...@gmail.com wrote:

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: Sounds reasonable, for

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote:

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 2:34 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 23, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu,

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 1:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: Sounds reasonable, for

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 10:16 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Oct 23, 2014 at 1:45 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: That's not right. Should check *after* the write if the segment was completed, and close it if so. Like the attached. Just

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Heikki Linnakangas
On 10/20/2014 09:26 AM, Michael Paquier wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: In this case, the patch seems to make the restartpoint recycle even WAL files which have

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Michael Paquier
On Fri, Oct 17, 2014 at 2:23 PM, Fujii Masao masao.fu...@gmail.com wrote: I found one problem in the 0002 patch. The patch changes the recovery so that it creates .done files for every WAL files which exist in pg_xlog directory at the end of recovery. But even WAL files which will have to be

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Heikki Linnakangas
On 10/22/2014 04:24 PM, Michael Paquier wrote: On Wed, Oct 22, 2014 at 1:53 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I think we should take a more wholesale approach to this. We should enforce the rule that the server only ever archives WAL files belonging to the same timeline

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-20 Thread Michael Paquier
On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: In this case, the patch seems to make the restartpoint recycle even WAL files which have .ready files and will have to be archived

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-20 Thread Michael Paquier
On Fri, Oct 17, 2014 at 10:12 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Fujii Masao
On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier michael.paqu...@gmail.com wrote: The additional process at promotion sounds like a good idea, I'll try to get a patch done tomorrow. This would result as well in

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Fujii Masao
On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier michael.paqu...@gmail.com wrote: The additional process at promotion sounds like a

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Michael Paquier
On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao masao.fu...@gmail.com wrote: In this case, the patch seems to make the restartpoint recycle even WAL files which have .ready files and will have to be archived later. Thought? The real problem currently is that it is possible to have a segment file

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier michael.paqu...@gmail.com wrote: The additional process at promotion sounds like a good idea, I'll try to get a patch done tomorrow. This would result as well in removing the XLogArchiveForceDone stuff. Either way, not that I have been able to

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Where do the FF files come from? In 9.2, FF-segments are not supposed to created, ever. I think we should add a check in walreceiver, to throw an error if the master sends an invalid WAL pointer, pointing to

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais j...@dalibo.com wrote: We kept the WAL files and log files for further analysis. How can we help regarding this issue? Commit c2f79ba has added as assumption that the WAL receiver should always enforce the create of .done files

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Heikki Linnakangas
On 10/08/2014 10:44 AM, Michael Paquier wrote: On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais j...@dalibo.com wrote: We kept the WAL files and log files for further analysis. How can we help regarding this issue? Commit c2f79ba has added as assumption that the WAL receiver

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Where do the FF files come from? In 9.2, FF-segments are not supposed to created, ever. Since this only happens with streaming replication, the FF segments are probably being created by walreceiver.

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Fujii Masao
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/08/2014 10:44 AM, Michael Paquier wrote: On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais j...@dalibo.com wrote: We kept the WAL files and log files for further analysis. How can we help

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Heikki Linnakangas
On 10/08/2014 04:59 PM, Fujii Masao wrote: On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Instead of creating any .done files during recovery, we could scan pg_xlog at promotion, and create a .done file for every WAL segment that's present at that point. That

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Wed, Oct 8, 2014 at 11:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/08/2014 04:59 PM, Fujii Masao wrote: On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Instead of creating any .done files during recovery, we could scan pg_xlog at

[HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-09-18 Thread Jehan-Guillaume de Rorthais
Hi hackers, We spend some time with Guillaume Lelarge studying this issue. CreateRestartPoint() calls RemoveOldXlogFiles() to drop/recycle old WALs. This one is calling XLogArchiveCheckDone() to check if the given WAL can be dropped. As our slave has archive_mode archive_command set,