Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-30 Thread Fujii Masao
On Wed, Mar 30, 2011 at 11:39 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 30, 2011 at 12:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, why did we change that? I'm not sure, but I guess that's because I missed the case where crash recovery starts from

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-30 Thread Heikki Linnakangas
On 30.03.2011 09:25, Fujii Masao wrote: On Wed, Mar 30, 2011 at 11:39 AM, Fujii Masaomasao.fu...@gmail.com wrote: On Wed, Mar 30, 2011 at 12:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, why did we change that? I'm not sure, but I guess that's because I missed

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-30 Thread Fujii Masao
On Wed, Mar 30, 2011 at 4:56 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Attached patch reverts that. Comments? Looks good, committed. Thanks! We could also improve the error message. If we haven't reached the end-of-backup location, we could say something along the

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 05:29:22PM -0500, Kevin Grittner wrote: I have a theory. Can you try it in what would be the failure case, but run an explicit a CHECKPOINT on the master, wait for pg_controldata to show that checkpoint on the slave, and (as soon as you see that) try to trigger the

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread hubert depesz lubaczewski
On Tue, Mar 29, 2011 at 11:20:48AM +0900, Fujii Masao wrote: On Tue, Mar 29, 2011 at 12:11 AM, hubert depesz lubaczewski dep...@depesz.com wrote: On Mon, Mar 28, 2011 at 01:48:13PM +0900, Fujii Masao wrote: In 9.0, recovery doesn't read a backup history file. That FATAL error happens if

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread hubert depesz lubaczewski
On Tue, Mar 29, 2011 at 11:13:07AM +0900, Fujii Masao wrote: Yes, it's intentional. In streaming replication, at first the master must stream a backup history file to the standby in order to let it know the recovery ending position. But streaming replication doesn't have ability to send a

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread Fujii Masao
On Tue, Mar 29, 2011 at 6:46 PM, hubert depesz lubaczewski dep...@depesz.com wrote: Did you use recovery.conf to start standalone PostgreSQL? If not, recovery doesn't check whether it reaches the recovery ending position or not. So I guess no problem didn't happen. no, i don't use. hmm .. i

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread Kevin Grittner
hubert depesz lubaczewski dep...@depesz.com wrote: it worked. now the slave2 is working as stand alone. what does it tell us? will any work happening after checkpoint break it anyway? I'm less sure about what will put it into a bad state again than I was that an immediate checkpoint would

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread Heikki Linnakangas
On 29.03.2011 14:27, Fujii Masao wrote: On Tue, Mar 29, 2011 at 6:46 PM, hubert depesz lubaczewski dep...@depesz.com wrote: Did you use recovery.conf to start standalone PostgreSQL? If not, recovery doesn't check whether it reaches the recovery ending position or not. So I guess no problem

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-29 Thread Fujii Masao
On Wed, Mar 30, 2011 at 12:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Hmm, why did we change that? I'm not sure, but I guess that's because I missed the case where crash recovery starts from the backup :( It seems like a mistake, the database is not consistent until

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Robert Haas
On Mon, Mar 28, 2011 at 12:48 AM, Fujii Masao masao.fu...@gmail.com wrote: If you want to take hot backup from the standby, you need to do the procedure explained in http://wiki.postgresql.org/wiki/Incrementally_Updated_Backups It'd be nice to improve this in 9.2. Relying on users to get this

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 01:48:13PM +0900, Fujii Masao wrote: In 9.0, recovery doesn't read a backup history file. That FATAL error happens if recovery ends before it reads the WAL record which was generated by pg_stop_backup(). IOW, recovery gets the recovery ending location from WAL record

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 01:48:13PM +0900, Fujii Masao wrote: In 9.0, recovery doesn't read a backup history file. That FATAL error happens if recovery ends before it reads the WAL record which was generated by pg_stop_backup(). IOW, recovery gets the recovery ending location from WAL record

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Kevin Grittner
hubert depesz lubaczewski dep...@depesz.com wrote: how come that I can use this backup to make standalone pg, and it starts without any problem, but when I start it as sr slave, let it run for some time, and then promote to standalone, it breaks? We need more detail to make much of a guess

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 04:24:23PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: how come that I can use this backup to make standalone pg, and it starts without any problem, but when I start it as sr slave, let it run for some time, and then promote to

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Kevin Grittner
hubert depesz lubaczewski dep...@depesz.com wrote: On Mon, Mar 28, 2011 at 04:24:23PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: how come that I can use this backup to make standalone pg, and it starts without any problem, but when I start it as sr

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 04:53:37PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: On Mon, Mar 28, 2011 at 04:24:23PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: how come that I can use this backup to make standalone

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Kevin Grittner
hubert depesz lubaczewski dep...@depesz.com wrote: have you seen this mail - http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php Ah, OK. I have a theory. Can you try it in what would be the failure case, but run an explicit a CHECKPOINT on the master, wait for pg_controldata

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Kevin Grittner
hubert depesz lubaczewski dep...@depesz.com wrote: have you seen this mail - http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php One more thing: Am I correct in understanding that you are trying to do a PITR-style backup without using pg_start_backup() and pg_stop_backup()? If

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 05:29:22PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: have you seen this mail - http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php Ah, OK. I have a theory. Can you try it in what would be the failure case,

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread hubert depesz lubaczewski
On Mon, Mar 28, 2011 at 05:43:15PM -0500, Kevin Grittner wrote: hubert depesz lubaczewski dep...@depesz.com wrote: have you seen this mail - http://archives.postgresql.org/pgsql-hackers/2011-03/msg01490.php One more thing: Am I correct in understanding that you are trying to do a

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Fujii Masao
On Mon, Mar 28, 2011 at 9:19 PM, hubert depesz lubaczewski dep...@depesz.com wrote: On Mon, Mar 28, 2011 at 01:48:13PM +0900, Fujii Masao wrote: In 9.0, recovery doesn't read a backup history file. That FATAL error happens if recovery ends before it reads the WAL record which was generated by

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-28 Thread Fujii Masao
On Tue, Mar 29, 2011 at 12:11 AM, hubert depesz lubaczewski dep...@depesz.com wrote: On Mon, Mar 28, 2011 at 01:48:13PM +0900, Fujii Masao wrote: In 9.0, recovery doesn't read a backup history file. That FATAL error happens if recovery ends before it reads the WAL record which was generated by

Re: [HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-27 Thread Fujii Masao
On Sat, Mar 26, 2011 at 5:31 AM, hubert depesz lubaczewski dep...@depesz.com wrote: I can also setup streaming slave, and it also works, but when I create trigger file to promote this slave to master it fails with error: 2011-03-24 21:01:58.051 CET @ 9680  LOG:  trigger file found:

[HACKERS] Problem with streaming replication, backups, and recovery (9.0.x)

2011-03-25 Thread hubert depesz lubaczewski
hi, So, I hit a strange problem with Streaming Replication, that I cannot explain. Executive summary: when using hot backup made on straming replication slave, sometimes (depending on load) generated backup is created in such a way, that while it can be brough back as standalone Pg, and it can