Re: [HACKERS] Incrementally Updated Backups and restartpoints
On Thu, Mar 4, 2010 at 9:00 PM, Fujii Masao masao.fu...@gmail.com wrote: And, when we start an archive recovery from the backup from the standby, we seem to reach a safe starting point before database has actually become consistent. It's because backupStartLoc is zero. Isn't this an issue? This issue seems to still happen. So should this be fixed for 9.0? Or only writing a note in document is enough for 9.0? I'm leaning towards the latter. I'm thinking of adding something like the following to the section 25.6. Incrementally Updated Backups. Thought? The pg_control file must be backed up first. This avoids the problem that we might fail to restore a consistent database state because recovery starts from the later restart point than the start of the backup. When recovering from the incrementally updated backup, the server can begin accepting connections and complete the recovery successfully before the database has become consistent. To avoid these problems, you must check whether the database has been consistent by comparing the progress of the recovery with the backup ending WAL location before your users try to connect to the server and when archive recovery ends. So, in advance, the backup ending WAL location must be taken by calling the pg_last_xlog_replay_location function at the end of the backup. The progress of the recovery is also taken from the pg_last_xlog_replay_location function. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incrementally Updated Backups and restartpoints
Hi, I thought of this issue again since the related question arrived. http://archives.postgresql.org/pgsql-admin/2010-03/msg00036.php On Thu, Jan 14, 2010 at 7:13 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Jan 13, 2010 at 9:34 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: No, that's not an issue. We only wait for the backup-end record if we haven't seen yet since we started recovery from the base backup. Assuming the standby had reached that point already before the new backup from the standby started, backupStartLoc is zero in the control file. OK. That assumption should be documented? This comment is meaningless. Sorry for noise. And, when we start an archive recovery from the backup from the standby, we seem to reach a safe starting point before database has actually become consistent. It's because backupStartLoc is zero. Isn't this an issue? This issue seems to still happen. So should this be fixed for 9.0? Or only writing a note in document is enough for 9.0? I'm leaning towards the latter. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Incrementally Updated Backups and restartpoints
Our documentation suggests that you can take a base backup of a warm standby server while it's running: If we take a backup of the standby server's data directory while it is processing logs shipped from the primary, we will be able to reload that data and restart the standby's recovery process from the last restart point. We no longer need to keep WAL files from before the restart point. If we need to recover, it will be faster to recover from the incrementally updated backup than from the original base backup. That doesn't seem safe. If the server makes a new restartpoint while the backup is running, and pg_control is backed up after the new restartpoint is made, recovery will restart from the new restartpoint. That is wrong; recovery needs to restart at the restartpoint that was most recent when the backup started. This is basically the same issue we have solved in master with the backup_label file. I wonder if it would be enough to document that pg_control must be backed up first? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incrementally Updated Backups and restartpoints
On Wed, Jan 13, 2010 at 8:36 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Our documentation suggests that you can take a base backup of a warm standby server while it's running: If we take a backup of the standby server's data directory while it is processing logs shipped from the primary, we will be able to reload that data and restart the standby's recovery process from the last restart point. We no longer need to keep WAL files from before the restart point. If we need to recover, it will be faster to recover from the incrementally updated backup than from the original base backup. That doesn't seem safe. If the server makes a new restartpoint while the backup is running, and pg_control is backed up after the new restartpoint is made, recovery will restart from the new restartpoint. That is wrong; recovery needs to restart at the restartpoint that was most recent when the backup started. This is basically the same issue we have solved in master with the backup_label file. Right. I wonder if it would be enough to document that pg_control must be backed up first? Probably No. The archive recovery from such base backup would always fail at the end of recovery because there is no backup-end record, i.e., pg_stop_backup() is not executed in that case. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incrementally Updated Backups and restartpoints
Fujii Masao wrote: On Wed, Jan 13, 2010 at 8:36 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: I wonder if it would be enough to document that pg_control must be backed up first? Probably No. The archive recovery from such base backup would always fail at the end of recovery because there is no backup-end record, i.e., pg_stop_backup() is not executed in that case. No, that's not an issue. We only wait for the backup-end record if we haven't seen yet since we started recovery from the base backup. Assuming the standby had reached that point already before the new backup from the standby started, backupStartLoc is zero in the control file. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Incrementally Updated Backups and restartpoints
On Wed, Jan 13, 2010 at 9:34 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: No, that's not an issue. We only wait for the backup-end record if we haven't seen yet since we started recovery from the base backup. Assuming the standby had reached that point already before the new backup from the standby started, backupStartLoc is zero in the control file. OK. That assumption should be documented? And, when we start an archive recovery from the backup from the standby, we seem to reach a safe starting point before database has actually become consistent. It's because backupStartLoc is zero. Isn't this an issue? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
