Re: [HACKERS] Recovery to backup point
On Sat, Jan 25, 2014 at 5:10 AM, Heikki Linnakangas wrote: > Thanks, committed! It seems that this patch has not been pushed :) Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
On 01/24/2014 01:37 PM, MauMau wrote: Hi, Heiki-san, From: "MauMau" From: "Heikki Linnakangas" After some refactoring and fixing bugs in the existing code, I came up with the attached patch. I called the option simply "recovery_target", with the only allowed value of "immediate". IOW, if you want to stop recovery as early as possible, you add recovery_target='immediate' to recovery.conf. Now that we have four different options to set the recovery target with, I rearranged the docs slightly. How does this look to you? I'm almost comfortable with your patch. There are two comments: C1. The following parts seem to be mistakenly taken from my patch. These are not necessary for your patch, aren't they? I'm going to add the attached new revision of the patch soon, which is almost based on yours. All what I modified is removal of parts I mentioned above. I confirmed that the original problem could be solved. Thanks. Thanks, committed! - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
Hi, Heiki-san, From: "MauMau" From: "Heikki Linnakangas" After some refactoring and fixing bugs in the existing code, I came up with the attached patch. I called the option simply "recovery_target", with the only allowed value of "immediate". IOW, if you want to stop recovery as early as possible, you add recovery_target='immediate' to recovery.conf. Now that we have four different options to set the recovery target with, I rearranged the docs slightly. How does this look to you? I'm almost comfortable with your patch. There are two comments: C1. The following parts seem to be mistakenly taken from my patch. These are not necessary for your patch, aren't they? I'm going to add the attached new revision of the patch soon, which is almost based on yours. All what I modified is removal of parts I mentioned above. I confirmed that the original problem could be solved. Thanks. Regards MauMau recover_to_backup_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
From: "Michael Paquier" On Fri, Jan 10, 2014 at 12:08 AM, MauMau wrote: C2. "recovery_target = 'immediate'" sounds less intuitive than my suggestion "recovery_target_time = 'backup_point'", at least for those who want to recover to the backup point. Although I don't have a good naming sense in English, the value should be a noun, not an adjective like "immediate", because the value specifies the "target (point)" of recovery. "immediate" is perfectly fine IMO, it fits with what this recovery target aims at: an immediate consistency point. My 2c on that. OK, I believe the naming sense of people whose mother tongue is English. I thought the value should be a noun like "earliest_consistency_point" or "earliest_consistency" (I don't these are good, though). Regards MauMau -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
The documentation doesn't build. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
On Fri, Jan 10, 2014 at 12:08 AM, MauMau wrote: > C2. "recovery_target = 'immediate'" sounds less intuitive than my suggestion > "recovery_target_time = 'backup_point'", at least for those who want to > recover to the backup point. > Although I don't have a good naming sense in English, the value should be a > noun, not an adjective like "immediate", because the value specifies the > "target (point)" of recovery. "immediate" is perfectly fine IMO, it fits with what this recovery target aims at: an immediate consistency point. My 2c on that. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
From: "Heikki Linnakangas"
After some refactoring and fixing bugs in the existing code, I came up
with the attached patch. I called the option simply "recovery_target",
with the only allowed value of "immediate". IOW, if you want to stop
recovery as early as possible, you add recovery_target='immediate' to
recovery.conf. Now that we have four different options to set the
recovery target with, I rearranged the docs slightly. How does this look
to you?
I'm almost comfortable with your patch. There are two comments:
C1. The following parts seem to be mistakenly taken from my patch. These
are not necessary for your patch, aren't they?
@@ -6238,6 +6277,10 @@ StartupXLOG(void)
ereport(LOG,
(errmsg("starting point-in-time recovery to XID %u",
recoveryTargetXid)));
+ else if (recoveryTarget == RECOVERY_TARGET_TIME &&
+ recoveryTargetTime == 0)
+ ereport(LOG,
+ (errmsg("starting point-in-time recovery to backup point")));
else if (recoveryTarget == RECOVERY_TARGET_TIME)
ereport(LOG,
(errmsg("starting point-in-time recovery to %s",
@@ -6971,6 +7017,22 @@ StartupXLOG(void)
if (switchedTLI && AllowCascadeReplication())
WalSndWakeup();
+/*
+ * If we have reached the end of base backup during recovery
+ * to the backup point, exit redo loop.
+ */
+if (recoveryTarget == RECOVERY_TARGET_TIME &&
+ recoveryTargetTime == 0 && reachedConsistency)
+{
+ if (recoveryPauseAtTarget)
+ {
+ SetRecoveryPause(true);
+ recoveryPausesHere();
+ }
+ reachedStopPoint = true;
+ break;
+}
+
/* Exit loop if we reached inclusive recovery target */
if (recoveryStopsAfter(record))
{
@@ -7116,6 +7178,9 @@ StartupXLOG(void)
"%s transaction %u",
recoveryStopAfter ? "after" : "before",
recoveryStopXid);
+ else if (recoveryTarget == RECOVERY_TARGET_TIME &&
+ recoveryStopTime == 0)
+ snprintf(reason, sizeof(reason), "at backup point");
else if (recoveryTarget == RECOVERY_TARGET_TIME)
snprintf(reason, sizeof(reason),
"%s %s\n",
C2. "recovery_target = 'immediate'" sounds less intuitive than my suggestion
"recovery_target_time = 'backup_point'", at least for those who want to
recover to the backup point.
Although I don't have a good naming sense in English, the value should be a
noun, not an adjective like "immediate", because the value specifies the
"target (point)" of recovery.
Being related to C2, I wonder if users would understand the following part
in the documentation.
+This parameter specifies that recovery should end as soon as a
+consistency is reached, ie. as early as possible.
The subsequent sentence clarifies the use case for recovery from an online
backup, but in what use cases do they specify this parameter? For example,
when do the users face the following situation?
I was thinking that you have a warm standby server, and you decide to
stop using it as a warm standby, and promote it. You'd do that by
stopping it, modifying recovery.conf to remove standby_mode, and set a
recovery target, and then restart.
Regards
MauMau
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
On 12/09/2013 03:05 PM, MauMau wrote: From: "Heikki Linnakangas" Thanks. Looks sane, although I don't much like the proposed interface to trigger this, setting recovery_target_time='backup_point'. What the code actually does is to stop recovery as soon as you reach consistency, which might not have anything to do with a backup. If you set it on a warm standby server, for example, it will end recovery as soon as it reaches consistency, but there was probably no backup taken at that point. Thank you for reviewing so rapidly. I thought I would check the end of backup in recoveryStopsHere(), by matching XLOG_BACKUP_END and ControlFile->backupStartPoint for backups taken on the primary, and comparing the current redo location with ControlFile->backupEndPoint for backups taken on the standby. However, that would duplicate much code in XLOG_BACKUP_END redo processing and checkRecoveryConsistency(). Besides, the code works only when the user explicitly requests recovery to backup point, not when he starts the warm standby server. (I wonder I'm answering correctly.) I was thinking that you have a warm standby server, and you decide to stop using it as a warm standby, and promote it. You'd do that by stopping it, modifying recovery.conf to remove standby_mode, and set a recovery target, and then restart. After some refactoring and fixing bugs in the existing code, I came up with the attached patch. I called the option simply "recovery_target", with the only allowed value of "immediate". IOW, if you want to stop recovery as early as possible, you add recovery_target='immediate' to recovery.conf. Now that we have four different options to set the recovery target with, I rearranged the docs slightly. How does this look to you? - Heikki diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml index a2361d7..854b5fd 100644 --- a/doc/src/sgml/backup.sgml +++ b/doc/src/sgml/backup.sgml @@ -1124,7 +1124,7 @@ restore_command = 'cp /mnt/server/archivedir/%f %p' If you want to recover to some previous point in time (say, right before the junior DBA dropped your main transaction table), just specify the -required stopping point in recovery.conf. You can specify +required stopping point in recovery.conf. You can specify the stop point, known as the recovery target, either by date/time, named restore point or by completion of a specific transaction ID. As of this writing only the date/time and named restore point options diff --git a/doc/src/sgml/recovery-config.sgml b/doc/src/sgml/recovery-config.sgml index 550cdce..a723338 100644 --- a/doc/src/sgml/recovery-config.sgml +++ b/doc/src/sgml/recovery-config.sgml @@ -199,8 +199,33 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows Recovery Target Settings + + By default, recovery will recover to the end of the WAL log. The + following parameters can be used to specify an earlier stopping point. + At most one of recovery_target, + recovery_target_name, recovery_target_time, or + recovery_target_xid can be specified. + + + recovery_target = 'immediate' + +recovery_target recovery parameter + + + +This parameter specifies that recovery should end as soon as a +consistency is reached, ie. as early as possible. When restoring from an +online backup, this means the point where taking the backup ended. + + +Technically, this is a string parameter, but 'immediate' +is currently the only allowed value. + + + + recovery_target_name (string) @@ -212,10 +237,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows This parameter specifies the named restore point, created with pg_create_restore_point() to which recovery will proceed. -At most one of recovery_target_name, - or - can be specified. The default is to -recover to the end of the WAL log. @@ -231,10 +252,6 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows This parameter specifies the time stamp up to which recovery will proceed. -At most one of recovery_target_time, - or - can be specified. -The default is to recover to the end of the WAL log. The precise stopping point is also influenced by . @@ -254,15 +271,18 @@ restore_command = 'copy "C:\\server\\archivedir\\%f" "%p"' # Windows start, transactions can complete in a different numeric order. The transactions that will be recovered are those that committed before (and optionally including) the specified one. -At most one of recovery_target_xid, - or - can be specified. -The default is to recover to the end of the WAL log.
Re: [HACKERS] Recovery to backup point
From: "Heikki Linnakangas" Thanks. Looks sane, although I don't much like the proposed interface to trigger this, setting recovery_target_time='backup_point'. What the code actually does is to stop recovery as soon as you reach consistency, which might not have anything to do with a backup. If you set it on a warm standby server, for example, it will end recovery as soon as it reaches consistency, but there was probably no backup taken at that point. Thank you for reviewing so rapidly. I thought I would check the end of backup in recoveryStopsHere(), by matching XLOG_BACKUP_END and ControlFile->backupStartPoint for backups taken on the primary, and comparing the current redo location with ControlFile->backupEndPoint for backups taken on the standby. However, that would duplicate much code in XLOG_BACKUP_END redo processing and checkRecoveryConsistency(). Besides, the code works only when the user explicitly requests recovery to backup point, not when he starts the warm standby server. (I wonder I'm answering correctly.) Hmm. I guess it's a nice work-around to use this option, but it doesn't really solve the underlying issue. The system might well reach consistency between deleting database files and the transaction commit, in which case you still have the same problem. Yes, you're right. But I believe the trouble can be avoided most of the time. It would be nice to have a more robust fix for that. Perhaps we could use the safe_restartpoint machinery we have to not allow recovery to end until we see the commit record. I was really hoping to get rid of that machinery in 9.4, though, as it won't be needed for GIN and B-tree after the patches I have in the current commitfest are committed. In any case, that's a separate discussion and separate patch. I think so, too. That still seems a bit difficult for what I am now. If someone starts a discussion in a separate thread, I'd like to join it. Regards MauMau -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
On 12/09/2013 02:03 PM, MauMau wrote: From: "Michael Paquier" As far as I recall, I don't think so. The problem and the way to solve that are clear. The only trick is to be sure that recovery is done just until a consistent point is reached, and to implement that cleanly. May I implement this feature and submit a patch for the next commitfest if I have time? Please feel free. I might as well participate in the review. I've done with the attached patch. Thanks. Looks sane, although I don't much like the proposed interface to trigger this, setting recovery_target_time='backup_point'. What the code actually does is to stop recovery as soon as you reach consistency, which might not have anything to do with a backup. If you set it on a warm standby server, for example, it will end recovery as soon as it reaches consistency, but there was probably no backup taken at that point. I also confirmed that the problem I raised in the first mail of the below thread was solved with this patch. [bug fix] PITR corrupts the database cluster http://www.postgresql.org/message-id/F93E42280A9A4A5EB74FC7350C801A20%40maumau Hmm. I guess it's a nice work-around to use this option, but it doesn't really solve the underlying issue. The system might well reach consistency between deleting database files and the transaction commit, in which case you still have the same problem. It would be nice to have a more robust fix for that. Perhaps we could use the safe_restartpoint machinery we have to not allow recovery to end until we see the commit record. I was really hoping to get rid of that machinery in 9.4, though, as it won't be needed for GIN and B-tree after the patches I have in the current commitfest are committed. In any case, that's a separate discussion and separate patch. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
From: "Michael Paquier" As far as I recall, I don't think so. The problem and the way to solve that are clear. The only trick is to be sure that recovery is done just until a consistent point is reached, and to implement that cleanly. May I implement this feature and submit a patch for the next commitfest if I have time? Please feel free. I might as well participate in the review. I've done with the attached patch. I also confirmed that the problem I raised in the first mail of the below thread was solved with this patch. [bug fix] PITR corrupts the database cluster http://www.postgresql.org/message-id/F93E42280A9A4A5EB74FC7350C801A20@maumau I'm wondering if I can do this with cleaner and less code. It would be grateful if you could give me any advice. Regards MauMau recover_to_backup.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
From: "Michael Paquier" On Sat, Dec 7, 2013 at 9:06 AM, MauMau wrote: Recovery target 'immediate' http://www.postgresql.org/message-id/51703751.2020...@vmware.com May I implement this feature and submit a patch for the next commitfest if I have time? Please feel free. I might as well participate in the review. Thanks. I'm feeling incliend to make the configuration "recovery_target = 'backup_point'" instead of "recovery_target = 'immediate'", because: * The meaning of this feature for usrs is to recover the database to the backup point. * it doesn't seem to need a new parameter. recovery_target_time sounds appropriate because users want to restore the database at the "time" of backup. Regards MauMau -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Recovery to backup point
On Sat, Dec 7, 2013 at 9:06 AM, MauMau wrote: > It seems that Everyone welcomed the following functionality, and I also want > it to solve some problem. But this doesn't appear to be undertaken. Indeed, nobody has really showed up to implement that. > > Recovery target 'immediate' > http://www.postgresql.org/message-id/51703751.2020...@vmware.com > Is there any technical difficulty? As far as I recall, I don't think so. The problem and the way to solve that are clear. The only trick is to be sure that recovery is done just until a consistent point is reached, and to implement that cleanly. > May I implement this feature and submit a patch for the next commitfest if I > have time? Please feel free. I might as well participate in the review. Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
