Re: [PATCHES] [HACKERS] Minor changes to Recovery related code
Nice, applied. I only modified some of the documentation wording. I was a little worried that statement_timeout might cancel pg_stop_backup() _before_ it had gotten to waiting for the archive logs to be saved, but based on what little code there is before that block, I think we are OK. --- Simon Riggs wrote: On Thu, 2008-03-27 at 17:34 +, Simon Riggs wrote: Follow-up during March 2008 CommitFest On Thu, 2007-06-07 at 21:53 +0100, Simon Riggs wrote: On Sat, 2007-03-31 at 00:51 +0200, Florian G. Pflug wrote: - pg_stop_backup() will wait until the WAL file that ends the backup is safely archived, even if a failure to archive occurs. This is a change to current behaviour, but since it implements the originally *expected* behaviour IMHO it should be the default. The most straightforward thing is to make pg_stop_backup() wait as described above. If people want it to timeout, they can use a statement_timeout as suggested by Florian. This can be implemented by having the function poll in an infinite loop for archive_status/LOG.done. Also, as Florian suggests, we should have it output a WARNING message every 60 seconds. I'll write a patch now. Patch to implement waiting pg_stop_backup(). Tested and ready to apply, includes docs. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk [ Attachment, skipping... ] -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + Index: doc/src/sgml/backup.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/backup.sgml,v retrieving revision 2.116 diff -c -c -r2.116 backup.sgml *** doc/src/sgml/backup.sgml 28 Mar 2008 15:00:28 - 2.116 --- doc/src/sgml/backup.sgml 5 Apr 2008 01:28:09 - *** *** 761,772 para Once the WAL segment files used during the backup are archived, you are done. The file identified by functionpg_stop_backup/'s result is ! the last segment that needs to be archived to complete the backup. ! Archival of these files will happen automatically, since you have ! already configured varnamearchive_command/. In many cases, this ! happens fairly quickly, but you are advised to monitor your archival ! system to ensure this has taken place so that you can be certain you ! have a complete backup. /para /listitem /orderedlist --- 761,779 para Once the WAL segment files used during the backup are archived, you are done. The file identified by functionpg_stop_backup/'s result is ! the last segment that is required to form a complete set of backup files. ! functionpg_stop_backup/ does not return until the last segment has ! been archived. ! Archiving of these files happens automatically since you have ! already configured varnamearchive_command/. In most cases this ! happens quickly, but you are advised to monitor your archive ! system to ensure there are no delays. ! If the archive process has fallen behind ! because of failures of the archive command, it will keep retrying ! until the archive succeeds and the backup is complete. ! If you wish to place a time limit on the execution of ! functionpg_stop_backup/, set an appropriate ! varnamestatement_timeout/varname value. /para /listitem /orderedlist *** *** 1044,1050 note para The stop point must be after the ending time of the base backup, i.e., ! the time of functionpg_stop_backup/. You cannot use a base backup to recover to a time when that backup was still going on. (To recover to such a time, you must go back to your previous base backup and roll forward from there.) --- 1051,1057 note para The stop point must be after the ending time of the base backup, i.e., ! the end time of functionpg_stop_backup/. You cannot use a base backup to recover to a time when that backup was still going on. (To recover to such a time, you must go back to your previous base backup and roll forward from there.) *** *** 1322,1327 --- 1329,1335 After the backup the switch file is removed. Archived WAL files are then added to the backup so that both base backup and all required WAL files are part of the same applicationtar/ file. + Please remember to add error handling to your backup scripts. /para /sect3 Index: src/backend/access/transam/xlog.c === RCS file:
Re: [PATCHES] [HACKERS] Minor changes to Recovery related code
On Thu, 2008-03-27 at 17:34 +, Simon Riggs wrote: Follow-up during March 2008 CommitFest On Thu, 2007-06-07 at 21:53 +0100, Simon Riggs wrote: On Sat, 2007-03-31 at 00:51 +0200, Florian G. Pflug wrote: - pg_stop_backup() will wait until the WAL file that ends the backup is safely archived, even if a failure to archive occurs. This is a change to current behaviour, but since it implements the originally *expected* behaviour IMHO it should be the default. The most straightforward thing is to make pg_stop_backup() wait as described above. If people want it to timeout, they can use a statement_timeout as suggested by Florian. This can be implemented by having the function poll in an infinite loop for archive_status/LOG.done. Also, as Florian suggests, we should have it output a WARNING message every 60 seconds. I'll write a patch now. Patch to implement waiting pg_stop_backup(). Tested and ready to apply, includes docs. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com PostgreSQL UK 2008 Conference: http://www.postgresql.org.uk Index: doc/src/sgml/backup.sgml === RCS file: /home/sriggs/pg/REPOSITORY/pgsql/doc/src/sgml/backup.sgml,v retrieving revision 2.116 diff -c -r2.116 backup.sgml *** doc/src/sgml/backup.sgml 28 Mar 2008 15:00:28 - 2.116 --- doc/src/sgml/backup.sgml 31 Mar 2008 14:11:59 - *** *** 761,772 para Once the WAL segment files used during the backup are archived, you are done. The file identified by functionpg_stop_backup/'s result is ! the last segment that needs to be archived to complete the backup. Archival of these files will happen automatically, since you have already configured varnamearchive_command/. In many cases, this happens fairly quickly, but you are advised to monitor your archival ! system to ensure this has taken place so that you can be certain you ! have a complete backup. /para /listitem /orderedlist --- 761,779 para Once the WAL segment files used during the backup are archived, you are done. The file identified by functionpg_stop_backup/'s result is ! the last segment that is required to form a complete set of backup files. ! functionpg_stop_backup/ does not return until the last segment has ! been archived. Archival of these files will happen automatically, since you have already configured varnamearchive_command/. In many cases, this happens fairly quickly, but you are advised to monitor your archival ! system to ensure there is no delay. ! If the archival process has fallen significantly behind ! because of failures of the archive command then we will keep waiting ! and retrying until the archive succeeds and the backup is complete. ! If you wish to place a time limit on the execution of ! functionpg_stop_backup/ then you should set an appropriate ! varnamestatement_timeout/varname. /para /listitem /orderedlist *** *** 1044,1050 note para The stop point must be after the ending time of the base backup, i.e., ! the time of functionpg_stop_backup/. You cannot use a base backup to recover to a time when that backup was still going on. (To recover to such a time, you must go back to your previous base backup and roll forward from there.) --- 1051,1057 note para The stop point must be after the ending time of the base backup, i.e., ! the end time of functionpg_stop_backup/. You cannot use a base backup to recover to a time when that backup was still going on. (To recover to such a time, you must go back to your previous base backup and roll forward from there.) *** *** 1322,1327 --- 1329,1335 After the backup the switch file is removed. Archived WAL files are then added to the backup so that both base backup and all required WAL files are part of the same applicationtar/ file. + Please remember to add error handling to your backup scripts. /para /sect3 Index: src/backend/access/transam/xlog.c === RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/transam/xlog.c,v retrieving revision 1.295 diff -c -r1.295 xlog.c *** src/backend/access/transam/xlog.c 25 Mar 2008 22:42:42 - 1.295 --- src/backend/access/transam/xlog.c 1 Apr 2008 16:08:07 - *** *** 382,388 static void XLogArchiveNotify(const char *xlog); static void XLogArchiveNotifySeg(uint32 log, uint32 seg); ! static bool XLogArchiveCheckDone(const char *xlog); static void XLogArchiveCleanup(const char *xlog); static void readRecoveryCommandFile(void); static void exitArchiveRecovery(TimeLineID endTLI,
