On 09/28/2015 06:33 AM, Michael Paquier wrote:
> On Sun, Sep 27, 2015 at 11:27 AM, Amir Rohan wrote:
>> Further editing. See attached V3.
>
> I think that you should consider adding this patch to the next commit
> fest so as we do not lose track of it:
> https://commitfest.postgresql.org/7/
>
I've recently picked up the base backup section of the documentation:
http://www.postgresql.org/docs/9.4/static/continuous-archiving.html
(Section 24.3.2) and went through like a tutorial. I'm a relative
newcomer to postgres, so this was the real deal.
The docs seem to be in need of some improvement, primarily because
it has the feel of having been monkey-patched repeatedly it in the
past. It needs to have someone with more experience read through it
and comment on technical accuracy and any knee-slapper error I
might have introduced.
One pain point in rewriting is the definition of terms, or lack
there of in the manual. I didn't find a section defining terms
conclusively for consistent referencing throughout the documentation.
For example, how should one refer to the directory housing
"postgresql.conf"?, is it:
1. The data directory
2. The storage directory
3. The PGDATA directory
4. The cluster directory
5. The server root directory
6. The config file directory
7. etc'
This lack of clear guidelines and uniformly applied terms
makes for some awkward verbal manoeuvring and compromises the
quality of the documentation (but admittedly, not its copiousness).
Amir
>From f763259e6e91d69dd1c41f67169fa0666b1f82d9 Mon Sep 17 00:00:00 2001
From: root <root@localhost.localdomain>
Date: Sat, 26 Sep 2015 11:50:43 +0300
Subject: [PATCH] Rewritten documentation on base backups V2
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index 7413666..e2727e1 100644
--- a/doc/src/sgml/backup.sgml
+++ b/doc/src/sgml/backup.sgml
@@ -427,8 +427,8 @@ tar -cf backup.tar /usr/local/pgsql/data
<para>
If simultaneous snapshots are not possible, one option is to shut down
the database server long enough to establish all the frozen snapshots.
- Another option is to perform a continuous archiving base backup (<xref
- linkend="backup-base-backup">) because such backups are immune to file
+ Another option is to create a base backup using the continuous archiving feature
+ (<xref linkend="backup-base-backup">) because such backups are immune to file
system changes during the backup. This requires enabling continuous
archiving just during the backup process; restore is done using
continuous archive recovery (<xref linkend="backup-pitr-recovery">).
@@ -752,60 +752,58 @@ test ! -f /mnt/server/archivedir/00000001000000A900000065 && cp pg_xlog/
<title>Making a Base Backup</title>
<para>
- The easiest way to perform a base backup is to use the
- <xref linkend="app-pgbasebackup"> tool. It can create
- a base backup either as regular files or as a tar archive. If more
- flexibility than <xref linkend="app-pgbasebackup"> can provide is
- required, you can also make a base backup using the low level API
- (see <xref linkend="backup-lowlevel-base-backup">).
+ A base backup consists of one or more WAL files and a <xref linkend="backup-file">.
+ Together they sufficient to recreate the database's state at some point in the past.
+ Once a base backup is made, the WAL files that precede its creation are no longer
+ necessary in order to recover the database to some later point in time.
+ </para>
+
+ <para>
+ The interval between base backups should usually be
+ chosen based on how much storage you want to expend on archived WAL
+ files, since you must keep all the archived WAL files back to your
+ last base backup.
+ You should also consider how long you are prepared to spend
+ recovering, if recovery should be necessary — the system will have to
+ replay all those WAL segments, and that could take awhile if it has
+ been a long time since the last base backup.
</para>
<para>
- It is not necessary to be concerned about the amount of time it takes
- to make a base backup. However, if you normally run the
- server with <varname>full_page_writes</> disabled, you might notice a drop
- in performance while the backup runs since <varname>full_page_writes</> is
- effectively forced on during backup mode.
+ Creating a base backup may be a lengthy process if you have a lots of data.
+ Be aware that If you normally run the server with <varname>full_page_writes</>
+ disabled, you might notice a drop in performance while the backup runs since
+ <varname>full_page_writes</> is effectively forced on during backup mode.
</para>
<para>
To make use of the backup, you will need to keep all the WAL
segment files generated during and after the file system backup.
- To aid you in doing this, the base backup process
- creates a <firstterm>backup history file</> that is immediately
- stored into the WAL archive area. This file is named after the first
- WAL segment file that you need for the file system backup.
- For example, if the starting WAL file is
- <literal>0000000100001234000055CD</> the backup history file will be
- named something like
- <literal>0000000100001234000055CD.007C9330.backup</>. (The second
- part of the file name stands for an exact position within the WAL
- file, and can ordinarily be ignored.) Once you have safely archived
- the file system backup and the WAL segment files used during the
- backup (as specified in the backup history file), all archived WAL
- segments with names numerically less are no longer needed to recover
- the file system backup and can be deleted. However, you should
- consider keeping several backup sets to be absolutely certain that
- you can recover your data.
+ To aid you in doing this, the base backup process creates a
+ a textual file called <filename>backup_label</> when you initiate the
+ base backup, it in the same directory as your server's
+ configuration files. Note that if you use the low-level API described
+ below, the file is only available between the issuing of the <function>pg_start_backup</>
+ command and the issuing of the <function>pg_stop_backup</> command.
+
+ The contents of <filename>backup_label</> include details about the range
+ of WAL files making up the base backup, together with other useful information
+ such as the date of the backup, and the text label associated with the backup
+ which you provide when initiating the backup.
</para>
<para>
- The backup history file is just a small text file. It contains the
- label string you gave to <xref linkend="app-pgbasebackup">, as well as
- the starting and ending times and WAL segments of the backup.
- If you used the label to identify the associated dump file,
- then the archived history file is enough to tell you which dump file to
- restore.
+ The simplest way to perform a base backup is by using <xref linkend="app-
+ pgbasebackup">, which uses the <literal>postgres</>'s replication mechanism to
+ connect to a running database and create a backup archive or directory. The
+ <filename>backup_label</> will be included in the root directory of the
+ archive/directory that <literal>pg_basebackup</> creates for you.
</para>
<para>
- Since you have to keep around all the archived WAL files back to your
- last base backup, the interval between base backups should usually be
- chosen based on how much storage you want to expend on archived WAL
- files. You should also consider how long you are prepared to spend
- recovering, if recovery should be necessary — the system will have to
- replay all those WAL segments, and that could take awhile if it has
- been a long time since the last base backup.
+ If more flexibility than <xref linkend="app-pgbasebackup"> can provide is
+ required, you can also make a base backup using the low level API
+ (see <xref linkend="backup-lowlevel-base-backup">).
</para>
</sect2>
@@ -833,82 +831,118 @@ SELECT pg_start_backup('label');
where <literal>label</> is any string you want to use to uniquely
identify this backup operation. (One good practice is to use the
full path where you intend to put the backup dump file.)
- <function>pg_start_backup</> creates a <firstterm>backup label</> file,
- called <filename>backup_label</>, in the cluster directory with
- information about your backup, including the start time and label
- string. The function also creates a <firstterm>tablespace map</> file,
- called <filename>tablespace_map</>, in the cluster directory with
- information about tablespace symbolic links in <filename>pg_tblspc/</>
- if one or more such link is present. Both files are critical to the
- integrity of the backup, should you need to restore from it.
+ This places the database in backup mode, creating the
+ <filename>backup_label</>
+ file described earlier, as well as the <firstterm>tablespace map</>
+ file called
+ <filename>tablespace_map</> which contains information about tablespace
+ symbolic links in <filename>pg_tblspc/</>, if such links are present.
+ Both files are created in your cluster directory, alongside
+ your configuration files, and are critical to the integrity of
+ the backup.
</para>
<para>
- It does not matter which database within the cluster you connect to to
- issue this command. You can ignore the result returned by the function;
- but if it reports an error, deal with that before proceeding.
+ It does not matter which database within the cluster you are connected
+ to when you issue this command, as the base backup will alway back up
+ the entire server.
+ The result returned by the function is
+ an identifier for the first WAL file in the base backup. You can
+ use the <function>pg_xlogfile_name</> function to get the filename
+ for the actual WAL file it identifies.
+ If <function>pg_start_backup</> reports an error, you should resolve the
+ issue before proceeding.
</para>
<para>
- By default, <function>pg_start_backup</> can take a long time to finish.
- This is because it performs a checkpoint, and the I/O
- required for the checkpoint will be spread out over a significant
- period of time, by default half your inter-checkpoint interval
+ The <function>pg_start_backup</> function performs a checkpoint in preparation
+ for the backup, spreading the I/O involved over a a period
+ of time that is set at half your inter-checkpoint interval
(see the configuration parameter
- <xref linkend="guc-checkpoint-completion-target">). This is
- usually what you want, because it minimizes the impact on query
- processing. If you want to start the backup as soon as
- possible, use:
+ <xref linkend="guc-checkpoint-completion-target">) by default.
+ This can make the backup process lengthy, but is usually desirable
+ because it minimizes the impact on query processing during the backup.
+ If you prefer to start the backup as soon as possible, which can
+ result in a larger negative impact on your server's performance during
+ the backup, use:
<programlisting>
SELECT pg_start_backup('label', true);
</programlisting>
- This forces the checkpoint to be done as quickly as possible.
+ Which forces the checkpoint to be done as quickly as possible.
</para>
</listitem>
<listitem>
<para>
- Perform the backup, using any convenient file-system-backup tool
+ Once <function>pg_start_backup</> returns, you are ready to perform a
+ file system level backup (See <xref linkend="backup-file"> for more details)
+ of your server's entire cluster directory.
+ Note that, when making a file system level backup in conjunction with a
+ base backup, it is neither necessary nor desirable to stop normal operation
+ of the database while you create the file system backup.
+ </para>
+ <para>
+ You can use any convenient file-system-backup tool
such as <application>tar</> or <application>cpio</> (not
<application>pg_dump</application> or
- <application>pg_dumpall</application>). It is neither
- necessary nor desirable to stop normal operation of the database
- while you do this.
- </para>
- </listitem>
+ <application>pg_dumpall</application>) to make the copy.
+ </para>
+ </listitem>
<listitem>
<para>
- Again connect to the database as a superuser, and issue the command:
+ Once <function>pg_start_backup</> finishes, and you've created a backup of
+ your cluster directory, connect to the database again as a superuser again,
+ and issue the command:
<programlisting>
SELECT pg_stop_backup();
</programlisting>
- This terminates the backup mode and performs an automatic switch to
- the next WAL segment. The reason for the switch is to arrange for
- the last WAL segment file written during the backup interval to be
- ready to archive.
+ This tells postgres that you have completed the file system level backup
+ and instructs it to forces a switch to the next WAL segment, allowing
+ the last WAL segment file in the base backup to be archived.
+
+ <function>pg_stop_backup</> will
+ not return until the last segment has been archived. In most cases this
+ happens quickly, but you are advised to monitor your archive system to
+ nsure there are no delays. If the archive process has fallen behind because
+ of failures of the archive command, it will keep retrying until the archive
+ succeeds and the backup is complete.
+ If you wish to place a time limit on the execution of
+ <function>pg_stop_backup</>, set an appropriate
+ <varname>statement_timeout</varname> value.
+
+ The result returned by <function>pg_stop_backup</> is the identifier for the
+ last WAL file included in the base backup
+
</para>
</listitem>
<listitem>
<para>
- Once the WAL segment files active during the backup are archived, you are
- done. The file identified by <function>pg_stop_backup</>'s result is
- the last segment that is required to form a complete set of backup files.
- If <varname>archive_mode</> is enabled,
- <function>pg_stop_backup</> does not return until the last segment has
- been archived.
- Archiving of these files happens automatically since you have
- already configured <varname>archive_command</>. In most cases this
- happens quickly, but you are advised to monitor your archive
- system to ensure there are no delays.
- If the archive process has fallen behind
- because of failures of the archive command, it will keep retrying
- until the archive succeeds and the backup is complete.
- If you wish to place a time limit on the execution of
- <function>pg_stop_backup</>, set an appropriate
- <varname>statement_timeout</varname> value.
+ During archiving of the base backup WALs, postgres will also creates
+ a <firsttrerm>backup history file</> in your archive directory.
+ This is a textual file very similar to the <filename>backup_label file</>.
+ It contains the backup label you set when initiating the backup,
+ the backup date, as well as the identifiers for the first and last WAL
+ in the backup. backup history files have names that match the first WAL
+ segment file in the base backup they are associated with, and have
+ the file extension <literal>.backup</>.
+ For example, if the first WAL file in the range is
+ <literal>0000000100001234000055CD</> the backup history file will be
+ named similarly to <literal>0000000100001234000055CD.007C9330.backup</>.
+ (The second part of the file name stands for an exact position within
+ the WAL file, and can ordinarily be ignored.)
</para>
- </listitem>
- </orderedlist>
+
+ <para>
+ Once all the WAL files in your base backup have been archived, and
+ you've created a file system level backup, the base backup is complete.
+ All archived WAL segments with names numerically less are no longer needed
+ now to recover the file system backup and can be deleted.
+ However, you should consider keeping
+ several backup sets to be absolutely certain that you can recover
+ your data.
</para>
+ </listitem>
+ </orderedlist>
+ </para>
<para>
Some file system backup tools emit warnings or errors
@@ -929,8 +963,11 @@ SELECT pg_stop_backup();
</para>
<para>
- Be certain that your backup dump includes all of the files under
+ Make sure that your backup dump includes all of the files under
the database cluster directory (e.g., <filename>/usr/local/pgsql/data</>).
+ This includes the <filename>backup_label</> and
+ <filename>tablespace_map</> files described earlier. They are crucial
+ parts of your backup.
If you are using tablespaces that do not reside underneath this directory,
be careful to include them as well (and be sure that your backup dump
archives symbolic links as links, otherwise the restore will corrupt
@@ -967,24 +1004,6 @@ SELECT pg_stop_backup();
</para>
<para>
- It's also worth noting that the <function>pg_start_backup</> function
- makes files named <filename>backup_label</> and
- <filename>tablespace_map</> in the database cluster directory,
- which are removed by <function>pg_stop_backup</>. These files will of
- course be archived as a part of your backup dump file. The backup label
- file includes the label string you gave to <function>pg_start_backup</>,
- as well as the time at which <function>pg_start_backup</> was run, and
- the name of the starting WAL file. In case of confusion it is therefore
- possible to look inside a backup dump file and determine exactly which
- backup session the dump file came from. The tablespace map file includes
- the symbolic link names as they exist in the directory
- <filename>pg_tblspc/</> and the full path of each symbolic link.
- These files are not merely for your information; their presence and
- contents are critical to the proper operation of the system's recovery
- process.
- </para>
-
- <para>
It is also possible to make a backup dump while the server is
stopped. In this case, you obviously cannot use
<function>pg_start_backup</> or <function>pg_stop_backup</>, and
--
2.4.3
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
