On Wed, Sep 13, 2017 at 01:35:17AM +0200, Andreas Joseph Krogh wrote:
> På onsdag 13. september 2017 kl. 01:00:20, skrev Bruce Momjian <
> br...@momjian.us>:
> (I know this isn't exactly -hackers food, but it seems natural to end this
> thread here)
>
> Ok, thanks.
> It is clearer what happens now that you've explained that there's a clever
> "rsync-trick" involving 2 directories and making rsync preserving
> hard-links that way on the destination-server. Maybe it's because I'm not a
> native English speaker but it wasn't obvious to me...
>
> I have my tablespaces laid out like this:
> /storage/fast_ssd/9.6/tablespaces/<customer>
> which you correctly say that in practice means that 9.6 files are (I see now
> that I don't need the pg-version in my directory-structure):
> /storage/fast_ssd/9.6/tablespaces/<customer>/PG_9.6_201608131
>
> I understand, I hope, that without link-mode rsyncing tablespaces would be
> like
> this:
> rsync --archive /path/to/tablespace_basedir
> standby:/path/to/tablespace_basedir
>
> What would the equivalent be in link-mode, for transferring most efficiently?
> The reason I ask is that it's not immediately obvious to me what "old_datadir"
> and "new_datadir" when rsync'ing tablespaces and pg_wal dirs outside the
> "pg-dirs".
>
> Speaking of pg_wal, how should this be rsynced now that it's changed its name
> (from pg_xlog), just rsync pg_xlog and rename it?
>
> I know I'm being a little nitty-gritty here, but if it helps me understand it
> might help others.
I have applied the attached patch to show examples of using rsync on
PGDATA and tablespaces, documented that rsync is only useful when in
link mode, and explained more clearly how rsync handles links. You can
see the results here:
http://momjian.us/pgsql_docs/pgupgrade.html
Any more improvements?
--
Bruce Momjian <br...@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
new file mode 100644
index f8d9630..60011d8
*** a/doc/src/sgml/ref/pgupgrade.sgml
--- b/doc/src/sgml/ref/pgupgrade.sgml
*************** pg_upgrade.exe
*** 421,432 ****
<title>Upgrade Streaming Replication and Log-Shipping standby
servers</title>
<para>
! If you have Streaming Replication (see <xref
linkend="streaming-replication">) or Log-Shipping (see <xref
linkend="warm-standby">) standby servers, follow these steps to
! upgrade them. You will not be running <application>pg_upgrade</>
! on the standby servers, but rather <application>rsync</> on the
! primary. Do not start any servers yet.
</para>
<substeps>
--- 421,434 ----
<title>Upgrade Streaming Replication and Log-Shipping standby
servers</title>
<para>
! If you used link mode and have Streaming Replication (see <xref
linkend="streaming-replication">) or Log-Shipping (see <xref
linkend="warm-standby">) standby servers, follow these steps to
! upgrade them. You will not be running <application>pg_upgrade</> on
! the standby servers, but rather <application>rsync</> on the primary.
! Do not start any servers yet. If you did <emphasis>not</> use link
! mode, skip the instructions in this section and simply recreate the
! standby servers.
</para>
<substeps>
*************** pg_upgrade.exe
*** 482,490 ****
<title>Run <application>rsync</></title>
<para>
! From a directory on the primary server that is above the old and
! new database cluster directories, run this on the
! <emphasis>primary</> for each standby server:
<programlisting>
rsync --archive --delete --hard-links --size-only old_pgdata new_pgdata
remote_dir
--- 484,494 ----
<title>Run <application>rsync</></title>
<para>
! When using link mode, standby servers can be quickly upgraded using
! <application>rsync</>. To accomplish this, from a directory on
! the primary server that is above the old and new database cluster
! directories, run this on the <emphasis>primary</> for each standby
! server:
<programlisting>
rsync --archive --delete --hard-links --size-only old_pgdata new_pgdata
remote_dir
*************** rsync --archive --delete --hard-links --
*** 492,521 ****
where <option>old_pgdata</> and <option>new_pgdata</> are relative
to the current directory on the primary, and <option>remote_dir</>
! is <emphasis>above</> the old and new cluster directories on
! the standby. The old and new relative cluster paths
! must match on the primary and standby server. Consult the
<application>rsync</> manual page for details on specifying the
! remote directory, e.g. <literal>standbyhost:/opt/PostgreSQL/</>.
</para>
<para>
! What <application>rsync</> does is to copy files from the
! primary to the standby, and, if <application>pg_upgrade</>'s
! <option>--link</> mode was used, link files from the old to
! new clusters on the standby. It links the same files that
! <application>pg_upgrade</> linked in the primary old and new
! clusters. (Of course, linking speeds up <application>rsync</>.)
! Unfortunately, <application>rsync</> needlessly copies files
! associated with temporary and unlogged tables because these files
! don't normally exist on standby servers.
</para>
<para>
If you have tablespaces, you will need to run a similar
! <application>rsync</> command for each tablespace directory. If you
! have relocated <filename>pg_wal</> outside the data directories,
! <application>rsync</> must be run on those directories too.
</para>
</step>
--- 496,539 ----
where <option>old_pgdata</> and <option>new_pgdata</> are relative
to the current directory on the primary, and <option>remote_dir</>
! is <emphasis>above</> the old and new cluster directories
! on the standby. The directory structure under the specified
! directories on the primary and standbys must match. Consult the
<application>rsync</> manual page for details on specifying the
! remote directory, e.g.
!
! <programlisting>
! rsync --archive --delete --hard-links --size-only /opt/PostgreSQL/9.5/data \
! /opt/PostgreSQL/9.6/data standby.example.com:/opt/PostgreSQL
! </programlisting>
!
</para>
<para>
! What this does is to record the links created by
! <application>pg_upgrade</>'s link mode that connect files in the
! old and new clusters on the primary server. It then finds matching
! files in the standby's old cluster and creates links for them in the
! standby's new cluster. Files that were not linked on the primary
! are copied from the primary to the standby. (They are usually
! small.) This provides rapid standby upgrades. Unfortunately,
! <application>rsync</> needlessly copies files associated with
! temporary and unlogged tables because these files don't normally
! exist on standby servers.
</para>
<para>
If you have tablespaces, you will need to run a similar
! <application>rsync</> command for each tablespace directory, e.g.:
!
! <programlisting>
! rsync --archive --delete --hard-links --size-only
/vol1/pg_tblsp/PG_9.5_201510051 \
! /vol1/pg_tblsp/PG_9.6_201608131 standby.example.com:/vol1/pg_tblsp
! </programlisting>
!
! If you have relocated <filename>pg_wal</> outside the data
! directories, <application>rsync</> must be run on those directories
! too.
</para>
</step>
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
