pg_upgrade: Add --swap for faster file transfer. This new option instructs pg_upgrade to move the data directories from the old cluster to the new cluster and then to replace the catalog files with those generated for the new cluster. This mode can outperform --link, --clone, --copy, and --copy-file-range, especially on clusters with many relations.
However, this mode creates many garbage files in the old cluster, which can prolong the file synchronization step if --sync-method=syncfs is used. To handle that, we recommend using --sync-method=fsync with this mode, and pg_upgrade internally uses "initdb --sync-only --no-sync-data-files" for file synchronization. pg_upgrade will synchronize the catalog files as they are transferred. We assume that the database files transferred from the old cluster were synchronized prior to upgrade. This mode also complicates reverting to the old cluster, so we recommend restoring from backup upon failure during or after file transfer. We did consider teaching pg_upgrade how to generate a revert script for such failures, but we decided against it due to the rarity of failing during file transfer, the complexity of generating the script, and the potential for misusing the script. The new mode is limited to clusters located in the same file system. With some effort, we could probably support upgrades between different file systems, but this mode is unlikely to offer much benefit if we have to copy the files across file system boundaries. It is also limited to upgrades from version 10 or newer. There are a few known obstacles for using swap mode to upgrade from older versions. For example, the visibility map format changed in v9.6, and the sequence tuple format changed in v10. In fact, swap mode omits the --sequence-data option in its uses of pg_dump and instead reuses the old cluster's sequence data files. While teaching swap mode to deal with these kinds of changes is surely possible (and we may have to deal with similar problems in the future, anyway), it doesn't seem worth the effort to support upgrades from long-unsupported versions. Reviewed-by: Greg Sabino Mullane <htamf...@gmail.com> Reviewed-by: Robert Haas <robertmh...@gmail.com> Discussion: https://postgr.es/m/Zyvop-LxLXBLrZil%40nathan Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/626d7236b65da50423df7de035e86f273cd36b49 Modified Files -------------- doc/src/sgml/ref/pgupgrade.sgml | 59 ++++- src/bin/pg_upgrade/TESTING | 6 +- src/bin/pg_upgrade/check.c | 29 ++- src/bin/pg_upgrade/controldata.c | 21 +- src/bin/pg_upgrade/dump.c | 4 +- src/bin/pg_upgrade/file.c | 14 +- src/bin/pg_upgrade/info.c | 4 +- src/bin/pg_upgrade/option.c | 7 + src/bin/pg_upgrade/pg_upgrade.c | 16 +- src/bin/pg_upgrade/pg_upgrade.h | 5 +- src/bin/pg_upgrade/relfilenumber.c | 384 +++++++++++++++++++++++++++++ src/bin/pg_upgrade/t/006_transfer_modes.pl | 10 + src/common/file_utils.c | 14 +- src/include/common/file_utils.h | 1 + 14 files changed, 540 insertions(+), 34 deletions(-)
