On Fri, Feb 5, 2016 at 4:17 AM, Michael Paquier wrote:
> Thanks for your enthusiasm. Now, to do an auto-critic of my patch:
> + if ($params{allows_streaming})
> + {
> + print $conf "wal_level = hot_standby\n";
> + print $conf "max_wal_senders = 5\n";
> + print $conf "wal_keep_segments = 20\n";
> + print $conf "max_wal_size = 128MB\n";
> + print $conf "shared_buffers = 1MB\n";
> + print $conf "wal_log_hints = on\n";
> + print $conf "hot_standby = on\n";
> + }
> This could have more thoughts, particularly for wal_log_hints which is
> not used all the time, I think that we'd actually want to complete
> that with an optional hash of parameter/values that get appended at
> the end of the configuration file, then pass wal_log_hints in the
> tests where it is needed. The default set of parameter is maybe fine
> if done this way, still wal_keep_segments could be removed.
At the end I have refrained from doing that, and refactoring
[email protected] to use the new option allows_streaming,
simplifying a bit the code. The introduction of allows_streaming could
be done in a separate patch, though it did not seem worth the
complication when hacking at that. The split is simple, though.
> +# Tets for timeline switch
> +# Encure that a standby is able to follow a newly-promoted standby
> Two typos in two lines.
Fixed.
I also found an issue with the use of application_name causing test
001 to fail, bug squashed on the way. The generation of paths for
archive_command and restore_command was incorrect on Windows. Those
need to use two backslashes (to detect correctly files) and need to be
double-quoted (to avoid errors with command copy should a space be
included in the path). I have added as well a new subcommand in
vcregress.pl called recoverycheck where one can run the recovery test
suite on Windows using MSVC.
Attached are rebased patches, split into 3 parts doing the following:
- 0001, fix default configuration of MSVC builds ignoring TAP tests
- 0002, add a promote routine in PostgresNode.pm. pg_rewind's tests
can make immediate use of that.
- 0003, the actual test suite.
This is registered in CF 2016-03 as well for further consideration.
--
Michael
From 0231866094b793ccd7c6941a19d5565c3d5811b6 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Mon, 21 Dec 2015 16:28:34 +0900
Subject: [PATCH 1/3] Fix default configuration of MSVC builds ignoring TAP
tests
MSVC build scripts use a flag to track if TAP tests are supported or not
but this was not configured correctly. By default, like the other build
types using ./configure, this is disabled.
---
src/tools/msvc/Solution.pm | 1 +
src/tools/msvc/config_default.pl | 27 ++++++++++++++-------------
2 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index ac116b7..c5a43f9 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -643,6 +643,7 @@ sub GetFakeConfigure
$cfg .= ' --enable-integer-datetimes'
if ($self->{options}->{integer_datetimes});
$cfg .= ' --enable-nls' if ($self->{options}->{nls});
+ $cfg .= ' --enable-tap-tests' if ($self->{options}->{tap_tests});
$cfg .= ' --with-ldap' if ($self->{options}->{ldap});
$cfg .= ' --without-zlib' unless ($self->{options}->{zlib});
$cfg .= ' --with-extra-version' if ($self->{options}->{extraver});
diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index b9f2ff4..e50be7e 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -3,7 +3,7 @@ use strict;
use warnings;
our $config = {
- asserts => 0, # --enable-cassert
+ asserts => 0, # --enable-cassert
# integer_datetimes=>1, # --enable-integer-datetimes - on is now default
# float4byval=>1, # --disable-float4-byval, on by default
@@ -13,18 +13,19 @@ our $config = {
# blocksize => 8, # --with-blocksize, 8kB by default
# wal_blocksize => 8, # --with-wal-blocksize, 8kB by default
# wal_segsize => 16, # --with-wal-segsize, 16MB by default
- ldap => 1, # --with-ldap
- extraver => undef, # --with-extra-version=<string>
- nls => undef, # --enable-nls=<path>
- tcl => undef, # --with-tls=<path>
- perl => undef, # --with-perl
- python => undef, # --with-python=<path>
- openssl => undef, # --with-openssl=<path>
- uuid => undef, # --with-ossp-uuid
- xml => undef, # --with-libxml=<path>
- xslt => undef, # --with-libxslt=<path>
- iconv => undef, # (not in configure, path to iconv)
- zlib => undef # --with-zlib=<path>
+ ldap => 1, # --with-ldap
+ extraver => undef, # --with-extra-version=<string>
+ nls => undef, # --enable-nls=<path>
+ tap_tests => undef, # --enable-tap-tests
+ tcl => undef, # --with-tls=<path>
+ perl => undef, # --with-perl
+ python => undef, # --with-python=<path>
+ openssl => undef, # --with-openssl=<path>
+ uuid => undef, # --with-ossp-uuid
+ xml => undef, # --with-libxml=<path>
+ xslt => undef, # --with-libxslt=<path>
+ iconv => undef, # (not in configure, path to iconv)
+ zlib => undef # --with-zlib=<path>
};
1;
--
2.7.1
From f866f5a211ba1fc3c34b4272c9b752ced90684ae Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Wed, 17 Feb 2016 15:21:32 +0900
Subject: [PATCH 2/3] Add routine for node promotion in PostgresNode
This is useful for tests to trigger a promotion on a node via pg_ctl,
and pg_rewind can make immediate use of it.
---
src/bin/pg_rewind/RewindTest.pm | 2 +-
src/test/perl/PostgresNode.pm | 12 ++++++++++++
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/src/bin/pg_rewind/RewindTest.pm b/src/bin/pg_rewind/RewindTest.pm
index 68834cd..e204830 100644
--- a/src/bin/pg_rewind/RewindTest.pm
+++ b/src/bin/pg_rewind/RewindTest.pm
@@ -177,7 +177,7 @@ sub promote_standby
# Now promote slave and insert some new data on master, this will put
# the master out-of-sync with the standby. Wait until the standby is
# out of recovery mode, and is ready to accept read-write connections.
- system_or_bail('pg_ctl', '-w', '-D', $node_standby->data_dir, 'promote');
+ $node_standby->promote;
$node_standby->poll_query_until('postgres',
"SELECT NOT pg_is_in_recovery()")
or die "Timed out while waiting for promotion of standby";
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 2ab9aee..5a57e31 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -294,6 +294,18 @@ sub restart
$self->_update_pid;
}
+sub promote
+{
+ my ($self) = @_;
+ my $port = $self->port;
+ my $pgdata = $self->data_dir;
+ my $logfile = $self->logfile;
+ my $name = $self->name;
+ print "### Promoting node \"$name\"\n";
+ TestLib::system_log('pg_ctl', '-D', $pgdata, '-w', '-l', $logfile,
+ 'promote');
+}
+
sub _update_pid
{
my $self = shift;
--
2.7.1
From 5959031af49436262788fede65c3fcb60fb343b7 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Wed, 17 Feb 2016 21:50:19 +0900
Subject: [PATCH 3/3] Add recovery test suite
This includes basic tests maipulating standbys, be they archiving or
streaming nodes, and some basic sanity checks around them. PostgresNode
is extended with a couple of routines allowing to set up WAL archiving,
WAL streaming or WAL restore on a node, as well as a commodity routine
to allow a promotion.
---
doc/src/sgml/install-windows.sgml | 4 +-
src/bin/pg_rewind/RewindTest.pm | 17 +---
src/test/Makefile | 2 +-
src/test/perl/PostgresNode.pm | 126 +++++++++++++++++++++++++++-
src/test/recovery/.gitignore | 3 +
src/test/recovery/Makefile | 17 ++++
src/test/recovery/README | 19 +++++
src/test/recovery/t/001_stream_rep.pl | 62 ++++++++++++++
src/test/recovery/t/002_archiving.pl | 46 ++++++++++
src/test/recovery/t/003_recovery_targets.pl | 125 +++++++++++++++++++++++++++
src/test/recovery/t/004_timeline_switch.pl | 67 +++++++++++++++
src/test/recovery/t/005_replay_delay.pl | 43 ++++++++++
src/tools/msvc/config_default.pl | 2 +-
src/tools/msvc/vcregress.pl | 13 ++-
14 files changed, 524 insertions(+), 22 deletions(-)
create mode 100644 src/test/recovery/.gitignore
create mode 100644 src/test/recovery/Makefile
create mode 100644 src/test/recovery/README
create mode 100644 src/test/recovery/t/001_stream_rep.pl
create mode 100644 src/test/recovery/t/002_archiving.pl
create mode 100644 src/test/recovery/t/003_recovery_targets.pl
create mode 100644 src/test/recovery/t/004_timeline_switch.pl
create mode 100644 src/test/recovery/t/005_replay_delay.pl
diff --git a/doc/src/sgml/install-windows.sgml b/doc/src/sgml/install-windows.sgml
index ba60a6b..8e87b70 100644
--- a/doc/src/sgml/install-windows.sgml
+++ b/doc/src/sgml/install-windows.sgml
@@ -440,6 +440,7 @@ $ENV{CONFIG}="Debug";
<userinput>vcregress ecpgcheck</userinput>
<userinput>vcregress isolationcheck</userinput>
<userinput>vcregress bincheck</userinput>
+<userinput>vcregress recoverycheck</userinput>
<userinput>vcregress upgradecheck</userinput>
</screen>
@@ -455,7 +456,8 @@ $ENV{CONFIG}="Debug";
<para>
Running the regression tests on client programs, with "vcregress bincheck",
- requires an additional Perl module to be installed:
+ or on recovery tests, with "vceregress recoverycheck" requires an additional
+ Perl module to be installed:
<variablelist>
<varlistentry>
<term><productname>IPC::Run</productname></term>
diff --git a/src/bin/pg_rewind/RewindTest.pm b/src/bin/pg_rewind/RewindTest.pm
index e204830..bda0516 100644
--- a/src/bin/pg_rewind/RewindTest.pm
+++ b/src/bin/pg_rewind/RewindTest.pm
@@ -114,24 +114,9 @@ sub check_query
sub setup_cluster
{
-
# Initialize master, data checksums are mandatory
$node_master = get_new_node('master');
- $node_master->init;
-
- # Custom parameters for master's postgresql.conf
- $node_master->append_conf(
- "postgresql.conf", qq(
-wal_level = hot_standby
-max_wal_senders = 2
-wal_keep_segments = 20
-max_wal_size = 200MB
-shared_buffers = 1MB
-wal_log_hints = on
-hot_standby = on
-autovacuum = off
-max_connections = 10
-));
+ $node_master->init(allows_streaming => 1);
}
sub start_master
diff --git a/src/test/Makefile b/src/test/Makefile
index b713c2c..7f7754f 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,7 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = regress isolation modules
+SUBDIRS = regress isolation modules recovery
# We don't build or execute examples/, locale/, or thread/ by default,
# but we do want "make clean" etc to recurse into them. Likewise for ssl/,
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 5a57e31..82a0d30 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -171,6 +171,10 @@ sub init
$params{hba_permit_replication} = 1
if (!defined($params{hba_permit_replication}));
+ $params{has_archiving} = 0
+ if (!defined($params{has_archiving}));
+ $params{allows_streaming} = 0
+ if (!defined($params{allows_streaming}));
mkdir $self->backup_dir;
mkdir $self->archive_dir;
@@ -183,6 +187,20 @@ sub init
print $conf "fsync = off\n";
print $conf "log_statement = all\n";
print $conf "port = $port\n";
+
+ if ($params{allows_streaming})
+ {
+ print $conf "wal_level = hot_standby\n";
+ print $conf "max_wal_senders = 5\n";
+ print $conf "wal_keep_segments = 20\n";
+ print $conf "max_wal_size = 128MB\n";
+ print $conf "shared_buffers = 1MB\n";
+ print $conf "autovacuum = off\n";
+ print $conf "wal_log_hints = on\n";
+ print $conf "hot_standby = on\n";
+ print $conf "max_connections = 10\n";
+ }
+
if ($TestLib::windows_os)
{
print $conf "listen_addresses = '$host'\n";
@@ -195,6 +213,7 @@ sub init
close $conf;
$self->set_replication_conf if ($params{hba_permit_replication});
+ $self->enable_archiving if ($params{has_archiving});
}
sub append_conf
@@ -220,12 +239,19 @@ sub backup
sub init_from_backup
{
- my ($self, $root_node, $backup_name) = @_;
+ my ($self, $root_node, $backup_name, %params) = @_;
my $backup_path = $root_node->backup_dir . '/' . $backup_name;
my $port = $self->port;
my $node_name = $self->name;
my $root_name = $root_node->name;
+ $params{hba_permit_replication} = 1
+ if (!defined($params{hba_permit_replication}));
+ $params{has_streaming} = 0
+ if (!defined($params{has_streaming}));
+ $params{has_restoring} = 0
+ if (!defined($params{has_restoring}));
+
print
"# Initializing node \"$node_name\" from backup \"$backup_name\" of node \"$root_name\"\n";
die "Backup \"$backup_name\" does not exist at $backup_path"
@@ -245,7 +271,10 @@ sub init_from_backup
qq(
port = $port
));
- $self->set_replication_conf;
+
+ $self->set_replication_conf if ($params{hba_permit_replication});
+ $self->enable_restoring($root_node) if ($params{has_restoring});
+ $self->enable_streaming($root_node) if ($params{has_streaming});
}
sub start
@@ -306,6 +335,99 @@ sub promote
'promote');
}
+#
+# Set of routines for replication and recovery
+#
+sub enable_streaming
+{
+ my ($self, $root_node) = @_;
+ my $root_connstr = $root_node->connstr;
+ my $name = $self->name;
+ my $pgdata = $self->data_dir;
+ my $port = $self->port;
+
+ print "### Enabling streaming replication for node in $pgdata with port $port\n";
+ $self->append_conf('recovery.conf', qq(
+primary_conninfo='$root_connstr application_name=$name'
+standby_mode=on
+));
+}
+
+sub enable_restoring
+{
+ my ($self, $root_node) = @_;
+ my $path = $root_node->archive_dir;
+ my $pgdata = $self->data_dir;
+ my $port = $self->port;
+
+ print "### Enabling restoring for node in $pgdata with port $port\n";
+
+ # On Windows, the path specified in the restore command needs to use
+ # double back-slashes to work properly and to be able to detect properly
+ # the file targetted by the copy command, so the directory value used
+ # in this routine, using only one back-slash, need to be properly changed
+ # first. Paths also need to be double-quoted to prevent failures where
+ # the path contains spaces.
+ $path =~ s{\\}{\\\\}g if ($TestLib::windows_os);
+ my $copy_command = $TestLib::windows_os ?
+ qq{copy "$path\\\\%f" "%p"} :
+ qq{cp $path/%f %p};
+
+ $self->append_conf('recovery.conf', qq(
+restore_command = '$copy_command'
+standby_mode = on
+));
+}
+
+sub enable_archiving
+{
+ my ($self) = @_;
+ my $path = $self->archive_dir;
+ my $pgdata = $self->data_dir;
+ my $port = $self->port;
+
+ print "### Enabling archiving for node in $pgdata with port $port\n";
+
+ # On Windows, the path specified in the restore command needs to use
+ # double back-slashes to work properly and to be able to detect properly
+ # the file targetted by the copy command, so the directory value used
+ # in this routine, using only one back-slash, need to be properly changed
+ # first. Paths also need to be double-quoted to prevent failures where
+ # the path contains spaces.
+ $path =~ s{\\}{\\\\}g if ($TestLib::windows_os);
+ my $copy_command = $TestLib::windows_os ?
+ qq{copy "%p" "$path\\\\%f"} :
+ qq{cp %p $path/%f};
+
+ # Enable archive_mode and archive_command on node
+ $self->append_conf('postgresql.conf', qq(
+archive_mode = on
+archive_command = '$copy_command'
+));
+}
+
+# Wait until a node is able to accept queries. Useful when putting a node
+# in recovery and wait for it to be able to work particularly on slow
+# machines.
+sub wait_for_access
+{
+ my ($self) = @_;
+ my $max_attempts = 30;
+ my $attempts = 0;
+ while ($attempts < $max_attempts)
+ {
+ if (run_log(['pg_isready', '-d', $self->connstr('postgres')]))
+ {
+ return 1;
+ }
+
+ # Wait a second before retrying.
+ sleep 1;
+ $attempts++;
+ }
+ return 0;
+}
+
sub _update_pid
{
my $self = shift;
diff --git a/src/test/recovery/.gitignore b/src/test/recovery/.gitignore
new file mode 100644
index 0000000..499fa7d
--- /dev/null
+++ b/src/test/recovery/.gitignore
@@ -0,0 +1,3 @@
+# Generated by test suite
+/regress_log/
+/tmp_check/
diff --git a/src/test/recovery/Makefile b/src/test/recovery/Makefile
new file mode 100644
index 0000000..16c063a
--- /dev/null
+++ b/src/test/recovery/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/recovery
+#
+# Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/recovery/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/recovery
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+check:
+ $(prove_check)
diff --git a/src/test/recovery/README b/src/test/recovery/README
new file mode 100644
index 0000000..20b98e0
--- /dev/null
+++ b/src/test/recovery/README
@@ -0,0 +1,19 @@
+src/test/recovery/README
+
+Regression tests for recovery and replication
+=============================================
+
+This directory contains a test suite for recovery and replication,
+testing mainly the interactions of recovery.conf with cluster
+instances by providing a simple set of routines that can be used
+to define a custom cluster for a test, including backup, archiving,
+and streaming configuration.
+
+Running the tests
+=================
+
+ make check
+
+NOTE: This creates a temporary installation, and some tests may
+create one or multiple nodes, be they master or standby(s) for the
+purpose of the tests.
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
new file mode 100644
index 0000000..3ed9be3
--- /dev/null
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -0,0 +1,62 @@
+# Minimal test testing streaming replication
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 4;
+
+# Initialize master node
+my $node_master = get_new_node('master');
+$node_master->init(allows_streaming => 1);
+$node_master->start;
+my $backup_name = 'my_backup';
+
+# Take backup
+$node_master->backup($backup_name);
+
+# Create streaming standby linking to master
+my $node_standby_1 = get_new_node('standby_1');
+$node_standby_1->init_from_backup($node_master, $backup_name,
+ has_streaming => 1);
+$node_standby_1->start;
+
+# Take backup of standby 1 (not mandatory, but useful to check if
+# pg_basebackup works on a standby).
+$node_standby_1->backup($backup_name);
+
+# Create second standby node linking to standby 1
+my $node_standby_2 = get_new_node('standby_2');
+$node_standby_2->init_from_backup($node_standby_1, $backup_name,
+ has_streaming => 1);
+$node_standby_2->start;
+
+# Create some content on master and check its presence in standby 1
+$node_master->psql('postgres', "CREATE TABLE tab_int AS SELECT generate_series(1,1002) AS a");
+
+# Wait for standbys to catch up
+my $applname_1 = $node_standby_1->name;
+my $applname_2 = $node_standby_2->name;
+my $caughtup_query = "SELECT pg_current_xlog_location() = write_location FROM pg_stat_replication WHERE application_name = '$applname_1';";
+$node_master->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby 1 to catch up";
+$caughtup_query = "SELECT pg_last_xlog_replay_location() = write_location FROM pg_stat_replication WHERE application_name = '$applname_2';";
+$node_standby_1->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby 2 to catch up";
+
+my $result = $node_standby_1->psql('postgres', "SELECT count(*) FROM tab_int");
+print "standby 1: $result\n";
+is($result, qq(1002), 'check streamed content on standby 1');
+
+$result = $node_standby_2->psql('postgres', "SELECT count(*) FROM tab_int");
+print "standby 2: $result\n";
+is($result, qq(1002), 'check streamed content on standby 2');
+
+# Check that only READ-only queries can run on standbys
+$node_standby_1->command_fails(['psql', '-A', '-t', '--no-psqlrc',
+ '-d', $node_standby_1->connstr, '-c',
+ "INSERT INTO tab_int VALUES (1)"],
+ 'Read-only queries on standby 1');
+$node_standby_2->command_fails(['psql', '-A', '-t', '--no-psqlrc',
+ '-d', $node_standby_2->connstr, '-c',
+ "INSERT INTO tab_int VALUES (1)"],
+ 'Read-only queries on standby 2');
diff --git a/src/test/recovery/t/002_archiving.pl b/src/test/recovery/t/002_archiving.pl
new file mode 100644
index 0000000..930125c
--- /dev/null
+++ b/src/test/recovery/t/002_archiving.pl
@@ -0,0 +1,46 @@
+# test for archiving with warm standby
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 1;
+use File::Copy;
+
+# Initialize master node, doing archives
+my $node_master = get_new_node('master');
+$node_master->init(has_archiving => 1,
+ allows_streaming => 1);
+my $backup_name = 'my_backup';
+
+# Start it
+$node_master->start;
+
+# Take backup for slave
+$node_master->backup($backup_name);
+
+# Initialize standby node from backup, fetching WAL from archives
+my $node_standby = get_new_node('standby');
+$node_standby->init_from_backup($node_master, $backup_name,
+ has_restoring => 1);
+$node_standby->append_conf('postgresql.conf', qq(
+wal_retrieve_retry_interval = '100ms'
+));
+$node_standby->start;
+
+# Create some content on master
+$node_master->psql('postgres', "CREATE TABLE tab_int AS SELECT generate_series(1,1000) AS a");
+my $current_lsn = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+
+# Force archiving of WAL file to make it present on master
+$node_master->psql('postgres', "SELECT pg_switch_xlog()");
+
+# Add some more content, it should not be present on standby
+$node_master->psql('postgres', "INSERT INTO tab_int VALUES (generate_series(1001,2000))");
+
+# Wait until necessary replay has been done on standby
+my $caughtup_query = "SELECT '$current_lsn'::pg_lsn <= pg_last_xlog_replay_location()";
+$node_standby->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby to catch up";
+
+my $result = $node_standby->psql('postgres', "SELECT count(*) FROM tab_int");
+is($result, qq(1000), 'check content from archives');
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
new file mode 100644
index 0000000..293603a
--- /dev/null
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -0,0 +1,125 @@
+# Test for recovery targets: name, timestamp, XID
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 7;
+
+# Create and test a standby from given backup, with a certain
+# recovery target.
+sub test_recovery_standby
+{
+ my $test_name = shift;
+ my $node_name = shift;
+ my $node_master = shift;
+ my $recovery_params = shift;
+ my $num_rows = shift;
+ my $until_lsn = shift;
+
+ my $node_standby = get_new_node($node_name);
+ $node_standby->init_from_backup($node_master, 'my_backup',
+ has_restoring => 1);
+
+ foreach my $param_item (@$recovery_params)
+ {
+ $node_standby->append_conf('recovery.conf',
+ qq($param_item
+));
+ }
+
+ $node_standby->start;
+
+ # Wait until standby has replayed enough data
+ my $caughtup_query = "SELECT '$until_lsn'::pg_lsn <= pg_last_xlog_replay_location()";
+ $node_standby->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby to catch up";
+
+ # Create some content on master and check its presence in standby
+ my $result = $node_standby->psql('postgres', "SELECT count(*) FROM tab_int");
+ is($result, qq($num_rows), "check standby content for $test_name");
+
+ # Stop standby node
+ $node_standby->teardown_node;
+}
+
+# Initialize master node
+my $node_master = get_new_node('master');
+$node_master->init(has_archiving => 1, allows_streaming => 1);
+
+# Start it
+$node_master->start;
+
+# Create data before taking the backup, aimed at testing
+# recovery_target = 'immediate'
+$node_master->psql('postgres', "CREATE TABLE tab_int AS SELECT generate_series(1,1000) AS a");
+my $lsn1 = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+
+# Take backup from which all operations will be run
+$node_master->backup('my_backup');
+
+# Insert some data with used as a replay reference, with a recovery
+# target TXID.
+$node_master->psql('postgres', "INSERT INTO tab_int VALUES (generate_series(1001,2000))");
+my $recovery_txid = $node_master->psql('postgres', "SELECT txid_current()");
+my $lsn2 = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+
+# More data, with recovery target timestamp
+$node_master->psql('postgres', "INSERT INTO tab_int VALUES (generate_series(2001,3000))");
+my $recovery_time = $node_master->psql('postgres', "SELECT now()");
+my $lsn3 = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+
+# Even more data, this time with a recovery target name
+$node_master->psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(3001,4000))");
+my $recovery_name = "my_target";
+my $lsn4 = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+$node_master->psql('postgres', "SELECT pg_create_restore_point('$recovery_name'");
+
+# Force archiving of WAL file
+$node_master->psql('postgres', "SELECT pg_switch_xlog()");
+
+# Test recovery targets
+my @recovery_params = ( "recovery_target = 'immediate'" );
+test_recovery_standby('immediate target', 'standby_1', $node_master,
+ \@recovery_params,
+ "1000", $lsn1);
+@recovery_params = ( "recovery_target_xid = '$recovery_txid'" );
+test_recovery_standby('XID', 'standby_2', $node_master,
+ \@recovery_params,
+ "2000", $lsn2);
+@recovery_params = ( "recovery_target_time = '$recovery_time'" );
+test_recovery_standby('Time', 'standby_3', $node_master,
+ \@recovery_params,
+ "3000", $lsn3);
+@recovery_params = ( "recovery_target_name = '$recovery_name'" );
+test_recovery_standby('Name', 'standby_4', $node_master,
+ \@recovery_params,
+ "4000", $lsn4);
+
+# Multiple targets
+# Last entry has priority (note that an array respects the order of items
+# not hashes).
+@recovery_params = (
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'"
+);
+test_recovery_standby('Name + XID + Time', 'standby_5', $node_master,
+ \@recovery_params,
+ "3000", $lsn3);
+@recovery_params = (
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'",
+ "recovery_target_xid = '$recovery_txid'"
+);
+test_recovery_standby('Time + Name + XID', 'standby_6', $node_master,
+ \@recovery_params,
+ "2000", $lsn2);
+@recovery_params = (
+ "recovery_target_xid = '$recovery_txid'",
+ "recovery_target_time = '$recovery_time'",
+ "recovery_target_name = '$recovery_name'"
+);
+test_recovery_standby('XID + Time + Name', 'standby_7', $node_master,
+ \@recovery_params,
+ "4000", $lsn4);
diff --git a/src/test/recovery/t/004_timeline_switch.pl b/src/test/recovery/t/004_timeline_switch.pl
new file mode 100644
index 0000000..dc08ec1
--- /dev/null
+++ b/src/test/recovery/t/004_timeline_switch.pl
@@ -0,0 +1,67 @@
+# Test for timeline switch
+# Ensure that a standby is able to follow a newly-promoted standby
+# on a new timeline.
+use strict;
+use warnings;
+use File::Path qw(remove_tree);
+use PostgresNode;
+use TestLib;
+use Test::More tests => 1;
+
+$ENV{PGDATABASE} = 'postgres';
+
+# Initialize master node
+my $node_master = get_new_node('master');
+$node_master->init(allows_streaming => 1);
+$node_master->start;
+
+# Take backup
+my $backup_name = 'my_backup';
+$node_master->backup($backup_name);
+
+# Create two standbys linking to it
+my $node_standby_1 = get_new_node('standby_1');
+$node_standby_1->init_from_backup($node_master, $backup_name,
+ has_streaming => 1);
+$node_standby_1->start;
+my $node_standby_2 = get_new_node('standby_2');
+$node_standby_2->init_from_backup($node_master, $backup_name,
+ has_streaming => 1);
+$node_standby_2->start;
+
+# Create some content on master
+$node_master->psql('postgres',
+ "CREATE TABLE tab_int AS SELECT generate_series(1,1000) AS a");
+my $until_lsn = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+
+# Wait until standby has replayed enough data on standby 1
+my $caughtup_query = "SELECT '$until_lsn'::pg_lsn <= pg_last_xlog_replay_location()";
+$node_standby_1->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby to catch up";
+
+# Stop and remove master, and promote standby 1, switching it to a new timeline
+$node_master->teardown_node;
+$node_standby_1->promote;
+
+# Switch standby 2 to replay from standby 1
+remove_tree($node_standby_2->data_dir . '/recovery.conf');
+my $connstr_1 = $node_standby_1->connstr;
+$node_standby_2->append_conf('recovery.conf', qq(
+primary_conninfo='$connstr_1'
+standby_mode=on
+recovery_target_timeline='latest'
+));
+$node_standby_2->restart;
+
+# Insert some data in standby 1 and check its presence in standby 2
+# to ensure that the timeline switch has been done. Standby 1 needs
+# to exit recovery first before moving on with the test.
+$node_standby_1->poll_query_until('postgres', "SELECT pg_is_in_recovery() <> true");
+$node_standby_1->psql('postgres', "INSERT INTO tab_int VALUES (generate_series(1001,2000))");
+$until_lsn = $node_standby_1->psql('postgres', "SELECT pg_current_xlog_location();");
+$caughtup_query = "SELECT '$until_lsn'::pg_lsn <= pg_last_xlog_replay_location()";
+$node_standby_2->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby to catch up";
+
+my $result = $node_standby_2->psql('postgres', "SELECT count(*) FROM tab_int");
+is($result, qq(2000), 'check content of standby 2');
diff --git a/src/test/recovery/t/005_replay_delay.pl b/src/test/recovery/t/005_replay_delay.pl
new file mode 100644
index 0000000..2d8c690
--- /dev/null
+++ b/src/test/recovery/t/005_replay_delay.pl
@@ -0,0 +1,43 @@
+# Checks for recovery_min_apply_delay
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 2;
+
+# Initialize master node
+my $node_master = get_new_node('master');
+$node_master->init(allows_streaming => 1);
+$node_master->start;
+
+# And some content
+$node_master->psql('postgres', "CREATE TABLE tab_int AS SELECT generate_series(1,10) AS a");
+
+# Take backup
+my $backup_name = 'my_backup';
+$node_master->backup($backup_name);
+
+# Create streaming standby from backup
+my $node_standby = get_new_node('standby');
+$node_standby->init_from_backup($node_master, $backup_name,
+ has_streaming => 1);
+$node_standby->append_conf('recovery.conf', qq(
+recovery_min_apply_delay = '2s'
+));
+$node_standby->start;
+
+# Make new content on master and check its presence in standby
+# depending on the delay of 2s applied above.
+$node_master->psql('postgres', "INSERT INTO tab_int VALUES (generate_series(11,20))");
+sleep 1;
+# Here we should have only 10 rows
+my $result = $node_standby->psql('postgres', "SELECT count(*) FROM tab_int");
+is($result, qq(10), 'check content with delay of 1s');
+
+# Now wait for replay to complete on standby
+my $until_lsn = $node_master->psql('postgres', "SELECT pg_current_xlog_location();");
+my $caughtup_query = "SELECT '$until_lsn'::pg_lsn <= pg_last_xlog_replay_location()";
+$node_standby->poll_query_until('postgres', $caughtup_query)
+ or die "Timed out while waiting for standby to catch up";
+$result = $node_standby->psql('postgres', "SELECT count(*) FROM tab_int");
+is($result, qq(20), 'check content with delay of 2s');
diff --git a/src/tools/msvc/config_default.pl b/src/tools/msvc/config_default.pl
index e50be7e..636beea 100644
--- a/src/tools/msvc/config_default.pl
+++ b/src/tools/msvc/config_default.pl
@@ -16,7 +16,7 @@ our $config = {
ldap => 1, # --with-ldap
extraver => undef, # --with-extra-version=<string>
nls => undef, # --enable-nls=<path>
- tap_tests => undef, # --enable-tap-tests
+ tap_tests => 1, # --enable-tap-tests
tcl => undef, # --with-tls=<path>
perl => undef, # --with-perl
python => undef, # --with-python=<path>
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index df1348b..3d14544 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -34,7 +34,7 @@ if (-e "src/tools/msvc/buildenv.pl")
my $what = shift || "";
if ($what =~
-/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck)$/i
+/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck)$/i
)
{
$what = uc $what;
@@ -89,6 +89,7 @@ my %command = (
MODULESCHECK => \&modulescheck,
ISOLATIONCHECK => \&isolationcheck,
BINCHECK => \&bincheck,
+ RECOVERYCHECK => \&recoverycheck,
UPGRADECHECK => \&upgradecheck,);
my $proc = $command{$what};
@@ -360,6 +361,16 @@ sub modulescheck
exit $mstat if $mstat;
}
+sub recoverycheck
+{
+ InstallTemp();
+
+ my $mstat = 0;
+ my $dir = "$topdir/src/test/recovery";
+ my $status = tap_check($dir);
+ exit $status if $status;
+}
+
# Run "initdb", then reconfigure authentication.
sub standard_initdb
{
--
2.7.1
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
