Hi Michael,
On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier <mich...@paquier.xyz> wrote:
>
> On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote:
> > Okay, thanks for the feedback. We have been relying on diff -u for
> > the parts of the tests touched by 0001 for some time now, so if there
> > are no objections I would like to apply 0001 in a couple of days.
>
> This part has been applied as 169208092f5c.
Thanks. PFA rebased patches.
I have added another diff adjustment to adjust_regress_dumpfile().
It's introduced by 83ea6c54025bea67bcd4949a6d58d3fc11c3e21b.
On Thu, Feb 6, 2025 at 11:32 AM Michael Paquier <mich...@paquier.xyz> wrote:
>
> On Wed, Feb 05, 2025 at 03:28:04PM +0900, Michael Paquier wrote:
> > Hmm. I was reading through the patch and there is something that
> > clearly stands out IMO: the new compare_dumps(). It is in Utils.pm,
> > and it acts as a wrapper of `diff` with its formalized output format.
> > It is not really about dumps, but about file comparisons. This should
> > be renamed compare_files(), with internals adjusted as such, and
> > reused in all the existing tests. Good idea to use that in
> > 027_stream_regress.pl, actually. I'll go extract that first, to
> > reduce the presence of `diff` in the whole set of TAP tests.
>
> The result of this part is pretty neat, resulting in 0001 where it is
> possible to use the refactored routine as well in pg_combinebackup
> where there is a piece comparing dumps. There are three more tests
> with diff commands and assumptions of their own, that I've left out.
> This has the merit of unifying the output generated should any diffs
> show up, while removing a nice chunk from the main patch.
>
> > AdjustDump.pm looks like a fine concept as it stands. I still need to
> > think more about it. It feels like we don't have the most optimal
> > interface, though, but perhaps that will be clearer once
> > compare_dumps() is moved out of the way.
Without knowing what makes the interface suboptimal, it's hard to make
it optimal. I did think about getting rid of adjust_child_columns
flag. But that either means we adjust CREATE TABLE ... INHERIT
statements from both the dump outputs from original and the restored
database to a canonical form or get rid of the tests in that function
to make sure that the adjustment is required. The first seems more
work (coding and run time). The tests look useful to detect when the
adjustment won't be required.
I also looked at the routines which adjust the dumps from upgrade
tests. They seem to be specific to the older versions and lack the
extensibility you mentioned earlier.
The third thing I looked at was the possibility of applying the
adjustments to only the dump from the original database where it is
required by passing the newline adjustments to compare_files().
However 0002 in the attached set of patches adds more logic,
applicable to both the original and restored dump outputs, to
AdjustDump.pm. So we can't do that either.
I am clueless as to what could be improved here.
>
> + my %dump_formats = ('plain' => 'p', 'tar' => 't', 'directory' => 'd',
> 'custom' => 'c');
>
> No need for this mapping, let's just use the long options.
Hmm, didn't realize -F accepts whole format name as well. pg_dump
--help doesn't give that impression but user facing documentation
mentions it. Done.
>
> + # restore data as well to catch any errors while doing so.
> + command_ok(
> + [
> + 'pg_dump', "-F$format_spec", '--no-sync',
> + '-d', $src_node->connstr('regression'),
> + '-f', $dump_file
> + ],
> + "pg_dump on source instance in $format format");
>
> The use of command_ok() looks incorrect here. Shouldn't we use
> $src_node->command_ok() here to ensure a correct PATH? That would be
> more consistent with the other dump commands. Same remark about
> @restore_command.
Cluster::command_ok's prologue doesn't mention PATH but mentions
PGHOST and PGPORT.
```
Runs a shell command like PostgreSQL::Test::Utils::command_ok, but
with PGHOST and PGPORT set
so that the command will default to connecting to this PostgreSQL::Test::Cluster
```
According to sub _get_env(), PATH is set only when custom install path
is provided. In the absence of that, build path is used. In this case,
the source and destination nodes are created from the build itself, so
no separate path is required. PGHOST and PGPORT are anyway overridden
by the connection string fetched from the node. So I don't think
there's any correctness issue here, but it's better to use
Cluster::command_ok() just for better readability. Done
>
> + # The order of columns in COPY statements dumped from the original
> database
> + # and that from the restored database differs. These differences are
> hard to
>
> What are the relations we are talking about here?
These are the child tables whose parent has added a column after being
inherited. Now that I have more expertise with perl regex, I have
added code in AdjustDump.pm to remove only the COPY statements where
we see legitimate difference. Added a comment explaining the cause
behind the difference. This patch is supposed to be merged into 0001
before committing the change.
--
Best Wishes,
Ashutosh Bapat
From 42ddf5e9fd6ea64c04e8c45033001fc834cd8039 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <ashutosh.ba...@enterprisedb.com>
Date: Thu, 27 Jun 2024 10:03:53 +0530
Subject: [PATCH 1/2] Test pg_dump/restore of regression objects
002_pg_upgrade.pl tests pg_upgrade of the regression database left
behind by regression run. Modify it to test dump and restore of the
regression database as well.
Regression database created by regression run contains almost all the
database objects supported by PostgreSQL in various states. Hence the
new testcase covers dump and restore scenarios not covered by individual
dump/restore cases. Till now 002_pg_upgrade only tested dump/restore
through pg_upgrade which only uses binary mode. Many regression tests
mention that they leave objects behind for dump/restore testing but they
are not tested in a non-binary mode. The new testcase closes that
gap.
Testing dump and restore of regression database makes this test run
longer for a relatively smaller benefit. Hence run it only when
explicitly requested by user by specifying "regress_dump_test" in
PG_TEST_EXTRA.
Note For the reviewers:
The new test has uncovered two bugs so far in one year.
1. Introduced by 14e87ffa5c54. Fixed in fd41ba93e4630921a72ed5127cd0d552a8f3f8fc.
2. Introduced by 0413a556990ba628a3de8a0b58be020fd9a14ed0. Reverted in 74563f6b90216180fc13649725179fc119dddeb5.
Author: Ashutosh Bapat
Reviewed by: Michael Pacquire, Daniel Gustafsson, Tom Lane
Discussion: https://www.postgresql.org/message-id/CAExHW5uF5V=Cjecx3_Z=7xfh4rg2wf61pt+hfquzjbqourz...@mail.gmail.com
---
doc/src/sgml/regress.sgml | 12 ++
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 140 +++++++++++++++++++-
src/test/perl/Makefile | 2 +
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 134 +++++++++++++++++++
src/test/perl/meson.build | 1 +
5 files changed, 287 insertions(+), 2 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/AdjustDump.pm
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 7c474559bdf..3061ce42fd1 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -347,6 +347,18 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance libpq_encryption'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>regress_dump_test</literal></term>
+ <listitem>
+ <para>
+ When enabled, <filename>src/bin/pg_upgrade/t/002_pg_upgrade.pl</filename>
+ tests dump and restore of regression database left behind by the
+ regression run. Not enabled by default because it is time and resource
+ consuming.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 68516fa486a..0d636529d74 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -12,6 +12,7 @@ use File::Path qw(rmtree);
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::AdjustUpgrade;
+use PostgreSQL::Test::AdjustDump;
use Test::More;
# Can be changed to test the other modes.
@@ -35,8 +36,8 @@ sub generate_db
"created database with ASCII characters from $from_char to $to_char");
}
-# Filter the contents of a dump before its use in a content comparison.
-# This returns the path to the filtered dump.
+# Filter the contents of a dump before its use in a content comparison for
+# upgrade testing. This returns the path to the filtered dump.
sub filter_dump
{
my ($is_old, $old_version, $dump_file) = @_;
@@ -261,6 +262,21 @@ else
}
}
is($rc, 0, 'regression tests pass');
+
+ # Test dump/restore of the objects left behind by regression. Ideally it
+ # should be done in a separate TAP test, but doing it here saves us one full
+ # regression run.
+ #
+ # This step takes several extra seconds and some extra disk space, so
+ # requires an opt-in with the PG_TEST_EXTRA environment variable.
+ #
+ # Do this while the old cluster is running before it is shut down by the
+ # upgrade test.
+ if ( $ENV{PG_TEST_EXTRA}
+ && $ENV{PG_TEST_EXTRA} =~ /\bregress_dump_test\b/)
+ {
+ test_regression_dump_restore($oldnode, %node_params);
+ }
}
# Initialize a new node for the upgrade.
@@ -517,4 +533,124 @@ my $dump2_filtered = filter_dump(0, $oldnode->pg_version, $dump2_file);
compare_files($dump1_filtered, $dump2_filtered,
'old and new dumps match after pg_upgrade');
+# Test dump and restore of objects left behind by the regression run.
+#
+# It is expected that regression tests, which create `regression` database, are
+# run on `src_node`, which in turn, is left in running state. The dump from
+# `src_node` is restored on a fresh node created using given `node_params`.
+# Plain dumps from both the nodes are compared to make sure that all the dumped
+# objects are restored faithfully.
+sub test_regression_dump_restore
+{
+ my ($src_node, %node_params) = @_;
+ my $dst_node = PostgreSQL::Test::Cluster->new('dst_node');
+
+ # Make sure that the source and destination nodes have the same version and
+ # do not use custom install paths. In both the cases, the dump files may
+ # require additional adjustments unknown to code here. Do not run this test
+ # in such a case to avoid utilizing the time and resources unnecessarily.
+ if ($src_node->pg_version != $dst_node->pg_version
+ or defined $src_node->{_install_path})
+ {
+ fail("same version dump and restore test using default installation");
+ return;
+ }
+
+ # Dump the original database for comparison later.
+ my $src_dump =
+ get_dump_for_comparison($src_node, 'regression', 'src_dump', 1);
+
+ # Setup destination database cluster
+ $dst_node->init(%node_params);
+ $dst_node->start;
+
+ for my $format ('plain', 'tar', 'directory', 'custom')
+ {
+ my $dump_file = "$tempdir/regression_dump.$format";
+ my $restored_db = 'regression_' . $format;
+
+ # Even though we compare only schema from the original and the restored
+ # database (See get_dump_for_comparison() for details.), we dump and
+ # restore data as well to catch any errors while doing so.
+ $src_node->command_ok(
+ [
+ 'pg_dump', "-F$format", '--no-sync',
+ '-d', $src_node->connstr('regression'),
+ '-f', $dump_file
+ ],
+ "pg_dump on source instance in $format format");
+
+ # Create a new database for restoring dump from every format so that it
+ # is available for debugging in case the test fails.
+ $dst_node->command_ok([ 'createdb', $restored_db ],
+ "created destination database '$restored_db'");
+
+ # Restore into destination database.
+ my @restore_command;
+ if ($format eq 'plain')
+ {
+ # Restore dump in "plain" format using `psql`.
+ @restore_command = [
+ 'psql', '-d', $dst_node->connstr($restored_db),
+ '-f', $dump_file
+ ];
+ }
+ else
+ {
+ @restore_command = [
+ 'pg_restore', '-d',
+ $dst_node->connstr($restored_db), $dump_file
+ ];
+ }
+ $dst_node->command_ok(@restore_command,
+ "restored dump taken in $format format on destination instance");
+
+ my $dst_dump =
+ get_dump_for_comparison($dst_node, $restored_db,
+ 'dest_dump.' . $format, 0);
+
+ compare_files($src_dump, $dst_dump,
+ "dump outputs from original and restored regression database (using $format format) match"
+ );
+ }
+}
+
+# Dump database `db` from the given `node` in plain format and adjust it for
+# comparing dumps from the original and the restored database.
+#
+# `file_prefix` is used to create unique names for all dump files so that they
+# remain available for debugging in case the test fails.
+#
+# `adjust_child_columns` is passed to adjust_regress_dumpfile() which actually
+# adjusts the dump output.
+#
+# The name of the file containting adjusted dump is returned.
+sub get_dump_for_comparison
+{
+ my ($node, $db, $file_prefix, $adjust_child_columns) = @_;
+
+ my $dumpfile = $tempdir . '/' . $file_prefix . '.sql';
+ my $dump_adjusted = "${dumpfile}_adjusted";
+
+
+ # The order of columns in COPY statements dumped from the original database
+ # and that from the restored database differs. These differences are hard to
+ # adjust. Hence we compare only schema dumps for now.
+ $node->command_ok(
+ [
+ 'pg_dump', '-s', '--no-sync', '-d',
+ $node->connstr($db), '-f', $dumpfile
+ ],
+ 'dump for comparison succeeded');
+
+ open(my $dh, '>', $dump_adjusted)
+ || die
+ "could not open $dump_adjusted for writing the adjusted dump: $!";
+ print $dh adjust_regress_dumpfile(slurp_file($dumpfile),
+ $adjust_child_columns);
+ close($dh);
+
+ return $dump_adjusted;
+}
+
done_testing();
diff --git a/src/test/perl/Makefile b/src/test/perl/Makefile
index d82fb67540e..def89650ead 100644
--- a/src/test/perl/Makefile
+++ b/src/test/perl/Makefile
@@ -26,6 +26,7 @@ install: all installdirs
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/Cluster.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/BackgroundPsql.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustUpgrade.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ $(INSTALL_DATA) $(srcdir)/PostgreSQL/Test/AdjustDump.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
$(INSTALL_DATA) $(srcdir)/PostgreSQL/Version.pm '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
uninstall:
@@ -36,6 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/Cluster.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/BackgroundPsql.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustUpgrade.pm'
+ rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Test/AdjustDump.pm'
rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/PostgreSQL/Version.pm'
endif
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
new file mode 100644
index 00000000000..e3e152b88fa
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -0,0 +1,134 @@
+
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::AdjustDump - helper module for dump and restore tests
+
+=head1 SYNOPSIS
+
+ use PostgreSQL::Test::AdjustDump;
+
+ # Adjust contents of dump output file so that dump output from original
+ # regression database and that from the restored regression database match
+ $dump = adjust_regress_dumpfile($dump, $adjust_child_columns);
+
+=head1 DESCRIPTION
+
+C<PostgreSQL::Test::AdjustDump> encapsulates various hacks needed to
+compare the results of dump and restore tests
+
+=cut
+
+package PostgreSQL::Test::AdjustDump;
+
+use strict;
+use warnings FATAL => 'all';
+
+use Exporter 'import';
+use Test::More;
+
+our @EXPORT = qw(
+ adjust_regress_dumpfile
+);
+
+=pod
+
+=head1 ROUTINES
+
+=over
+
+=item $dump = adjust_regress_dumpfile($dump, $adjust_child_columns)
+
+If we take dump of the regression database left behind after running regression
+tests, restore the dump, and take dump of the restored regression database, the
+outputs of both the dumps differ. Some regression tests purposefully create
+some child tables in such a way that their column orders differ from column
+orders of their respective parents. In the restored database, however, their
+column orders are same as that of their respective parents. Thus the column
+orders of these child tables in the original database and those in the restored
+database differ, causing difference in the dump outputs. See MergeAttributes()
+and dumpTableSchema() for details.
+
+This routine rearranges the column declarations in the relevant
+C<CREATE TABLE... INHERITS> statements in the dump file from original database
+to match those from the restored database. We could instead adjust the
+statements in the dump from the restored database to match those from original
+database or adjust both to a canonical order. But we have chosen to adjust the
+statements in the dump from original database for no particular reason.
+
+Additionally it adjusts blank and new lines to avoid noise.
+
+Arguments:
+
+=over
+
+=item C<dump>: Contents of dump file
+
+=item C<adjust_child_columns>: 1 indicates that the given dump file requires
+adjusting columns in the child tables; usually when the dump is from original
+database. 0 indicates no such adjustment is needed; usually when the dump is
+from restored database.
+
+=back
+
+Returns the adjusted dump text.
+
+=cut
+
+sub adjust_regress_dumpfile
+{
+ my ($dump, $adjust_child_columns) = @_;
+
+ # use Unix newlines
+ $dump =~ s/\r\n/\n/g;
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
+ # Adjust the CREATE TABLE ... INHERITS statements.
+ if ($adjust_child_columns)
+ {
+ my $saved_dump = $dump;
+
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_stored_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_stored_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\sgenerated_virtual_tests\.gtestxx_4\s\()
+ (\n\s+b\sinteger),
+ (\n\s+a\sinteger\sNOT\sNULL)/$1$3,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied generated_virtual_tests.gtestxx_4 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c1\s\()
+ (\n\s+int_four\sbigint),
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint)/$1$4,$2,$3/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c1 adjustments');
+
+ $saved_dump = $dump;
+ $dump =~ s/(^CREATE\sTABLE\spublic\.test_type_diff2_c2\s\()
+ (\n\s+int_eight\sbigint),
+ (\n\s+int_two\ssmallint),
+ (\n\s+int_four\sbigint)/$1$3,$4,$2/mgx;
+ ok($saved_dump ne $dump,
+ 'applied public.test_type_diff2_c2 adjustments');
+ }
+
+ return $dump;
+}
+
+=pod
+
+=back
+
+=cut
+
+1;
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
index 58e30f15f9d..492ca571ff8 100644
--- a/src/test/perl/meson.build
+++ b/src/test/perl/meson.build
@@ -14,4 +14,5 @@ install_data(
'PostgreSQL/Test/Cluster.pm',
'PostgreSQL/Test/BackgroundPsql.pm',
'PostgreSQL/Test/AdjustUpgrade.pm',
+ 'PostgreSQL/Test/AdjustDump.pm',
install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
base-commit: 6998db59c2959c4f280a9088054e6dbf7178efe0
--
2.34.1
From 6976ebad1e4fa717cd8fa2f70fbe73cd476e7caf Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <ashutosh.bapat....@gmail.com>
Date: Tue, 11 Feb 2025 16:31:10 +0530
Subject: [PATCH 2/2] Filter COPY statements with differing column order
---
src/bin/pg_upgrade/t/002_pg_upgrade.pl | 10 +---
src/test/perl/PostgreSQL/Test/AdjustDump.pm | 59 +++++++++++++++------
2 files changed, 45 insertions(+), 24 deletions(-)
diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 0d636529d74..b3e1573ec34 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -569,9 +569,6 @@ sub test_regression_dump_restore
my $dump_file = "$tempdir/regression_dump.$format";
my $restored_db = 'regression_' . $format;
- # Even though we compare only schema from the original and the restored
- # database (See get_dump_for_comparison() for details.), we dump and
- # restore data as well to catch any errors while doing so.
$src_node->command_ok(
[
'pg_dump', "-F$format", '--no-sync',
@@ -633,13 +630,10 @@ sub get_dump_for_comparison
my $dump_adjusted = "${dumpfile}_adjusted";
- # The order of columns in COPY statements dumped from the original database
- # and that from the restored database differs. These differences are hard to
- # adjust. Hence we compare only schema dumps for now.
$node->command_ok(
[
- 'pg_dump', '-s', '--no-sync', '-d',
- $node->connstr($db), '-f', $dumpfile
+ 'pg_dump', '--no-sync', '-d', $node->connstr($db), '-f',
+ $dumpfile
],
'dump for comparison succeeded');
diff --git a/src/test/perl/PostgreSQL/Test/AdjustDump.pm b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
index e3e152b88fa..e00a00d1b2c 100644
--- a/src/test/perl/PostgreSQL/Test/AdjustDump.pm
+++ b/src/test/perl/PostgreSQL/Test/AdjustDump.pm
@@ -44,22 +44,36 @@ our @EXPORT = qw(
If we take dump of the regression database left behind after running regression
tests, restore the dump, and take dump of the restored regression database, the
-outputs of both the dumps differ. Some regression tests purposefully create
-some child tables in such a way that their column orders differ from column
-orders of their respective parents. In the restored database, however, their
-column orders are same as that of their respective parents. Thus the column
+outputs of both the dumps differ in the following cases. This routine adjusts
+the given dump so that dump outputs from the original and restored database,
+respectively, match.
+
+Case 1: Some regression tests purposefully create child tables in such a way
+that the order of their inherited columns differ from column orders of their
+respective parents. In the restored database, however, the order of their
+inherited columns are same as that of their respective parents. Thus the column
orders of these child tables in the original database and those in the restored
database differ, causing difference in the dump outputs. See MergeAttributes()
-and dumpTableSchema() for details.
-
-This routine rearranges the column declarations in the relevant
-C<CREATE TABLE... INHERITS> statements in the dump file from original database
-to match those from the restored database. We could instead adjust the
-statements in the dump from the restored database to match those from original
-database or adjust both to a canonical order. But we have chosen to adjust the
-statements in the dump from original database for no particular reason.
-
-Additionally it adjusts blank and new lines to avoid noise.
+and dumpTableSchema() for details. This routine rearranges the column
+declarations in the relevant C<CREATE TABLE... INHERITS> statements in the dump
+file from original database to match those from the restored database. We could,
+instead, adjust the statements in the dump from the restored database to match
+those from original database or adjust both to a canonical order. But we have
+chosen to adjust the statements in the dump from original database for no
+particular reason.
+
+Case 2: When dumping COPY statements the columns are ordered by their attribute
+number by fmtCopyColumnList(). If a column is added to a parent table after a
+child has inherited the parent and the child has its own columns, the attribute
+number of the column changes after restoring the child table. This is because
+when executing the dumped C<CREATE TABLE... INHERITS> statement all the parent
+attributes are created before any child attributes. Thus the order of columns in
+COPY statements dumped from the original and the restored databases,
+respectively, differs. Such tables in regression tests are listed below. It is
+hard to adjust the column order in the COPY statement along with the data. Hence
+we just remove such COPY statements from the dump output.
+
+Additionally the routine adjusts blank and new lines to avoid noise.
Arguments:
@@ -84,8 +98,6 @@ sub adjust_regress_dumpfile
# use Unix newlines
$dump =~ s/\r\n/\n/g;
- # Suppress blank lines, as some places in pg_dump emit more or fewer.
- $dump =~ s/\n\n+/\n/g;
# Adjust the CREATE TABLE ... INHERITS statements.
if ($adjust_child_columns)
@@ -122,6 +134,21 @@ sub adjust_regress_dumpfile
'applied public.test_type_diff2_c2 adjustments');
}
+ # Remove COPY statements with differing column order
+ for my $table (
+ 'public\.b_star', 'public\.c_star',
+ 'public\.cc2', 'public\.d_star',
+ 'public\.e_star', 'public\.f_star',
+ 'public\.renamecolumnanother', 'public\.renamecolumnchild',
+ 'public\.test_type_diff2_c1', 'public\.test_type_diff2_c2',
+ 'public\.test_type_diff_c')
+ {
+ $dump =~ s/^COPY\s$table\s\(.+?^\\\.$//sm;
+ }
+
+ # Suppress blank lines, as some places in pg_dump emit more or fewer.
+ $dump =~ s/\n\n+/\n/g;
+
return $dump;
}
--
2.34.1
