> On 20 Nov 2023, at 06:20, Pavel Stehule <pavel.steh...@gmail.com> wrote:
> I was pondering replacing the is_include handling with returning an enum for > the operation, to keep things more future proof in case we add more operations > (and also a bit less magic IMHO). > > +1 > > I did it. Nice, I think it's an improvement. + <literal>extension</literal>: data on foreign servers, works like + <option>--extension</option>. This keyword can only be + used with the <literal>include</literal> keyword. This seems like a copy-pasteo, fixed in the attached. I've spent some time polishing this version of the patch, among other things trying to make the docs and --help screen consistent across the tools. I've added the diff as a txt file to this email (to keep the CFbot from applying it), it's mainly reformatting a few comments and making things consistent. The attached is pretty close to a committable patch IMO, review is welcome on both the patch and commit message. I tried to identify all reviewers over the past 3+ years but I might have missed someone. -- Daniel Gustafsson
commit 4a3c0bdaf3fd21b75e17244691fbeb9340e960e1
Author: Daniel Gustafsson <dgustafs...@postgresql.org>
Date: Tue Nov 21 15:08:27 2023 +0100
Fixups and tweaks
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index e2f100d552..0e5ba4f712 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -873,49 +873,52 @@ PostgreSQL documentation
<itemizedlist>
<listitem>
<para>
- <literal>extension</literal>: data on foreign servers, works like
- <option>--extension</option>. This keyword can only be
+ <literal>extension</literal>: extensions, works like the
+ <option>--extension</option> option. This keyword can only be
used with the <literal>include</literal> keyword.
</para>
</listitem>
<listitem>
<para>
<literal>foreign_data</literal>: data on foreign servers, works like
- <option>--include-foreign-data</option>. This keyword can only be
- used with the <literal>include</literal> keyword.
+ the <option>--include-foreign-data</option> option. This keyword can
+ only be used with the <literal>include</literal> keyword.
</para>
</listitem>
<listitem>
<para>
- <literal>table</literal>: tables, works like
- <option>-t</option>/<option>--table</option>
+ <literal>table</literal>: tables, works like the
+ <option>-t</option>/<option>--table</option> option.
</para>
</listitem>
<listitem>
<para>
- <literal>table_and_children</literal>: tables, works like
- <option>--table-and-children</option>
+ <literal>table_and_children</literal>: tables including any
partitions
+ or inheritance child tables, works like the
+ <option>--table-and-children</option> option.
</para>
</listitem>
<listitem>
<para>
- <literal>table_data</literal>: table data, works like
- <option>--exclude-table-data</option>. This keyword can only be
- used with the <literal>exclude</literal> keyword.
+ <literal>table_data</literal>: table data of any tables matching
+ <replaceable>pattern</replaceable>, works like the
+ <option>--exclude-table-data</option> option. This keyword can only
+ be used with the <literal>exclude</literal> keyword.
</para>
</listitem>
<listitem>
<para>
- <literal>table_data_and_children</literal>: table data of any
- partitions or inheritance child, works like
- <option>--exclude-table-data-and-children</option>. This keyword
can only be
- used with the <literal>exclude</literal> keyword.
+ <literal>table_data_and_children</literal>: table data of any tables
+ matching <replaceable>pattern</replaceable> as well as any
partitions
+ or inheritance children of the table(s), works like the
+ <option>--exclude-table-data-and-children</option> option. This
+ keyword can only be used with the <literal>exclude</literal>
keyword.
</para>
</listitem>
<listitem>
<para>
- <literal>schema</literal>: schemas, works like
- <option>-n</option>/<option>--schema</option>
+ <literal>schema</literal>: schemas, works like the
+ <option>-n</option>/<option>--schema</option> option.
</para>
</listitem>
</itemizedlist>
@@ -923,9 +926,9 @@ PostgreSQL documentation
<para>
Lines starting with <literal>#</literal> are considered comments and
- ignored. Comments can be placed after filter as well. Blank lines
- are also ignored. See <xref linkend="app-psql-patterns"/> for how to
- perform quoting in patterns.
+ ignored. Comments can be placed after an object pattern row as well.
+ Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+ for how to perform quoting in patterns.
</para>
<para>
@@ -1715,8 +1718,8 @@ CREATE DATABASE foo WITH TEMPLATE template0;
</screen></para>
<para>
- To dump all tables with names starting with mytable, except for table
- <literal>mytable2</literal>, specify a filter file
+ To dump all tables whose names start with <literal>mytable</literal>, except
+ for table <literal>mytable2</literal>, specify a filter file
<filename>filter.txt</filename> like:
<programlisting>
include table mytable*
diff --git a/doc/src/sgml/ref/pg_dumpall.sgml b/doc/src/sgml/ref/pg_dumpall.sgml
index 75ba03f3ad..4d7c046468 100644
--- a/doc/src/sgml/ref/pg_dumpall.sgml
+++ b/doc/src/sgml/ref/pg_dumpall.sgml
@@ -130,20 +130,29 @@ PostgreSQL documentation
<listitem>
<para>
Specify a filename from which to read patterns for databases excluded
- from the dump. The patterns are interpretted according to the same
rules
+ from the dump. The patterns are interpreted according to the same rules
as <option>--exclude-database</option>.
To read from <literal>STDIN</literal>, use <filename>-</filename> as
the
filename. The <option>--filter</option> option can be specified in
- conjunction with the above listed options for excluding databases,
- and can also be specified more than once for multiple filter files.
+ conjunction with <option>--exclude-database</option> for excluding
+ databases, and can also be specified more than once for multiple filter
+ files.
</para>
<para>
The file lists one database pattern per row, with the following format:
<synopsis>
-exclude database <replaceable class="parameter">PATTERN</replaceable>
+exclude database <replaceable class="parameter">PATTERN</replaceable>
</synopsis>
</para>
+
+ <para>
+ Lines starting with <literal>#</literal> are considered comments and
+ ignored. Comments can be placed after an object pattern row as well.
+ Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+ for how to perform quoting in patterns.
+ </para>
+
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/pg_restore.sgml b/doc/src/sgml/ref/pg_restore.sgml
index 64f7c5dc4d..1a23874da6 100644
--- a/doc/src/sgml/ref/pg_restore.sgml
+++ b/doc/src/sgml/ref/pg_restore.sgml
@@ -195,10 +195,14 @@ PostgreSQL documentation
<listitem>
<para>
Specify a filename from which to read patterns for objects excluded
- or included from restore. The patterns are interpretted according to
the
- same rules as <option>--schema</option>,
<option>--exclude-schema</option>,
- <option>--function</option>, <option>--index</option>,
<option>--table</option>
- or <option>--trigger</option>.
+ or included from restore. The patterns are interpreted according to the
+ same rules as
+ <option>-n</option>/<option>--schema</option> for including objects in
schemas,
+ <option>-N</option>/<option>--exclude-schema</option>for excluding
objects in schemas,
+ <option>-P</option>/<option>--function</option> for restoring named
functions,
+ <option>-I</option>/<option>--index</option> for restoring named
indexes,
+ <option>-t</option>/<option>--table</option> for restoring named tables
+ or <option>-T</option>/<option>--trigger</option> for restoring
triggers.
To read from <literal>STDIN</literal>, use <filename>-</filename> as
the
filename. The <option>--filter</option> option can be specified in
conjunction with the above listed options for including or excluding
@@ -212,6 +216,57 @@ PostgreSQL documentation
{ include | exclude } { function | index | schema | table | trigger }
<replaceable class="parameter">PATTERN</replaceable>
</synopsis>
</para>
+
+ <para>
+ The first keyword specifies whether the objects matched by the pattern
+ are to be included or excluded. The second keyword specifies the type
+ of object to be filtered using the pattern:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>function</literal>: functions, works like the
+ <option>-P</option>/<option>--function</option> option. This keyword
+ can only be used with the <literal>include</literal> keyword.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>index</literal>: indexes, works like the
+ <option>-I</option>/<option>--indexes</option> option. This keyword
+ can only be used with the <literal>include</literal> keyword.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>schema</literal>: schemas, works like the
+ <option>-n</option>/<option>--schema</option> and
+ <option>-N</option>/<option>--exclude-schema</option> options.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>table</literal>: tables, works like the
+ <option>-t</option>/<option>--table</option> option. This keyword
+ can only be used with the <literal>include</literal> keyword.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>trigger</literal>: triggers, works like the
+ <option>-T</option>/<option>--trigger</option> option. This keyword
+ can only be used with the <literal>include</literal> keyword.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <para>
+ Lines starting with <literal>#</literal> are considered comments and
+ ignored. Comments can be placed after an object pattern row as well.
+ Blank lines are also ignored. See <xref linkend="app-psql-patterns"/>
+ for how to perform quoting in patterns.
+ </para>
+
</listitem>
</varlistentry>
diff --git a/src/bin/pg_dump/filter.c b/src/bin/pg_dump/filter.c
index 9b5e45c722..b64822b6b7 100644
--- a/src/bin/pg_dump/filter.c
+++ b/src/bin/pg_dump/filter.c
@@ -23,15 +23,13 @@
/*
* Following routines are called from pg_dump, pg_dumpall and pg_restore.
- * Unfortunately, the implementation of exit_nicely in pg_dump and pg_restore
is
- * different from the one in pg_dumpall, so instead of calling exit_nicely we
- * have to return some error flag (in this case NULL), and exit_nicely will be
- * executed from caller's routine.
+ * Since the implementation of exit_nicely is application specific, each
+ * application need to pass a function pointer to the exit_nicely function to
+ * use for exiting on errors.
*/
/*
* Opens filter's file and initialize fstate structure.
- * Returns true on success.
*/
void
filter_init(FilterStateData *fstate, const char *filename, exit_function
f_exit)
@@ -76,8 +74,8 @@ filter_free(FilterStateData *fstate)
}
/*
- * Translate FilterObjectType enum to string. It is designed for formatting
- * of error message in log_unsupported_filter_object_type routine.
+ * Translate FilterObjectType enum to string. The main purpose is for error
+ * message formatting.
*/
const char *
filter_object_type_name(FilterObjectType fot)
@@ -181,7 +179,7 @@ filter_get_keyword(const char **line, int *size)
const char *ptr = *line;
const char *result = NULL;
- /* Set returnlength preemptively in case no keyword is found */
+ /* Set returned length preemptively in case no keyword is found */
*size = 0;
/* Skip initial whitespace */
@@ -382,8 +380,7 @@ read_pattern(FilterStateData *fstate, const char *str,
PQExpBuffer pattern)
* Returns true when one filter item was successfully read and parsed. When
* object name contains \n chars, then more than one line from input file can
* be processed. Returns false when the filter file reaches EOF. In case of
- * error, the function will emit an appropriate error message before returning
- * false.
+ * error, the function will emit an appropriate error message and exit.
*/
bool
filter_read_item(FilterStateData *fstate,
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1e7756beba..64e2d754d1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -436,6 +436,7 @@ main(int argc, char **argv)
{"exclude-table-data-and-children", required_argument, NULL,
14},
{"sync-method", required_argument, NULL, 15},
{"filter", required_argument, NULL, 16},
+
{NULL, 0, NULL, 0}
};
@@ -666,7 +667,7 @@ main(int argc, char **argv)
exit_nicely(1);
break;
- case 16: /* object filters from
file */
+ case 16: /* read object filters
from file */
read_dump_filters(optarg, &dopt);
break;
@@ -1117,8 +1118,8 @@ help(const char *progname)
" do NOT dump data for
the specified table(s),\n"
" including child and
partition tables\n"));
printf(_(" --extra-float-digits=NUM override default setting for
extra_float_digits\n"));
- printf(_(" --filter=FILENAME dump objects and data based on
the filter expressions\n"
- " in specified file\n"));
+ printf(_(" --filter=FILENAME include or exclude objects and
data from dump\n"
+ " based expressions in
FILENAME\n"));
printf(_(" --if-exists use IF EXISTS when dropping
objects\n"));
printf(_(" --include-foreign-data=PATTERN\n"
" include data of
foreign tables on foreign\n"
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index fe2a541c56..1b974cf7e8 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -160,7 +160,6 @@ main(int argc, char *argv[])
{"disable-triggers", no_argument, &disable_triggers, 1},
{"exclude-database", required_argument, NULL, 6},
{"extra-float-digits", required_argument, NULL, 5},
- {"filter", required_argument, NULL, 8},
{"if-exists", no_argument, &if_exists, 1},
{"inserts", no_argument, &inserts, 1},
{"lock-wait-timeout", required_argument, NULL, 2},
@@ -180,6 +179,7 @@ main(int argc, char *argv[])
{"no-unlogged-table-data", no_argument,
&no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument,
&on_conflict_do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 7},
+ {"filter", required_argument, NULL, 8},
{NULL, 0, NULL, 0}
};
@@ -660,7 +660,7 @@ help(void)
printf(_(" --disable-triggers disable triggers during
data-only restore\n"));
printf(_(" --exclude-database=PATTERN exclude databases whose name
matches PATTERN\n"));
printf(_(" --extra-float-digits=NUM override default setting for
extra_float_digits\n"));
- printf(_(" --filter=FILENAME exclude databases specified in
filter file\n"));
+ printf(_(" --filter=FILENAME exclude databases specified in
FILENAME\n"));
printf(_(" --if-exists use IF EXISTS when dropping
objects\n"));
printf(_(" --inserts dump data as INSERT commands,
rather than COPY\n"));
printf(_(" --load-via-partition-root load partitions via the root
table\n"));
@@ -1921,6 +1921,7 @@ executeCommand(PGconn *conn, const char *query)
PQclear(res);
}
+
/*
* dumpTimestamp
*/
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 0cca9ee612..1459e02263 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -470,7 +470,8 @@ usage(const char *progname)
printf(_(" -1, --single-transaction restore as a single
transaction\n"));
printf(_(" --disable-triggers disable triggers during
data-only restore\n"));
printf(_(" --enable-row-security enable row security\n"));
- printf(_(" --filter=FILE restore objects based on
filter expressions\n"));
+ printf(_(" --filter=FILENAME restore or skip objects based
on expressions\n"
+ " in FILENAME\n"));
printf(_(" --if-exists use IF EXISTS when dropping
objects\n"));
printf(_(" --no-comments do not restore comments\n"));
printf(_(" --no-data-for-failed-tables do not restore data of tables
that could not be\n"
diff --git a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
index 09d3262b8b..d4c2c2340d 100644
--- a/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
+++ b/src/bin/pg_dump/t/005_pg_dump_filterfile.pl
@@ -1,12 +1,12 @@
-# Copyright (c) 2021, PostgreSQL Global Development Group
+# Copyright (c) 2023, PostgreSQL Global Development Group
use strict;
use warnings;
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
-use Test::More tests => 106;
+use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;;
my $inputfile;
@@ -421,7 +421,7 @@ command_ok(
"--filter=$tempdir/inputfile.txt",
'--strict-names', 'postgres'
],
- "strict names with matching mattern");
+ "strict names with matching pattern");
$dump = slurp_file($plainfile);
@@ -546,7 +546,7 @@ command_fails_like(
"--filter=$tempdir/inputfile.txt"
],
qr/include filter for "table data" is not allowed/,
- "invalid syntax: inclusion of non allowed object"
+ "invalid syntax: inclusion of unallowed object"
);
open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -560,7 +560,7 @@ command_fails_like(
"--filter=$tempdir/inputfile.txt"
],
qr/include filter for "extension" is not allowed/,
- "invalid syntax: inclusion of non allowed object"
+ "invalid syntax: inclusion of unallowed object"
);
open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -574,7 +574,7 @@ command_fails_like(
"--filter=$tempdir/inputfile.txt"
],
qr/exclude filter for "extension" is not allowed/,
- "invalid syntax: exclusion of non allowed object"
+ "invalid syntax: exclusion of unallowed object"
);
open $inputfile, '>', "$tempdir/inputfile.txt"
@@ -587,8 +587,8 @@ command_fails_like(
'pg_restore', '-p', $port, '-f', $plainfile,
"--filter=$tempdir/inputfile.txt"
],
- qr/xclude filter for "table data" is not allowed/,
- "invalid syntax: exclusion of non allowed object"
+ qr/exclude filter for "table data" is not allowed/,
+ "invalid syntax: exclusion of unallowed object"
);
#########################################
@@ -771,3 +771,6 @@ command_fails_like(
],
qr/pg_dump: error: no matching extensions were found/,
"dump nonexisting extension");
+
+
+done_testing();
v20231121-0001-Read-include-exclude-commands-for-dump-res.patch
Description: Binary data
