pgsql: Support adding partitioned tables to publication
Support adding partitioned tables to publication When a partitioned table is added to a publication, changes of all of its partitions (current or future) are published via that publication. This change only affects which tables a publication considers as its members. The receiving side still sees the data coming from the individual leaf partitions. So existing restrictions that partition hierarchies can only be replicated one-to-one are not changed by this. Author: Amit Langote Reviewed-by: Rafia Sabih Reviewed-by: Peter Eisentraut Discussion: https://www.postgresql.org/message-id/flat/CA+HiwqH=Y85vRK3mOdjEkqFK+E=ST=eQiHdpj43L=_ejmoo...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/17b9e7f9fe238eeb5f6b40061b444ebf28d9e06f Modified Files -- doc/src/sgml/logical-replication.sgml | 17 +-- doc/src/sgml/ref/create_publication.sgml| 19 ++- src/backend/catalog/pg_publication.c| 85 ++--- src/backend/commands/publicationcmds.c | 23 +++- src/backend/replication/logical/tablesync.c | 1 + src/backend/replication/pgoutput/pgoutput.c | 19 ++- src/bin/pg_dump/pg_dump.c | 8 +- src/include/catalog/pg_publication.h| 19 ++- src/test/regress/expected/publication.out | 34 +- src/test/regress/sql/publication.sql| 23 +++- src/test/subscription/t/013_partition.pl| 178 11 files changed, 382 insertions(+), 44 deletions(-)
pgsql: Remove utils/acl.h from catalog/objectaddress.h
Remove utils/acl.h from catalog/objectaddress.h The need for this was removed by 8b9e9644dc6a9bd4b7a97950e6212f63880cf18b. A number of files now need to include utils/acl.h or parser/parse_node.h explicitly where they previously got it indirectly somehow. Since parser/parse_node.h already includes nodes/parsenodes.h, the latter is then removed where the former was added. Also, remove nodes/pg_list.h from objectaddress.h, since that's included via nodes/parsenodes.h. Reviewed-by: Tom Lane Reviewed-by: Alvaro Herrera Discussion: https://www.postgresql.org/message-id/flat/7601e258-26b2-8481-36d0-dc9dca6f28f1%402ndquadrant.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3c173a53a825075f3efe32b9917eff5063e81f4d Modified Files -- contrib/adminpack/adminpack.c | 1 + contrib/file_fdw/file_fdw.c| 1 + src/backend/access/brin/brin.c | 1 + src/backend/catalog/dependency.c | 1 + src/backend/catalog/objectaddress.c| 1 + src/backend/commands/collationcmds.c | 1 + src/backend/commands/conversioncmds.c | 1 + src/backend/commands/copy.c| 1 + src/backend/commands/dropcmds.c| 1 + src/backend/commands/operatorcmds.c| 1 + src/backend/commands/publicationcmds.c | 1 + src/backend/commands/subscriptioncmds.c| 1 + src/backend/executor/execExpr.c| 1 + src/backend/executor/execPartition.c | 1 + src/backend/replication/walreceiver.c | 1 + src/backend/replication/walsender.c| 1 + src/backend/statistics/extended_stats.c| 1 + src/backend/storage/ipc/procarray.c| 1 + src/backend/storage/large_object/inv_api.c | 1 + src/backend/utils/adt/genfile.c| 1 + src/backend/utils/adt/selfuncs.c | 1 + src/backend/utils/misc/guc.c | 1 + src/include/catalog/objectaddress.h| 4 ++-- src/include/catalog/pg_namespace.h | 1 + src/include/commands/collationcmds.h | 2 +- src/include/commands/createas.h| 2 +- src/include/commands/dbcommands.h | 2 +- src/include/commands/defrem.h | 2 +- src/include/commands/extension.h | 2 +- src/include/commands/vacuum.h | 2 +- src/pl/tcl/pltcl.c | 1 + 31 files changed, 32 insertions(+), 8 deletions(-)
pgsql: Remove win32ver.rc from version_stamp.pl
Remove win32ver.rc from version_stamp.pl This removes another relic from the old nmake-based Windows build. version_stamp.pl put version number information into win32ver.rc. But win32ver.rc already gets other version number information from the preprocessor at build time, so it would make more sense if all version number information would be handled in the same way and we don't have two places that do it. What we need for this is having the major version number and the minor version number as separate integer symbols. Both configure and Solution.pm already have that logic, because they compute PG_VERSION_NUM. So we just keep all the logic there now. Put the minor version number into a new symbol PG_MINORVERSION_NUM. Also, add a symbol PG_MAJORVERSION_NUM, which is a number, alongside the existing PG_MAJORVERSION, which is a string. Reviewed-by: Tom Lane Discussion: https://www.postgresql.org/message-id/flat/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0a42a2e9ce8481a024d085f2cc526a366db8df59 Modified Files -- configure | 15 +-- configure.in | 7 +-- src/include/pg_config.h.in | 6 ++ src/port/win32ver.rc | 4 ++-- src/tools/msvc/Solution.pm | 16 +--- src/tools/version_stamp.pl | 14 +- 6 files changed, 36 insertions(+), 26 deletions(-)
Re: pgsql: Remove win32ver.rc from version_stamp.pl
On 2020-03-10 11:22, Peter Eisentraut wrote: What we need for this is having the major version number and the minor version number as separate integer symbols. Both configure and Solution.pm already have that logic, because they compute PG_VERSION_NUM. So we just keep all the logic there now. Put the minor version number into a new symbol PG_MINORVERSION_NUM. Also, add a symbol PG_MAJORVERSION_NUM, which is a number, alongside the existing PG_MAJORVERSION, which is a string. This created some seemingly unrelated build farm failures. I suspect this must have something to do with the new shell code in configure +PG_MINORVERSION=`expr "$PACKAGE_VERSION" : '.*\.\([0-9][0-9]*\)'` +test -n "$PG_MINORVERSION" || PG_MINORVERSION=0 ending up with some nonsense being assigned to PG_MINORVERSION, perhaps due to a not-quite-mainstream Bourne shell being used. Can anyone test this locally? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: pgsql: Remove win32ver.rc from version_stamp.pl
Peter Eisentraut writes: > This created some seemingly unrelated build farm failures. I suspect > this must have something to do with the new shell code in configure > +PG_MINORVERSION=`expr "$PACKAGE_VERSION" : '.*\.\([0-9][0-9]*\)'` > +test -n "$PG_MINORVERSION" || PG_MINORVERSION=0 > ending up with some nonsense being assigned to PG_MINORVERSION, perhaps > due to a not-quite-mainstream Bourne shell being used. Looking at the configure log on gharial, the only incorrect symbol seems to be PG_VERSION_NUM: #define PG_MAJORVERSION "13" #define PG_MAJORVERSION_NUM 13 #define PG_MINORVERSION_NUM 0 #define PG_VERSION "13devel" #define PG_VERSION_STR "PostgreSQL 13devel on ia64-hp-hpux11.31, compiled by gcc (GCC) 4.6.0, 64-bit" #define PG_VERSION_NUM which squares with the compile error that's being thrown (it's pointing at a macro that uses PG_VERSION_NUM). I'd suggest reverting the change in how PG_VERSION_NUM is computed in configure.in --- that was certainly not necessary to the patch, and evidently you're falling foul of some weird behavior around nesting double-quotes and backquotes. regards, tom lane
Re: pgsql: Remove win32ver.rc from version_stamp.pl
I wrote:
> I'd suggest reverting the change in how PG_VERSION_NUM is computed
> in configure.in --- that was certainly not necessary to the patch,
> and evidently you're falling foul of some weird behavior around
> nesting double-quotes and backquotes.
Actually, you can just get rid of the nesting, because there's no
need to be picky about what white-space gets fed to awk. The
attached seems to be enough to fix it (tested on gaur).
regards, tom lane
diff --git a/configure b/configure
index d6d3f26..1a0aca9 100755
--- a/configure
+++ b/configure
@@ -18887,7 +18887,7 @@ _ACEOF
# Supply a numeric version string for use by 3rd party add-ons
# awk -F is a regex on some platforms, and not on others, so make "." a tab
-PG_VERSION_NUM="`echo "$PG_MAJORVERSION $PG_MINORVERSION" |
+PG_VERSION_NUM="`echo $PG_MAJORVERSION $PG_MINORVERSION |
$AWK '{printf "%d%04d", $1, $2}'`"
cat >>confdefs.h <<_ACEOF
diff --git a/configure.in b/configure.in
index 78902fb..b27708e 100644
--- a/configure.in
+++ b/configure.in
@@ -2322,7 +2322,7 @@ AC_DEFINE_UNQUOTED(PG_VERSION_STR,
# Supply a numeric version string for use by 3rd party add-ons
# awk -F is a regex on some platforms, and not on others, so make "." a tab
-[PG_VERSION_NUM="`echo "$PG_MAJORVERSION $PG_MINORVERSION" |
+[PG_VERSION_NUM="`echo $PG_MAJORVERSION $PG_MINORVERSION |
$AWK '{printf "%d%04d", $1, $2}'`"]
AC_DEFINE_UNQUOTED(PG_VERSION_NUM, $PG_VERSION_NUM, [PostgreSQL version as a number])
AC_SUBST(PG_VERSION_NUM)
pgsql: Split out CreateCast into src/backend/catalog/pg_cast.c
Split out CreateCast into src/backend/catalog/pg_cast.c This catalog-handling code was previously together with the rest of CastCreate() in src/backend/commands/functioncmds.c. A future patch will need a way to add casts internally, so this will be useful to have separate. Also, move the nearby get_cast_oid() function from functioncmds.c to lsyscache.c, which seems a more natural place for it. Author: Paul Jungwirth, minor edits by Álvaro Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/40b3e2c201af10c27a2c4c5bfcd029026b4cdff3 Modified Files -- src/backend/catalog/Makefile| 1 + src/backend/catalog/pg_cast.c | 123 src/backend/commands/functioncmds.c | 100 + src/backend/commands/typecmds.c | 1 + src/backend/utils/cache/lsyscache.c | 26 src/include/catalog/pg_cast.h | 9 +++ src/include/commands/defrem.h | 1 - src/include/utils/lsyscache.h | 1 + 8 files changed, 164 insertions(+), 98 deletions(-)
pgsql: Preserve integer and float values accurately in (de)serialize_de
Preserve integer and float values accurately in (de)serialize_deflist. Previously, this code just smashed all types of DefElem values to strings, cavalierly reasoning that nobody would care. But in point of fact, most of the defGetFoo functions do distinguish among different input syntaxes; for instance defGetBoolean will accept 1 as an integer but not "1" as a string. This led to CREATE/ALTER TEXT SEARCH DICTIONARY accepting 0 and 1 as values for boolean dictionary properties, only to have the dictionary fail at runtime. We can upgrade this behavior by teaching serialize_deflist that it does not need to quote T_Integer or T_Float nodes' values on output, and then teaching deserialize_deflist to restore unquoted integer or float values as the appropriate node type. This should not break anything using pg_ts_dict.dictinitoption, since that field is just defined as being something valid to include in CREATE TEXT SEARCH DICTIONARY. deserialize_deflist is also used to parse the options arguments for the ts_headline family of functions, but so far as I can see this won't cause any problems there either: the only consumer of that output is prsd_headline which always uses defGetString. (Really that's a bad idea, but I won't risk changing it here.) This is surely a bug fix, but given the lack of field complaints I don't think it's necessary to back-patch. Discussion: https://postgr.es/m/CAMkU=1xrcs_bupzr0+v3wndacav0e_m3h6auej8nf-sy1nn...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d01f03a495294f726001a90ec27a675428d9fc45 Modified Files -- contrib/dict_int/expected/dict_int.out | 29 ++- contrib/dict_int/sql/dict_int.sql | 10 +++- src/backend/commands/tsearchcmds.c | 90 ++ src/test/regress/expected/tsdicts.out | 35 + src/test/regress/sql/tsdicts.sql | 13 + 5 files changed, 154 insertions(+), 23 deletions(-)
pgsql: Avoid portability problem introduced in 0a42a2e9c.
Avoid portability problem introduced in 0a42a2e9c. Apparently not all shells handle nested quotes quite the same. But we don't need the inner double quotes in this case, so just drop them. Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/dbf05a14399430751e54cd0b4bd8fb0e4f1fb309 Modified Files -- configure| 2 +- configure.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
pgsql: Ensure that CREATE TABLE LIKE copies any NO INHERIT constraint p
Ensure that CREATE TABLE LIKE copies any NO INHERIT constraint property. Since the documentation about LIKE doesn't say that a copied constraint has properties different from the original, it seems that ignoring a NO INHERIT property doesn't meet the principle of least surprise. So make it copy that. (Note, however, that we still don't copy a NOT VALID property; CREATE TABLE offers no way to do that, plus it seems pointless.) Arguably this is a bug fix; but no back-patch, as it seems barely possible somebody is depending on the current behavior. Ildar Musin and Chris Travers; reviewed by Amit Langote and myself Discussion: https://postgr.es/m/CAONYFtMC6C+3AWCVp7Yd8H87Zn0GxG1_iQG6_bQKbaqYZY0=-g...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/cacef172237fd3426b578f81b7414c0de56cbaaf Modified Files -- src/backend/parser/parse_utilcmd.c | 11 +++ src/test/regress/expected/create_table_like.out | 16 src/test/regress/sql/create_table_like.sql | 11 +++ 3 files changed, 34 insertions(+), 4 deletions(-)
pgsql: nbtree: Demote minus infinity "can't happen" error.
nbtree: Demote minus infinity "can't happen" error. Only a very basic logic bug in a _bt_insertonpg() caller could lead to a violation of this invariant. Besides, any newitemoff used for an internal page is sanitized using other "can't happen" errors in _bt_getstackbuf() or its callers, before _bt_insertonpg() even gets called. Also, move the error/assertion from the insert-without-split path of _bt_insertonpg() to the top of the same function. There is no reason why this invariant only applies to insertions that happen to not result in a page split; cover every insertion. The assertion naturally belongs next to the existing generic assertions that document relatively high-level invariants for the item being inserted. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d1e241c22602a5369b54e29b7430553445f6a52b Modified Files -- src/backend/access/nbtree/nbtinsert.c | 20 1 file changed, 8 insertions(+), 12 deletions(-)
pgsql: Marginal comments and docs cleanup.
Marginal comments and docs cleanup. Fix up some imprecise comments and poor markup from ba79cb5dc. Also try to convert the documentation of log_min_duration_sample and friends into passable English. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c8e8b2f9dfee21fa4ef1ec7da5c10e6ef706df36 Modified Files -- doc/src/sgml/config.sgml | 97 src/backend/nodes/params.c | 11 ++-- src/backend/utils/mb/stringinfo_mb.c | 2 +- 3 files changed, 49 insertions(+), 61 deletions(-)
pgsql: nbtree: Move fastpath NULL descent stack assertion.
nbtree: Move fastpath NULL descent stack assertion. Commit 074251db added an assertion that verified the fastpath/rightmost page insert optimization's assumption about free space: There should always be enough free space on the page to insert the new item without splitting the page. Otherwise, we end up using the "concurrent root page split" phony/fake stack path in _bt_insert_parent(). This does not lead to incorrect behavior, but it is likely to be far slower than simply using the regular _bt_search() path. The assertion catches serious performance bugs that would probably take a long time to detect any other way. It seems much more natural to make this assertion just before the point that we generate a fake/phony descent stack. Move the assert there. This also makes _bt_insertonpg() a bit more readable. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/39eabec90451d8badbba9b3e938d6d05432a0517 Modified Files -- src/backend/access/nbtree/nbtinsert.c | 46 +++ 1 file changed, 20 insertions(+), 26 deletions(-)
pgsql: Remove stray parenthesis in nbtree.h.
Remove stray parenthesis in nbtree.h. Oversight in commit 0d861bbb702. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a88a285c7e343502b55204a211818d12b1b7d6ff Modified Files -- src/include/access/nbtree.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
