pgsql: Raise a WARNING for missing publications.
Raise a WARNING for missing publications. When we create or alter a subscription to add publications raise a warning for non-existent publications. We don't want to give an error here because it is possible that users can later create the missing publications. Author: Vignesh C Reviewed-by: Bharath Rupireddy, Japin Li, Dilip Kumar, Euler Taveira, Ashutosh Sharma, Amit Kapila Discussion: https://postgr.es/m/CALDaNm0f4YujGW+q-Di0CbZpnQKFFrXntikaQQKuEmGG0=Zw=q...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8f2e2bbf145384784bad07a96d461c6bbd91f597 Modified Files -- doc/src/sgml/ref/alter_subscription.sgml | 4 +- doc/src/sgml/ref/create_subscription.sgml | 7 ++ src/backend/commands/subscriptioncmds.c | 133 +- src/test/subscription/t/007_ddl.pl| 37 + 4 files changed, 160 insertions(+), 21 deletions(-)
Re: pgsql: Make PostgreSQL::Test::Cluster compatible with all live branches
> On Mar 30, 2022, at 9:12 PM, Tom Lane wrote: > > Andrew Dunstan writes: On Mar 30, 2022, at 8:19 PM, Tom Lane wrote: >>> I think this means that its old Perl version misinterprets >>> use parent -norequire, qw(PostgreSQL::Test::Cluster); >>> as a request to include "parent.pm". Is there a more >>> backwards-compatible way to spell that? > >> Not that I know of. Googling tells me it was a core module from 5.10.1. I >> can revert tomorrow if necessary :-( > > Hmm. I checked not too long ago, and these are the buildfarm > animals that would be broken: > > prairiedog| 2022-03-23 05:04:02 | configure: using perl 5.8.3 > anole | 2022-03-24 10:40:08 | configure: using perl 5.8.8 > gharial | 2022-03-24 18:32:22 | configure: using perl 5.8.8 > locust| 2022-03-19 22:17:53 | configure: using perl 5.8.8 > gaur | 2022-03-19 15:00:54 | configure: using perl 5.8.9 > > I could update prairiedog and gaur (... probably), but hard to say > about the others. > > It seems kind of sad for a backwards-compatibility patch to force a > major move of our minimum Perl version. OTOH, if there's no other > good way to do it, maybe we gotta. > > Another possibility is that maybe parent.pm could be successfully > installed into a 5.8.x installation. I can't speak for those > other animals, but mine at least are already frankencritters. > > I think it can be salvaged. See https://fastapi.metacpan.org/source/CORION/parent-0.238/lib/parent.pm Cheers Andrew
pgsql: Clean up some dead code in pg_dump with tar format and gzip comp
Clean up some dead code in pg_dump with tar format and gzip compression Compression with gzip has never been supported in the tar format of pg_dump since this code has been introduced in c3e18804, as the use of buffered I/O in gzdopen() changes the file positioning that tar requires. The original idea behind the use of compression with the tar mode is to be able to include compressed data files (named %u.dat.gz) and blob files (blob_%u.dat.gz) in the tarball generated by the dump, with toc.dat, that tracks down if compression is used in the dump, always uncompressed. Note that this commit removes the dump part of the code as well as the restore part, removing any dependency to zlib in pg_backup_tar.c. There could be an argument behind keeping around the restore part, but this would require one to change the internals of a tarball previously dumped so as data and blob files are compressed with toc.dat itself changed to track down if compression is enabled. However, the argument about gzdopen() still holds in the read case with pg_restore. Removing this code simplifies future additions related to compression in pg_dump. Author: Georgios Kokolatos, Rachel Heaton Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8ac4c25a05d1e491a51d5390aaae405600e8e466 Modified Files -- src/bin/pg_dump/pg_backup_tar.c | 89 + 1 file changed, 11 insertions(+), 78 deletions(-)
Re: pgsql: Make PostgreSQL::Test::Cluster compatible with all live branches
Andrew Dunstan writes: >> On Mar 30, 2022, at 8:19 PM, Tom Lane wrote: >> I think this means that its old Perl version misinterprets >> use parent -norequire, qw(PostgreSQL::Test::Cluster); >> as a request to include "parent.pm". Is there a more >> backwards-compatible way to spell that? > Not that I know of. Googling tells me it was a core module from 5.10.1. I can > revert tomorrow if necessary :-( Hmm. I checked not too long ago, and these are the buildfarm animals that would be broken: prairiedog| 2022-03-23 05:04:02 | configure: using perl 5.8.3 anole | 2022-03-24 10:40:08 | configure: using perl 5.8.8 gharial | 2022-03-24 18:32:22 | configure: using perl 5.8.8 locust| 2022-03-19 22:17:53 | configure: using perl 5.8.8 gaur | 2022-03-19 15:00:54 | configure: using perl 5.8.9 I could update prairiedog and gaur (... probably), but hard to say about the others. It seems kind of sad for a backwards-compatibility patch to force a major move of our minimum Perl version. OTOH, if there's no other good way to do it, maybe we gotta. Another possibility is that maybe parent.pm could be successfully installed into a 5.8.x installation. I can't speak for those other animals, but mine at least are already frankencritters. regards, tom lane
Re: pgsql: Make PostgreSQL::Test::Cluster compatible with all live branches
> On Mar 30, 2022, at 8:19 PM, Tom Lane wrote: > > Andrew Dunstan writes: >> Make PostgreSQL::Test::Cluster compatible with all live branches > > prairiedog is unhappy [1], seemingly as a result of this patch: > > t/001_repl_statsCan't locate parent.pm in @INC (@INC contains: > ../../src/test/perl/ . /usr/local/perl5.8.3/lib/5.8.3/darwin-2level > /usr/local/perl5.8.3/lib/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/5.8.3 > /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level > /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level > /usr/local/perl5.8.3/lib/site_perl/5.8.3 > /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level > /usr/local/perl5.8.3/lib/site_perl/5.8.3 /usr/local/perl5.8.3/lib/site_perl . > /usr/local/perl5.8.3/lib/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/5.8.3 > /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level > /usr/local/perl5.8.3/lib/site_perl/5.8.3 /usr/local/perl5.8.3/lib/site_perl > .) at ../../src/test/perl//PostgreSQL/Test/Cluster.pm line 2921. > BEGIN failed--compilation aborted at > ../../src/test/perl//PostgreSQL/Test/Cluster.pm line 2921. > > I think this means that its old Perl version misinterprets > > use parent -norequire, qw(PostgreSQL::Test::Cluster); > > as a request to include "parent.pm". Is there a more > backwards-compatible way to spell that? Not that I know of. Googling tells me it was a core module from 5.10.1. I can revert tomorrow if necessary :-( Cheers Andrew
Re: pgsql: Make PostgreSQL::Test::Cluster compatible with all live branches
Andrew Dunstan writes: > Make PostgreSQL::Test::Cluster compatible with all live branches prairiedog is unhappy [1], seemingly as a result of this patch: t/001_repl_statsCan't locate parent.pm in @INC (@INC contains: ../../src/test/perl/ . /usr/local/perl5.8.3/lib/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/5.8.3 /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/site_perl/5.8.3 /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/site_perl/5.8.3 /usr/local/perl5.8.3/lib/site_perl . /usr/local/perl5.8.3/lib/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/5.8.3 /usr/local/perl5.8.3/lib/site_perl/5.8.3/darwin-2level /usr/local/perl5.8.3/lib/site_perl/5.8.3 /usr/local/perl5.8.3/lib/site_perl .) at ../../src/test/perl//PostgreSQL/Test/Cluster.pm line 2921. BEGIN failed--compilation aborted at ../../src/test/perl//PostgreSQL/Test/Cluster.pm line 2921. I think this means that its old Perl version misinterprets use parent -norequire, qw(PostgreSQL::Test::Cluster); as a request to include "parent.pm". Is there a more backwards-compatible way to spell that? regards, tom lane [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog=2022-03-30%2021%3A23%3A58
pgsql: Add .gitignore for basebackup_to_shell.
Add .gitignore for basebackup_to_shell. Nathan Bossart Discussion: https://postgr.es/m/20220330223531.GA134543@nathanxps13 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f8e0d900afed933d8f5d3f189fdb141924a8e518 Modified Files -- contrib/basebackup_to_shell/.gitignore | 4 1 file changed, 4 insertions(+)
pgsql: Optimize order of GROUP BY keys
Optimize order of GROUP BY keys When evaluating a query with a multi-column GROUP BY clause using sort, the cost may be heavily dependent on the order in which the keys are compared when building the groups. Grouping does not imply any ordering, so we're allowed to compare the keys in arbitrary order, and a Hash Agg leverages this. But for Group Agg, we simply compared keys in the order as specified in the query. This commit explores alternative ordering of the keys, trying to find a cheaper one. In principle, we might generate grouping paths for all permutations of the keys, and leave the rest to the optimizer. But that might get very expensive, so we try to pick only a couple interesting orderings based on both local and global information. When planning the grouping path, we explore statistics (number of distinct values, cost of the comparison function) for the keys and reorder them to minimize comparison costs. Intuitively, it may be better to perform more expensive comparisons (for complex data types etc.) last, because maybe the cheaper comparisons will be enough. Similarly, the higher the cardinality of a key, the lower the probability we’ll need to compare more keys. The patch generates and costs various orderings, picking the cheapest ones. The ordering of group keys may interact with other parts of the query, some of which may not be known while planning the grouping. E.g. there may be an explicit ORDER BY clause, or some other ordering-dependent operation, higher up in the query, and using the same ordering may allow using either incremental sort or even eliminate the sort entirely. The patch generates orderings and picks those minimizing the comparison cost (for various pathkeys), and then adds orderings that might be useful for operations higher up in the plan (ORDER BY, etc.). Finally, it always keeps the ordering specified in the query, on the assumption the user might have additional insights. This introduces a new GUC enable_group_by_reordering, so that the optimization may be disabled if needed. The original patch was proposed by Teodor Sigaev, and later improved and reworked by Dmitry Dolgov. Reviews by a number of people, including me, Andrey Lepikhov, Claudio Freire, Ibrar Ahmed and Zhihong Yu. Author: Dmitry Dolgov, Teodor Sigaev, Tomas Vondra Reviewed-by: Tomas Vondra, Andrey Lepikhov, Claudio Freire, Ibrar Ahmed, Zhihong Yu Discussion: https://postgr.es/m/7c79e6a5-8597-74e8-0671-1c39d124c9d6%40sigaev.ru Discussion: https://postgr.es/m/CA%2Bq6zcW_4o2NC0zutLkOJPsFt80megSpX_dVRo6GK9PC-Jx_Ag%40mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/db0d67db2401eb6238ccc04c6407a4fd4f985832 Modified Files -- contrib/postgres_fdw/expected/postgres_fdw.out| 15 +- doc/src/sgml/config.sgml | 14 + src/backend/optimizer/path/costsize.c | 366 +++- src/backend/optimizer/path/equivclass.c | 13 +- src/backend/optimizer/path/pathkeys.c | 569 +++ src/backend/optimizer/plan/planner.c | 649 +- src/backend/optimizer/util/pathnode.c | 2 +- src/backend/utils/adt/selfuncs.c | 38 +- src/backend/utils/misc/guc.c | 10 + src/backend/utils/misc/postgresql.conf.sample | 1 + src/include/nodes/nodes.h | 1 + src/include/nodes/pathnodes.h | 10 + src/include/optimizer/cost.h | 4 +- src/include/optimizer/paths.h | 7 + src/include/utils/selfuncs.h | 5 + src/test/regress/expected/aggregates.out | 244 +++- src/test/regress/expected/incremental_sort.out| 2 +- src/test/regress/expected/join.out| 51 +- src/test/regress/expected/partition_aggregate.out | 136 +++-- src/test/regress/expected/partition_join.out | 75 ++- src/test/regress/expected/sysviews.out| 3 +- src/test/regress/expected/union.out | 60 +- src/test/regress/sql/aggregates.sql | 99 src/test/regress/sql/incremental_sort.sql | 2 +- 24 files changed, 1882 insertions(+), 494 deletions(-)
Re: pgsql: Add function to pump IPC process until string match
> On 30 Mar 2022, at 18:09, Andres Freund wrote: >> Would adding back something like the (right now untested) below be what >> you're >> after? Looking at the perldoc I didn't see any other debugging aids we can >> emit other than the stream and type of error. > > I found showing the regex from before useful. Often there's many > pump_until()'s, and the regex makes it easier to figure out what errored > out. And it shows quoting problem. Since it only happens when there was a > failure, it's not like it'll bloat the log unduly. I'll add diag output on failures along the lines of the diff from earlier today but with the regex as well (which Alvarao also requested), but tomorrow morning once coffee has been consumed. -- Daniel Gustafsson https://vmware.com/
pgsql: SQL JSON functions
SQL JSON functions This Patch introduces three SQL standard JSON functions: JSON() (incorrectly mentioned in my commit message for f4fb45d15c) JSON_SCALAR() JSON_SERIALIZE() JSON() produces json values from text, bytea, json or jsonb values, and has facilitites for handling duplicate keys. JSON_SCALAR() produces a json value from any scalar sql value, including json and jsonb. JSON_SERIALIZE() produces text or bytea from input which containis or represents json or jsonb; For the most part these functions don't add any significant new capabilities, but they will be of use to users wanting standard compliant JSON handling. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886dea...@postgrespro.ru Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/606948b058dc16bce494270eea577011a602810e Modified Files -- doc/src/sgml/keywords/sql2016-02-reserved.txt | 3 + src/backend/executor/execExpr.c | 45 + src/backend/executor/execExprInterp.c | 42 +++- src/backend/nodes/copyfuncs.c | 53 + src/backend/nodes/equalfuncs.c| 38 src/backend/nodes/nodeFuncs.c | 14 ++ src/backend/parser/gram.y | 56 +- src/backend/parser/parse_expr.c | 152 +-- src/backend/parser/parse_target.c | 9 + src/backend/utils/adt/format_type.c | 4 + src/backend/utils/adt/json.c | 51 +++-- src/backend/utils/adt/jsonb.c | 64 +++--- src/backend/utils/adt/ruleutils.c | 13 +- src/include/executor/execExpr.h | 5 + src/include/nodes/nodes.h | 3 + src/include/nodes/parsenodes.h| 35 src/include/nodes/primnodes.h | 5 +- src/include/parser/kwlist.h | 4 +- src/include/utils/json.h | 21 +- src/include/utils/jsonb.h | 21 ++ src/test/regress/expected/sqljson.out | 267 ++ src/test/regress/sql/sqljson.sql | 57 ++ 22 files changed, 880 insertions(+), 82 deletions(-)
pgsql: Fix possible NULL-pointer-deference in backup_compression.c.
Fix possible NULL-pointer-deference in backup_compression.c. Per Coverity and Tom Lane. Reviewed by Tom Lane and Justin Pryzby. Discussion: http://postgr.es/m/384291.1648403...@sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/8e053dc6dfbee4ae412e98ad73cfd4662d7453ac Modified Files -- src/common/backup_compression.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-)
pgsql: basebackup_to_shell: Add TAP test.
basebackup_to_shell: Add TAP test. Per gripe from Andres Freund. Thanks to Andres Freund, Thomas Munro, and Andrew Dunstan for advice on how to make this test work even on Windows. Discussion: https://postgr.es/m/CA+Tgmoat+zbzzZQJ7poXyUwiqxQxTaUid=aub4fejz15vvd...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/027fa0fd72619a56d9d8877eedcb514331b63fa4 Modified Files -- contrib/basebackup_to_shell/Makefile | 5 ++ contrib/basebackup_to_shell/t/001_basic.pl | 129 + 2 files changed, 134 insertions(+)
pgsql: Document basebackup_to_shell.required_role.
Document basebackup_to_shell.required_role. Omission noted by Joe Conway. Discussion: https://postgr.es/m/CA+Tgmoat+zbzzZQJ7poXyUwiqxQxTaUid=aub4fejz15vvd...@mail.gmail.com Discussion: https://postgr.es/m/744cf762-47d3-050f-5fa1-d4f9e8dba...@joeconway.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/26a0c025e233c3d4333f071bde4ac970ba1ec643 Modified Files -- doc/src/sgml/basebackup-to-shell.sgml | 17 + 1 file changed, 17 insertions(+)
pgsql: Add range_agg with multirange inputs
Add range_agg with multirange inputs range_agg for normal ranges already existed. A lot of code can be shared. Author: Paul Jungwirth Reviewed-by: Chapman Flack Discussion: https://www.postgresql.org/message-id/flat/007ef255-35ef-fd26-679c-f97e7a7f3...@illuminatedcomputing.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7ae1619bc5b1794938c7387a766b8cae34e38d8a Modified Files -- doc/src/sgml/func.sgml| 5 +++ src/backend/utils/adt/multirangetypes.c | 61 +++ src/include/catalog/catversion.h | 2 +- src/include/catalog/pg_aggregate.dat | 3 ++ src/include/catalog/pg_proc.dat | 11 + src/test/regress/expected/multirangetypes.out | 61 +++ src/test/regress/expected/opr_sanity.out | 3 +- src/test/regress/sql/multirangetypes.sql | 12 ++ 8 files changed, 156 insertions(+), 2 deletions(-)
pgsql: ci: enable zstd where available.
ci: enable zstd where available. Since zstd is now used in a bunch of places, enable it in CI. The windows image unfortunately doesn't yet contain zstd, so it's not enabled there. Discussion: https://postgr.es/m/20220330155017.lfnlzt3m42nk7...@alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ff50baec65bba1a839322ba8bcb3efcd08f40621 Modified Files -- .cirrus.yml | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-)
Re: pgsql: Add function to pump IPC process until string match
Hi, On 2022-03-30 09:10:17 +0200, Daniel Gustafsson wrote: > > On 30 Mar 2022, at 00:58, Andres Freund wrote: > > On 2022-02-23 13:32:03 +, Daniel Gustafsson wrote: > > >> Add function to pump IPC process until string match > >> > >> Refactor the recovery tests to not carry a local duplicated copy of > >> the pump_until function which pumps a process until a defined string > >> is seen on a stream. This reduces duplication, and is in preparation > >> for another patch which will also use this functionality. > > > > I'm a bit disappointed by the moved function not having the diagnostic > > output > > that at least the version in 013_crash_restart.pl had. How is one supposed > > to > > figure out what caused a timeout with the new central version? > > Thats my bad then. Since we don't really have diag output in any module I was > going off that "precedent" when moving this over. We have a few places that manually output diagnostic stuff by printing with # etc. > > Given that timeouts are the only way tests using pump_until() fail, that's > > not > > great. > > They can also fail on the pumped process crashing/exiting before the timeout > without emitting the expected output. Fair. > Would adding back something like the (right now untested) below be what you're > after? Looking at the perldoc I didn't see any other debugging aids we can > emit other than the stream and type of error. I found showing the regex from before useful. Often there's many pump_until()'s, and the regex makes it easier to figure out what errored out. And it shows quoting problem. Since it only happens when there was a failure, it's not like it'll bloat the log unduly. Greetings, Andres Freund
pgsql: Change some internal error messages to elogs
Change some internal error messages to elogs Author: Paul Jungwirth Reviewed-by: Chapman Flack Discussion: https://www.postgresql.org/message-id/flat/007ef255-35ef-fd26-679c-f97e7a7f3...@illuminatedcomputing.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f453d684ecf9f6cd872d4bb43fe385c2c56126bd Modified Files -- src/backend/utils/adt/multirangetypes.c | 8 ++-- src/backend/utils/adt/rangetypes.c | 2 +- 2 files changed, 3 insertions(+), 7 deletions(-)
pgsql: Make PostgreSQL::Test::Cluster compatible with all live branches
Make PostgreSQL::Test::Cluster compatible with all live branches We do this via a subclass for any branch older than the minimum known to be compatible with the main package (currently release 12). This should be useful for constructing cross-version tests. In theory this could be extended back any number of versions, with varying degrees of compatibility. Reviewed by Michael Paquier and Dagfinn Ilmari Mannsåker Discussion: https://postgr.es/m/a3efd19a-d5c9-fdf2-6094-4cde056a2...@dunslane.net Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/fb16d2c6588446b00534d90958e6dc312ae52a2f Modified Files -- src/test/perl/PostgreSQL/Test/Cluster.pm | 79 1 file changed, 70 insertions(+), 9 deletions(-)
pgsql: Additional tests for range_intersect_agg(anymultirange)
Additional tests for range_intersect_agg(anymultirange) Author: Paul Jungwirth Reviewed-by: Chapman Flack Discussion: https://www.postgresql.org/message-id/flat/007ef255-35ef-fd26-679c-f97e7a7f3...@illuminatedcomputing.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/cd7ea75e4b1b0c44476bef4f00075b9a4b07733e Modified Files -- src/test/regress/expected/multirangetypes.out | 39 +++ src/test/regress/sql/multirangetypes.sql | 9 +++ 2 files changed, 48 insertions(+)
pgsql: doc: Document range_intersect_agg(anymultirange)
doc: Document range_intersect_agg(anymultirange) It already existed but was not mentioned in the documentation. (Only the anyrange variant was listed.) Author: Paul Jungwirth Reviewed-by: Chapman Flack Discussion: https://www.postgresql.org/message-id/flat/007ef255-35ef-fd26-679c-f97e7a7f3...@illuminatedcomputing.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b21c4cf95103fae63aeb534f8ee37fbf90a1f907 Modified Files -- doc/src/sgml/func.sgml | 5 + 1 file changed, 5 insertions(+)
pgsql: Allow parallel zstd compression when taking a base backup.
Allow parallel zstd compression when taking a base backup. libzstd allows transparent parallel compression just by setting an option when creating the compression context, so permit that for both client and server-side backup compression. To use this, use something like pg_basebackup --compress WHERE-zstd:workers=N where WHERE is "client" or "server" and N is an integer. When compression is performed on the server side, this will spawn threads inside the PostgreSQL backend. While there is almost no PostgreSQL server code which is thread-safe, the threads here are used internally by libzstd and touch only data structures controlled by libzstd. Patch by me, based in part on earlier work by Dipesh Pandit and Jeevan Ladhe. Reviewed by Justin Pryzby. Discussion: http://postgr.es/m/CA+Tgmobj6u-nWF-j=femyguhobhrylxf9h-wjn7w-2rssse...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/51c0d186d99a18e6aae53003f5138f20991e15a6 Modified Files -- doc/src/sgml/protocol.sgml| 12 +-- doc/src/sgml/ref/pg_basebackup.sgml | 4 +-- src/backend/replication/basebackup_zstd.c | 45 +-- src/bin/pg_basebackup/bbstreamer_zstd.c | 40 +--- src/bin/pg_basebackup/t/010_pg_basebackup.pl | 5 +++ src/bin/pg_verifybackup/t/009_extract.pl | 29 +++-- src/bin/pg_verifybackup/t/010_client_untar.pl | 33 ++-- src/common/backup_compression.c | 16 ++ src/include/common/backup_compression.h | 2 ++ 9 files changed, 147 insertions(+), 39 deletions(-)
pgsql: Simplify a needlessly-complicated regular expression.
Simplify a needlessly-complicated regular expression. Dilip Kumar Discussion: https://postgr.es/m/cafitn-uv_u1lgbn_caigyfgpxp+bfbuvqg5mz24nqc8e_yb...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/c6863b85829149e2241faafa161b6c5af1f06cb9 Modified Files -- src/bin/scripts/t/020_createdb.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: doc: Clarify when SSL actually means TLS
doc: Clarify when SSL actually means TLS SSL has become the de facto term to mean an end-to-end encrypted channel regardless of protocol used, even though the SSL protocol is deprecated. Clarify what we mean with SSL in our documentation, especially for new users who might be looking for TLS. Reviewed-by: Robert Haas Discussion: https://postgr.es/m/d4abb281-6cfd-46c6-a4e0-8ec23a297...@yesql.se Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/860ea46ba7be69c46c37a96983e1ddca9d630c2e Modified Files -- doc/src/sgml/config.sgml | 8 +++- doc/src/sgml/libpq.sgml | 6 -- doc/src/sgml/runtime.sgml | 15 ++- 3 files changed, 25 insertions(+), 4 deletions(-)
Re: pgsql: Add function to pump IPC process until string match
On 2022-Mar-30, Daniel Gustafsson wrote: > Would adding back something like the (right now untested) below be what you're > after? Looking at the perldoc I didn't see any other debugging aids we can > emit other than the stream and type of error. I think you could also show the $until that wasn't found. The old .pl versions of this did that. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/81fd8085de0974e0d4ed932d8dd861a159eb279d Modified Files -- src/backend/commands/copy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/166f143869c98492a41288bf8175521cd9ce7c24 Modified Files -- src/backend/commands/copy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/f042dc6a415decc4e2a9707de5f3fc737e42e1ed Modified Files -- src/backend/commands/copy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f505bec711f602c6bd08a88e8ad894b611e7e8a1 Modified Files -- src/backend/commands/copyfrom.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/0598bed99efa6e2d5fa43bbccbcca0358c6e1e75 Modified Files -- src/backend/commands/copy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Fix typo in comment.
Fix typo in comment. Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/637afee327f4690afc3f16fd76123cd4b7af1d95 Modified Files -- src/backend/commands/copyfrom.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Re: pgsql: Add function to pump IPC process until string match
> On 30 Mar 2022, at 00:58, Andres Freund wrote:
> On 2022-02-23 13:32:03 +, Daniel Gustafsson wrote:
>> Add function to pump IPC process until string match
>>
>> Refactor the recovery tests to not carry a local duplicated copy of
>> the pump_until function which pumps a process until a defined string
>> is seen on a stream. This reduces duplication, and is in preparation
>> for another patch which will also use this functionality.
>
> I'm a bit disappointed by the moved function not having the diagnostic output
> that at least the version in 013_crash_restart.pl had. How is one supposed to
> figure out what caused a timeout with the new central version?
Thats my bad then. Since we don't really have diag output in any module I was
going off that "precedent" when moving this over.
> Given that timeouts are the only way tests using pump_until() fail, that's not
> great.
They can also fail on the pumped process crashing/exiting before the timeout
without emitting the expected output.
Would adding back something like the (right now untested) below be what you're
after? Looking at the perldoc I didn't see any other debugging aids we can
emit other than the stream and type of error.
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm
b/src/test/perl/PostgreSQL/Test/Utils.pm
index 15b314d1f8..693f2f02e5 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -426,8 +426,16 @@ sub pump_until
while (1)
{
last if $$stream =~ /$until/;
- return 0 if ($timeout->is_expired);
- return 0 if (not $proc->pumpable());
+ if ($timeout->is_expired)
+ {
+ diag("pump_until: timeout expired with stream:
\"$$stream\"");
+ return 0;
+ }
+ if (not $proc->pumpable())
+ {
+ diag("pump_until: process terminated unexpectedly with
stream: \"$$stream\"");
+ return 0
+ }
$proc->pump();
}
return 1;
--
Daniel Gustafsson https://vmware.com/
pgsql: Add header matching mode to COPY FROM
Add header matching mode to COPY FROM COPY FROM supports the HEADER option to silently discard the header line from a CSV or text file. It is possible to load by mistake a file that matches the expected format, for example, if two text columns have been swapped, resulting in garbage in the database. This adds a new option value HEADER MATCH that checks the column names in the header line against the actual column names and errors out if they do not match. Author: Rémi Lapeyre Reviewed-by: Daniel Verite Reviewed-by: Peter Eisentraut Discussion: https://www.postgresql.org/message-id/flat/caf1-j-0ptcwmeltswwgv2m70u26n4g33gpe1rckqqe6wvqd...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/072132f04e55c1c3b0f1a582318da78de7334379 Modified Files -- contrib/file_fdw/expected/file_fdw.out | 19 ++- contrib/file_fdw/sql/file_fdw.sql | 9 + doc/src/sgml/ref/copy.sgml | 8 +++-- src/backend/commands/copy.c| 60 +- src/backend/commands/copyfromparse.c | 53 -- src/include/commands/copy.h| 13 +++- src/test/regress/expected/copy.out | 23 + src/test/regress/sql/copy.sql | 33 +++ 8 files changed, 210 insertions(+), 8 deletions(-)
