pgsql: Fix old-fd issues using global barriers everywhere.
Fix old-fd issues using global barriers everywhere. Commits 4eb21763 and b74e94dc introduced a way to force every backend to close all relation files, to fix an ancient Windows-only bug. This commit extends that behavior to all operating systems and adds a couple of extra barrier points, to fix a totally different class of bug: the reuse of relfilenodes in scenarios that have no other kind of cache invalidation to prevent file descriptor mix-ups. In all releases, data corruption could occur when you moved a database to another tablespace and then back again. Despite that, no back-patch for now as the infrastructure required is too new and invasive. In master only, since commit aa010514, it could also happen when using CREATE DATABASE with a user-supplied OID or via pg_upgrade. Author: Andres Freund Reviewed-by: Robert Haas Reviewed-by: Thomas Munro Discussion: https://postgr.es/m/20220209220004.kb3dgtn2x2k2gtdm%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e2f65f42555ff531c6d7c8f151526b4ef7c016f8 Modified Files -- src/backend/commands/dbcommands.c| 9 +- src/backend/commands/tablespace.c| 11 +- src/include/pg_config_manual.h | 11 -- src/test/recovery/Makefile | 2 +- src/test/recovery/t/032_relfilenode_reuse.pl | 233 +++ 5 files changed, 241 insertions(+), 25 deletions(-)
pgsql: Rethink PROCSIGNAL_BARRIER_SMGRRELEASE.
Rethink PROCSIGNAL_BARRIER_SMGRRELEASE. With sufficiently bad luck, it was possible for IssuePendingWritebacks() to reopen a file after we'd processed PROCSIGNAL_BARRIER_SMGRRELEASE and before the file was unlinked by some other backend. That left a small hole in commit 4eb21763's plan to fix all spurious errors from DROP TABLESPACE and similar on Windows. Fix by closing md.c's segments, instead of just closing fd.c's descriptors, and then teaching smgrwriteback() not to open files that aren't already open. Reported-by: Andres Freund Reviewed-by: Robert Haas Discussion: https://postgr.es/m/20220209220004.kb3dgtn2x2k2gtdm%40alap3.anarazel.de Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b74e94dc27fdbb13954f230b1d1298430afa6c0c Modified Files -- src/backend/storage/smgr/md.c | 22 +++- src/backend/storage/smgr/smgr.c | 45 + src/include/storage/md.h| 1 - src/include/storage/smgr.h | 2 ++ 4 files changed, 52 insertions(+), 18 deletions(-)
Re: pgsql: Remove command checks in tests of pg_basebackup and pg_receivewa
On Fri, May 06, 2022 at 10:12:36AM -0400, Andrew Dunstan wrote: > On 2022-05-02 Mo 03:06, Michael Paquier wrote: > vcpkg doesn't install any.exe files AFAICS, just DLLs, headers, .lib > files etc. Okay, thanks for confirming. >>> I don't think we should do that check for every time we call >>> vc_regress.pl, that seems wasteful. Maybe do it if the command is one >>> that might require these commands, which I think would be bincheck or >>> taptest. And/Or stash some status somewhere? >> >> Mostly. We also care about gzip in contrib/basebackup_to_shell/, >> though contribcheck does not run the TAP tests of contrib/ modules. >> As you say, setting up those variables in bincheck and taptest would >> be enough. > > That would be a start. Maybe meson will us get rid of some of this mess ... Let's do that for now then as we need a middle ground for HEAD. I'll come up with something at the beginning of next week. -- Michael signature.asc Description: PGP signature
pgsql: Temporarily skip recovery deadlock test in back branches.
Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylw...@alap3.anarazel.de Backpatch: 10-14 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/f40d362a667f6cf36f17d7565e4d7565fb7910f3 Modified Files -- src/test/recovery/t/031_recovery_conflict.pl | 5 + 1 file changed, 5 insertions(+)
pgsql: Temporarily skip recovery deadlock test in back branches.
Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylw...@alap3.anarazel.de Backpatch: 10-14 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/26f9e21cb63e0737f6abcc106e31313ae94d3b8b Modified Files -- src/test/recovery/t/031_recovery_conflict.pl | 5 + 1 file changed, 5 insertions(+)
pgsql: Temporarily skip recovery deadlock test in back branches.
Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylw...@alap3.anarazel.de Backpatch: 10-14 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/4e39957e6eb52887c2d631e9edf08a07c306ec9f Modified Files -- src/test/recovery/t/031_recovery_conflict.pl | 5 + 1 file changed, 5 insertions(+)
pgsql: Temporarily skip recovery deadlock test in back branches.
Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylw...@alap3.anarazel.de Backpatch: 10-14 Branch -- REL_11_STABLE Details --- https://git.postgresql.org/pg/commitdiff/fb965ab0f64198c7b97fd83e125860a0b4b944d7 Modified Files -- src/test/recovery/t/031_recovery_conflict.pl | 5 + 1 file changed, 5 insertions(+)
pgsql: Temporarily skip recovery deadlock test in back branches.
Temporarily skip recovery deadlock test in back branches. The recovery deadlock test has a timing issue that was fixed in 5136967f1eb in HEAD. Unfortunately the same fix doesn't quite work in the back branches: 1) adjust_conf() doesn't exist, which is easy enough to work around 2) a restart cleares the recovery conflict stats < 15. These issues can be worked around, but given the upcoming set of minor releases, skip the problematic test for now. The buildfarm doesn't show failures in other parts of 031_recovery_conflict.pl. Discussion: https://postgr.es/m/20220506155827.dfnaheq6ufylw...@alap3.anarazel.de Backpatch: 10-14 Branch -- REL_10_STABLE Details --- https://git.postgresql.org/pg/commitdiff/441fa7a63ad4ed9377e08c6295b9f94f16a4e61e Modified Files -- src/test/recovery/t/031_recovery_conflict.pl | 5 + 1 file changed, 5 insertions(+)
Re: pgsql: Backpatch addition of wait_for_log(), pump_until().
Hi, On 2022-05-06 15:29:43 +0200, Peter Eisentraut wrote: > On 03.05.22 03:11, Andres Freund wrote: > > Backpatch addition of wait_for_log(), pump_until(). > > > > These were originally introduced in a2ab9c06ea1 and a2ab9c06ea1, as they are > > needed by a about-to-be-backpatched test. > > I see this warning: > > Subroutine pump_until redefined at t/013_crash_restart.pl line 246. Yea :(. See bottom of https://postgr.es/m/20220506030927.6qehktk6bodv4ft3%40alap3.anarazel.de > Also, the local definition has different arguments than the backpatched one. > This should be double-checked. That's as it's in the "original" commit... Greetings, Andres Freund
pgsql: Backpatch addition of pump_until() more completely.
Backpatch addition of pump_until() more completely. In a2ab9c06ea1 I just backpatched the introduction of pump_until(), without changing the existing local definitions (as 6da65a3f9a9). The necessary changes seemed more verbose than desirable. However, that leads to warnings, as I failed to realize... Backpatch to all versions containing pump_until() calls before f74496dd611 (there's none in 10). Discussion: https://postgr.es/m/2808491.1651802...@sss.pgh.pa.us Discussion: https://postgr.es/m/18b37361-b482-b9d8-f30d-6115cd5ce...@enterprisedb.com Backpatch: 11-14 Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/7fa95bb0ac8e98799a98a08006a11feefcb614a6 Modified Files -- src/test/recovery/t/013_crash_restart.pl| 46 +++-- src/test/recovery/t/022_crash_temp_files.pl | 45 +--- 2 files changed, 18 insertions(+), 73 deletions(-)
pgsql: Backpatch addition of pump_until() more completely.
Backpatch addition of pump_until() more completely. In a2ab9c06ea1 I just backpatched the introduction of pump_until(), without changing the existing local definitions (as 6da65a3f9a9). The necessary changes seemed more verbose than desirable. However, that leads to warnings, as I failed to realize... Backpatch to all versions containing pump_until() calls before f74496dd611 (there's none in 10). Discussion: https://postgr.es/m/2808491.1651802...@sss.pgh.pa.us Discussion: https://postgr.es/m/18b37361-b482-b9d8-f30d-6115cd5ce...@enterprisedb.com Backpatch: 11-14 Branch -- REL_12_STABLE Details --- https://git.postgresql.org/pg/commitdiff/fbf659bc40409be03758d9fe2535c82895dd5d45 Modified Files -- src/test/recovery/t/013_crash_restart.pl | 46 +++- 1 file changed, 10 insertions(+), 36 deletions(-)
pgsql: Backpatch addition of pump_until() more completely.
Backpatch addition of pump_until() more completely. In a2ab9c06ea1 I just backpatched the introduction of pump_until(), without changing the existing local definitions (as 6da65a3f9a9). The necessary changes seemed more verbose than desirable. However, that leads to warnings, as I failed to realize... Backpatch to all versions containing pump_until() calls before f74496dd611 (there's none in 10). Discussion: https://postgr.es/m/2808491.1651802...@sss.pgh.pa.us Discussion: https://postgr.es/m/18b37361-b482-b9d8-f30d-6115cd5ce...@enterprisedb.com Backpatch: 11-14 Branch -- REL_13_STABLE Details --- https://git.postgresql.org/pg/commitdiff/6ab90d215afcec4c8d6583cb359ee0c1c1530a20 Modified Files -- src/test/recovery/t/013_crash_restart.pl | 46 +++- 1 file changed, 10 insertions(+), 36 deletions(-)
gcc 12.1.0 warning
Hi, Not sure if these compiler-mutterings are worth reporting but I guess we're trying to get a silent compile. System: Debian 4.9.303-1 (2022-03-07) x86_64 GNU/Linux Compiling with gcc 12.1.0 causes the below 'warning' and 'note'. Compiling with --enable-cassert --enable-debug is silent, no warnings) In function ‘guc_var_compare’, inlined from ‘bsearch’ at /usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h:33:23, inlined from ‘find_option’ at guc.c:5649:35: guc.c:5736:38: warning: array subscript ‘const struct config_generic[0]’ is partly outside array bounds of ‘const char[8]’ [-Warray-bounds] 5736 | return guc_name_compare(confa->name, confb->name); | ~^~ guc.c: In function ‘find_option’: guc.c:5636:25: note: object ‘name’ of size 8 5636 | find_option(const char *name, bool create_placeholders, bool skip_errors, | ^~~~ (Compiling with gcc 6.3.0 does not complain.) Below are the two configure lines, FWIW. Erik Rijkers # cassert-build: no warning/note ./configure \ --prefix=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD \ --bindir=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD/bin \ --libdir=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD/lib \ --with-pgport=6515 --quiet --enable-depend \ --enable-cassert --enable-debug \ --with-openssl --with-perl --with-libxml --with-libxslt --with-zlib \ --enable-tap-tests --with-extra-version=_0506_HEAD_701d --with-lz4 # normal build: causes warning/note: ./configure \ --prefix=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD \ --bindir=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD/bin.fast \ --libdir=/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD/lib.fast \ --with-pgport=6515 --quiet --enable-depend \ --with-openssl --with-perl --with-libxml --with-libxslt --with-zlib \ --enable-tap-tests --with-extra-version=_0506_HEAD_701d --with-lz4
Re: pgsql: Remove command checks in tests of pg_basebackup and pg_receivewa
On 2022-05-02 Mo 03:06, Michael Paquier wrote: > On Sun, May 01, 2022 at 10:18:37AM -0400, Andrew Dunstan wrote: >> IIRC we know that tar will be available on Windows. > What about gzip? Are the binaries and the libraries split into > different packages for this package manager on Windows? We've never > assumed that this is possible on ./configure, but it would make sense > to make MSVC able to manage this case. vcpkg doesn't install any.exe files AFAICS, just DLLs, headers, .lib files etc. > >> I don't think we should do that check for every time we call >> vc_regress.pl, that seems wasteful. Maybe do it if the command is one >> that might require these commands, which I think would be bincheck or >> taptest. And/Or stash some status somewhere? > Mostly. We also care about gzip in contrib/basebackup_to_shell/, > though contribcheck does not run the TAP tests of contrib/ modules. > As you say, setting up those variables in bincheck and taptest would > be enough. That would be a start. Maybe meson will us get rid of some of this mess ... cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
pgsql: Fix misleading comments about background worker registration.
Fix misleading comments about background worker registration. Since 6bc8ef0b7f1f1df3998745a66e1790e27424aa0c, the maximum number of backends can't change as background workers are registered, but these comments still reflect the way things worked prior to that. Also, per recent discussion, some modules call SetConfigOption() from _PG_init(). It's not entirely clear to me whether we want to regard that as a fully supported operation, but since we know it's a thing that happens, it at least deserves a mention in the comments, so add that. Nathan Bossart, reviewed by Anton A. Melnikov Discussion: http://postgr.es/m/20220419154658.GA2487941@nathanxps13 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/701d918a426b394620ce4d046533f77262c70829 Modified Files -- src/backend/postmaster/postmaster.c | 10 -- src/backend/utils/init/postinit.c | 5 ++--- 2 files changed, 6 insertions(+), 9 deletions(-)
Re: pgsql: Backpatch addition of wait_for_log(), pump_until().
On 03.05.22 03:11, Andres Freund wrote: Backpatch addition of wait_for_log(), pump_until(). These were originally introduced in a2ab9c06ea1 and a2ab9c06ea1, as they are needed by a about-to-be-backpatched test. I see this warning: Subroutine pump_until redefined at t/013_crash_restart.pl line 246. (There are a varying number of these warnings across the backbranches.) Also, the local definition has different arguments than the backpatched one. This should be double-checked.
pgsql: pgcrypto: report init errors as PXE_CIPHER_INIT
pgcrypto: report init errors as PXE_CIPHER_INIT Report OpenSSL errors during initialization as PXE_CIPHER_INIT since that's just what they were, and not generic unknown errors. This also removes the last users of the generic error, and thus it can be removed. Discussion: http://postgr.es/m/c89d932c-501e-4473-9750-638cfcd90...@yesql.se Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0250a167a07d4d2cff932e467d84306cbd3704fe Modified Files -- contrib/pgcrypto/openssl.c | 4 ++-- contrib/pgcrypto/px.c | 1 - contrib/pgcrypto/px.h | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-)
pgsql: Clear the OpenSSL error queue before cryptohash operations
Clear the OpenSSL error queue before cryptohash operations
Setting up an EVP context for ciphers banned under FIPS generate
two OpenSSL errors in the queue, and as we only consume one from
the queue the other is at the head for the next invocation:
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: unsupported
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: initialization error
Clearing the error queue when creating the context ensures that
we don't pull in an error from an earlier operation.
Discussion: https://postgr.es/m/c89d932c-501e-4473-9750-638cfcd90...@yesql.se
Branch
--
master
Details
---
https://git.postgresql.org/pg/commitdiff/17ec5fa502d299b1919b1afacda839fb7d8206ad
Modified Files
--
src/common/cryptohash_openssl.c | 9 +
src/common/hmac_openssl.c | 4
2 files changed, 13 insertions(+)
pgsql: pgcrypto: remove questionmark from error message
pgcrypto: remove questionmark from error message The PXE_CIPHER_INIT error is used to report initialization errors, so appending a questionmark to the error isn't entirely accurate (using a space before the questionmark doubly so). Discussion: https://postgr.es/m/c89d932c-501e-4473-9750-638cfcd90...@yesql.se Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ee97d46cdb7e23f8ee98fe22648a1c3c75a80261 Modified Files -- contrib/pgcrypto/expected/blowfish_1.out | 32 contrib/pgcrypto/expected/cast5_1.out| 20 ++-- contrib/pgcrypto/expected/des_1.out | 16 contrib/pgcrypto/px.c| 2 +- 4 files changed, 35 insertions(+), 35 deletions(-)
pgsql: Fix typo in origin.c
Fix typo in origin.c Introduced in 5aa2350. Author: Peter Smith Discussion: https://postgr.es/m/cahut+psuwz6_7acmivnu5fahgqxdutqtc3+__xnwkbzqcfn...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/59a32f00937c85fe944cf1fac3e8b98d091e2bc6 Modified Files -- src/backend/replication/logical/origin.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Update SQL features
Update SQL features Update a few items that have become supported or mostly supported but weren't updated at the time. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7e367924e33e47e0b0b135b0649c406a589bd496 Modified Files -- src/backend/catalog/sql_features.txt | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-)
pgsql: Fix some whitespace in documentation markup
Fix some whitespace in documentation markup Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/7d6a4fdf3243a7a18e2cfc6cf03bce01cc6ce651 Modified Files -- doc/src/sgml/func.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: doc: Fix typos
doc: Fix typos introduced by 222b697ec077047024a96392a2f5cb9b1803ccf7 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a6f12520bed073d1d548a691e85d7f5371cc4ed4 Modified Files -- doc/src/sgml/func.sgml | 8 1 file changed, 4 insertions(+), 4 deletions(-)
