pgsql: Unify parsing logic for command-line integer options

2021-07-24 Thread Michael Paquier
Unify parsing logic for command-line integer options

Most of the integer options for command-line binaries now make use of a
single routine able to do the job, fixing issues with the detection of
sloppy values caused for example by the use of atoi(), that fails on
strings beginning with numerical characters with junk trailing
characters.

This commit cuts down the number of strings requiring translation by 26
per my count, switching the code to have two error types for invalid and
out-of-range values instead.

Much more could be done here, with float or even int64 options, but
int32 was the most appealing case as it is possible to rely on strtol()
to do the job reliably.  Note that there are some exceptions for now,
like pg_ctl or pg_upgrade that use their own logging logic.  A couple of
negative TAP tests required some adjustments for the new errors
generated.

pg_dump and pg_restore tracked the maximum number of parallel jobs
within the option parsing.  The code is refactored a bit to track that
in the code dedicated to parallelism instead.

Author: Kyotaro Horiguchi, Michael Paquier
Reviewed-by: David Rowley, Álvaro Herrera
Discussion: 
https://postgr.es/m/CALj2ACXqdG9WhqVoJ9zYf-iZt7sgK7Szv5USs=he6nnwq2o...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/6f164e6d17616a157ea5d9e34dbb1b211c080c41

Modified Files
--
src/bin/pg_amcheck/pg_amcheck.c|  8 ++--
src/bin/pg_basebackup/pg_basebackup.c  | 17 -
src/bin/pg_basebackup/pg_receivewal.c  | 23 
src/bin/pg_basebackup/pg_recvlogical.c | 25 +
src/bin/pg_checksums/Makefile  |  3 ++
src/bin/pg_checksums/pg_checksums.c|  9 +++--
src/bin/pg_dump/parallel.h | 13 +++
src/bin/pg_dump/pg_dump.c  | 47 ++-
src/bin/pg_dump/pg_restore.c   | 22 +++
src/bin/pg_dump/t/001_basic.pl | 20 +-
src/bin/pgbench/pgbench.c  | 60 ++
src/bin/pgbench/t/002_pgbench_no_server.pl | 34 +++--
src/bin/scripts/createuser.c   | 13 +++
src/bin/scripts/reindexdb.c|  9 ++---
src/bin/scripts/vacuumdb.c | 31 +--
src/fe_utils/option_utils.c| 39 +++
src/include/fe_utils/option_utils.h|  3 ++
17 files changed, 177 insertions(+), 199 deletions(-)



pgsql: Add missing header declarations for pg_basebackup and pg_{dump,r

2021-07-24 Thread Michael Paquier
Add missing header declarations for pg_basebackup and pg_{dump,restore}

This fixes two compilation failures caused by 6f164e6.  Interesting to
see that missing  dies not fail in Linux or even Windows.  On
MacOS, it fails, though.

Per various buildfarm members.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/bc0cc68f8a1648029424e9300b2e4457acd474a6

Modified Files
--
src/bin/pg_basebackup/pg_basebackup.c | 1 +
src/bin/pg_dump/parallel.h| 2 ++
2 files changed, 3 insertions(+)



pgsql: Fix failure of some headers to compile "standalone".

2021-07-24 Thread Tom Lane
Fix failure of some headers to compile "standalone".

Recently-added references to ParseState weren't covered by #include
references, creating unwanted ordering dependencies for users of
these headers.

Oversight in commit 2bfb50b3d.  Per headerscheck/cpluspluscheck.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/678f5448c2d86976a98b402ef14482a8ba3b159b

Modified Files
--
src/include/commands/publicationcmds.h  | 2 +-
src/include/commands/subscriptioncmds.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)



pgsql: Remove configure-time thread safety checking (thread_test.c).

2021-07-24 Thread Tom Lane
Remove configure-time thread safety checking (thread_test.c).

This testing was useful when it was written, nigh twenty years ago,
but it seems fairly pointless for any platform built in the last
dozen or more years.  (Compare also the comments at 8a2121185.)
Also we now have reports that the test program itself fails under
ThreadSanitizer.  Rather than invest effort in fixing it, let's
just drop it, and assume that the few people who still care
already know they need to use --disable-thread-safety.

Back-patch into v14, for consistency with 8a2121185.

Discussion: 
https://postgr.es/m/CADhDkKzPSiNvA3Hyq+wSR_icuPmazG0cFe=ync3u-cfcylc...@mail.gmail.com

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d5e913a81cbab063c53b0a06dc8894da89390676

Modified Files
--
config/thread_test.c | 437 ---
configure| 194 +--
configure.ac |  38 +
3 files changed, 43 insertions(+), 626 deletions(-)



pgsql: Remove configure-time thread safety checking (thread_test.c).

2021-07-24 Thread Tom Lane
Remove configure-time thread safety checking (thread_test.c).

This testing was useful when it was written, nigh twenty years ago,
but it seems fairly pointless for any platform built in the last
dozen or more years.  (Compare also the comments at 8a2121185.)
Also we now have reports that the test program itself fails under
ThreadSanitizer.  Rather than invest effort in fixing it, let's
just drop it, and assume that the few people who still care
already know they need to use --disable-thread-safety.

Back-patch into v14, for consistency with 8a2121185.

Discussion: 
https://postgr.es/m/CADhDkKzPSiNvA3Hyq+wSR_icuPmazG0cFe=ync3u-cfcylc...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/76fa3db33654e543b5c796e28c6fc5b505a19c2a

Modified Files
--
config/thread_test.c | 433 ---
configure| 194 +--
configure.ac |  38 +
3 files changed, 43 insertions(+), 622 deletions(-)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5a435289d1a8842317630d0f6435987851899897

Modified Files
--
src/port/snprintf.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/3779ac62d709467fe6331c8f0285d42e7487a01c

Modified Files
--
src/port/snprintf.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4c8a14e8d993e10b9851eb6be225b151bc4e233b

Modified Files
--
src/port/snprintf.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/89ad14cd7870bce199448c527e3130c36f80d2bf

Modified Files
--
src/port/snprintf.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c0a6f83deba9c55459e52ad4475207f62354a1f1

Modified Files
--
src/port/snprintf.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/7e09b504d0232c81654d4da1b8d0ab724d941fa9

Modified Files
--
src/port/snprintf.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make printf("%s", NULL) print "(null)" instead of crashing.

2021-07-24 Thread Tom Lane
Make printf("%s", NULL) print "(null)" instead of crashing.

We previously took a hard-line attitude that callers should never print
a null string pointer, and doing so is worthy of an assertion failure
or crash.  However, we've long since flushed out any easy-to-find bugs
of that nature.  What remains is a lot of code that perhaps could fail
that way in hard-to-reach corner cases.  For example, in something as
simple as
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
 errmsg("constraint \"%s\" for table \"%s\" does not exist",
conname, get_rel_name(relid;
one must wonder whether it's completely guaranteed that get_rel_name
cannot return NULL in this context.  If such a situation did occur,
the existing policy converts what might be a pretty minor bug into
a server crash condition.  This is not good for robustness.

Hence, let's follow the lead of glibc and print "(null)" instead
of failing.  We should, of course, still consider it a bug if that
behavior is reachable in ordinary use; but crashing seems less
desirable than not crashing.

This fix works across-the-board in v12 and up, where we always use
src/port/snprintf.c.  Before that, on most platforms we're at the mercy
of the local libc, but it appears that Solaris 10 is the only supported
platform where we'd still get a crash.  Most other platforms such as
*BSD, macOS, and Solaris 11 have adopted glibc's behavior at some
point.  (AIX and HPUX just print "" not "(null)", but that's close
enough.)  I've not checked what Windows' native printf would do, but
it doesn't matter because we've long used snprintf.c on that platform.

In v12 and up, also const-ify related code so that we're not casting
away const on the constant string.  This is just neatnik-ism, since
next to no compilers will warn about that.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/9329b923542f9f6efecb575d5b931d2fe1c90191

Modified Files
--
src/port/snprintf.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/712ba6b8b73870fa9e3336df8d88f4bc5f824112

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/f47408cdc19c2bbe357703afd81037b6d12ed3ba

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1861390e6cf52bd36e100a64bfc5449dd82377fb

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/899785b4f6c8c732c233e23c31a93405cff5ec34

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/654372169b8935ef1a4233522d711fe5e556e280

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/6310809c4aa146b3996a35524955c6c6943d241a

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Fix check for conflicting session- vs transaction-level locks.

2021-07-24 Thread Tom Lane
Fix check for conflicting session- vs transaction-level locks.

We have an implementation restriction that PREPARE TRANSACTION can't
handle cases where both session-lifespan and transaction-lifespan locks
are held on the same lockable object.  (That's because we'd otherwise
need to acquire a new PROCLOCK entry during post-prepare cleanup, which
is an operation that might fail.  The situation can only arise with odd
usages of advisory locks, so removing the restriction is probably not
worth the amount of effort it would take.)  AtPrepare_Locks attempted
to enforce this, but its logic was many bricks shy of a load, because
it only detected cases where the session and transaction locks had the
same lockmode.  Locks of different modes on the same object would lead
to the rather unhelpful message "PANIC: we seem to have dropped a bit
somewhere".

To fix, build a transient hashtable with one entry per locktag,
not one per locktag + mode, and use that to detect conflicts.

Per bug #17122 from Alexander Pyhalov.  This bug is ancient,
so back-patch to all supported branches.

Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/7b2262a21cde4e943cf220e201ace3f61623bdfe

Modified Files
--
src/backend/storage/lmgr/lock.c| 129 -
src/test/regress/expected/prepared_xacts.out   |  16 +++
src/test/regress/expected/prepared_xacts_1.out |  17 
src/test/regress/sql/prepared_xacts.sql|   6 ++
4 files changed, 144 insertions(+), 24 deletions(-)



pgsql: Make the standby server promptly handle interrupt signals.

2021-07-24 Thread Fujii Masao
Make the standby server promptly handle interrupt signals.

This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.

Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.

Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: 
https://postgr.es/m/[email protected]

Per discussion of BUG #17073, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8e5be9cfe70e2b7042c7e29f859a05abad57d5e1

Modified Files
--
src/backend/access/transam/xlog.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make the standby server promptly handle interrupt signals.

2021-07-24 Thread Fujii Masao
Make the standby server promptly handle interrupt signals.

This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.

Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.

Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: 
https://postgr.es/m/[email protected]

Per discussion of BUG #17073, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/710fabfa2e4ec88a357dc36209547948545414c9

Modified Files
--
src/backend/access/transam/xlog.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make the standby server promptly handle interrupt signals.

2021-07-24 Thread Fujii Masao
Make the standby server promptly handle interrupt signals.

This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.

Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.

Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: 
https://postgr.es/m/[email protected]

Per discussion of BUG #17073, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/9c83398f8743181ed6e877bbb35bcfafc1cb82b2

Modified Files
--
src/backend/access/transam/xlog.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make the standby server promptly handle interrupt signals.

2021-07-24 Thread Fujii Masao
Make the standby server promptly handle interrupt signals.

This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.

Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.

Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: 
https://postgr.es/m/[email protected]

Per discussion of BUG #17073, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1bcfda30fb64eb9ed423e0652e02ce04e2dcaad0

Modified Files
--
src/backend/access/transam/xlog.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Make the standby server promptly handle interrupt signals.

2021-07-24 Thread Fujii Masao
Make the standby server promptly handle interrupt signals.

This commit changes the startup process in the standby server so that
it handles the interrupt signals after waiting for wal_retrieve_retry_interval
on the latch and resetting it, before entering another wait on the latch.
This change causes the standby server to promptly handle interrupt signals.

Otherwise, previously, there was the case where the standby needs to
wait extra five seconds to shutdown when the shutdown request arrived
while the startup process was waiting for wal_retrieve_retry_interval
on the latch.

Author: Fujii Masao, but implementation idea is from Soumyadeep Chakraborty
Reviewed-by: Soumyadeep Chakraborty
Discussion: 
https://postgr.es/m/[email protected]

Per discussion of BUG #17073, back-patch to all supported versions.
Discussion: https://postgr.es/m/[email protected]

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8d091922ffcc27323ffe2127e228e302b9f153f4

Modified Files
--
src/backend/access/transam/xlog.c | 3 +++
1 file changed, 3 insertions(+)



pgsql: Deduplicate choice of horizon for a relation procarray.c.

2021-07-24 Thread Andres Freund
Deduplicate choice of horizon for a relation procarray.c.

5a1e1d83022 was a minimal bug fix for dc7420c2c92. To avoid future bugs of
that kind, deduplicate the choice of a relation's horizon into a new helper,
GlobalVisHorizonKindForRel().

As the code in question was only introduced in dc7420c2c92 it seems worth
backpatching this change as well, otherwise 14 will look different from all
other branches.

A different approach to this was suggested by Matthias van de Meent.

Author: Andres Freund
Discussion: 
https://postgr.es/m/[email protected]
Backpatch: 14, like 5a1e1d83022

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3d0a4636aa4c976e971c05c77e162fc70c61f40b

Modified Files
--
src/backend/storage/ipc/procarray.c | 98 -
1 file changed, 64 insertions(+), 34 deletions(-)



pgsql: Deduplicate choice of horizon for a relation procarray.c.

2021-07-24 Thread Andres Freund
Deduplicate choice of horizon for a relation procarray.c.

5a1e1d83022 was a minimal bug fix for dc7420c2c92. To avoid future bugs of
that kind, deduplicate the choice of a relation's horizon into a new helper,
GlobalVisHorizonKindForRel().

As the code in question was only introduced in dc7420c2c92 it seems worth
backpatching this change as well, otherwise 14 will look different from all
other branches.

A different approach to this was suggested by Matthias van de Meent.

Author: Andres Freund
Discussion: 
https://postgr.es/m/[email protected]
Backpatch: 14, like 5a1e1d83022

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/d9d8aa9bb9aa2a850f53facd2c91e454129af57b

Modified Files
--
src/backend/storage/ipc/procarray.c | 101 +++-
1 file changed, 66 insertions(+), 35 deletions(-)