pgsql: Renumber cursor option flags

2021-04-05 Thread Peter Eisentraut
Renumber cursor option flags Move the planner-control flags up so that there is more room for parse options. Some pending patches need some room there, so do this renumbering separately so that there is less potential for conflicts. Branch -- master Details --- https://git.postgresql.or

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
Michael Paquier writes: > Refactor all TAP test suites doing connection checks Some of the buildfarm seems not to like this. gaur failed here: ==~_~===-=-===~_~== pgsql.build/src/test/authentication/tmp_check/log/regress_log_001_password ==~_~===-=-===~_~== ... ok 8 - authentication success f

pgsql: Align some terms in arch-dev.sgml to glossary

2021-04-05 Thread Alvaro Herrera
Align some terms in arch-dev.sgml to glossary This mostly adds links to the glossary to the existing text, instead of using . Heikki left this out of 29ad6595ef7f out of stylistic concerns; these have since been addressed. Author: Jürgen Purtz Discussion: https://postgr.es/m/67d7240f-8596-83fc-

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Shut down transaction tracking at startup process exit.

2021-04-05 Thread Fujii Masao
Shut down transaction tracking at startup process exit. Maxim Orlov reported that the shutdown of standby server could result in the following assertion failure. The cause of this issue was that, when the shutdown caused the startup process to exit, recovery-time transaction tracking was not shut

pgsql: Add pg_read_all_data and pg_write_all_data roles

2021-04-05 Thread Stephen Frost
Add pg_read_all_data and pg_write_all_data roles A commonly requested use-case is to have a role who can run an unfettered pg_dump without having to explicitly GRANT that user access to all tables, schemas, et al, without that role being a superuser. This address that by adding a "pg_read_all_data

pgsql: Simplify state managed by VACUUM.

2021-04-05 Thread Peter Geoghegan
Simplify state managed by VACUUM. Reorganize the state struct used by VACUUM -- group related items together to make it easier to understand. Also stop relying on stack variables inside lazy_scan_heap() -- move those into the state struct instead. Doing things this way simplifies large groups of

Re: pgsql: Simplify state managed by VACUUM.

2021-04-05 Thread Peter Geoghegan
On Mon, Apr 5, 2021 at 1:27 PM Peter Geoghegan wrote: > Simplify state managed by VACUUM. I'm investigating the buildfarm failures on elver, curculio, and morepork. -- Peter Geoghegan

pgsql: Propagate parallel VACUUM's buffer access strategy.

2021-04-05 Thread Peter Geoghegan
Propagate parallel VACUUM's buffer access strategy. Parallel VACUUM relied on global variable state from the leader process being propagated to workers on fork(). Commit b4af70cb removed most uses of global variables inside vacuumlazy.c, but did not account for the buffer access strategy state.

pgsql: Support INCLUDE'd columns in SP-GiST.

2021-04-05 Thread Tom Lane
Support INCLUDE'd columns in SP-GiST. Not much to say here: does what it says on the tin. We steal a previously-always-zero bit from the nextOffset field of leaf index tuples in order to track whether there is a nulls bitmap. Otherwise it works about like included columns in other index types. P

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Michael Paquier
On Mon, Apr 05, 2021 at 10:57:22AM -0400, Tom Lane wrote: > Some of the buildfarm seems not to like this. gaur failed here: Thanks. I did not notice these. > I didn't look at the code, but I bet this is similar to the race > conditions we've seen before where a server process may exit before >

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 05, 2021 at 10:57:22AM -0400, Tom Lane wrote: >> I didn't look at the code, but I bet this is similar to the race >> conditions we've seen before where a server process may exit before >> the perl test script finishes stuffing a command down the pipe. > Hmm.

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Michael Paquier
On Mon, Apr 05, 2021 at 06:57:47PM -0400, Tom Lane wrote: > No, I bet it is with "now defines a SQL query". That means there's > something to be stuffed down the pipe. > > I find I can reproduce this pretty reliably on slower machines > here, if you want me to dig into it more carefully. Now tha

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
I wrote: > Michael Paquier writes: >> So, the problem is with on_error_stop. > No, I bet it is with "now defines a SQL query". That means there's > something to be stuffed down the pipe. I can confirm that removing the on_error_stop parameter as you propose does *not* fix it.

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
What does seem to fix it reliably is the attached. regards, tom lane diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm index bbde34c..07a9167 100644 --- a/src/test/perl/PostgresNode.pm +++ b/src/test/perl/PostgresNode.pm @@ -1938,7 +1938,7 @@ sub c

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Michael Paquier
On Mon, Apr 05, 2021 at 07:23:43PM -0400, Tom Lane wrote: > What does seem to fix it reliably is the attached. Thanks for confirming. I am still surprised that this has not popped up as an issue until now. Could that be because those animals have never run the LDAP, SSL or kerberos test suites o

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 05, 2021 at 07:23:43PM -0400, Tom Lane wrote: >> What does seem to fix it reliably is the attached. > Thanks for confirming. I am still surprised that this has not popped > up as an issue until now. Could that be because those animals have > never run the L

pgsql: Allocate access strategy in parallel VACUUM workers.

2021-04-05 Thread Peter Geoghegan
Allocate access strategy in parallel VACUUM workers. Commit 49f49def took entirely the wrong approach to fixing this issue. Just allocate a local buffer access strategy in each individual worker instead of trying to propagate state. This state was never propagated by parallel VACUUM in the first

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Michael Paquier
On Mon, Apr 05, 2021 at 08:15:31PM -0400, Tom Lane wrote: > Dunno. Certainly most animals that enable TAP at all should be > running the "authentication" test, but those others all need to > be manually enabled, I believe. My own animals run the SSL test > but not LDAP or kerberos. The SSL tests

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Thomas Munro
On Tue, Apr 6, 2021 at 12:15 PM Tom Lane wrote: > Michael Paquier writes: > > On Mon, Apr 05, 2021 at 07:23:43PM -0400, Tom Lane wrote: > >> What does seem to fix it reliably is the attached. > > > Thanks for confirming. I am still surprised that this has not popped > > up as an issue until now.

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 05, 2021 at 08:15:31PM -0400, Tom Lane wrote: >> Dunno. Certainly most animals that enable TAP at all should be >> running the "authentication" test, but those others all need to >> be manually enabled, I believe. My own animals run the SSL test >> but not L

pgsql: Change PostgresNode::connect_fails() to never send down queries

2021-04-05 Thread Michael Paquier
Change PostgresNode::connect_fails() to never send down queries This type of failure is similar to what has been fixed in c757a3da, where an authentication failure combined with psql pushing a command down its communication pipe causes a test failure. This routine is designed to fail, so sending

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Michael Paquier
On Tue, Apr 06, 2021 at 12:30:48PM +1200, Thomas Munro wrote: > Commit f44b9b62 springs to mind. Right. The kerberos suite does not do that though, so wouldn't it be better to use the same trick with -c? For example, the attached for ~13 does so. -- Michael diff --git a/src/test/kerberos/t/001_a

Re: pgsql: Refactor all TAP test suites doing connection checks

2021-04-05 Thread Tom Lane
I wrote: > Well, since it's a race condition it's obviously going to be sensitive to > machine speed. I think that Perl version, or more likely version of > IPC::Run or a related module, may enter into it too. prairiedog has > gotten through the tests several times since yesterday without failing

pgsql: Refactor function parse_output_parameters.

2021-04-05 Thread Amit Kapila
Refactor function parse_output_parameters. Instead of using multiple parameters in parse_ouput_parameters function signature, use the struct PGOutputData that encapsulates all pgoutput options. It will be useful for future work where we need to add other options in pgoutput. Author: Euler Taveira

pgsql: Allow pgoutput to send logical decoding messages.

2021-04-05 Thread Amit Kapila
Allow pgoutput to send logical decoding messages. The output plugin accepts a new parameter (messages) that controls if logical decoding messages are written into the replication stream. It is useful for those clients that use pgoutput as an output plugin and needs to process messages that were wr

pgsql: Fix some issues with SSL and Kerberos tests

2021-04-05 Thread Michael Paquier
Fix some issues with SSL and Kerberos tests The recent refactoring done in c50624c accidentally broke a portion of the kerberos tests checking after a query, so add its functionality back. Some inactive SSL tests had their arguments in an incorrect order, which would cause them to fail if they we

Re: pgsql: Improve PQtrace() output format

2021-04-05 Thread David Rowley
On Wed, 31 Mar 2021 at 12:17, Alvaro Herrera wrote: > > Improve PQtrace() output format I see the Visual Studio buildfarm members have been getting a compiler warning since this commit [1] (ClCompile target) -> src/interfaces/libpq/fe-trace.c(87): warning C4133: 'function': incompatible types

pgsql: Add function to log the memory contexts of specified backend pro

2021-04-05 Thread Fujii Masao
Add function to log the memory contexts of specified backend process. Commit 3e98c0bafb added pg_backend_memory_contexts view to display the memory contexts of the backend process. However its target process is limited to the backend that is accessing to the view. So this is not so convenient when

Re: pgsql: Improve PQtrace() output format

2021-04-05 Thread Tom Lane
David Rowley writes: > I see the Visual Studio buildfarm members have been getting a compiler > warning since this commit [1] > (ClCompile target) -> > src/interfaces/libpq/fe-trace.c(87): warning C4133: 'function': > incompatible types - from 'long *' to 'const time_t *const ' > [C:\\pgbuildfar

pgsql: Fix typo in pgstat.c.

2021-04-05 Thread Fujii Masao
Fix typo in pgstat.c. Introduced by 9868167500. Author: Vignesh C Discussion: https://postgr.es/m/caldanm1dqgalbajrtgznkk1sr1mh-augmp7lfgvxwwtuhah...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f5d94e405e17a49487672316610630be2f9d0bb7 Modified

Re: pgsql: Improve PQtrace() output format

2021-04-05 Thread Tom Lane
I wrote: > As best I can tell from these warnings, VS has the tv_sec field > of struct timeval defined as something other than time_t. Because > VS would never stoop to actually complying with the plain text of > the POSIX standard. Ah, seems like it's really Winsock's fault: https://docs.micros

pgsql: Change return type of EXTRACT to numeric

2021-04-05 Thread Peter Eisentraut
Change return type of EXTRACT to numeric The previous implementation of EXTRACT mapped internally to date_part(), which returned type double precision (since it was implemented long before the numeric type existed). This can lead to imprecise output in some cases, so returning numeric would be pr

pgsql: Fix compiler warning in fe-trace.c for MSVC

2021-04-05 Thread David Rowley
Fix compiler warning in fe-trace.c for MSVC It seems that in MSVC timeval's tv_sec field is of type long. localtime() takes a time_t pointer. Since long is 32-bit even on 64-bit builds in MSVC, passing a long pointer instead of the correct time_t pointer generated a compiler warning. Fix that.

Re: pgsql: Improve PQtrace() output format

2021-04-05 Thread David Rowley
On Tue, 6 Apr 2021 at 17:21, Tom Lane wrote: > > I wrote: > > As best I can tell from these warnings, VS has the tv_sec field > > of struct timeval defined as something other than time_t. Because > > VS would never stoop to actually complying with the plain text of > > the POSIX standard. > > Ah,