Re: pgsql: Transform OR clauses to ANY expression

2024-04-07 Thread Kyotaro Horiguchi
At Sun, 07 Apr 2024 22:28:06 +, Alexander Korotkov wrote in > Transform OR clauses to ANY expression This commit introduces a message like this: > gettext_noop("Set the minimum length of the list of OR clauses to attempt the > OR-to-ANY transformation."), Unlike the usual phrasing of

pgsql: Remove references to old function name

2024-04-07 Thread Andres Freund
Remove references to old function name In a97bbe1f1df I accidentally referenced heapgetpage(), both in a function name and a comment. But since 44086b09753 the relevant function is named heap_prepare_pagescan(). Rename the new function to page_collect_tuples(). Reported-by: Melanie Plageman

pgsql: Add pg_buffercache_evict() function for testing.

2024-04-07 Thread Thomas Munro
Add pg_buffercache_evict() function for testing. When testing buffer pool logic, it is useful to be able to evict arbitrary blocks. This function can be used in SQL queries over the pg_buffercache view to set up a wide range of buffer pool states. Of course, buffer mappings might change

pgsql: Fix alignment of stack variable

2024-04-07 Thread John Naylor
Fix alignment of stack variable Declare with union similar to PGAlignedBlock. Report and fix by Andres Freund Discussion: https://postgr.es/m/20240407190731.izm3mdazednrsiqk%40awork3.anarazel.de Branch -- master Details ---

pgsql: Add more tab completion support for ALTER DEFAULT PRIVILEGES in

2024-04-07 Thread Masahiko Sawada
Add more tab completion support for ALTER DEFAULT PRIVILEGES in psql. This adds tab completion of "GRANT" and "REVOKE [GRANT OPTION FOR]" for ALTER DEFAULT PRIVILEGES, and adds "WITH GRANT OPTION" for ALTER DEFAULT PRIVILEGES ... GRANT ... TO role. Author: Vignesh C, with cosmetic adjustments by

pgsql: Remove redundant nbtree preprocessing assertions.

2024-04-07 Thread Peter Geoghegan
Remove redundant nbtree preprocessing assertions. One of the assertions was the subject of a false positive complaint from Coverity, but none of the assertions added much, so get rid of them. Reported-By: Tom Lane Discussion: https://postgr.es/m/3000247.1712537...@sss.pgh.pa.us Branch --

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

pgsql: simplehash: Free collisions array in SH_STAT

2024-04-07 Thread Andres Freund
simplehash: Free collisions array in SH_STAT While SH_STAT() is only used for debugging, the allocated array can be large, and therefore should be freed. It's unclear why coverity started warning now. Reported-by: Tom Lane Reported-by: Coverity Discussion:

Re: pgsql: Send ALPN in TLS handshake, require it in direct SSL connections

2024-04-07 Thread Heikki Linnakangas
On 08/04/2024 04:50, Andres Freund wrote: On 2024-04-08 01:25:40 +, Heikki Linnakangas wrote: Send ALPN in TLS handshake, require it in direct SSL connections libpq now always tries to send ALPN. With the traditional negotiated SSL connections, the server accepts the ALPN, and refuses the

pgsql: Fix check for 'outlen' return from SSL_select_next_proto()

2024-04-07 Thread Heikki Linnakangas
Fix check for 'outlen' return from SSL_select_next_proto() Fixes compiler warning reported by Andres Freund. Discusssion: https://www.postgresql.org/message-id/20240408015055.xsuahullywpfw...@awork3.anarazel.de Branch -- master Details ---

Re: pgsql: Send ALPN in TLS handshake, require it in direct SSL connections

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 01:25:40 +, Heikki Linnakangas wrote: > Send ALPN in TLS handshake, require it in direct SSL connections > > libpq now always tries to send ALPN. With the traditional negotiated > SSL connections, the server accepts the ALPN, and refuses the > connection if it's not what we

pgsql: Silence perlcritic warnings in new libpq tests

2024-04-07 Thread Heikki Linnakangas
Silence perlcritic warnings in new libpq tests Per buildfarm member 'koel'. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/d60ab76f630921dc4deb958fb7892ef6b0e4a7ee Modified Files -- src/test/libpq_encryption/t/001_negotiate_encryption.pl | 5 +++-- 1

pgsql: Refactor libpq state machine for negotiating encryption

2024-04-07 Thread Heikki Linnakangas
Refactor libpq state machine for negotiating encryption This fixes the few corner cases noted in commit 705843d294, as shown by the changes in the test. Author: Heikki Linnakangas, Matthias van de Meent Reviewed-by: Jacob Champion Branch -- master Details ---

pgsql: Support TLS handshake directly without SSLRequest negotiation

2024-04-07 Thread Heikki Linnakangas
Support TLS handshake directly without SSLRequest negotiation By skipping SSLRequest, you can eliminate one round-trip when establishing a TLS connection. It is also more friendly to generic TLS proxies that don't understand the PostgreSQL protocol. This is disabled by default in libpq, because

pgsql: Send ALPN in TLS handshake, require it in direct SSL connections

2024-04-07 Thread Heikki Linnakangas
Send ALPN in TLS handshake, require it in direct SSL connections libpq now always tries to send ALPN. With the traditional negotiated SSL connections, the server accepts the ALPN, and refuses the connection if it's not what we expect, but connecting without ALPN is still OK. With the new direct

pgsql: Use streaming I/O in ANALYZE.

2024-04-07 Thread Thomas Munro
Use streaming I/O in ANALYZE. The ANALYZE command prefetches and reads sample blocks chosen by a BlockSampler algorithm. Instead of calling [Prefetch|Read]Buffer() for each block, ANALYZE now uses the streaming API introduced in b5a9b18cd0. Author: Nazir Bilal Yavuz Reviewed-by: Melanie

pgsql: injection_points: Introduce runtime conditions

2024-04-07 Thread Michael Paquier
injection_points: Introduce runtime conditions This adds a new SQL function injection_points_set_local() that can be used to force injection points to be run only in the process where they are attached. This is handy for SQL tests to: - Detach automatically injection points when the process

pgsql: Enhance libpq encryption negotiation tests with new GUC

2024-04-07 Thread Heikki Linnakangas
Enhance libpq encryption negotiation tests with new GUC The new "log_connection_negotiation" server option causes the server to print messages to the log when it receives a SSLRequest or GSSENCRequest packet from the client. Together with "log_connections", it gives a trace of how a connection

pgsql: With gssencmode='require', check credential cache before connect

2024-04-07 Thread Heikki Linnakangas
With gssencmode='require', check credential cache before connecting Previously, libpq would establish the TCP connection, and then immediately disconnect if the credentials were not available. The same thing happened if you tried to use a Unix domain socket with gssencmode=require. Check those

pgsql: Add tests for libpq gssencmode and sslmode options

2024-04-07 Thread Heikki Linnakangas
Add tests for libpq gssencmode and sslmode options Test all combinations of gssencmode, sslmode, whether the server supports SSL and/or GSSAPI encryption, and whether they are accepted by pg_hba.conf. This is in preparation for refactoring that code in libpq, and for adding a new option for

pgsql: Move Kerberos module

2024-04-07 Thread Heikki Linnakangas
Move Kerberos module So that we can reuse it in new tests. Discussion: https://www.postgresql.org/message-id/a3af4070-3556-461d-aec8-a8d794f94...@iki.fi Reviewed-by: Jacob Champion, Matthias van de Meent Branch -- master Details ---

pgsql: Make GIN test using injection points repeatable

2024-04-07 Thread Michael Paquier
Make GIN test using injection points repeatable As written, the test would fail when run repeatedly because one of the injection points attached was not detached. This would not matter if the test is rewritten to be concurrently safe, but let's be clean and it is a good practice. Oversight in

pgsql: Fix incorrect KeeperBlock macro in bump.c

2024-04-07 Thread David Rowley
Fix incorrect KeeperBlock macro in bump.c The macro was missing a MAXALIGN around the sizeof(BumpContext) which would cause problems detecting the keeper block on 32-bit systems that have a MAXALIGN value of 8. Thank you to Andres Freund, Tomas Vondra and Tom Lane for investigating and testing.

pgsql: Fix usage of same ListCell transform_or_to_any()'s in nested loo

2024-04-07 Thread Alexander Korotkov
Fix usage of same ListCell transform_or_to_any()'s in nested loops Discussion: https://postgr.es/m/CAAKRu_b4SXNW4GAM0bv3e6wcL5ODSXg1ZdRCn6uyLLjSPbveBg%40mail.gmail.com Author: Melanie Plageman Branch -- master Details ---

Re: pgsql: Transform OR clauses to ANY expression

2024-04-07 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 1:35 AM Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov > wrote: > > > > Transform OR clauses to ANY expression > > > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, > > ...]) > > on the preliminary stage of

Re: pgsql: Transform OR clauses to ANY expression

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 6:28 PM Alexander Korotkov wrote: > > Transform OR clauses to ANY expression > > Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) > on the preliminary stage of optimization when we are still working with the > expression tree. > > Here Cn is a

pgsql: Transform OR clauses to ANY expression

2024-04-07 Thread Alexander Korotkov
Transform OR clauses to ANY expression Replace (expr op C1) OR (expr op C2) ... with expr op ANY(ARRAY[C1, C2, ...]) on the preliminary stage of optimization when we are still working with the expression tree. Here Cn is a n-th constant expression, 'expr' is non-constant expression, 'op' is an

pgsql: Change debug printing to log filename

2024-04-07 Thread Daniel Gustafsson
Change debug printing to log filename When restarting the cluster fails the code introduced in 33774978c78 printed the full log contents to aid debugging. For cases when the logfile is large this adds unnecessary overhead. Reduce to printing the logfile path instead. Reported-by: Andres Freund

pgsql: Doc: clarify behavior of boolean options in replication protocol

2024-04-07 Thread Tom Lane
Doc: clarify behavior of boolean options in replication protocol commands. Same idea as ec7e053a9, but applying to the walsender commands described in protocol.sgml. Peter Smith Discussion: https://postgr.es/m/CAHut+PvwjZfdGt2R8HTXgSZft=jzkymrs8kug31ps7zqaaw...@mail.gmail.com Branch --

Re: pgsql: Use bump memory context for tuplesorts

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 13:52:28 -0400, Tom Lane wrote: > Melanie Plageman writes: > > Looks like this assert is tripping on grison [1]. > > > running bootstrap script ... TRAP: failed Assert("total_allocated == > > context->mem_allocated"), File: "bump.c", Line: 808, PID: 30248 > > The same on mamba.

pgsql: Remove useless duplicate call of defGetBoolean().

2024-04-07 Thread Tom Lane
Remove useless duplicate call of defGetBoolean(). Seems to be a copy-and-paste error dating to dc2123400. Noted while reviewing a related documentation patch. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0c66a164e75163f5a850fb89d13ec0b213753d8d Modified Files

pgsql: Doc: show how to get the equivalent of LIMIT for UPDATE/DELETE.

2024-04-07 Thread Tom Lane
Doc: show how to get the equivalent of LIMIT for UPDATE/DELETE. Add examples showing use of a CTE and a self-join to perform partial UPDATEs and DELETEs. Corey Huinker, reviewed by Laurenz Albe Discussion: https://postgr.es/m/CADkLM=caneqsuwpwnfi2jr4ix99e0ejm_3jtce-yjneqc7r...@mail.gmail.com

pgsql: Doc: update documentation about EXCLUDE constraint elements.

2024-04-07 Thread Tom Lane
Doc: update documentation about EXCLUDE constraint elements. What the documentation calls an exclude_element is an index_elem according to gram.y, and it allows all the same options that a CREATE INDEX column specification does. The COLLATE patch neglected to update the CREATE/ALTER TABLE docs

pgsql: Doc: update documentation about EXCLUDE constraint elements.

2024-04-07 Thread Tom Lane
Doc: update documentation about EXCLUDE constraint elements. What the documentation calls an exclude_element is an index_elem according to gram.y, and it allows all the same options that a CREATE INDEX column specification does. The COLLATE patch neglected to update the CREATE/ALTER TABLE docs

pgsql: Doc: update documentation about EXCLUDE constraint elements.

2024-04-07 Thread Tom Lane
Doc: update documentation about EXCLUDE constraint elements. What the documentation calls an exclude_element is an index_elem according to gram.y, and it allows all the same options that a CREATE INDEX column specification does. The COLLATE patch neglected to update the CREATE/ALTER TABLE docs

pgsql: Doc: update documentation about EXCLUDE constraint elements.

2024-04-07 Thread Tom Lane
Doc: update documentation about EXCLUDE constraint elements. What the documentation calls an exclude_element is an index_elem according to gram.y, and it allows all the same options that a CREATE INDEX column specification does. The COLLATE patch neglected to update the CREATE/ALTER TABLE docs

pgsql: Doc: update documentation about EXCLUDE constraint elements.

2024-04-07 Thread Tom Lane
Doc: update documentation about EXCLUDE constraint elements. What the documentation calls an exclude_element is an index_elem according to gram.y, and it allows all the same options that a CREATE INDEX column specification does. The COLLATE patch neglected to update the CREATE/ALTER TABLE docs

pgsql: Use conditional variable to wait for next MultiXact offset

2024-04-07 Thread Alvaro Herrera
Use conditional variable to wait for next MultiXact offset In one multixact.c edge case, we need a mechanism to wait for one multixact offset to be written before being allowed to read the next one. We used to handle this case by sleeping for one millisecond and retrying, but such sleeps have

pgsql: Avoid extra lookups with nbtree array inequalities.

2024-04-07 Thread Peter Geoghegan
Avoid extra lookups with nbtree array inequalities. nbtree index scans with SAOP inequalities (but no SAOP equalities) performed extra ORDER proc lookups for any remaining equality strategy scan keys. This could waste cycles, and caused assertion failures. Keeping around a separate ORDER proc is

Re: pgsql: Use bump memory context for tuplesorts

2024-04-07 Thread Tom Lane
Melanie Plageman writes: > Looks like this assert is tripping on grison [1]. > running bootstrap script ... TRAP: failed Assert("total_allocated == > context->mem_allocated"), File: "bump.c", Line: 808, PID: 30248 The same on mamba. However, I failed to duplicate it in an x86 (32-bit) VM. I

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Improve check in LDAP test to find the OpenLDAP installation

2024-04-07 Thread Heikki Linnakangas
Improve check in LDAP test to find the OpenLDAP installation If the OpenLDAP installation directory is not found, set $setup to 0 so that the LDAP tests are skipped. The macOS checks were already doing that, but the checks on other OS's were not. While we're at it, improve the error message when

pgsql: Don't clobber test exit code at cleanup in LDAP/Kerberors tests

2024-04-07 Thread Heikki Linnakangas
Don't clobber test exit code at cleanup in LDAP/Kerberors tests If the test script die()d before running the first test, the whole test was interpreted as SKIPped rather than failed. The PostgreSQL::Cluster module got this right. Backpatch to all supported versions. Discussion:

Re: pgsql: Reduce branches in heapgetpage()'s per-tuple loop

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 09:14:57 -0400, Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 3:29 AM Andres Freund wrote: > > > > Reduce branches in heapgetpage()'s per-tuple loop > > > > Until now, heapgetpage()'s loop over all tuples performed some conditional > > checks for each tuple, even though

Re: pgsql: Use bump memory context for tuplesorts

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 8:32 AM David Rowley wrote: > > Use bump memory context for tuplesorts > > 29f6a959c added a bump allocator type for efficient compact allocations. > Here we make use of this for non-bounded tuplesorts to store tuples. > This is very space efficient when storing narrow

pgsql: Use streaming I/O in sequential scans.

2024-04-07 Thread Thomas Munro
Use streaming I/O in sequential scans. Instead of calling ReadBuffer() for each block, heap sequential scans and TID range scans now use the streaming API introduced in b5a9b18cd0. Author: Melanie Plageman Reviewed-by: Andres Freund Reviewed-by: Thomas Munro Discussion:

Re: pgsql: Reduce branches in heapgetpage()'s per-tuple loop

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 3:29 AM Andres Freund wrote: > > Reduce branches in heapgetpage()'s per-tuple loop > > Until now, heapgetpage()'s loop over all tuples performed some conditional > checks for each tuple, even though condition did not change across the loop. I haven't reviewed exactly what

pgsql: Use bump memory context for tuplesorts

2024-04-07 Thread David Rowley
Use bump memory context for tuplesorts 29f6a959c added a bump allocator type for efficient compact allocations. Here we make use of this for non-bounded tuplesorts to store tuples. This is very space efficient when storing narrow tuples due to bump.c not having chunk headers. This means we can

pgsql: Add XLogCtl->logInsertResult

2024-04-07 Thread Alvaro Herrera
Add XLogCtl->logInsertResult This tracks the position of WAL that's been fully copied into WAL buffers by all processes emitting WAL. (For some reason we call that "WAL insertion"). This is updated using atomic monotonic advance during WaitXLogInsertionsToFinish, which is not when the

pgsql: Introduce a bump memory allocator

2024-04-07 Thread David Rowley
Introduce a bump memory allocator This introduces a bump MemoryContext type. The bump context is best suited for short-lived memory contexts which require only allocations of memory and never a pfree or repalloc, which are unsupported. Memory palloc'd into a bump context has no chunk header.

pgsql: Enlarge bit-space for MemoryContextMethodID

2024-04-07 Thread David Rowley
Enlarge bit-space for MemoryContextMethodID Reserve 4 bits for MemoryContextMethodID rather than 3. 3 bits did technically allow a maximum of 8 memory context types, however, we've opted to reserve some bit patterns which left us with only 4 slots, all of which were used. Here we add another

pgsql: Avoid needless large memcpys in libpq socket writing

2024-04-07 Thread David Rowley
Avoid needless large memcpys in libpq socket writing Until now, when calling pq_putmessage to write new data to a libpq socket, all writes are copied into a buffer and that buffer gets flushed when full to avoid having to perform small writes to the socket. There are cases where we must write

pgsql: Reduce branches in heapgetpage()'s per-tuple loop

2024-04-07 Thread Andres Freund
Reduce branches in heapgetpage()'s per-tuple loop Until now, heapgetpage()'s loop over all tuples performed some conditional checks for each tuple, even though condition did not change across the loop. This commit fixes that by moving the loop into an inline function. By calling it with