Re: pgsql: Prevent redeclaration of typedef TocEntry.

2025-04-04 Thread Tom Lane
=?utf-8?Q?=C3=81lvaro?= Herrera writes: > ... AFAICS we could move both DefnDumperPtr and > DataDumperPtr typedefs to pg_backup_archiver.h, together with struct > _tocEntry (the only place where they are used), and we'd have less of a > mess here. Then we don't need struct TocEntry in pg_backup.h

pgsql: pg_dump: Fix query for gathering attribute stats on older versio

2025-04-04 Thread Nathan Bossart
pg_dump: Fix query for gathering attribute stats on older versions. Commit 9c02e3a986 taught pg_dump to retrieve attribute statistics for 64 relations at a time. pg_dump supports dumping from v9.2 and newer versions, but our query for retrieving statistics for multiple relations uses WITH ORDINAL

pgsql: Repair misbehavior with duplicate entries in FK SET column lists

2025-04-04 Thread Tom Lane
Repair misbehavior with duplicate entries in FK SET column lists. Since v15 we've had an option to apply a foreign key constraint's ON DELETE SET DEFAULT or SET NULL action to just some of the referencing columns. There was not a check for duplicate entries in the list of columns-to-set, though.

pgsql: Repair misbehavior with duplicate entries in FK SET column lists

2025-04-04 Thread Tom Lane
Repair misbehavior with duplicate entries in FK SET column lists. Since v15 we've had an option to apply a foreign key constraint's ON DELETE SET DEFAULT or SET NULL action to just some of the referencing columns. There was not a check for duplicate entries in the list of columns-to-set, though.

pgsql: Repair misbehavior with duplicate entries in FK SET column lists

2025-04-04 Thread Tom Lane
Repair misbehavior with duplicate entries in FK SET column lists. Since v15 we've had an option to apply a foreign key constraint's ON DELETE SET DEFAULT or SET NULL action to just some of the referencing columns. There was not a check for duplicate entries in the list of columns-to-set, though.

pgsql: Repair misbehavior with duplicate entries in FK SET column lists

2025-04-04 Thread Tom Lane
Repair misbehavior with duplicate entries in FK SET column lists. Since v15 we've had an option to apply a foreign key constraint's ON DELETE SET DEFAULT or SET NULL action to just some of the referencing columns. There was not a check for duplicate entries in the list of columns-to-set, though.

pgsql: functions.c: copy trees from source_list before parse analysis e

2025-04-04 Thread Tom Lane
functions.c: copy trees from source_list before parse analysis etc. This is yet another bit of fallout from the fact that backend/parser (like other code) feels free to scribble on the parse tree it's handed. In this case that resulted in modifying the relatively-short-lived copy in the cached fu

Re: pgsql: Prevent redeclaration of typedef TocEntry.

2025-04-04 Thread Álvaro Herrera
On 2025-Apr-04, Nathan Bossart wrote: > Prevent redeclaration of typedef TocEntry. > > Commit 9c02e3a986 added a forward declaration for this typedef that > caused redeclarations, which is not valid in C99. To fix, add some > preprocessor guards to avoid a redefinition, as is done elsewhere > (e

pgsql: Fix a couple of error messages and tests for them

2025-04-04 Thread Andrew Dunstan
Fix a couple of error messages and tests for them oversights in 1495eff7bdb and 289f74d0cb2. Mea culpa. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2ef57908067ab29c22ae13f7775fe2afc330e8f6 Modified Files -- src/bin/pg_dump/pg_restore.c| 6 ++

pgsql: doc: Mention clock synchronization recommendation for hot_standb

2025-04-04 Thread Peter Eisentraut
doc: Mention clock synchronization recommendation for hot_standby_feedback hot_standby_feedback mechanics assume that clocks are synchronized, but it was not clear from documentation. Author: Jakub Wartak Reviewed-by: Euler Taveira Reviewed-by: Amit Kapila Reviewed-by: vignesh C Discussion:

pgsql: Prevent redeclaration of typedef TocEntry.

2025-04-04 Thread Nathan Bossart
Prevent redeclaration of typedef TocEntry. Commit 9c02e3a986 added a forward declaration for this typedef that caused redeclarations, which is not valid in C99. To fix, add some preprocessor guards to avoid a redefinition, as is done elsewhere (e.g., commit 382092a0cd). Per buildfarm. Branch --

pgsql: Add errhint_internal()

2025-04-04 Thread Andres Freund
Add errhint_internal() We have errmsg_internal(), errdetail_internal(), but not errhint_internal(). Sometimes it is useful to output a hint with already translated format string (e.g. because there different messages depending on the condition). For message/detail we do that with the _internal()

pgsql: Improve accounting for memory used by shared hash tables

2025-04-04 Thread Tomas Vondra
Improve accounting for memory used by shared hash tables pg_shmem_allocations tracks the memory allocated by ShmemInitStruct(), but for shared hash tables that covered only the header and hash directory. The remaining parts (segments and buckets) were allocated later using ShmemAlloc(), which doe

pgsql: libpq: Add min/max_protocol_version connection options

2025-04-04 Thread Heikki Linnakangas
libpq: Add min/max_protocol_version connection options All supported version of the PostgreSQL server send the NegotiateProtocolVersion message when an unsupported minor protocol version is requested by a client. But many other applications that implement the PostgreSQL protocol (connection pooler

Re: pgsql: Add vacuum_truncate configuration parameter.

2025-04-04 Thread Tom Lane
Nathan Bossart writes: > Since there's presently no way to determine whether a Boolean > storage parameter is explicitly set or has just picked up the > default value, this commit also introduces an isset_offset member > to relopt_parse_elt. Uh, what? Why is it a good idea to distinguish those s

pgsql: doc: Correct description of values used in FSM for indexes

2025-04-04 Thread Michael Paquier
doc: Correct description of values used in FSM for indexes The implementation of FSM for indexes is simpler than heap, where 0 is used to track if a page is in-use and (BLCKSZ - 1) if a page is free. One comment in indexfsm.c and one description in the documentation of pg_freespacemap were incorre

Re: pgsql: Add more TAP tests for pg_dumpall

2025-04-04 Thread Andrew Dunstan
On 2025-04-04 Fr 4:09 PM, Andrew Dunstan wrote: Add more TAP tests for pg_dumpall I changed an error message at the last moment and forgot to fix the test ... will fix cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com

pgsql: Non text modes for pg_dumpall, correspondingly change pg_restore

2025-04-04 Thread Andrew Dunstan
Non text modes for pg_dumpall, correspondingly change pg_restore pg_dumpall acquires a new -F/--format option, with the same meanings as pg_dump. The default is p, meaning plain text. For any other value, a directory is created containing two files, globals.data and map.dat. The first contains SQL

pgsql: Add more TAP tests for pg_dumpall

2025-04-04 Thread Andrew Dunstan
Add more TAP tests for pg_dumpall Author: Matheus Alcantara Author: Mahendra Singh Thalor Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/289f74d0cb247ebdb443fec65fb2500531c961b4 Modified Files -- src/bin/pg_dump/meson.build | 1 + src/bin/pg

pgsql: add new list type simple_oid_string_list to fe-utils/simple_list

2025-04-04 Thread Andrew Dunstan
add new list type simple_oid_string_list to fe-utils/simple_list This type contains both an oid and a string. This will be used in forthcoming changes to pg_restore. Author: Andrew Dunstan Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2b69afbe50d5e39cc7d9703b3a

pgsql: Move common pg_dump code related to connections to a new file

2025-04-04 Thread Andrew Dunstan
Move common pg_dump code related to connections to a new file ConnectDatabase is used by pg_dumpall, pg_restore and pg_dump so move common code to new file. new file name: connectdb.c Author:Mahendra Singh Thalor Branch -- master Details --- https://git.postgresql.org/pg/commitdif

Re: pgsql: pg_dump: Retrieve attribute statistics in batches.

2025-04-04 Thread Nathan Bossart
On Fri, Apr 04, 2025 at 07:51:56PM +, Nathan Bossart wrote: > pg_dump: Retrieve attribute statistics in batches. > > [...] > > Discussion: > https://postgr.es/m/CADkLM%3Dc%2Br05srPy9w%2B-%2BnbmLEo15dKXYQ03Q_xyK%2BriJerigLQ%40mail.gmail.com I see the buildfarm failure and am working on a fix.

pgsql: Remove unused function parameters in pg_backup_archiver.c.

2025-04-04 Thread Nathan Bossart
Remove unused function parameters in pg_backup_archiver.c. Thanks to commit 9c02e3a986, which modified some of the changes from commit a0a4601765, we can remove the now-unused ArchiveHandle parameter from _tocEntryRestorePass() and move_to_ready_heap(). Reviewed-by: Jeff Davis Discussion: https:

pgsql: pg_dump: Reduce memory usage of dumps with statistics.

2025-04-04 Thread Nathan Bossart
pg_dump: Reduce memory usage of dumps with statistics. Right now, pg_dump stores all generated commands for statistics in memory. These commands can be quite large and therefore can significantly increase pg_dump's memory footprint. To fix, wait until we are about to write out the commands befor

pgsql: pg_dump: Retrieve attribute statistics in batches.

2025-04-04 Thread Nathan Bossart
pg_dump: Retrieve attribute statistics in batches. Currently, pg_dump gathers attribute statistics with a query per relation, which can cause pg_dump to take significantly longer, especially when there are many relations. This commit addresses this by teaching pg_dump to gather attribute statisti

pgsql: Skip second WriteToc() call for custom-format dumps without data

2025-04-04 Thread Nathan Bossart
Skip second WriteToc() call for custom-format dumps without data. Presently, "pg_dump --format=custom" calls WriteToc() twice. The second call updates the data offset information, which allegedly makes parallel pg_restore significantly faster. However, if we're not dumping any data, there are no

pgsql: Remove superfluous autoprewarm check

2025-04-04 Thread Melanie Plageman
Remove superfluous autoprewarm check autoprewarm_database_main() prewarms blocks from the same database. It is passed an array of sorted BlockInfoRecords and a start and stop index into the array. The range represented should include only blocks belonging to global objects or blocks from a single

pgsql: Refactor autoprewarm_database_main() in preparation for read str

2025-04-04 Thread Melanie Plageman
Refactor autoprewarm_database_main() in preparation for read stream Autoprewarm prewarms blocks from a dump file representing the contents of shared buffers at the time it was dumped. It uses a sorted array of BlockInfoRecords, each representing a block from one of the cluster's databases and tabl

pgsql: Use streaming read I/O in autoprewarm

2025-04-04 Thread Melanie Plageman
Use streaming read I/O in autoprewarm Make a read stream for each valid fork of each valid relation represented in the autoprewarm dump file and prewarm those blocks through the read stream API instead of by directly invoking ReadBuffer(). Co-authored-by: Nazir Bilal Yavuz Co-authored-by: Melani

pgsql: Update Unicode data to Unicode 16.0.0

2025-04-04 Thread Peter Eisentraut
Update Unicode data to Unicode 16.0.0 Reviewed-by: Jeff Davis Discussion: https://www.postgresql.org/message-id/flat/146349e4-4687-4321-91af-f23557249...@eisentraut.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/82a46cca99fa967cc3c6c2cf55254dd44eebbcfd Modif

pgsql: Change one loop in ATRewriteTable to use 1-based attnums

2025-04-04 Thread Álvaro Herrera
Change one loop in ATRewriteTable to use 1-based attnums All TupleDescAttr() calls in tablecmds.c that aren't in loops across all attributes use AttrNumber-style indexes (1-based); there was only one place in ATRewriteTable that was stashing 0-based indexes in a list for later processing. Switch

pgsql: Fix code comment

2025-04-04 Thread Peter Eisentraut
Fix code comment The changes made in commit d2b4b4c2259 contained incorrect comments: They said that certain forward declarations were necessary to "avoid including pathnodes.h here", but the file is itself pathnodes.h! So change the comment to just say it's a forward declaration in one case, and

pgsql: Need to do CommandCounterIncrement after StoreAttrMissingVal.

2025-04-04 Thread Tom Lane
Need to do CommandCounterIncrement after StoreAttrMissingVal. Without this, an additional change to the same pg_attribute row within the same command will fail. This is possible at least with ALTER TABLE ADD COLUMN on a multiple-inheritance-pathway structure. (Another potential hazard is that imm

pgsql: bufmgr: Improve stats when a buffer is read in concurrently

2025-04-04 Thread Andres Freund
bufmgr: Improve stats when a buffer is read in concurrently Previously we would have the following inaccuracies when a backend tried to read in a buffer, but that buffer was read in concurrently by another backend: - the read IO was double-counted in the global buffer access stats (pgBufferUsage)

pgsql: Fix plpgsql's handling of simple expressions in scrollable curso

2025-04-04 Thread Tom Lane
Fix plpgsql's handling of simple expressions in scrollable cursors. exec_save_simple_expr did not account for the possibility that standard_planner would stick a Materialize node atop the plan of even a simple Result, if CURSOR_OPT_SCROLL is set. This led to an "unexpected plan node type" error.

pgsql: Avoid extra index searches through preprocessing.

2025-04-04 Thread Peter Geoghegan
Avoid extra index searches through preprocessing. Transform low_compare and high_compare nbtree skip array inequalities (with opclasses that offer skip support) in such a way as to allow _bt_first to consistently apply later keys when it descends the tree. This can lower the number of index search

pgsql: Improve nbtree skip scan primitive scan scheduling.

2025-04-04 Thread Peter Geoghegan
Improve nbtree skip scan primitive scan scheduling. Don't allow nbtree scans with skip arrays to end any primitive scan on its first leaf page without giving some consideration to how many times the scan's arrays advanced while changing at least one skip array (though continue not caring about the

pgsql: pg_recvlogical: Add --failover option.

2025-04-04 Thread Masahiko Sawada
pg_recvlogical: Add --failover option. This new option instructs pg_recvlogical to create the logical replication slot with the failover option enabled. It can be used in conjunction with the --create-slot option. Author: Hayato Kuroda Reviewed-by: Michael Banck Reviewed-by: Masahiko Sawada Di

pgsql: Oversight in commit b81ffa13e3.

2025-04-04 Thread Jeff Davis
Oversight in commit b81ffa13e3. Should warn if a materialized view may be affected, as well. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3556c89321e8baa2242288bd4f015efd1e9d6be0 Modified Files -- src/bin/pg_upgrade/check.c | 4 1 file changed, 4

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate

2025-04-04 Thread Alexander Korotkov
Convert 'x IN (VALUES ...)' to 'x = ANY ...' then appropriate This commit implements the automatic conversion of 'x IN (VALUES ...)' into ScalarArrayOpExpr. That simplifies the query tree, eliminating the appearance of an unnecessary join. Since VALUES describes a relational table, and the value

pgsql: Add nbtree skip scan optimization.

2025-04-04 Thread Peter Geoghegan
Add nbtree skip scan optimization. Teach nbtree multi-column index scans to opportunistically skip over irrelevant sections of the index given a query with no "=" conditions on one or more prefix index columns. When nbtree is passed input scan keys derived from a predicate "WHERE b = 5", new nbtr

pgsql: Restrict copying of invalidated replication slots.

2025-04-04 Thread Masahiko Sawada
Restrict copying of invalidated replication slots. Previously, invalidated logical and physical replication slots could be copied using the pg_copy_logical_replication_slot and pg_copy_physical_replication_slot functions. Replication slots that were invalidated for reasons other than WAL removal r

pgsql: Further optimize nbtree search scan key comparisons.

2025-04-04 Thread Peter Geoghegan
Further optimize nbtree search scan key comparisons. Postgres 17 commit e0b1ee17 added two complementary optimizations to nbtree: the "prechecked" and "firstmatch" optimizations. _bt_readpage was made to avoid needlessly evaluating keys that are guaranteed to be satisfied by applying page-level c

pgsql: Fix logical decoding test to correctly check slot removal on sta

2025-04-04 Thread Fujii Masao
Fix logical decoding test to correctly check slot removal on standby. The regression test for logical decoding verifies whether a logical slot is correctly dropped on a standby when its associated database is dropped. However, the test mistakenly retrieved slot information from the primary instead

pgsql: docs: Clarify that NULL arg to set_config() means reset to defau

2025-04-04 Thread Heikki Linnakangas
docs: Clarify that NULL arg to set_config() means reset to default Author: David G. Johnston Reviewed-by: Zhang Mingli Discussion: https://www.postgresql.org/message-id/cakfquwy0sk6jdcci1vjx6xsztrxggevey-grkenzx%2b3czpy...@mail.gmail.com Branch -- master Details --- https://git.postgr

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Stabilize regression test from c0962a113.

2025-04-04 Thread Tom Lane
Stabilize regression test from c0962a113. Per buildfarm. Co-authored-by: Alena Rybakina Co-authored-by: Tom Lane Discussion: https://postgr.es/m/srnuqlttuimzmvoulhsrbgvj4vnul6b65osswvua7sfkqsvmuy@yg7apybpxp34 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/3ba2c

pgsql: Fix autoprewarm neglect of tablespaces

2025-04-04 Thread Melanie Plageman
Fix autoprewarm neglect of tablespaces While prewarming blocks from a dump file, autoprewarm_database_main() mistakenly ignored tablespace when detecting the beginning of the next relation to prewarm. Because RelFileNumbers are only unique within a tablespace, autoprewarm could miss prewarming blo

pgsql: Fix crash/valgrind error

2025-04-04 Thread Peter Eisentraut
Fix crash/valgrind error Fix for commit 9ef1851685b: We have to skip indexes where sortopfamily is NULL. This takes the place of the previous btree check. Detected by valgrind on the buildfarm. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/ee1ae8b99f964ca1edc038

pgsql: Add commit e1a8b1ad58 to .git-blame-ignore-revs.

2025-04-04 Thread Nathan Bossart
Add commit e1a8b1ad58 to .git-blame-ignore-revs. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/742317a80f89a5d6476c20b3d07330f3ec0d4357 Modified Files -- .git-blame-ignore-revs | 3 +++ 1 file changed, 3 insertions(+)

pgsql: Re-pgindent pg_largeobject.c after commit 0d6c477664.

2025-04-04 Thread Nathan Bossart
Re-pgindent pg_largeobject.c after commit 0d6c477664. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/e1a8b1ad587112e67fdc5aa7b388631dde4dbdda Modified Files -- src/backend/catalog/pg_largeobject.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

pgsql: Extract make_SAOP_expr() function from match_orclause_to_indexco

2025-04-04 Thread Alexander Korotkov
Extract make_SAOP_expr() function from match_orclause_to_indexcol() This commit extracts the code to generate ScalarArrayOpExpr on top of the list of expressions from match_orclause_to_indexcol() into a separate function make_SAOP_expr(). This function was extracted to be used in optimization for

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Relax assertion in finding correct GiST parent

2025-04-04 Thread Heikki Linnakangas
Relax assertion in finding correct GiST parent Commit 28d3c2ddcf introduced an assertion that if the memorized downlink location in the insertion stack isn't valid, the parent's LSN should've changed too. Turns out that was too strict. In gistFindCorrectParent(), if we walk right, we update the pa

pgsql: Use standard die() signal handler in walreceiver

2025-04-04 Thread Heikki Linnakangas
Use standard die() signal handler in walreceiver This gets rid of the bespoken ProcessWalRcvInterrupts() function, which lets walreceiver terminate at any CHECK_FOR_INTERRUPTS() call. And it's less code anyway. We can now use the standard libpqsrv_connect_params() libpq wrapper from libpq-be-fe-h

pgsql: Allow "COPY table TO" command to copy rows from materialized vie

2025-04-04 Thread Fujii Masao
Allow "COPY table TO" command to copy rows from materialized views. Previously, "COPY table TO" command worked only with plain tables and did not support materialized views, even when they were populated and had physical storage. To copy rows from materialized views, "COPY (query) TO" command had

pgsql: Support non-btree indexes in get_actual_variable_range()

2025-04-04 Thread Peter Eisentraut
Support non-btree indexes in get_actual_variable_range() This was previously not supported because the btree strategy numbers were hardcoded. Now we can support this for any index that has the required strategy mapping support and the required operators. If an index scan used for get_actual_vari

pgsql: Extend ALTER DEFAULT PRIVILEGES to define default privileges for

2025-04-04 Thread Fujii Masao
Extend ALTER DEFAULT PRIVILEGES to define default privileges for large objects. Previously, ALTER DEFAULT PRIVILEGES did not support large objects. This meant that to grant privileges to users other than the owner, permissions had to be manually assigned each time a large object was created, which

pgsql: Convert PathKey to use CompareType

2025-04-04 Thread Peter Eisentraut
Convert PathKey to use CompareType Change the PathKey struct to use CompareType to record the sort direction instead of hardcoding btree strategy numbers. The CompareType is then converted to the index-type-specific strategy when the plan is created. This reduces the number of places btree strat

pgsql: doc: Clarify the system value for sslrootcert

2025-04-04 Thread Daniel Gustafsson
doc: Clarify the system value for sslrootcert The documentation for the special value "system" for sslrootcert could be misinterpreted to mean the default operating system CA store, which it may be, but it's defined to be the default CA store of the SSL lib used. Backpatch down to v16 where suppo

pgsql: doc: Clarify the system value for sslrootcert

2025-04-04 Thread Daniel Gustafsson
doc: Clarify the system value for sslrootcert The documentation for the special value "system" for sslrootcert could be misinterpreted to mean the default operating system CA store, which it may be, but it's defined to be the default CA store of the SSL lib used. Backpatch down to v16 where suppo

pgsql: doc: Clarify the system value for sslrootcert

2025-04-04 Thread Daniel Gustafsson
doc: Clarify the system value for sslrootcert The documentation for the special value "system" for sslrootcert could be misinterpreted to mean the default operating system CA store, which it may be, but it's defined to be the default CA store of the SSL lib used. Backpatch down to v16 where suppo

pgsql: Fix logical decoding regression tests to correctly check slot ex

2025-04-04 Thread Fujii Masao
Fix logical decoding regression tests to correctly check slot existence. The regression tests for logical decoding verify whether a logical slot exists or has been dropped. Previously, these tests attempted to retrieve "slot_name" from the result of slot(), but since "slot_name" was not included i