pgsql: Add parameter "connstr" to PostgreSQL::Test::Cluster::background

2024-09-19 Thread Michael Paquier
Add parameter "connstr" to PostgreSQL::Test::Cluster::background_psql Like for Cluster::psql, this can be handy to force the use of a connection string with some values overriden, like a "host". Author: Aidar Imamov Discussion: https://postgr.es/m/ecacb079efc533aed3c234cbcb5b0...@postgrespro.ru

pgsql: psql: Add tests for repeated calls of \bind[_named]

2024-09-19 Thread Michael Paquier
nnema-Nio, Michael Paquier Discussion: https://postgr.es/m/CAGECzQSTE7CoM=gst56xj8povjapr09+7jjtwqtc40pgety...@mail.gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/91287b5f5da324ac24678f556962e1b95e52240c Modified Files -- src/test/regress/exp

pgsql: psql: Fix memory leak with repeated calls of \bind

2024-09-19 Thread Michael Paquier
psql: Fix memory leak with repeated calls of \bind Calling \bind repeatedly would cause the memory allocated for the list of bind parameters to be leaked after each call, as the list is reset when beginning a single call. This issue is fixed by making the cleanup of the bind parameter list more a

pgsql: psql: Fix memory leak with repeated calls of \bind

2024-09-19 Thread Michael Paquier
psql: Fix memory leak with repeated calls of \bind Calling \bind repeatedly would cause the memory allocated for the list of bind parameters to be leaked after each call, as the list is reset when beginning a single call. This issue is fixed by making the cleanup of the bind parameter list more a

pgsql: psql: Clean up more aggressively state of \bind[_named], \parse

2024-09-18 Thread Michael Paquier
psql: Clean up more aggressively state of \bind[_named], \parse and \close This fixes a couple of issues with the psql meta-commands mentioned above when called repeatedly: - The statement name is reset for each call. If a command errors out, its send_mode would still be set, causing an incorrect

pgsql: Introduce ATT_PARTITIONED_TABLE in tablecmds.c

2024-09-18 Thread Michael Paquier
ch to restrict SET LOGGED/UNLOGGED for partitioned tables is under discussion, and introducing ATT_PARTITIONED_TABLE makes subcommand restrictions for partitioned tables easier to deal with, so let's add one. There is no functional change. Author: Michael Paquier Reviewed-by: Nathan Bossart

pgsql: Add some sanity checks in executor for query ID reporting

2024-09-17 Thread Michael Paquier
Add some sanity checks in executor for query ID reporting This commit adds three sanity checks in code paths of the executor where it is possible to use hooks, checking that a query ID is reported in pg_stat_activity if compute_query_id is enabled: - ExecutorRun() - ExecutorFinish() - ExecutorEnd(

pgsql: Extend PgStat_HashKey.objid from 4 to 8 bytes

2024-09-17 Thread Michael Paquier
Extend PgStat_HashKey.objid from 4 to 8 bytes This opens the possibility to define keys for more types of statistics kinds in PgStat_HashKey, the first case being 8-byte query IDs for statistics like pg_stat_statements. This increases the size of PgStat_HashKey from 12 to 16 bytes, while PgStatSh

pgsql: Add missing query ID reporting in extended query protocol

2024-09-17 Thread Michael Paquier
Haas. This uses psql's \bind metacommand, hence this part is backpatched down to v16. Reported-by: Kaido Vaikla, Erik Wienhold Author: Sami Imseih Reviewed-by: Jian He, Andrei Lepikhov, Michael Paquier Discussion: https://postgr.es/m/ca+427g8diw3az6popvgkpbqk97oubdf18vlihfesea2juk3...@mail

pgsql: Add missing query ID reporting in extended query protocol

2024-09-17 Thread Michael Paquier
Haas. This uses psql's \bind metacommand, hence this part is backpatched down to v16. Reported-by: Kaido Vaikla, Erik Wienhold Author: Sami Imseih Reviewed-by: Jian He, Andrei Lepikhov, Michael Paquier Discussion: https://postgr.es/m/ca+427g8diw3az6popvgkpbqk97oubdf18vlihfesea2juk3...@mail

pgsql: Add missing query ID reporting in extended query protocol

2024-09-17 Thread Michael Paquier
Haas. This uses psql's \bind metacommand, hence this part is backpatched down to v16. Reported-by: Kaido Vaikla, Erik Wienhold Author: Sami Imseih Reviewed-by: Jian He, Andrei Lepikhov, Michael Paquier Discussion: https://postgr.es/m/ca+427g8diw3az6popvgkpbqk97oubdf18vlihfesea2juk3...@mail

pgsql: Add missing query ID reporting in extended query protocol

2024-09-17 Thread Michael Paquier
Haas. This uses psql's \bind metacommand, hence this part is backpatched down to v16. Reported-by: Kaido Vaikla, Erik Wienhold Author: Sami Imseih Reviewed-by: Jian He, Andrei Lepikhov, Michael Paquier Discussion: https://postgr.es/m/ca+427g8diw3az6popvgkpbqk97oubdf18vlihfesea2juk3...@mail

pgsql: Add missing query ID reporting in extended query protocol

2024-09-17 Thread Michael Paquier
Haas. This uses psql's \bind metacommand, hence this part is backpatched down to v16. Reported-by: Kaido Vaikla, Erik Wienhold Author: Sami Imseih Reviewed-by: Jian He, Andrei Lepikhov, Michael Paquier Discussion: https://postgr.es/m/ca+427g8diw3az6popvgkpbqk97oubdf18vlihfesea2juk3...@mail

pgsql: pg_stat_statements: Add tests with extended query protocol

2024-09-12 Thread Michael Paquier
pg_stat_statements: Add tests with extended query protocol There are currently no tests in the tree checking that queries using the extended query protocol are able to map with their query ID. This can be achieved for some paths of the extended query protocol with the psql meta-commands \bind or

pgsql: Move logic related to WAL replay of Heap/Heap2 into its own file

2024-09-11 Thread Michael Paquier
, cutting half of the headers required. Author: Li Yong Reviewed-by: Sutou Kouhei, Michael Paquier Discussion: https://postgr.es/m/efe55e65-d7bd-4c6a-b630-91f43fd07...@ebay.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/00c76cf21c42c17e60e73a87dea0d1b4e234d9da

pgsql: Remove emode argument from XLogFileRead() and XLogFileReadAnyTLI

2024-09-09 Thread Michael Paquier
Remove emode argument from XLogFileRead() and XLogFileReadAnyTLI() This change makes the code slightly easier to reason about, because there is actually no need to know if a specific caller of one of these routines should fail hard on a PANIC, or just let it go through with a DEBUG2. The only cal

pgsql: Fix waits of REINDEX CONCURRENTLY for indexes with predicates or

2024-09-08 Thread Michael Paquier
Fix waits of REINDEX CONCURRENTLY for indexes with predicates or expressions As introduced by f9900df5f94, a REINDEX CONCURRENTLY job done for an index with predicates or expressions would set PROC_IN_SAFE_IC in its MyProc->statusFlags, causing it to be ignored by other concurrent operations. Suc

pgsql: Fix waits of REINDEX CONCURRENTLY for indexes with predicates or

2024-09-08 Thread Michael Paquier
Fix waits of REINDEX CONCURRENTLY for indexes with predicates or expressions As introduced by f9900df5f94, a REINDEX CONCURRENTLY job done for an index with predicates or expressions would set PROC_IN_SAFE_IC in its MyProc->statusFlags, causing it to be ignored by other concurrent operations. Suc

pgsql: Fix waits of REINDEX CONCURRENTLY for indexes with predicates or

2024-09-08 Thread Michael Paquier
Fix waits of REINDEX CONCURRENTLY for indexes with predicates or expressions As introduced by f9900df5f94, a REINDEX CONCURRENTLY job done for an index with predicates or expressions would set PROC_IN_SAFE_IC in its MyProc->statusFlags, causing it to be ignored by other concurrent operations. Suc

pgsql: Fix waits of REINDEX CONCURRENTLY for indexes with predicates or

2024-09-08 Thread Michael Paquier
Fix waits of REINDEX CONCURRENTLY for indexes with predicates or expressions As introduced by f9900df5f94, a REINDEX CONCURRENTLY job done for an index with predicates or expressions would set PROC_IN_SAFE_IC in its MyProc->statusFlags, causing it to be ignored by other concurrent operations. Suc

pgsql: Fix waits of REINDEX CONCURRENTLY for indexes with predicates or

2024-09-08 Thread Michael Paquier
Fix waits of REINDEX CONCURRENTLY for indexes with predicates or expressions As introduced by f9900df5f94, a REINDEX CONCURRENTLY job done for an index with predicates or expressions would set PROC_IN_SAFE_IC in its MyProc->statusFlags, causing it to be ignored by other concurrent operations. Suc

pgsql: Add callbacks to control flush of fixed-numbered stats

2024-09-08 Thread Michael Paquier
Add callbacks to control flush of fixed-numbered stats This commit adds two callbacks in pgstats to have a better control of the flush timing of pgstat_report_stat(), whose operation depends on the three PGSTAT_*_INTERVAL variables: - have_fixed_pending_cb(), to check if a stats kind has any pendi

pgsql: Update extension lookup routines to use the syscache

2024-09-07 Thread Michael Paquier
Update extension lookup routines to use the syscache The following routines are changed to use the syscache entries added for pg_extension in 490f869d92e5: - get_extension_oid() - get_extension_name() - get_extension_schema() A catalog scan is costly and could easily lead to a noticeable performa

pgsql: Add callback for backend initialization in pgstats

2024-09-05 Thread Michael Paquier
Add callback for backend initialization in pgstats pgstat_initialize() is currently used by the WAL stats as a code path to take some custom actions when a backend starts. A callback is added to generalize the concept so as all stats kinds can do the same, for builtin and custom kinds, if set. R

pgsql: Fix two NULL pointer dereferences when reading custom pgstats fr

2024-09-04 Thread Michael Paquier
Fix two NULL pointer dereferences when reading custom pgstats from file There were two spots in pgstat_read_statsfile() where is was possible to finish with a null-pointer-dereference crash for custom pgstats kinds: - When reading stats for a fixed-numbered stats entry. - When reading a variable s

pgsql: Check availability of module injection_points in TAP tests

2024-09-04 Thread Michael Paquier
Check availability of module injection_points in TAP tests This fixes defects with installcheck for TAP tests that expect the module injection_points to exist in an installation, but the contents of src/test/modules are not installed by default with installcheck. This would cause, for example, fa

pgsql: Unify some error messages to ease work of translators

2024-09-03 Thread Michael Paquier
Unify some error messages to ease work of translators This commit updates a couple of error messages around control file data, GUCs and server settings, unifying to the same message where possible. This reduces the translation burden a bit. Author: Peter Smith Discussion: https://postgr.es/m/cah

pgsql: Apply more quoting to GUC names in messages

2024-09-03 Thread Michael Paquier
Apply more quoting to GUC names in messages This is a continuation of 17974ec25946. More quotes are applied to GUC names in error messages and hints, taking care of what seems to be all the remaining holes currently in the tree for the GUCs. Author: Peter Smith Discussion: https://postgr.es/m/c

pgsql: Fix inconsistent LWLock tranche name "CommitTsSLRU"

2024-09-03 Thread Michael Paquier
Fix inconsistent LWLock tranche name "CommitTsSLRU" This term was using an inconsistent casing between the code and the documentation, using "CommitTsSLRU" in wait_event_names.txt and "CommitTSSLRU" in the code. Let's update the term in the code to reflect what's in the documentation, "CommitTs"

pgsql: Fix inconsistent LWLock tranche name "CommitTsSLRU"

2024-09-03 Thread Michael Paquier
Fix inconsistent LWLock tranche name "CommitTsSLRU" This term was using an inconsistent casing between the code and the documentation, using "CommitTsSLRU" in wait_event_names.txt and "CommitTSSLRU" in the code. Let's update the term in the code to reflect what's in the documentation, "CommitTs"

pgsql: Avoid installcheck failure in TAP tests using injection_points

2024-09-03 Thread Michael Paquier
Avoid installcheck failure in TAP tests using injection_points These tests depend on the test module injection_points to be installed, but it may not be available as the contents of src/test/modules/ are not installed by default. This commit adds a workaround based on a scan of pg_available_exten

pgsql: Avoid installcheck failure in TAP tests using injection_points

2024-09-03 Thread Michael Paquier
Avoid installcheck failure in TAP tests using injection_points These tests depend on the test module injection_points to be installed, but it may not be available as the contents of src/test/modules/ are not installed by default. This commit adds a workaround based on a scan of pg_available_exten

pgsql: Simplify makefiles exporting twice enable_injection_points

2024-09-03 Thread Michael Paquier
Simplify makefiles exporting twice enable_injection_points This is confusing, as it exports twice the same variable. Oversight in 6782709df81f that has spread in more places afterwards. Reported-by: Alvaro Herrera, Tom Lane Discussion: https://postgr.es/m/202408201630.mn6vbohjh7hh@alvherre.pgsql

pgsql: Simplify makefiles exporting twice enable_injection_points

2024-09-03 Thread Michael Paquier
Simplify makefiles exporting twice enable_injection_points This is confusing, as it exports twice the same variable. Oversight in 6782709df81f that has spread in more places afterwards. Reported-by: Alvaro Herrera, Tom Lane Discussion: https://postgr.es/m/202408201630.mn6vbohjh7hh@alvherre.pgsql

pgsql: Fix typos and grammar in code comments and docs

2024-09-02 Thread Michael Paquier
Fix typos and grammar in code comments and docs Author: Alexander Lakhin Discussion: https://postgr.es/m/f7e514cf-2446-21f1-a5d2-8c089a6e2...@gmail.com Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4236825197e8b7f26f8fd2ce0ac287a696e7264e Modified Files -

pgsql: Define PG_TBLSPC_DIR for path pg_tblspc/ in data folder

2024-09-02 Thread Michael Paquier
duction of PG_TBLSPC_DIR_SLASH, required in two places for recovery and backups. Author: Bertrand Drouvot Reviewed-by: Ashutosh Bapat, Álvaro Herrera, Yugo Nagata, Michael Paquier Discussion: https://postgr.es/m/zryvvjqs9snv1...@ip-10-97-1-34.eu-west-3.compute.internal Branch -- master Details ---

Re: pgsql: Implement pg_wal_replay_wait() stored procedure

2024-09-01 Thread Michael Paquier
On Mon, Sep 02, 2024 at 02:55:50AM +0300, Alexander Korotkov wrote: > Could you, please, check the attached patch? The patch moving the code looks correct at quick glance. Now, I've been staring at this line, wondering why this is required while WaitForLSNReplay() does not return any status: +

Re: pgsql: Implement pg_wal_replay_wait() stored procedure

2024-09-01 Thread Michael Paquier
On Sun, Sep 01, 2024 at 10:35:27PM +0300, Alexander Korotkov wrote: > This path hasn't changes since the patch revision when it was a > utility command. I agree that this doesn't look like proper path for > stored procedure. But I don't think src/backend/utils/adt is > appropriate path either, be

pgsql: Rename enum labels of PG_Locale_Strategy

2024-09-01 Thread Michael Paquier
Rename enum labels of PG_Locale_Strategy PG_REGEX_BUILTIN was added in f69319f2f1fb but it did not follow the same pattern as the previous labels, i.e. PG_LOCALE_*. In addition to this, the two libc strategies did not include in the name that they were related to this library. The enum labels ar

pgsql: Define PG_LOGICAL_DIR for path pg_logical/ in data folder

2024-08-29 Thread Michael Paquier
Define PG_LOGICAL_DIR for path pg_logical/ in data folder This is similar to 2065ddf5e34c, but this time for pg_logical/ itself and its contents, like the paths for snapshots, mappings or origin checkpoints. Author: Bertrand Drouvot Reviewed-by: Ashutosh Bapat, Yugo Nagata, Michael Paquier

pgsql: Define PG_REPLSLOT_DIR for path pg_replslot/ in data folder

2024-08-29 Thread Michael Paquier
e. Author: Bertrand Drouvot Reviewed-by: Ashutosh Bapat, Yugo Nagata, Michael Paquier Discussion: https://postgr.es/m/zryvvjqs9snv1...@ip-10-97-1-34.eu-west-3.compute.internal Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2065ddf5e34ce098f549c4279ee3ab33c188a764

pgsql: Rename pg_sequence_read_tuple() to pg_get_sequence_data()

2024-08-29 Thread Michael Paquier
Rename pg_sequence_read_tuple() to pg_get_sequence_data() This commit removes log_cnt from the tuple returned by the SQL function. This field is an internal counter that tracks when a WAL record should be generated for a sequence, and it is reset each time the sequence is restored or recovered. I

pgsql: Refactor some code for ALTER TABLE SET LOGGED/UNLOGGED in tablec

2024-08-28 Thread Michael Paquier
Refactor some code for ALTER TABLE SET LOGGED/UNLOGGED in tablecmds.c Both sub-commands use the same routine to switch the relpersistence of a relation, duplicated the same checks, and used a style inconsistent with access methods and tablespaces. SET LOGEED/UNLOGGED is refactored to avoid any du

Re: pgsql: Add injection-point test for new multixact CV usage

2024-08-22 Thread Michael Paquier
On Thu, Aug 22, 2024 at 01:24:33AM +0300, Alexander Korotkov wrote: > It seems that header files aren't alphabetically ordered here. Thanks. I've fixed this one in 94a3373ac5c3 while tweaking the area. -- Michael signature.asc Description: PGP signature

pgsql: Rework new SLRU test with injection points

2024-08-22 Thread Michael Paquier
in multixact.c, as pointed out by Alexander Korotkov. This was an oversight in 768a9fd5535f. Per discussion with Álvaro Herrera. Author: Michael Paquier Discussion: https://postgr.es/m/zsunjulsobnaz...@paquier.xyz Discussion: https://postgr.es/m/CAPpHfduzaBz7KMhwuVOZMTpG=jnipg4auosxpzcxzydmzq_

pgsql: injection_point: Add injection_points.stats

2024-08-22 Thread Michael Paquier
will need to be loaded with shared_preload_libraries there. Per discussion with Álvaro Herrera. Author: Michael Paquier Discussion: https://postgr.es/m/zsunjulsobnaz...@paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff

pgsql: injection_points: Add initialization of shmem state when loading

2024-08-22 Thread Michael Paquier
its allocation. If the module is not loaded at startup, it will use a GetNamedDSMSegment() instead to initialize its shmem state on-the-fly. Per discussion with Álvaro Herrera. Author: Michael Paquier Discussion: https://postgr.es/m/zsunjulsobnaz...@paquier.xyz Branch -- master Details

pgsql: psql: Add more meta-commands able to use the extended protocol

2024-08-22 Thread Michael Paquier
so as bind_flag is replaced by an enum in _psqlSettings that tracks the type of libpq routine to execute, based on the meta-command involved, with the default being PQsendQuery(). This refactoring piece has been written by me, while Anthonin has implemented the rest. Author: Anthonin Bonnefoy, Mi

pgsql: Create syscache entries for pg_extension

2024-08-21 Thread Michael Paquier
Create syscache entries for pg_extension Two syscache identifiers are added for extension names and OIDs. Shared libraries of extensions might want to invalidate or update their own caches whenever a CREATE, ALTER or DROP EXTENSION command is run for their extension (in any backend). Right now t

pgsql: Remove _PG_fini()

2024-08-20 Thread Michael Paquier
Remove _PG_fini() ab02d702ef08 has removed from the backend the code able to support the unloading of modules, because this has never worked. This removes the last references to _PG_fini(), that could be used as a callback for modules to manipulate the stack when unloading a library. The test mo

Re: pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-18 Thread Michael Paquier
On Thu, Aug 15, 2024 at 03:40:12AM +0300, Alexander Korotkov wrote: > On Thu, Aug 15, 2024 at 2:07 AM Michael Paquier wrote: >> I’d like to take a look at what you have here to get an opinion. Could you >> wait for a few days? > > Sure, NP. My apologies for the delay. I h

pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-18 Thread Michael Paquier
Fix more holes with SLRU code in need of int64 for segment numbers This is a continuation of c9e24573905b, containing changes included into the proposed patch that have been missed in the actual commit. I have managed to miss these diffs while doing a rebase of the original patch. Thanks to Noah

pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-18 Thread Michael Paquier
Fix more holes with SLRU code in need of int64 for segment numbers This is a continuation of c9e24573905b, containing changes included into the proposed patch that have been missed in the actual commit. I have managed to miss these diffs while doing a rebase of the original patch. Thanks to Noah

pgsql: injection_points: Add stats for point caching and loading

2024-08-18 Thread Michael Paquier
injection_points: Add stats for point caching and loading This adds two counters to the fixed-numbered stats of injection points to track the number of times injection points have been cached and loaded from the cache, as of the additions coming from a0a5869a8598 and 4b211003ecc2. These should ha

pgsql: doc: Fix typo in section for custom pgstats

2024-08-18 Thread Michael Paquier
doc: Fix typo in section for custom pgstats Per offline report from Erik Rijkers. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/a5f4ff6c806d359b3d2c28cf902047c830acc132 Modified Files -- doc/src/sgml/xfunc.sgml | 2 +- 1 file changed, 1 insertion(+), 1

Re: pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-14 Thread Michael Paquier
> On Aug 15, 2024, at 6:26, Alexander Korotkov wrote: > I mean, I'd like to push it if you don't object. > If you object and like to push it yourself, feel free to use this patch. I’d like to take a look at what you have here to get an opinion. Could you wait for a few days? -- Michael

Re: pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-12 Thread Michael Paquier
> On Aug 13, 2024, at 6:35, Alexander Korotkov wrote:. > > As pointed by Noah Misch [1], unlike the commit the patch [2] also > changed segment-returning functions to return int64. Thus, in the > patch output formats make much more sense, because they match the > input data types. Michael, ar

Re: pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-08-07 Thread Michael Paquier
 > On Aug 8, 2024, at 5:05, Alexander Korotkov wrote: > On Wed, Aug 7, 2024 at 10:52 PM Peter Eisentraut > wrote: >> It looks like the commit I'm talking about here is a subset of v55-0001 >> from that thread? > > Yes, looks like this. > >> So why is some of this being committed now into v17

pgsql: Remove unnecessary declaration of heapam_methods

2024-08-06 Thread Michael Paquier
Remove unnecessary declaration of heapam_methods This overlaps with the declaration at the end of heapam_handler.c that lists all the callback routines for the heap table AM. Author: Japin Li Discussion: https://postgr.es/m/me0p300mb04459456d5c4e70d48116896b6...@me0p300mb0445.ausp300.prod.outloo

pgsql: injection_points: Add some fixed-numbered statistics

2024-08-04 Thread Michael Paquier
ustom stats. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m/zmqm9j5eo0i4w...@paquier.xyz Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/f68cd847fa40ead44a786b9c34aff9ccc048004b Modified Files -- src

pgsql: injection_points: Add some cumulative stats for injection points

2024-08-04 Thread Michael Paquier
hash of the injection point name as object ID. A TAP test is added to provide coverage for the new custom cumulative stats APIs, showing the persistency of the data across restarts, for example. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m

pgsql: Add helper routines to retrieve data for custom fixed-numbered p

2024-08-04 Thread Michael Paquier
upcoming commit introducing an example template for custom cumulative stats with fixed-numbered objects will make use of these. I have noticed that this is useful for extension developers while hacking my own example, actually. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot

pgsql: Introduce pluggable APIs for Cumulative Statistics

2024-08-04 Thread Michael Paquier
r fixed-numbered and variable-numbered stats kinds will be added in some follow-up commits, with tests to provide coverage. Some documentation is added to explain how to use this plugin facility. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postg

pgsql: Fix incorrect format placeholders in pgstat.c

2024-08-03 Thread Michael Paquier
Fix incorrect format placeholders in pgstat.c These should have been switched from %d to %u in 3188a4582a8c in the debugging elogs added in ca1ba50fcb6f. PgStat_Kind should never be higher than INT32_MAX, but let's be clean. Issue noticed while hacking more on this area. Branch -- master D

pgsql: Switch PgStat_Kind from an enum to a uint32 type

2024-08-01 Thread Michael Paquier
on it, this commit switches pgstat_is_kind_valid() to use PgStat_Kind rather than an int, to be more consistent with its existing callers. Some loops based on the stats kind IDs are switched to use PgStat_Kind rather than int, for consistency with the new time. Author: Michael Paquier Reviewed-by

pgsql: Add redo LSN to pgstats files

2024-08-01 Thread Michael Paquier
Add redo LSN to pgstats files This is used in the startup process to check that the pgstats file we are reading includes the redo LSN referring to the shutdown checkpoint where it has been written. The redo LSN in the pgstats file needs to match with what the control file has. This is intended t

pgsql: Add more debugging information when failing to read pgstats file

2024-07-29 Thread Michael Paquier
Add more debugging information when failing to read pgstats files This is useful to know which part of a stats file is corrupted when reading it, adding to the server logs a WARNING with details about what could not be read before giving up with the remaining data in the file. Author: Michael

pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-07-26 Thread Michael Paquier
Fix more holes with SLRU code in need of int64 for segment numbers This is a continuation of 3937cadfd438, taking care of more areas I have managed to miss previously. Reported-by: Noah Misch Reviewed-by: Noah Misch Discussion: https://postgr.es/m/20240724130059.1f.nmi...@google.com Backpatch-thr

pgsql: Fix more holes with SLRU code in need of int64 for segment numbe

2024-07-26 Thread Michael Paquier
Fix more holes with SLRU code in need of int64 for segment numbers This is a continuation of 3937cadfd438, taking care of more areas I have managed to miss previously. Reported-by: Noah Misch Reviewed-by: Noah Misch Discussion: https://postgr.es/m/20240724130059.1f.nmi...@google.com Backpatch-thr

pgsql: doc: Decorate psql page with application markup tags

2024-07-24 Thread Michael Paquier
doc: Decorate psql page with application markup tags Noticed while looking at this area of the documentation for a separate patch. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b8aa44fd4f86ed1ba5798ad360423555865c437c Modified Files -- doc/src/sgml/re

pgsql: Use more consistently int64 for page numbers in SLRU-related cod

2024-07-23 Thread Michael Paquier
integers for SLRU page numbers, still forgot about the code paths updated by this commit. Reported-by: Noah Misch Author: Aleksander Alekseev, Michael Paquier Discussion: https://postgr.es/m/20240626002747.dc.nmi...@google.com Backpatch-through: 17 Branch -- master Details --- https

pgsql: Use more consistently int64 for page numbers in SLRU-related cod

2024-07-23 Thread Michael Paquier
integers for SLRU page numbers, still forgot about the code paths updated by this commit. Reported-by: Noah Misch Author: Aleksander Alekseev, Michael Paquier Discussion: https://postgr.es/m/20240626002747.dc.nmi...@google.com Backpatch-through: 17 Branch -- REL_17_STABLE Details

pgsql: Improve comments in slru.{c,h} about segment name format

2024-07-23 Thread Michael Paquier
Improve comments in slru.{c,h} about segment name format slru.h described incorrectly how SLRU segment names are formatted depending on the segment number and if long or short segment names are used. This commit closes the gap with a better description, fitting with the reality. Reported-by: Noa

pgsql: Improve comments in slru.{c,h} about segment name format

2024-07-23 Thread Michael Paquier
Improve comments in slru.{c,h} about segment name format slru.h described incorrectly how SLRU segment names are formatted depending on the segment number and if long or short segment names are used. This commit closes the gap with a better description, fitting with the reality. Reported-by: Noa

pgsql: Add new error code for "file name too long"

2024-07-21 Thread Michael Paquier
Add new error code for "file name too long" This new error code, named file_name_too_long, maps internally to the errno ENAMETOOLONG to produce a proper error code rather than an internal code under errcode_for_file_access(). This error code can be reached with some SQL command patterns, like a s

pgsql: Add more test coverage for jsonpath "$.*" with arrays

2024-07-19 Thread Michael Paquier
Add more test coverage for jsonpath "$.*" with arrays There was no coverage for the code path to unwrap an array before applying ".*" to it, so add tests to provide more coverage for both objects and arrays. This shows, for example, that no results are returned for an array of scalars, and what r

pgsql: Propagate query IDs of utility statements in functions

2024-07-18 Thread Michael Paquier
Propagate query IDs of utility statements in functions For utility statements defined within a function, the query tree is copied to a PlannedStmt as utility commands do not require planning. However, the query ID was missing from the information passed down. This leads to plugins relying on the

pgsql: Propagate query IDs of utility statements in functions

2024-07-18 Thread Michael Paquier
Propagate query IDs of utility statements in functions For utility statements defined within a function, the query tree is copied to a PlannedStmt as utility commands do not require planning. However, the query ID was missing from the information passed down. This leads to plugins relying on the

pgsql: Propagate query IDs of utility statements in functions

2024-07-18 Thread Michael Paquier
Propagate query IDs of utility statements in functions For utility statements defined within a function, the query tree is copied to a PlannedStmt as utility commands do not require planning. However, the query ID was missing from the information passed down. This leads to plugins relying on the

pgsql: Add INJECTION_POINT_CACHED() to run injection points directly fr

2024-07-17 Thread Michael Paquier
to describe in more details how to use this new macro with a load. Some tests are added to the module injection_points based on a new SQL function that acts as a wrapper of INJECTION_POINT_CACHED(). Based on a suggestion from Heikki Linnakangas. Author: Heikki Linnakangas, Michael Paquier

pgsql: Make write of pgstats file durable at shutdown

2024-07-16 Thread Michael Paquier
Make write of pgstats file durable at shutdown This switches the pgstats write code to use durable_rename() rather than rename(). This ensures that the stats file's data is durable when the statistics are written, which is something only happening at shutdown now with the checkpointer doing the j

Re: pgsql: Adjust recently added test for pg_signal_autovacuum role

2024-07-16 Thread Michael Paquier
On Tue, Jul 16, 2024 at 02:08:55PM +, Andrew Dunstan wrote: > Adjust recently added test for pg_signal_autovacuum role > > This test was added by commit d2b74882ca, but fails if > log_error_verbosity is set to verbose. Adjust the regex that checks the > error message to allow for it containing

pgsql: Add tap test for pg_signal_autovacuum role

2024-07-15 Thread Michael Paquier
worker is still alive when sending and processing the signal sent. Author: Anthony Leung, Michael Paquier, Kirill Reshke Reviewed-by: Andrey Borodin, Nathan Bossart Discussion: https://postgr.es/m/caldsspiqpuuqpokf7x0g2qka5ee-3xxt7hijfvshv1bhkdv...@mail.gmail.com Branch -- master Details

pgsql: Add assertion in pgstat_write_statsfile() about processes allowe

2024-07-11 Thread Michael Paquier
Add assertion in pgstat_write_statsfile() about processes allowed This routine can currently only be called from the postmaster in single-user mode or the checkpointer, but there was no sanity check to make sure that this was always the case. This has proved to be useful when hacking the zone (at

pgsql: Improve comment of pgstat_read_statsfile()

2024-07-11 Thread Michael Paquier
Improve comment of pgstat_read_statsfile() The comment at the top of pgstat_read_statsfile() mentioned that the stats are read from the on-disk file into the pgstats dshash. This is incorrect for fix-numbered stats as these are loaded directly into shared memory. This commit simplifies the comme

pgsql: Add a new 'F' entry type for fixed-numbered stats in pgstats fil

2024-07-11 Thread Michael Paquier
Add a new 'F' entry type for fixed-numbered stats in pgstats file This new entry type is used for all the fixed-numbered statistics, making possible support for custom pluggable stats. In short, we need to be able to detect more easily if a stats kind exists or not when reading back its data from

pgsql: Add PgStat_KindInfo.init_shmem_cb

2024-07-10 Thread Michael Paquier
Add PgStat_KindInfo.init_shmem_cb This new callback gives fixed-numbered stats the possibility to take actions based on the area of shared memory allocated for them. This removes from pgstat_shmem.c any knowledge specific to the types of fixed-numbered stats, and the initializations happen in the

pgsql: Extend pg_get_acl() to handle sub-object IDs

2024-07-09 Thread Michael Paquier
Extend pg_get_acl() to handle sub-object IDs This patch modifies the pg_get_acl() function to accept a third argument called "objsubid", bringing it on par with similar functions in this area like pg_describe_object(). This enables the retrieval of ACLs for relation attributes when scanning depen

pgsql: Use pgstat_kind_infos to write fixed shared statistics

2024-07-08 Thread Michael Paquier
Use pgstat_kind_infos to write fixed shared statistics This is similar to 9004abf6206e, but this time for the write part of the stats file. The code is changed so as, rather than referring to individual members of PgStat_Snapshot in an order based on their PgStat_Kind value, a loop based on pgsta

pgsql: Renumber pg_get_acl() in pg_proc.dat

2024-07-07 Thread Michael Paquier
Renumber pg_get_acl() in pg_proc.dat a6417078c414 has introduced as project policy that new features committed during the development cycle should use new OIDs in the [8000,] range. 4564f1cebd43 did not respect that rule, so let's renumber pg_get_acl() to use an OID in the correct range. Bum

pgsql: Re-enable autoruns for cmd.exe on Windows

2024-07-07 Thread Michael Paquier
Re-enable autoruns for cmd.exe on Windows This acts as a revert of b83747a8a65b and 9886744a361b. As pointed out by Noah, HEAD and REL_17_STABLE are in a weird state where the code paths adding /D would limit the spawn of child processes, but we still have code paths where the spawn of more than

pgsql: Re-enable autoruns for for cmd.exe on Windows

2024-07-07 Thread Michael Paquier
Re-enable autoruns for for cmd.exe on Windows This acts as a revert of b83747a8a65b and 9886744a361b. As pointed out by Noah, HEAD and REL_17_STABLE are in a weird state where the code paths adding /D would limit the spawn of child processes, but we still have code paths where the spawn of more t

pgsql: Support loading of injection points

2024-07-05 Thread Michael Paquier
Support loading of injection points This can be used to load an injection point and prewarm the backend-level cache before running it, to avoid issues if the point cannot be loaded due to restrictions in the code path where it would be run, like a critical section where no memory allocation can ha

pgsql: Add pg_get_acl() to get the ACL for a database object

2024-07-04 Thread Michael Paquier
object identification, when joined with pg_depend or pg_shdepend. Original idea by Álvaro Herrera. Bump catalog version. Author: Joel Jacobson Reviewed-by: Isaac Morland, Michael Paquier, Ranier Vilela Discussion: https://postgr.es/m/80b16434-b9b1-4c3d-8f28-569f21c2c...@app.fastmail.com Branch

pgsql: Assign error codes where missing for user-facing failures

2024-07-03 Thread Michael Paquier
Assign error codes where missing for user-facing failures All the errors triggered in the code paths patched here would cause the backend to issue an internal_error errcode, which is a state that should be used only for "can't happen" situations. However, these code paths are reachable by the reg

pgsql: Replace hardcoded identifiers of pgstats file by #defines

2024-07-02 Thread Michael Paquier
Replace hardcoded identifiers of pgstats file by #defines This changes pgstat.c so as the three types of entries that can exist in a pgstats file are not hardcoded anymore, replacing them with descriptively-named macros, when reading and writing stats files: - 'N' for named entries, like replicati

pgsql: Clean up more unused variables in perl code

2024-07-02 Thread Michael Paquier
Clean up more unused variables in perl code This is a continuation of 0c1aca461481, with some cleanup in: - msvc_gendef.pl - pgindent - 005_negotiate_encryption.pl, as of an oversight of d39a49c1e459 that has removed %params in test_matrix(), making also $server_config useless. Author: Dagfinn Il

pgsql: Cleanup perl code from unused variables and routines

2024-07-01 Thread Michael Paquier
Cleanup perl code from unused variables and routines This commit removes unused variables and routines from some perl code that have accumulated across the years. This touches the following areas: - Wait event generation script. - AdjustUpgrade.pm. - TAP perl code Author: Alexander Lakhin Review

pgsql: Add information about access method for partitioned relations in

2024-07-01 Thread Michael Paquier
Add information about access method for partitioned relations in \dP+ Since 374c7a229042, it is possible to set a table AM on a partitioned table. This information was showing up already in psql with \d+, while \dP+ provided no information. This commit extends \dP+ to show the access method used

pgsql: Use pgstat_kind_infos to read fixed shared statistics

2024-06-30 Thread Michael Paquier
future improvements aimed at making this area more pluggable for external modules. Original idea suggested by Andres Freund. Author: Tristan Partin Reviewed-by: Andres Freund, Michael Paquier Discussion: https://postgr.es/m/d12sq7oycd85.20buvf3dwu...@neon.tech Branch -- master Details

  1   2   3   4   5   6   7   8   9   10   >