Hi, Here's a v6 addressing the comments. I've split it into two parts:
0001 - While updating the "Impact on System of Online Operations" part, I realized it does not mention checkpoints at all. Which seems wrong, because even on 19 it can be a source of disruption. So 0001 adds this reference, and I intend to backpatch this to 19. FWIW the "Impact on System of Online Operations" seems a bit strange to me. It's as if it was about impact on "System of Online Operations", which does not seem right. I think it should be "Impact of Online Operations on System" instead. The other thing I noticed is that wal.sgml does not quite follow the terminology, it talks about "checksums" and not "data checksums" in a bunch of places. Maybe it should be cleaned up, but I haven't done anything about that. 0002 - This is the main part for 20. It now acquires the locks even in cases where I think it's not strictly necessary (but explaining the reasoning would be longer than just following the locking protocol). It also uses the launch_fast_checkpoint in launcher_exit, as explained in the previous message. It should also fix all the typos, etc. regards -- Tomas Vondra
From 9ed492a760f7d4c0fba3d4c7906eb540e6e3214f Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Mon, 6 Jul 2026 11:31:19 +0200 Subject: [PATCH v5 2/2] Allow using spread checkpoints when enabling/disabling checksums While enabling data checksums, the process has to rewrite all data pages, and then perform checkpoints at certain points. The writes due to updating data checksums on pages may be throttled (by using the cost_delay/cost_limit parameters), but the checkpoints were always requested as CHECKPOINT_FAST. That can be very disruptive. This applies even to disabling data checksums. In this case no pages need to be rewritten to update the data checksums, but the checkpoints are still needed and may trigger a lot of I/O. So a seemingly cheap operation (disabling checksums) may get rather disruptive. This commit allows requesting spread (throttled) checkpoints while enabling or disabling data checksums in a running cluster, to limit the impact on the rest of the system. It introduces a new "fast" parameter for functions initiating the data checksum state change. The default "fast=true" means the requested checkpoints are still "fast." Original patch by Daniel Gustafsson, part of the f19c0eccae96 work, but removed to reduce scope of the initial commit. Revived by me, with some minor cleanups and improvements. Author: Tomas Vondra <[email protected]> Author: Daniel Gustafsson <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Reviewed-by: Ewan Young <[email protected]> Reviewed-by: Fujii Masao <[email protected]> Reviewed-by: Rui Zhao <[email protected]> Discussion: https://postgr.es/m/[email protected] --- doc/src/sgml/func/func-admin.sgml | 22 ++++++- doc/src/sgml/wal.sgml | 5 +- src/backend/access/transam/xlog.c | 31 +++++++-- src/backend/postmaster/datachecksum_state.c | 64 ++++++++++++++++--- src/include/access/xlog.h | 4 +- src/include/catalog/pg_proc.dat | 12 ++-- .../test_checksums/t/DataChecksums/Utils.pm | 24 ++++++- 7 files changed, 134 insertions(+), 28 deletions(-) diff --git a/doc/src/sgml/func/func-admin.sgml b/doc/src/sgml/func/func-admin.sgml index 0eae1c1f616..56f82aa2c0f 100644 --- a/doc/src/sgml/func/func-admin.sgml +++ b/doc/src/sgml/func/func-admin.sgml @@ -3158,7 +3158,7 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8'); <indexterm> <primary>pg_enable_data_checksums</primary> </indexterm> - <function>pg_enable_data_checksums</function> ( <optional><parameter>cost_delay</parameter> <type>int</type>, <parameter>cost_limit</parameter> <type>int</type></optional> ) + <function>pg_enable_data_checksums</function> ( <optional><parameter>cost_delay</parameter> <type>int</type> <literal>DEFAULT</literal> <literal>0</literal>, <parameter>cost_limit</parameter> <type>int</type> <literal>DEFAULT</literal> <literal>100</literal>, <parameter>fast</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>true</literal></optional> ) <returnvalue>void</returnvalue> </para> <para> @@ -3175,6 +3175,15 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8'); specified, the process is throttled using the same principles as <link linkend="runtime-config-resource-vacuum-cost">Cost-based Vacuum Delay</link>. </para> + <para> + The <parameter>fast</parameter> parameter specifies whether the checkpoints + performed are fast or spread. + When set to <literal>true</literal> the checkpoints are fast, which may cause + a spike in I/O. When set to <literal>false</literal>, the checkpoints are + spread (throttled), which may increase the time needed to enable data checksums, + but it limits the impact on the system. + The default value is <literal>true</literal>. + </para> </entry> </row> @@ -3183,7 +3192,7 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8'); <indexterm> <primary>pg_disable_data_checksums</primary> </indexterm> - <function>pg_disable_data_checksums</function> () + <function>pg_disable_data_checksums</function> ( <optional><parameter>fast</parameter> <type>boolean</type> <literal>DEFAULT</literal> <literal>true</literal></optional> ) <returnvalue>void</returnvalue> </para> <para> @@ -3193,6 +3202,15 @@ SELECT convert_from(pg_read_binary_file('file_in_utf8.txt'), 'UTF8'); stopped validating data checksums, the data checksum state will be set to <literal>off</literal>. </para> + <para> + The <parameter>fast</parameter> parameter specifies whether the checkpoints + performed are fast or spread. + When set to <literal>true</literal> the checkpoints are fast, which may cause + a spike in I/O. When set to <literal>false</literal>, the checkpoints are + spread (throttled), which may increase the time needed to disable data checksums, + but it limits the impact on the system. + The default value is <literal>true</literal>. + </para> </entry> </row> </tbody> diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml index 1adfca01aa5..c150d55b012 100644 --- a/doc/src/sgml/wal.sgml +++ b/doc/src/sgml/wal.sgml @@ -360,7 +360,10 @@ <para> Checkpoints have to be performed at certain points - both when enabling and disabling data checksums, which may generate of lot of writes due to - flushing dirty pages. + flushing dirty pages. The impact may be reduced by requresting spread + checkpoints using the <parameter>fast</parameter> parameter of the + <function>pg_enable_data_checksums()</function> and + <function>pg_disable_data_checksums()</function> functions. </para> <para> diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index a8bbf6284a7..d1ad3747ad9 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -4794,13 +4794,17 @@ SetDataChecksumsOnInProgress(void) * state can be changed to "on". Once the state is "on" checksums will be both * written and verified. * + * 'fast' determines whether the checkpoints requested before progressing to + * the next phase are fast (CHECKPOINT_FAST) or spread. + * * This function blocks until all backends in the cluster have acknowledged the * state transition. */ void -SetDataChecksumsOn(void) +SetDataChecksumsOn(bool fast) { uint64 barrier; + int flags; SpinLockAcquire(&XLogCtl->info_lck); @@ -4814,7 +4818,7 @@ SetDataChecksumsOn(void) SpinLockRelease(&XLogCtl->info_lck); elog(WARNING, "cannot set data checksums to \"on\", current state is not \"inprogress-on\", disabling"); - SetDataChecksumsOff(); + SetDataChecksumsOff(fast); return; } @@ -4844,7 +4848,12 @@ SetDataChecksumsOn(void) MyProc->delayChkptFlags &= ~DELAY_CHKPT_START; END_CRIT_SECTION(); - RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | CHECKPOINT_FAST); + /* determine flags for the checkpoint */ + flags = CHECKPOINT_FORCE | CHECKPOINT_WAIT; + if (fast) + flags |= CHECKPOINT_FAST; + + RequestCheckpoint(flags); WaitForProcSignalBarrier(barrier); } @@ -4858,13 +4867,23 @@ SetDataChecksumsOn(void) * backends which have yet to observe the state change from "on" won't get * validation errors on concurrently modified pages. Once all backends have * changed to "inprogress-off", the barrier for moving to "off" can be emitted. + * + * 'fast' determines whether the checkpoints requested before progressing to + * the next phase are fast (CHECKPOINT_FAST) or spread. + * * This function blocks until all backends in the cluster have acknowledged the * state transition. */ void -SetDataChecksumsOff(void) +SetDataChecksumsOff(bool fast) { uint64 barrier; + int flags; + + /* determine flags for the checkpoint(s) */ + flags = CHECKPOINT_FORCE | CHECKPOINT_WAIT; + if (fast) + flags |= CHECKPOINT_FAST; SpinLockAcquire(&XLogCtl->info_lck); @@ -4906,7 +4925,7 @@ SetDataChecksumsOff(void) MyProc->delayChkptFlags &= ~DELAY_CHKPT_START; END_CRIT_SECTION(); - RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | CHECKPOINT_FAST); + RequestCheckpoint(flags); WaitForProcSignalBarrier(barrier); /* @@ -4944,7 +4963,7 @@ SetDataChecksumsOff(void) MyProc->delayChkptFlags &= ~DELAY_CHKPT_START; END_CRIT_SECTION(); - RequestCheckpoint(CHECKPOINT_FORCE | CHECKPOINT_WAIT | CHECKPOINT_FAST); + RequestCheckpoint(flags); WaitForProcSignalBarrier(barrier); } diff --git a/src/backend/postmaster/datachecksum_state.c b/src/backend/postmaster/datachecksum_state.c index 68557c16cb9..efed153dc44 100644 --- a/src/backend/postmaster/datachecksum_state.c +++ b/src/backend/postmaster/datachecksum_state.c @@ -307,6 +307,7 @@ typedef struct DataChecksumsStateStruct DataChecksumsWorkerOperation launch_operation; int launch_cost_delay; int launch_cost_limit; + bool launch_fast_checkpoint; /* * Is a launcher process currently running? This is set by the main @@ -340,6 +341,7 @@ typedef struct DataChecksumsStateStruct DataChecksumsWorkerOperation operation; int cost_delay; int cost_limit; + bool fast_checkpoint; /* * Signaling between the launcher and the worker process. Protected by @@ -382,7 +384,8 @@ static DataChecksumsWorkerOperation operation; /* Prototypes */ static void StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation op, int cost_delay, - int cost_limit); + int cost_limit, + bool fast); static void DataChecksumsShmemRequest(void *arg); static bool DatabaseExists(Oid dboid); static List *BuildDatabaseList(void); @@ -543,6 +546,8 @@ AbsorbDataChecksumsBarrier(ProcSignalBarrierType barrier) Datum disable_data_checksums(PG_FUNCTION_ARGS) { + bool fast = PG_GETARG_BOOL(0); + PreventCommandDuringRecovery("pg_disable_data_checksums()"); if (!superuser()) @@ -550,7 +555,7 @@ disable_data_checksums(PG_FUNCTION_ARGS) errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), errmsg("must be superuser to change data checksum state")); - StartDataChecksumsWorkerLauncher(DISABLE_DATACHECKSUMS, 0, 0); + StartDataChecksumsWorkerLauncher(DISABLE_DATACHECKSUMS, 0, 0, fast); PG_RETURN_VOID(); } @@ -564,6 +569,7 @@ enable_data_checksums(PG_FUNCTION_ARGS) { int cost_delay = PG_GETARG_INT32(0); int cost_limit = PG_GETARG_INT32(1); + bool fast = PG_GETARG_BOOL(2); PreventCommandDuringRecovery("pg_enable_data_checksums()"); @@ -582,7 +588,7 @@ enable_data_checksums(PG_FUNCTION_ARGS) errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("cost limit must be greater than zero")); - StartDataChecksumsWorkerLauncher(ENABLE_DATACHECKSUMS, cost_delay, cost_limit); + StartDataChecksumsWorkerLauncher(ENABLE_DATACHECKSUMS, cost_delay, cost_limit, fast); PG_RETURN_VOID(); } @@ -602,7 +608,8 @@ enable_data_checksums(PG_FUNCTION_ARGS) static void StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation op, int cost_delay, - int cost_limit) + int cost_limit, + bool fast) { BackgroundWorker bgw; BackgroundWorkerHandle *bgw_handle; @@ -622,6 +629,7 @@ StartDataChecksumsWorkerLauncher(DataChecksumsWorkerOperation op, DataChecksumState->launch_operation = op; DataChecksumState->launch_cost_delay = cost_delay; DataChecksumState->launch_cost_limit = cost_limit; + DataChecksumState->launch_fast_checkpoint = fast; /* Is the launcher already running? If so, what is it doing? */ running = DataChecksumState->launcher_running; @@ -989,7 +997,20 @@ launcher_exit(int code, Datum arg) * the state to off since processing cannot be resumed. */ if (DataChecksumsInProgressOn()) - SetDataChecksumsOff(); + { + bool fast_checkpoint; + + /* + * Get the current value of the launch_fast_checkpoint flag. It might + * have been updated while the launcher was already running, so make + * sure to follow the locking protocol. + */ + LWLockAcquire(DataChecksumsWorkerLock, LW_EXCLUSIVE); + fast_checkpoint = DataChecksumState->launch_fast_checkpoint; + LWLockRelease(DataChecksumsWorkerLock); + + SetDataChecksumsOff(fast_checkpoint); + } LWLockAcquire(DataChecksumsWorkerLock, LW_EXCLUSIVE); launcher_running = false; @@ -1130,6 +1151,7 @@ DataChecksumsWorkerLauncherMain(Datum arg) DataChecksumState->operation = operation; DataChecksumState->cost_delay = DataChecksumState->launch_cost_delay; DataChecksumState->cost_limit = DataChecksumState->launch_cost_limit; + DataChecksumState->fast_checkpoint = DataChecksumState->launch_fast_checkpoint; LWLockRelease(DataChecksumsWorkerLock); /* @@ -1145,6 +1167,8 @@ again: if (operation == ENABLE_DATACHECKSUMS) { + bool fast_checkpoint; + /* * If we are asked to enable checksums in a cluster which already has * checksums enabled, exit immediately as there is nothing more to do. @@ -1180,23 +1204,42 @@ again: errmsg("unable to enable data checksums in cluster")); } + /* + * Get the current value of the fast_checkpoint flag, in case it was + * updated while the worker was running. + */ + LWLockAcquire(DataChecksumsWorkerLock, LW_EXCLUSIVE); + fast_checkpoint = DataChecksumState->fast_checkpoint; + LWLockRelease(DataChecksumsWorkerLock); + /* * Data checksums have been set on all pages, set the state to on in * order to instruct backends to validate checksums on reading. */ - SetDataChecksumsOn(); + SetDataChecksumsOn(fast_checkpoint); ereport(LOG, errmsg("data checksums are now enabled")); } else if (operation == DISABLE_DATACHECKSUMS) { + bool fast_checkpoint; + ereport(LOG, errmsg("disabling data checksums requested")); pgstat_progress_update_param(PROGRESS_DATACHECKSUMS_PHASE, PROGRESS_DATACHECKSUMS_PHASE_DISABLING); - SetDataChecksumsOff(); + + /* + * Get the current value of the fast_checkpoint flag, in case it was + * updated while the worker was running. + */ + LWLockAcquire(DataChecksumsWorkerLock, LW_EXCLUSIVE); + fast_checkpoint = DataChecksumState->fast_checkpoint; + LWLockRelease(DataChecksumsWorkerLock); + + SetDataChecksumsOff(fast_checkpoint); ereport(LOG, errmsg("data checksums are now disabled")); } @@ -1224,6 +1267,7 @@ done: operation = DataChecksumState->launch_operation; DataChecksumState->cost_delay = DataChecksumState->launch_cost_delay; DataChecksumState->cost_limit = DataChecksumState->launch_cost_limit; + DataChecksumState->fast_checkpoint = DataChecksumState->launch_fast_checkpoint; LWLockRelease(DataChecksumsWorkerLock); goto again; } @@ -1313,7 +1357,7 @@ ProcessAllDatabases(void) * Disable checksums on cluster, because we failed one of the * databases and this is an all or nothing process. */ - SetDataChecksumsOff(); + SetDataChecksumsOff(DataChecksumState->fast_checkpoint); ereport(ERROR, errcode(ERRCODE_INSUFFICIENT_RESOURCES), errmsg("data checksums failed to get enabled in all databases, aborting"), @@ -1667,7 +1711,8 @@ DataChecksumsWorkerMain(Datum arg) break; } if ((DataChecksumState->launch_cost_delay != DataChecksumState->cost_delay) - || (DataChecksumState->launch_cost_limit != DataChecksumState->cost_limit)) + || (DataChecksumState->launch_cost_limit != DataChecksumState->cost_limit) + || (DataChecksumState->launch_fast_checkpoint != DataChecksumState->fast_checkpoint)) { costs_updated = true; VacuumCostDelay = DataChecksumState->launch_cost_delay; @@ -1676,6 +1721,7 @@ DataChecksumsWorkerMain(Datum arg) DataChecksumState->cost_delay = DataChecksumState->launch_cost_delay; DataChecksumState->cost_limit = DataChecksumState->launch_cost_limit; + DataChecksumState->fast_checkpoint = DataChecksumState->launch_fast_checkpoint; } else costs_updated = false; diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h index 4dd98624204..8a9e8961a6a 100644 --- a/src/include/access/xlog.h +++ b/src/include/access/xlog.h @@ -253,8 +253,8 @@ extern bool DataChecksumsOn(void); extern bool DataChecksumsOff(void); extern bool DataChecksumsInProgressOn(void); extern void SetDataChecksumsOnInProgress(void); -extern void SetDataChecksumsOn(void); -extern void SetDataChecksumsOff(void); +extern void SetDataChecksumsOn(bool fast); +extern void SetDataChecksumsOff(bool fast); extern const char *show_data_checksums(void); extern const char *get_checksum_state_string(uint32 state); extern void InitLocalDataChecksumState(void); diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3cb84359adf..c7ef5a9c930 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12415,13 +12415,15 @@ # data checksum management functions { oid => '6507', descr => 'disable data checksums', proname => 'pg_disable_data_checksums', provolatile => 'v', - proparallel => 'r', prorettype => 'void', proargtypes => '', - prosrc => 'disable_data_checksums', proacl => '{POSTGRES=X}' }, + proparallel => 'r', prorettype => 'void', proargtypes => 'bool', + proallargtypes => '{bool}', proargmodes => '{i}', proargnames => '{fast}', + proargdefaults => '{true}', prosrc => 'disable_data_checksums', + proacl => '{POSTGRES=X}' }, { oid => '6506', descr => 'enable data checksums', proname => 'pg_enable_data_checksums', provolatile => 'v', proparallel => 'r', - prorettype => 'void', proargtypes => 'int4 int4', - proallargtypes => '{int4,int4}', proargmodes => '{i,i}', - proargnames => '{cost_delay,cost_limit}', proargdefaults => '{0,100}', + prorettype => 'void', proargtypes => 'int4 int4 bool', + proallargtypes => '{int4,int4,bool}', proargmodes => '{i,i,i}', + proargnames => '{cost_delay,cost_limit,fast}', proargdefaults => '{0,100,true}', prosrc => 'enable_data_checksums', proacl => '{POSTGRES=X}' }, # collation management functions diff --git a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm index 3aa533d777f..77c208dd5fd 100644 --- a/src/test/modules/test_checksums/t/DataChecksums/Utils.pm +++ b/src/test/modules/test_checksums/t/DataChecksums/Utils.pm @@ -118,6 +118,11 @@ If defined, the function will wait for the state defined in this parameter, waiting timing out, before returning. The function will wait for $PostgreSQL::Test::Utils::timeout_default seconds before timing out. +=item fast + +The B<fast> determines if checkpoints will be fast or spread, default is +'true' (which means fast checkpoints). + =back =cut @@ -130,13 +135,14 @@ sub enable_data_checksums # Set sane defaults for the parameters $params{cost_delay} = 0 unless (defined($params{cost_delay})); $params{cost_limit} = 100 unless (defined($params{cost_limit})); + $params{fast} = 'true' unless (defined($params{fast})); my $query = <<'EOQ'; -SELECT pg_enable_data_checksums(%s, %s); +SELECT pg_enable_data_checksums(%s, %s, %s); EOQ $postgresnode->safe_psql('postgres', - sprintf($query, $params{cost_delay}, $params{cost_limit})); + sprintf($query, $params{cost_delay}, $params{cost_limit}, $params{fast})); if (defined($params{wait})) { @@ -166,6 +172,11 @@ waiting timing out, before returning. The function will wait for $PostgreSQL::Test::Utils::timeout_default seconds before timing out. Unlike in C<enable_data_checksums> the value of the parameter is discarded. +=item fast + +The B<fast> determines if checkpoints will be fast or spread, default is +'true' (which means fast checkpoints). + =back =cut @@ -175,8 +186,15 @@ sub disable_data_checksums my $postgresnode = shift; my %params = @_; + # Set sane defaults for the parameters + $params{fast} = 'true' unless (defined($params{fast})); + + my $query = <<'EOQ'; +SELECT pg_disable_data_checksums(%s); +EOQ + $postgresnode->safe_psql('postgres', - 'SELECT pg_disable_data_checksums();'); + sprintf($query, $params{fast})); if (defined($params{wait})) { -- 2.54.0
From 0660ead78d62660fa97e20fd101464762d081553 Mon Sep 17 00:00:00 2001 From: Tomas Vondra <[email protected]> Date: Mon, 13 Jul 2026 13:47:20 +0200 Subject: [PATCH v5 1/2] Mention checkpoints in online data checksums docs Mention checkpoints in the docs discussion impact of online data checksums changes on the system. Checkpoints (especially fast ones) are a major source of write activity, and can be rather disruptive. Backpatch to 19, where the online change of data checksums was introduced. Author: Tomas Vondra <[email protected]> Discussion: https://postgr.es/m/[email protected] Backpatch-through: 19 --- doc/src/sgml/wal.sgml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml index 646076f7e39..1adfca01aa5 100644 --- a/doc/src/sgml/wal.sgml +++ b/doc/src/sgml/wal.sgml @@ -357,12 +357,19 @@ parameters of the <function>pg_enable_data_checksums()</function> function. </para> + <para> + Checkpoints have to be performed at certain points - both when enabling + and disabling data checksums, which may generate of lot of writes due to + flushing dirty pages. + </para> + <para> <itemizedlist> <listitem><para> I/O: all pages need to have data checksums calculated and written which will generate a lot of dirty pages that will need to be flushed to disk, - as well as WAL logged. + as well as WAL logged. Both enabling and disabling data checksums requires + performing checkpoints, which may generate a lot of writes. </para></listitem> <listitem><para> Replication: When the standby receives the data checksum state change -- 2.54.0
