Hi all, Please find attached a patch for $subject, which is something that I had on my stack of things to look at for some time now.
My main argument regarding the removal of toast_max_chunk_size in the control file is that it is a redundant check, due to the fact that the definition of TOAST_OID_MAX_CHUNK_SIZE is tied to two fields that we already track in the control file: - BLCKSZ - MAXALIGN Then, I have this table, that shows the value of the max_chunk_size depending on both parameters, for all MAXALIGN and block sizes supported,assuming my maths are right: BLCKSZ MAXALIGN 4_byte_id 8_byte_id 1kB 4 208 204 1kB 8 204 200 2kB 4 464 460 2kB 8 460 456 4kB 4 976 972 4kB 8 972 968 8kB 4 2000 1996 8kB 8 1996 1992 16kB 4 4048 4044 16kB 8 4044 4040 32kB 4 8144 8140 32kB 8 8140 8136 8_byte_id is assuming a OID8 TOAST value, but look just at the 4_byte_id column for the existing OID case. That's where I can see that if either BLCKSZ or MAXALIGN is different, we would fail the early validity checks on a cluster if trying to copy a data folder with an incompatible set of any of (BLCKSZ,MAXALIGN), without max_chunk_size interfering at all. In terms of pg_upgrade, we check for got_align and got_blocksz (see pg_upgrade/controldata.c). I won't hide that this removal offers extra benefits for the other work I am doing now for TOAST with more external pointer types, because it makes the max_chunk_size kind of irrelevant anyway at cluster level, but freeing bytes from the control file is super nice as a change of its own, because it's more bytes for more useful things in the future. (Spoiler: I'd need these 4 bytes myself, but that's a separate discussion.) There may be an argument about somebody enforcing a new TOAST_OID_MAX_CHUNK_SIZE or EXTERN_TUPLES_PER_PAGE manually, of course, but while we claim that tweaks are possible in heaptoast.h, I've never seen that as an officially-supported documented option (right?), and I've never heard somebody actually doing that. So I see nothing that prevents this removal from the control file? Comments or opinions? Feel free to point out anything I may be missing, of course.. -- Michael
From 0dab8db3093b899ae747adbcf92ca842e063e963 Mon Sep 17 00:00:00 2001 From: Michael Paquier <[email protected]> Date: Mon, 7 Sep 2026 10:10:35 +0900 Subject: [PATCH] Remove toast_max_chunk_size from pg_control toast_max_chunk_size depends on two fields that we already track in the control file: - BLCKSZ (settable in meson/configure, from 1kB to 32kB) - MAXALIGN, 4 or 8 bytes If either of these fields change, toast_max_chunk_size also changes, making toast_max_chunk_size kind of irrelevant because the other two would fail before toast_max_chunk_size is checked. heaptoast.h claims that manual updates of TOAST_OID_MAX_CHUNK_SIZE, EXTERN_TUPLE_MAX_SIZE or even EXTERN_TUPLES_PER_PAGE are possible, but this has never been an official thing documented in any way. --- src/include/catalog/pg_control.h | 3 +-- src/include/catalog/pg_proc.dat | 6 +++--- src/backend/access/transam/xlog.c | 12 ------------ src/backend/utils/misc/pg_controldata.c | 15 ++++++--------- src/bin/pg_controldata/pg_controldata.c | 2 -- src/bin/pg_resetwal/pg_resetwal.c | 4 ---- src/bin/pg_upgrade/controldata.c | 21 +-------------------- src/bin/pg_upgrade/pg_upgrade.h | 1 - doc/src/sgml/func/func-info.sgml | 5 ----- 9 files changed, 11 insertions(+), 58 deletions(-) diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h index 7b5404460ec2..a19922a3de23 100644 --- a/src/include/catalog/pg_control.h +++ b/src/include/catalog/pg_control.h @@ -22,7 +22,7 @@ /* Version identifier for this pg_control format */ -#define PG_CONTROL_VERSION 1903 +#define PG_CONTROL_VERSION 2000 /* Nonce key length, see below */ #define MOCK_AUTH_NONCE_LEN 32 @@ -223,7 +223,6 @@ typedef struct ControlFileData uint32 nameDataLen; /* catalog name field width */ uint32 indexMaxKeys; /* max number of columns in an index */ - uint32 toast_max_chunk_size; /* chunk size in TOAST tables */ uint32 loblksize; /* chunk size in pg_largeobject */ bool float8ByVal; /* float8, int8, etc pass-by-value? */ diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 6979c7d11613..a81e0b6a0fcc 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12449,9 +12449,9 @@ descr => 'pg_controldata init state information as a function', proname => 'pg_control_init', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{int4,int4,int4,int4,int4,int4,int4,int4,int4,bool,int4,bool}', - proargmodes => '{o,o,o,o,o,o,o,o,o,o,o,o}', - proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,max_toast_chunk_size,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version,default_char_signedness}', + proallargtypes => '{int4,int4,int4,int4,int4,int4,int4,int4,bool,int4,bool}', + proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}', + proargnames => '{max_data_alignment,database_block_size,blocks_per_segment,wal_block_size,bytes_per_wal_segment,max_identifier_length,max_index_columns,large_object_chunk_size,float8_pass_by_value,data_page_checksum_version,default_char_signedness}', prosrc => 'pg_control_init' }, # subscripting support for built-in types diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 3203f2fd4ee2..1cffd7d580f6 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -48,7 +48,6 @@ #include "access/clog.h" #include "access/commit_ts.h" -#include "access/heaptoast.h" #include "access/multixact.h" #include "access/rewriteheap.h" #include "access/subtrans.h" @@ -4324,7 +4323,6 @@ WriteControlFile(void) ControlFile->nameDataLen = NAMEDATALEN; ControlFile->indexMaxKeys = INDEX_MAX_KEYS; - ControlFile->toast_max_chunk_size = TOAST_OID_MAX_CHUNK_SIZE; ControlFile->loblksize = LOBLKSIZE; ControlFile->float8ByVal = true; /* vestigial */ @@ -4577,16 +4575,6 @@ ReadControlFile(void) "INDEX_MAX_KEYS", ControlFile->indexMaxKeys, "INDEX_MAX_KEYS", INDEX_MAX_KEYS), errhint("It looks like you need to recompile or initdb."))); - if (ControlFile->toast_max_chunk_size != TOAST_OID_MAX_CHUNK_SIZE) - ereport(FATAL, - (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), - errmsg("database files are incompatible with server"), - /* translator: %s is a variable name and %d is its value */ - errdetail("The database cluster was initialized with %s %d," - " but the server was compiled with %s %d.", - "TOAST_OID_MAX_CHUNK_SIZE", ControlFile->toast_max_chunk_size, - "TOAST_OID_MAX_CHUNK_SIZE", (int) TOAST_OID_MAX_CHUNK_SIZE), - errhint("It looks like you need to recompile or initdb."))); if (ControlFile->loblksize != LOBLKSIZE) ereport(FATAL, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c index 9014f0953e99..f120f7f12da9 100644 --- a/src/backend/utils/misc/pg_controldata.c +++ b/src/backend/utils/misc/pg_controldata.c @@ -209,8 +209,8 @@ pg_control_recovery(PG_FUNCTION_ARGS) Datum pg_control_init(PG_FUNCTION_ARGS) { - Datum values[12]; - bool nulls[12]; + Datum values[11]; + bool nulls[11]; TupleDesc tupdesc; HeapTuple htup; ControlFileData *ControlFile; @@ -248,21 +248,18 @@ pg_control_init(PG_FUNCTION_ARGS) values[6] = Int32GetDatum(ControlFile->indexMaxKeys); nulls[6] = false; - values[7] = Int32GetDatum(ControlFile->toast_max_chunk_size); + values[7] = Int32GetDatum(ControlFile->loblksize); nulls[7] = false; - values[8] = Int32GetDatum(ControlFile->loblksize); + values[8] = BoolGetDatum(ControlFile->float8ByVal); nulls[8] = false; - values[9] = BoolGetDatum(ControlFile->float8ByVal); + values[9] = Int32GetDatum(ControlFile->data_checksum_version_init); nulls[9] = false; - values[10] = Int32GetDatum(ControlFile->data_checksum_version_init); + values[10] = BoolGetDatum(ControlFile->default_char_signedness); nulls[10] = false; - values[11] = BoolGetDatum(ControlFile->default_char_signedness); - nulls[11] = false; - htup = heap_form_tuple(tupdesc, values, nulls); PG_RETURN_DATUM(HeapTupleGetDatum(htup)); diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c index 6fc87ed114d8..039533b082f6 100644 --- a/src/bin/pg_controldata/pg_controldata.c +++ b/src/bin/pg_controldata/pg_controldata.c @@ -338,8 +338,6 @@ main(int argc, char *argv[]) ControlFile->nameDataLen); printf(_("Maximum columns in an index: %u\n"), ControlFile->indexMaxKeys); - printf(_("Maximum size of a TOAST chunk: %u\n"), - ControlFile->toast_max_chunk_size); printf(_("Size of a large-object chunk: %u\n"), ControlFile->loblksize); /* This is no longer configurable, but users may still expect to see it: */ diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c index 79f3085d7696..83b0c0d99c94 100644 --- a/src/bin/pg_resetwal/pg_resetwal.c +++ b/src/bin/pg_resetwal/pg_resetwal.c @@ -43,7 +43,6 @@ #include <time.h> #include <unistd.h> -#include "access/heaptoast.h" #include "access/multixact.h" #include "access/transam.h" #include "access/xlog.h" @@ -737,7 +736,6 @@ GuessControlValues(void) ControlFile.xlog_seg_size = DEFAULT_XLOG_SEG_SIZE; ControlFile.nameDataLen = NAMEDATALEN; ControlFile.indexMaxKeys = INDEX_MAX_KEYS; - ControlFile.toast_max_chunk_size = TOAST_OID_MAX_CHUNK_SIZE; ControlFile.loblksize = LOBLKSIZE; ControlFile.float8ByVal = true; /* vestigial */ @@ -812,8 +810,6 @@ PrintControlValues(bool guessed) ControlFile.nameDataLen); printf(_("Maximum columns in an index: %u\n"), ControlFile.indexMaxKeys); - printf(_("Maximum size of a TOAST chunk: %u\n"), - ControlFile.toast_max_chunk_size); printf(_("Size of a large-object chunk: %u\n"), ControlFile.loblksize); /* This is no longer configurable, but users may still expect to see it: */ diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c index 7543a9880451..47b512db3b07 100644 --- a/src/bin/pg_upgrade/controldata.c +++ b/src/bin/pg_upgrade/controldata.c @@ -57,7 +57,6 @@ get_control_data(ClusterInfo *cluster) bool got_walseg = false; bool got_ident = false; bool got_index = false; - bool got_toast = false; bool got_large_object = false; bool got_date_is_int = false; bool got_data_checksum_version = false; @@ -398,17 +397,6 @@ get_control_data(ClusterInfo *cluster) cluster->controldata.index = str2uint(p); got_index = true; } - else if ((p = strstr(bufin, "Maximum size of a TOAST chunk:")) != NULL) - { - p = strchr(p, ':'); - - if (p == NULL || strlen(p) <= 1) - pg_fatal("%d: controldata retrieval problem", __LINE__); - - p++; /* remove ':' char */ - cluster->controldata.toast = str2uint(p); - got_toast = true; - } else if ((p = strstr(bufin, "Size of a large-object chunk:")) != NULL) { p = strchr(p, ':'); @@ -527,8 +515,7 @@ get_control_data(ClusterInfo *cluster) !got_mxoff || (!live_check && !got_nextxlogfile) || !got_float8_pass_by_value || !got_align || !got_blocksz || !got_largesz || !got_walsz || !got_walseg || !got_ident || - !got_index || !got_toast || - !got_large_object || + !got_index || !got_large_object || !got_date_is_int || !got_data_checksum_version || (!got_default_char_signedness && cluster->controldata.cat_ver >= DEFAULT_CHAR_SIGNEDNESS_CAT_VER)) @@ -585,9 +572,6 @@ get_control_data(ClusterInfo *cluster) if (!got_index) pg_log(PG_REPORT, " maximum number of indexed columns"); - if (!got_toast) - pg_log(PG_REPORT, " maximum TOAST chunk size"); - if (!got_large_object) pg_log(PG_REPORT, " large-object chunk size"); @@ -638,9 +622,6 @@ check_control_data(ControlData *oldctrl, if (oldctrl->index == 0 || oldctrl->index != newctrl->index) pg_fatal("old and new pg_controldata maximum indexed columns are invalid or do not match"); - if (oldctrl->toast == 0 || oldctrl->toast != newctrl->toast) - pg_fatal("old and new pg_controldata maximum TOAST chunk sizes are invalid or do not match"); - if (oldctrl->large_object == 0 || oldctrl->large_object != newctrl->large_object) pg_fatal("old and new pg_controldata large-object chunk sizes are invalid or do not match"); diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h index c80e8fb4031c..fbdb8fe108a0 100644 --- a/src/bin/pg_upgrade/pg_upgrade.h +++ b/src/bin/pg_upgrade/pg_upgrade.h @@ -228,7 +228,6 @@ typedef struct uint32 walseg; uint32 ident; uint32 index; - uint32 toast; uint32 large_object; bool date_is_int; bool float8_pass_by_value; diff --git a/doc/src/sgml/func/func-info.sgml b/doc/src/sgml/func/func-info.sgml index 2f03766b67ab..a5093e37e902 100644 --- a/doc/src/sgml/func/func-info.sgml +++ b/doc/src/sgml/func/func-info.sgml @@ -3593,11 +3593,6 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres} <entry><type>integer</type></entry> </row> - <row> - <entry><structfield>max_toast_chunk_size</structfield></entry> - <entry><type>integer</type></entry> - </row> - <row> <entry><structfield>large_object_chunk_size</structfield></entry> <entry><type>integer</type></entry> -- 2.55.0
signature.asc
Description: PGP signature
