Hi all, During normal startup, we call CalculateShmemSize() twice: once to initialize the shared-memory-related GUCs and again to create the shared memory segment. We can avoid the second calculation by calling CalculateShmemSize() after ShmemCallRequestCallbacks() and passing the result to InitializeShmemGUCs() and CreateSharedMemoryAndSemaphores(). The attached patch does this, ensuring that GUC reporting and shared memory creation use the same calculated size.
I considered combining these four calls into a single function. However, when postgres -C reports a runtime-computed GUC, it exits after GUC initialization without creating shared memory. Keeping sizing and GUC initialization separate from shared memory creation preserves that distinction. We could also invoke ShmemCallRequestCallbacks() from CalculateShmemSize() to keep request collection and sizing together. That alone would not prevent additional requests after sizing, though. As a follow-up, would it make sense to introduce an SRS_SIZED state between SRS_REQUESTING and SRS_INITIALIZING? We could enter it when ShmemGetRequestedSize() finishes and adjust the state checks accordingly. This would prevent new requests from being registered after the required shared memory size has been calculated. EXEC_BACKEND child attachment and late allocation would retain their existing paths without entering this state. I can update the patch if this seems worthwhile. I’m proposing this change for master. It may also be worth considering for PG 19 to reduce conflicts when backpatching future changes to this code, rather than as a bug fix in itself. -- Best Wishes, Ashutosh Bapat
From 5eba2b8514f1436d129d8ac6bbe3c84c1344c03b Mon Sep 17 00:00:00 2001 From: Ashutosh Bapat <[email protected]> Date: Thu, 29 Oct 2026 15:12:37 +0530 Subject: [PATCH v20261030] Call CalculateShmemSize() only once when creating shared memory During normal startup, CalculateShmemSize() is called twice: first in InitializeShmemGUCs() and then in CreateSharedMemoryAndSemaphores(). When the server is invoked only to report a runtime-computed GUC, the second call is not reached. Calculate the size after collecting shared memory requests and pass it to both functions. Besides avoiding duplicate calculation, this ensures that GUC reporting and shared memory creation use the same calculated size. Note: It might be worth considering adding a separate state SRS_SIZING after SRS_REQUESTING and SRS_INITIALIZING to make sure that no new requests are added between calculating the total amount of memory required and allocating the shared memory. But it's not done in this commit. Author: Ashutosh Bapat <[email protected]> Reviewed-by: TBD Discussion: https://postgr.es/m/<message-id-TBD> Co-authored-by: Copilot <[email protected]> --- src/backend/bootstrap/bootstrap.c | 4 +++- src/backend/postmaster/postmaster.c | 11 ++++++++--- src/backend/storage/ipc/ipci.c | 21 +++++++++++---------- src/backend/storage/ipc/shmem.c | 10 ++++++---- src/backend/tcop/postgres.c | 6 ++++-- src/include/storage/ipc.h | 4 ++-- 6 files changed, 34 insertions(+), 22 deletions(-) diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 63e4d71b96b..4baf15e1e9e 100644 --- a/src/backend/bootstrap/bootstrap.c +++ b/src/backend/bootstrap/bootstrap.c @@ -243,6 +243,7 @@ BootstrapModeMain(int argc, char *argv[], bool check_only) char *userDoption = NULL; uint32 bootstrap_data_checksum_version = PG_DATA_CHECKSUM_OFF; yyscan_t scanner; + Size shmem_size; Assert(!IsUnderPostmaster); @@ -377,7 +378,8 @@ BootstrapModeMain(int argc, char *argv[], bool check_only) InitializeFastPathLocks(); ShmemCallRequestCallbacks(); - CreateSharedMemoryAndSemaphores(); + shmem_size = CalculateShmemSize(); + CreateSharedMemoryAndSemaphores(shmem_size); /* * Estimate number of openable files. This is essential too in --check diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c index f53aa9c404d..240a6048cf4 100644 --- a/src/backend/postmaster/postmaster.c +++ b/src/backend/postmaster/postmaster.c @@ -501,6 +501,7 @@ PostmasterMain(int argc, char *argv[]) char *userDoption = NULL; bool listen_addr_saved = false; char *output_config_variable = NULL; + Size shmem_size; InitProcessGlobals(); @@ -973,13 +974,14 @@ PostmasterMain(int argc, char *argv[]) * shared memory needs. */ ShmemCallRequestCallbacks(); + shmem_size = CalculateShmemSize(); /* * Now that loadable modules have had their chance to request additional * shared memory, determine the value of any runtime-computed GUCs that * depend on the amount of shared memory required. */ - InitializeShmemGUCs(); + InitializeShmemGUCs(shmem_size); /* * Now that modules have been loaded, we can process any custom resource @@ -1014,7 +1016,7 @@ PostmasterMain(int argc, char *argv[]) * normally choose the same IPC keys. This helps ensure that we will * clean up dead IPC objects if the postmaster crashes and is restarted. */ - CreateSharedMemoryAndSemaphores(); + CreateSharedMemoryAndSemaphores(shmem_size); /* * Estimate number of openable files. This must happen after setting up @@ -3257,6 +3259,8 @@ PostmasterStateMachine(void) */ if (FatalError && pmState == PM_NO_CHILDREN) { + Size shmem_size; + ereport(LOG, (errmsg("all server processes terminated; reinitializing"))); @@ -3280,7 +3284,8 @@ PostmasterStateMachine(void) */ ResetShmemAllocator(); ShmemCallRequestCallbacks(); - CreateSharedMemoryAndSemaphores(); + shmem_size = CalculateShmemSize(); + CreateSharedMemoryAndSemaphores(shmem_size); UpdatePMState(PM_STARTUP); diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c index e149a738c8d..90536f887f1 100644 --- a/src/backend/storage/ipc/ipci.c +++ b/src/backend/storage/ipc/ipci.c @@ -52,6 +52,10 @@ RequestAddinShmemSpace(Size size) /* * CalculateShmemSize * Calculates the amount of shared memory needed. + * + * Expected to be called only once after collecting all shared memory requests + * but before allocating the shared memory segment at the startup or server + * restart. */ Size CalculateShmemSize(void) @@ -114,19 +118,16 @@ AttachSharedMemoryStructs(void) /* * CreateSharedMemoryAndSemaphores - * Creates and initializes shared memory and semaphores. + * Creates and initializes shared memory of given size and semaphores. */ void -CreateSharedMemoryAndSemaphores(void) +CreateSharedMemoryAndSemaphores(Size size) { PGShmemHeader *shim; PGShmemHeader *seghdr; - Size size; Assert(!IsUnderPostmaster); - /* Compute the size of the shared-memory block */ - size = CalculateShmemSize(); elog(DEBUG3, "invoking IpcMemoryCreate(size=%zu)", size); /* @@ -183,20 +184,20 @@ RegisterBuiltinShmemCallbacks(void) * InitializeShmemGUCs * * This function initializes runtime-computed GUCs related to the amount of - * shared memory required for the current configuration. + * shared memory required for the current configuration. The input size is + * expected to be the total shared memory size computed for the current + * configuration. */ void -InitializeShmemGUCs(void) +InitializeShmemGUCs(Size size_b) { char buf[64]; - Size size_b; Size size_mb; Size hp_size; /* - * Calculate the shared memory size and round up to the nearest megabyte. + * Round the shared memory size up to the nearest megabyte. */ - size_b = CalculateShmemSize(); size_mb = add_size(size_b, (1024 * 1024) - 1) / (1024 * 1024); sprintf(buf, "%zu", size_mb); SetConfigOption("shared_memory_size", buf, diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c index 8d81893b470..497c0189cf0 100644 --- a/src/backend/storage/ipc/shmem.c +++ b/src/backend/storage/ipc/shmem.c @@ -389,17 +389,19 @@ ShmemRequestInternal(ShmemStructOpts *options, ShmemRequestKind kind) } /* - * ShmemGetRequestedSize() --- estimate the total size of all registered shared - * memory structures. + * ShmemGetRequestedSize() --- estimate the total size of all registered shared + * memory structures. * - * This is called at postmaster startup, before the shared memory segment has - * been created. + * This is called at postmaster startup or server restart, before the shared + * memory segment has been created. */ size_t ShmemGetRequestedSize(void) { size_t size; + Assert(shmem_request_state == SRS_REQUESTING); + /* memory needed for the ShmemIndex */ size = hash_estimate_size(list_length(pending_shmem_requests) + SHMEM_INDEX_ADDITIONAL_SIZE, sizeof(ShmemIndexEnt)); diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index b6bdfe213fe..944aefe3699 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -4222,6 +4222,7 @@ PostgresSingleUserMain(int argc, char *argv[], const char *username) { const char *dbname = NULL; + Size shmem_size; Assert(!IsUnderPostmaster); @@ -4303,13 +4304,14 @@ PostgresSingleUserMain(int argc, char *argv[], * add-ins, a chance to chance to adjust their requested shmem sizes. */ ShmemCallRequestCallbacks(); + shmem_size = CalculateShmemSize(); /* * Now that loadable modules have had their chance to request additional * shared memory, determine the value of any runtime-computed GUCs that * depend on the amount of shared memory required. */ - InitializeShmemGUCs(); + InitializeShmemGUCs(shmem_size); /* * Now that modules have been loaded, we can process any custom resource @@ -4321,7 +4323,7 @@ PostgresSingleUserMain(int argc, char *argv[], * Create shared memory etc. (Nothing's really "shared" in single-user * mode, but we must have these data structures anyway.) */ - CreateSharedMemoryAndSemaphores(); + CreateSharedMemoryAndSemaphores(shmem_size); /* * Estimate number of openable files. This must happen after setting up diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h index b205b00e7a1..db5c69a08db 100644 --- a/src/include/storage/ipc.h +++ b/src/include/storage/ipc.h @@ -79,10 +79,10 @@ extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook; extern void RegisterBuiltinShmemCallbacks(void); extern Size CalculateShmemSize(void); -extern void CreateSharedMemoryAndSemaphores(void); +extern void CreateSharedMemoryAndSemaphores(Size size); #ifdef EXEC_BACKEND extern void AttachSharedMemoryStructs(void); #endif -extern void InitializeShmemGUCs(void); +extern void InitializeShmemGUCs(Size size_b); #endif /* IPC_H */ base-commit: 4f89416c9dbc78da5679d32072315af56fbeb8ed -- 2.34.1
