On 09/02/2026 17:15, Ashutosh Bapat wrote:
On Sun, Feb 8, 2026 at 5:14 AM Heikki Linnakangas <[email protected]> wrote:
Could you write a standalone test module in src/test/modules to
demonstrate how to use the resizable shmem segments, please? That'd
allow focusing on that interface without worrying all the other
complexities of shared buffers.
From your writeup it seems like you are leaning towards creating the
shared memory segments on-demand rather than having predefined
segments as done in the patch.
Right. Same as with normal shared memory structs: there are
ShmemInitStruct calls scattered throughout the codebase, there's no need
to predefine them in a single header file or anything like that.
(Note: "on-demand" doesn't mean "on the fly", i.e. each shared memory
area still needs to be registered at postmaster startup.)
I think your proposal is interesting
and might create a possibility for extensions to be able to create
resizable shared memory structures.
Ah yes, good point. This facility should certainly be open for
extensions too. I didn't even realize that the predefined segments
scheme would not allow that.
Since the shared memory segments are predefined, a test module, as you
suggest, does not have a segment that it can use. That led me to think
that you are imagining some kind of on-demand shared memory segments
that extensions or test modules can use. I think that will be a useful
feature by itself. Just as an example, imagine an extension to provide
shared plan cache which resizes the plan cache as needed. However, we
need to make sure that these on-demand shared memory segments work
well with CalculateShmemSize(), PGSharedMemoryDetach(),
AnonymousShmemDetach() etc.
+1
2. There is no predefined limit on the number of segments. When
CalculateShmemSize() is called, BufferManagerShmemSize() outputs the
shared memory required in the main segment and total of shared memory
required in the other segments. Similarly shmem_request_hook of each
external module outputs the shared memory required in the main segment
and the total of shared memory required in the other segments. The
main segment is created in CreateSharedMemoryAndSemaphores() directly
using the requested size. The other output size is merely used for
reporting purposes in InitializeShmemGUCs(). When allocating shared
data structures, the main shared memory data structures are allocated
using ShmemInitStruct(), whereas the data structures in other memory
segments are allocated using ShmemInitStructExt() which takes
immediate allocation size and size of address space to be reserved as
arguments. It does not require segment_id though. For every new data
structure that ShmemInitStructExt() encounters, it a. creates a new
shared memory segment using PGSharedMemoryCreate(), b. allocates that
structure with the initial size. This function also needs to create
the PGShmemInfo and AnonShmemData entries corresponding to new
segments and make them available to PGSharedMemoryDetach(),
AnonymousShmemDetach() etc. For that we create a shared hash table in
the main shared memory segment (just like ShmemIndex) where we store
the metadata against each segment name (by coining it from the name of
the structure). We expect ShmemInitStructExt() to allocate structures
and segments only in the Postmaster and only at the beginning. When a
backend starts, it pulls the segment metadata from the shared hash
table which is ultimately used by PGSharedMemoryDetach(),
AnonymousShmemDetach(). At run time any backend which has access to
the shared memory should be able to call ShmemResizeStruct() given a
resizable structure and new size. Some higher level synchronization is
needed to ensure that the same structure is not resized simultaneously
by two backends.
The first approach is simple but has limited use given the fixed
number of segments. Second is more flexible but that's some work. I am
not sure whether it's worth doing all that if there are hardly any
extensions which could use resizable shared data structures.
It doesn't seem *that* much more work.
Putting this patch aside for a moment, I don't much like our current
interface for defining shared memory structs anyway. The
[SubSystem]ShmemSize() functions feel too detached from the
ShmemInitStruct() calls. For example, it's an easy mistake to return a
slightly different size in the FoobarShmemSize() call than what you use
in the ShmemInitStruct() call. And then there's the fact that the
initialization functions run both at postmaster startup but also at
backend startup in EXEC_BACKEND mode. There were reasons for that when
EXEC_BACKEND was introduced, but it's always felt awkward to me.
I feel that it'd be good to have a single definition of each shmem
struct, and derive all the other things from there.
Attached is a proof-of-concept of what I have in mind. Don't look too
closely at how it's implemented, it's very hacky and EXEC_BACKEND mode
is slightly broken, for example. The point is to demonstrate what the
callers would look like. I converted only a few subsystems to use the
new API, the rest still use ShmemInitStruct() and ShmemInitHash().
With this, initialization of a subsystem that defines a shared memory
area looks like this:
--------------
/* This struct lives in shared memory */
typedef struct
{
int field;
} FoobarSharedCtlData;
static void FoobarShmemInit(void *arg);
/* Descriptor for the shared memory area */
ShmemStructDesc FoobarShmemDesc = {
.name = "Foobar subsystem",
.size = sizeof(FoobarSharedCtlData),
.init_fn = FoobarShmemInit,
};
/* Pointer to the shared memory struct */
#define FoobarCtl ((FoobarSharedCtlData *) FoobarShmemDesc.ptr)
/*
* Register the shared memory struct. This is called once at
* postmaster startup, before the shared memory segment is allocated,
* and in EXEC_BACKEND mode also early at backend startup.
*
* For core subsystems, there's a list of all these functions in core
* in ipci.c, similar to all the *ShmemSize() and *ShmemInit() functions
* today. In an extension, this would be done in _PG_init() or in
* the shmem_request_hook, replacing the RequestAddinShmemSpace calls
* we have today.
*/
void
FoobarShmemRegister(void)
{
ShmemRegisterStruct(&FoobarShmemDesc);
}
/*
* This callback is called once at postmaster startup, to initialize
* the shared memory struct. FoobarShmemDesc.ptr has already been
* set when this is called.
*/
static void
FoobarShmemInit(void *arg)
{
memset(FoobarCtl, 0, sizeof(FoobarSharedCtlData));
FoobarCtl->field = 123;
}
--------------
The ShmemStructDesc provides room for extending the facility in the
future. For example, you could specify alignment there, or an additional
"attach" callback when you need to do more per-backend initialization in
EXEC_BACKEND mode. And with the resizeable shared memory, a max size.
Thoughts?
- Heikki
From c3c90517778b47156787f05a529096416dd67727 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <[email protected]>
Date: Mon, 9 Feb 2026 22:28:23 +0200
Subject: [PATCH 1/1] wip: Introduce a new way of registering shared memory
structs
---
.../pg_stat_statements/pg_stat_statements.c | 112 ++++-----
src/backend/access/transam/varsup.c | 32 +--
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/postmaster/launch_backend.c | 11 +-
src/backend/postmaster/postmaster.c | 2 +
src/backend/storage/ipc/dsm.c | 46 ++--
src/backend/storage/ipc/dsm_registry.c | 34 ++-
src/backend/storage/ipc/ipci.c | 51 ++--
src/backend/storage/ipc/pmsignal.c | 53 ++--
src/backend/storage/ipc/procarray.c | 127 +++++-----
src/backend/storage/ipc/procsignal.c | 63 +++--
src/backend/storage/ipc/shmem.c | 230 +++++++++++++++++-
src/backend/storage/ipc/sinvaladt.c | 39 +--
src/backend/storage/lmgr/proc.c | 156 ++++++------
src/backend/tcop/postgres.c | 2 +
src/include/access/transam.h | 12 +-
src/include/storage/dsm_registry.h | 3 +-
src/include/storage/ipc.h | 1 +
src/include/storage/pmsignal.h | 3 +-
src/include/storage/proc.h | 5 +-
src/include/storage/procarray.h | 3 +-
src/include/storage/procsignal.h | 3 +-
src/include/storage/shmem.h | 57 +++++
src/include/storage/sinvaladt.h | 3 +-
24 files changed, 662 insertions(+), 388 deletions(-)
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 4a427533bd8..71debc8b47f 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -258,6 +258,25 @@ typedef struct pgssSharedState
pgssGlobalStats stats; /* global statistics for pgss */
} pgssSharedState;
+static void pgss_shmem_init(void *arg);
+
+static ShmemStructDesc pgssSharedStateShmemDesc = {
+ .name = "pg_stat_statements",
+ .size = sizeof(pgssSharedState),
+ .init_fn = pgss_shmem_init,
+};
+
+static ShmemHashDesc pgssSharedHashDesc = {
+ .name = "pg_stat_statements hash",
+ .init_size = 0, /* set from 'pgss_max' */
+ .max_size = 0, /* set from 'pgss_max' */
+};
+
+/* Links to shared memory state */
+#define pgss ((pgssSharedState *) pgssSharedStateShmemDesc.ptr)
+#define pgss_hash (pgssSharedHashDesc.ptr)
+
+
/*---- Local variables ----*/
/* Current nesting depth of planner/ExecutorRun/ProcessUtility calls */
@@ -274,10 +293,6 @@ static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
static ProcessUtility_hook_type prev_ProcessUtility = NULL;
-/* Links to shared memory state */
-static pgssSharedState *pgss = NULL;
-static HTAB *pgss_hash = NULL;
-
/*---- GUC variables ----*/
typedef enum
@@ -365,7 +380,6 @@ static void pgss_store(const char *query, int64 queryId,
static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
pgssVersion api_version,
bool showtext);
-static Size pgss_memsize(void);
static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
int encoding, bool sticky);
static void entry_dealloc(void);
@@ -500,11 +514,39 @@ _PG_init(void)
static void
pgss_shmem_request(void)
{
+ HASHCTL info;
+
if (prev_shmem_request_hook)
prev_shmem_request_hook();
- RequestAddinShmemSpace(pgss_memsize());
RequestNamedLWLockTranche("pg_stat_statements", 1);
+
+ /*
+ * Register our shared memory state, including hash table
+ */
+ ShmemRegisterStruct(&pgssSharedStateShmemDesc);
+
+ info.keysize = sizeof(pgssHashKey);
+ info.entrysize = sizeof(pgssEntry);
+ pgssSharedHashDesc.init_size = pgss_max;
+ pgssSharedHashDesc.max_size = pgss_max;
+ ShmemRegisterHash(&pgssSharedHashDesc,
+ &info,
+ HASH_ELEM | HASH_BLOBS);
+}
+
+static void
+pgss_shmem_init(void *arg)
+{
+ pgss->lock = &(GetNamedLWLockTranche("pg_stat_statements"))->lock;
+ pgss->cur_median_usage = ASSUMED_MEDIAN_INIT;
+ pgss->mean_query_len = ASSUMED_LENGTH_INIT;
+ SpinLockInit(&pgss->mutex);
+ pgss->extent = 0;
+ pgss->n_writers = 0;
+ pgss->gc_count = 0;
+ pgss->stats.dealloc = 0;
+ pgss->stats.stats_reset = GetCurrentTimestamp();
}
/*
@@ -516,8 +558,6 @@ pgss_shmem_request(void)
static void
pgss_shmem_startup(void)
{
- bool found;
- HASHCTL info;
FILE *file = NULL;
FILE *qfile = NULL;
uint32 header;
@@ -530,42 +570,6 @@ pgss_shmem_startup(void)
if (prev_shmem_startup_hook)
prev_shmem_startup_hook();
- /* reset in case this is a restart within the postmaster */
- pgss = NULL;
- pgss_hash = NULL;
-
- /*
- * Create or attach to the shared memory state, including hash table
- */
- LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
-
- pgss = ShmemInitStruct("pg_stat_statements",
- sizeof(pgssSharedState),
- &found);
-
- if (!found)
- {
- /* First time through ... */
- pgss->lock = &(GetNamedLWLockTranche("pg_stat_statements"))->lock;
- pgss->cur_median_usage = ASSUMED_MEDIAN_INIT;
- pgss->mean_query_len = ASSUMED_LENGTH_INIT;
- SpinLockInit(&pgss->mutex);
- pgss->extent = 0;
- pgss->n_writers = 0;
- pgss->gc_count = 0;
- pgss->stats.dealloc = 0;
- pgss->stats.stats_reset = GetCurrentTimestamp();
- }
-
- info.keysize = sizeof(pgssHashKey);
- info.entrysize = sizeof(pgssEntry);
- pgss_hash = ShmemInitHash("pg_stat_statements hash",
- pgss_max, pgss_max,
- &info,
- HASH_ELEM | HASH_BLOBS);
-
- LWLockRelease(AddinShmemInitLock);
-
/*
* If we're in the postmaster (or a standalone backend...), set up a shmem
* exit hook to dump the statistics to disk.
@@ -573,12 +577,6 @@ pgss_shmem_startup(void)
if (!IsUnderPostmaster)
on_shmem_exit(pgss_shmem_shutdown, (Datum) 0);
- /*
- * Done if some other process already completed our initialization.
- */
- if (found)
- return;
-
/*
* Note: we don't bother with locks here, because there should be no other
* processes running when this code is reached.
@@ -2082,20 +2080,6 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
}
-/*
- * Estimate shared memory space needed.
- */
-static Size
-pgss_memsize(void)
-{
- Size size;
-
- size = MAXALIGN(sizeof(pgssSharedState));
- size = add_size(size, hash_estimate_size(pgss_max, sizeof(pgssEntry)));
-
- return size;
-}
-
/*
* Allocate a new hashtable entry.
* caller must hold an exclusive lock on pgss->lock
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 3e95d4cfd16..11ad90e7372 100644
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -30,35 +30,27 @@
/* Number of OIDs to prefetch (preallocate) per XLOG write */
#define VAR_OID_PREFETCH 8192
-/* pointer to variables struct in shared memory */
-TransamVariablesData *TransamVariables = NULL;
+static void VarsupShmemInit(void *arg);
+ShmemStructDesc TransamVariablesShmemDesc = {
+ .name = "TransamVariables",
+ .size = sizeof(TransamVariablesData),
+ .init_fn = VarsupShmemInit,
+};
/*
* Initialization of shared memory for TransamVariables.
*/
-Size
-VarsupShmemSize(void)
+void
+VarsupShmemRegister(void)
{
- return sizeof(TransamVariablesData);
+ ShmemRegisterStruct(&TransamVariablesShmemDesc);
}
-void
-VarsupShmemInit(void)
+static void
+VarsupShmemInit(void *arg)
{
- bool found;
-
- /* Initialize our shared state struct */
- TransamVariables = ShmemInitStruct("TransamVariables",
- sizeof(TransamVariablesData),
- &found);
- if (!IsUnderPostmaster)
- {
- Assert(!found);
- memset(TransamVariables, 0, sizeof(TransamVariablesData));
- }
- else
- Assert(found);
+ memset(TransamVariables, 0, sizeof(TransamVariablesData));
}
/*
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 7d32cd0e159..0ded7018e86 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -337,6 +337,8 @@ BootstrapModeMain(int argc, char *argv[], bool check_only)
InitializeFastPathLocks();
+ RegisterShmemStructs();
+
CreateSharedMemoryAndSemaphores();
/*
diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c
index 05b1feef3cf..5d397e3c619 100644
--- a/src/backend/postmaster/launch_backend.c
+++ b/src/backend/postmaster/launch_backend.c
@@ -49,6 +49,7 @@
#include "replication/walreceiver.h"
#include "storage/dsm.h"
#include "storage/io_worker.h"
+#include "storage/ipc.h"
#include "storage/pg_shmem.h"
#include "tcop/backend_startup.h"
#include "utils/memutils.h"
@@ -104,12 +105,10 @@ typedef struct
char **LWLockTrancheNames;
int *LWLockCounter;
LWLockPadded *MainLWLockArray;
- slock_t *ProcStructLock;
PROC_HDR *ProcGlobal;
PGPROC *AuxiliaryProcs;
PGPROC *PreparedXactProcs;
volatile PMSignalData *PMSignalState;
- ProcSignalHeader *ProcSignal;
pid_t PostmasterPid;
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
@@ -678,8 +677,12 @@ SubPostmasterMain(int argc, char *argv[])
/* Restore basic shared memory pointers */
if (UsedShmemSegAddr != NULL)
+ {
InitShmemAllocator(UsedShmemSegAddr);
+ RegisterShmemStructs();
+ }
+
/*
* Run the appropriate Main function
*/
@@ -735,12 +738,10 @@ save_backend_variables(BackendParameters *param,
param->LWLockTrancheNames = LWLockTrancheNames;
param->LWLockCounter = LWLockCounter;
param->MainLWLockArray = MainLWLockArray;
- param->ProcStructLock = ProcStructLock;
param->ProcGlobal = ProcGlobal;
param->AuxiliaryProcs = AuxiliaryProcs;
param->PreparedXactProcs = PreparedXactProcs;
param->PMSignalState = PMSignalState;
- param->ProcSignal = ProcSignal;
param->PostmasterPid = PostmasterPid;
param->PgStartTime = PgStartTime;
@@ -995,12 +996,10 @@ restore_backend_variables(BackendParameters *param)
LWLockTrancheNames = param->LWLockTrancheNames;
LWLockCounter = param->LWLockCounter;
MainLWLockArray = param->MainLWLockArray;
- ProcStructLock = param->ProcStructLock;
ProcGlobal = param->ProcGlobal;
AuxiliaryProcs = param->AuxiliaryProcs;
PreparedXactProcs = param->PreparedXactProcs;
PMSignalState = param->PMSignalState;
- ProcSignal = param->ProcSignal;
PostmasterPid = param->PostmasterPid;
PgStartTime = param->PgStartTime;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d6133bfebc6..f6d3369f917 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -968,6 +968,8 @@ PostmasterMain(int argc, char *argv[])
* shared memory, determine the value of any runtime-computed GUCs that
* depend on the amount of shared memory required.
*/
+ RegisterShmemStructs();
+
InitializeShmemGUCs();
/*
diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c
index 6a5b16392f7..55f46c7687e 100644
--- a/src/backend/storage/ipc/dsm.c
+++ b/src/backend/storage/ipc/dsm.c
@@ -108,7 +108,15 @@ static inline bool is_main_region_dsm_handle(dsm_handle handle);
static bool dsm_init_done = false;
/* Preallocated DSM space in the main shared memory region. */
-static void *dsm_main_space_begin = NULL;
+static void dsm_main_space_init(void *);
+
+static ShmemStructDesc dsm_main_space_shmem_desc = {
+ .name = "Preallocated DSM",
+ .size = 0, /* dynamic */
+ .init_fn = dsm_main_space_init,
+};
+
+#define dsm_main_space_begin (dsm_main_space_shmem_desc.ptr)
/*
* List of dynamic shared memory segments used by this backend.
@@ -479,27 +487,29 @@ void
dsm_shmem_init(void)
{
size_t size = dsm_estimate_size();
- bool found;
if (size == 0)
return;
- dsm_main_space_begin = ShmemInitStruct("Preallocated DSM", size, &found);
- if (!found)
- {
- FreePageManager *fpm = (FreePageManager *) dsm_main_space_begin;
- size_t first_page = 0;
- size_t pages;
-
- /* Reserve space for the FreePageManager. */
- while (first_page * FPM_PAGE_SIZE < sizeof(FreePageManager))
- ++first_page;
-
- /* Initialize it and give it all the rest of the space. */
- FreePageManagerInitialize(fpm, dsm_main_space_begin);
- pages = (size / FPM_PAGE_SIZE) - first_page;
- FreePageManagerPut(fpm, first_page, pages);
- }
+ ShmemRegisterStruct(&dsm_main_space_shmem_desc);
+}
+
+static void
+dsm_main_space_init(void *arg)
+{
+ size_t size = dsm_main_space_shmem_desc.size;
+ FreePageManager *fpm = (FreePageManager *) dsm_main_space_begin;
+ size_t first_page = 0;
+ size_t pages;
+
+ /* Reserve space for the FreePageManager. */
+ while (first_page * FPM_PAGE_SIZE < sizeof(FreePageManager))
+ ++first_page;
+
+ /* Initialize it and give it all the rest of the space. */
+ FreePageManagerInitialize(fpm, dsm_main_space_begin);
+ pages = (size / FPM_PAGE_SIZE) - first_page;
+ FreePageManagerPut(fpm, first_page, pages);
}
/*
diff --git a/src/backend/storage/ipc/dsm_registry.c b/src/backend/storage/ipc/dsm_registry.c
index 068c1577b12..882af83b7b2 100644
--- a/src/backend/storage/ipc/dsm_registry.c
+++ b/src/backend/storage/ipc/dsm_registry.c
@@ -54,7 +54,15 @@ typedef struct DSMRegistryCtxStruct
dshash_table_handle dshh;
} DSMRegistryCtxStruct;
-static DSMRegistryCtxStruct *DSMRegistryCtx;
+static void DSMRegistryCtxShmemInit(void *arg);
+
+static ShmemStructDesc DSMRegistryCtxShmemDesc = {
+ .name = "DSM Registry Data",
+ .size = sizeof(DSMRegistryCtxStruct),
+ .init_fn = DSMRegistryCtxShmemInit,
+};
+
+#define DSMRegistryCtx ((DSMRegistryCtxStruct *) DSMRegistryCtxShmemDesc.ptr)
typedef struct NamedDSMState
{
@@ -113,27 +121,17 @@ static const dshash_parameters dsh_params = {
static dsa_area *dsm_registry_dsa;
static dshash_table *dsm_registry_table;
-Size
-DSMRegistryShmemSize(void)
+void
+DSMRegistryShmemRegister(void)
{
- return MAXALIGN(sizeof(DSMRegistryCtxStruct));
+ ShmemRegisterStruct(&DSMRegistryCtxShmemDesc);
}
-void
-DSMRegistryShmemInit(void)
+static void
+DSMRegistryCtxShmemInit(void *)
{
- bool found;
-
- DSMRegistryCtx = (DSMRegistryCtxStruct *)
- ShmemInitStruct("DSM Registry Data",
- DSMRegistryShmemSize(),
- &found);
-
- if (!found)
- {
- DSMRegistryCtx->dsah = DSA_HANDLE_INVALID;
- DSMRegistryCtx->dshh = DSHASH_HANDLE_INVALID;
- }
+ DSMRegistryCtx->dsah = DSA_HANDLE_INVALID;
+ DSMRegistryCtx->dshh = DSHASH_HANDLE_INVALID;
}
/*
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 1f7e933d500..952988645d0 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -101,13 +101,14 @@ CalculateShmemSize(void)
size = add_size(size, hash_estimate_size(SHMEM_INDEX_SIZE,
sizeof(ShmemIndexEnt)));
size = add_size(size, dsm_estimate_size());
- size = add_size(size, DSMRegistryShmemSize());
+
+ size = add_size(size, ShmemRegisteredSize());
+
+ /* legacy subsystmes */
size = add_size(size, BufferManagerShmemSize());
size = add_size(size, LockManagerShmemSize());
size = add_size(size, PredicateLockShmemSize());
- size = add_size(size, ProcGlobalShmemSize());
size = add_size(size, XLogPrefetchShmemSize());
- size = add_size(size, VarsupShmemSize());
size = add_size(size, XLOGShmemSize());
size = add_size(size, XLogRecoveryShmemSize());
size = add_size(size, CLOGShmemSize());
@@ -117,11 +118,7 @@ CalculateShmemSize(void)
size = add_size(size, BackgroundWorkerShmemSize());
size = add_size(size, MultiXactShmemSize());
size = add_size(size, LWLockShmemSize());
- size = add_size(size, ProcArrayShmemSize());
size = add_size(size, BackendStatusShmemSize());
- size = add_size(size, SharedInvalShmemSize());
- size = add_size(size, PMSignalShmemSize());
- size = add_size(size, ProcSignalShmemSize());
size = add_size(size, CheckpointerShmemSize());
size = add_size(size, AutoVacuumShmemSize());
size = add_size(size, ReplicationSlotsShmemSize());
@@ -217,6 +214,10 @@ CreateSharedMemoryAndSemaphores(void)
*/
InitShmemAllocator(seghdr);
+ /* Reserve space for semaphores. */
+ if (!IsUnderPostmaster)
+ PGReserveSemaphores(ProcGlobalSemas());
+
/* Initialize subsystems */
CreateOrAttachShmemStructs();
@@ -230,6 +231,19 @@ CreateSharedMemoryAndSemaphores(void)
shmem_startup_hook();
}
+void
+RegisterShmemStructs(void)
+{
+ DSMRegistryShmemRegister();
+
+ ProcGlobalShmemRegister();
+ VarsupShmemRegister();
+ ProcArrayShmemRegister();
+ SharedInvalShmemRegister();
+ PMSignalShmemRegister();
+ ProcSignalShmemRegister();
+}
+
/*
* Initialize various subsystems, setting up their data structures in
* shared memory.
@@ -259,14 +273,23 @@ CreateOrAttachShmemStructs(void)
*/
InitShmemIndex();
+#ifdef EXEC_BACKEND
+ if (IsUnderPostmaster)
+ ShmemAttachRegistered();
+ else
+#endif
+ {
+ ShmemInitRegistered();
+ }
+
dsm_shmem_init();
- DSMRegistryShmemInit();
+ //DSMRegistryShmemInit();
/*
* Set up xlog, clog, and buffers
*/
- VarsupShmemInit();
XLOGShmemInit();
+
XLogPrefetchShmemInit();
XLogRecoveryShmemInit();
CLOGShmemInit();
@@ -288,23 +311,13 @@ CreateOrAttachShmemStructs(void)
/*
* Set up process table
*/
- if (!IsUnderPostmaster)
- InitProcGlobal();
- ProcArrayShmemInit();
BackendStatusShmemInit();
TwoPhaseShmemInit();
BackgroundWorkerShmemInit();
- /*
- * Set up shared-inval messaging
- */
- SharedInvalShmemInit();
-
/*
* Set up interprocess signaling mechanisms
*/
- PMSignalShmemInit();
- ProcSignalShmemInit();
CheckpointerShmemInit();
AutoVacuumShmemInit();
ReplicationSlotsShmemInit();
diff --git a/src/backend/storage/ipc/pmsignal.c b/src/backend/storage/ipc/pmsignal.c
index 4618820b337..23752500d16 100644
--- a/src/backend/storage/ipc/pmsignal.c
+++ b/src/backend/storage/ipc/pmsignal.c
@@ -80,9 +80,24 @@ struct PMSignalData
sig_atomic_t PMChildFlags[FLEXIBLE_ARRAY_MEMBER];
};
-/* PMSignalState pointer is valid in both postmaster and child processes */
+static void PMSignalShmemInit(void *);
+
+static ShmemStructDesc PMSignalShmemDesc = {
+ .name = "PMSignalState",
+ .size = 0, /* dynamic */
+ .init_fn = PMSignalShmemInit,
+};
+
+/*
+ * PMSignalState pointer is valid in both postmaster and child processes
+ *
+ * This is a stand-alone variable rather than just a #define over
+ * PMSignalShmemDesc.ptr because it is needed early at backend startup and
+ * passed as a backend parameter in EXEC_BACKEND mode
+ */
NON_EXEC_STATIC volatile PMSignalData *PMSignalState = NULL;
+
/*
* Local copy of PMSignalState->num_child_flags, only valid in the
* postmaster. Postmaster keeps a local copy so that it doesn't need to
@@ -123,39 +138,28 @@ postmaster_death_handler(SIGNAL_ARGS)
static void MarkPostmasterChildInactive(int code, Datum arg);
/*
- * PMSignalShmemSize
- * Compute space needed for pmsignal.c's shared memory
+ * PMSignalShmemRegister - Register our shared memory
*/
-Size
-PMSignalShmemSize(void)
+void
+PMSignalShmemRegister(void)
{
Size size;
size = offsetof(PMSignalData, PMChildFlags);
size = add_size(size, mul_size(MaxLivePostmasterChildren(),
sizeof(sig_atomic_t)));
-
- return size;
+ PMSignalShmemDesc.size = size;
+ ShmemRegisterStruct(&PMSignalShmemDesc);
}
-/*
- * PMSignalShmemInit - initialize during shared-memory creation
- */
-void
-PMSignalShmemInit(void)
+static void
+PMSignalShmemInit(void *arg)
{
- bool found;
-
- PMSignalState = (PMSignalData *)
- ShmemInitStruct("PMSignalState", PMSignalShmemSize(), &found);
-
- if (!found)
- {
- /* initialize all flags to zeroes */
- MemSet(unvolatize(PMSignalData *, PMSignalState), 0, PMSignalShmemSize());
- num_child_flags = MaxLivePostmasterChildren();
- PMSignalState->num_child_flags = num_child_flags;
- }
+ /* initialize all flags to zeroes */
+ PMSignalState = PMSignalShmemDesc.ptr;
+ MemSet(unvolatize(PMSignalData *, PMSignalState), 0, PMSignalShmemDesc.size);
+ num_child_flags = MaxLivePostmasterChildren();
+ PMSignalState->num_child_flags = num_child_flags;
}
/*
@@ -291,6 +295,7 @@ RegisterPostmasterChildActive(void)
{
int slot = MyPMChildSlot;
+ Assert(PMSignalState);
Assert(slot > 0 && slot <= PMSignalState->num_child_flags);
slot--;
Assert(PMSignalState->PMChildFlags[slot] == PM_CHILD_ASSIGNED);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 301f54fb5a8..08c63bcb2a7 100644
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -101,6 +101,18 @@ typedef struct ProcArrayStruct
int pgprocnos[FLEXIBLE_ARRAY_MEMBER];
} ProcArrayStruct;
+static void ProcArrayShmemInit(void *arg);
+static void ProcArrayShmemAttach(void *arg);
+
+static ShmemStructDesc ProcArrayShmemDesc = {
+ .name = "Proc Array",
+ .size = 0, /* dynamic */
+ .init_fn = ProcArrayShmemInit,
+ .attach_fn = ProcArrayShmemAttach,
+};
+
+#define procArray ((ProcArrayStruct *) ProcArrayShmemDesc.ptr)
+
/*
* State for the GlobalVisTest* family of functions. Those functions can
* e.g. be used to decide if a deleted row can be removed without violating
@@ -267,9 +279,6 @@ typedef enum KAXCompressReason
KAX_STARTUP_PROCESS_IDLE, /* startup process is about to sleep */
} KAXCompressReason;
-
-static ProcArrayStruct *procArray;
-
static PGPROC *allProcs;
/*
@@ -280,8 +289,23 @@ static TransactionId cachedXidIsNotInProgress = InvalidTransactionId;
/*
* Bookkeeping for tracking emulated transactions in recovery
*/
-static TransactionId *KnownAssignedXids;
-static bool *KnownAssignedXidsValid;
+
+static ShmemStructDesc KnownAssignedXidsShmemDesc = {
+ .name = "KnownAssignedXids",
+ .size = 0, /* dynamic */
+ .init_fn = NULL,
+};
+
+#define KnownAssignedXids ((TransactionId *) KnownAssignedXidsShmemDesc.ptr)
+
+static ShmemStructDesc KnownAssignedXidsValidShmemDesc = {
+ .name = "KnownAssignedXidsValid",
+ .size = 0, /* dynamic */
+ .init_fn = NULL,
+};
+
+#define KnownAssignedXidsValid ((bool *) KnownAssignedXidsValidShmemDesc.ptr)
+
static TransactionId latestObservedXid = InvalidTransactionId;
/*
@@ -372,18 +396,19 @@ static inline FullTransactionId FullXidRelativeTo(FullTransactionId rel,
static void GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons);
/*
- * Report shared-memory space needed by ProcArrayShmemInit
+ * Register the shared PGPROC array during postmaster startup.
*/
-Size
-ProcArrayShmemSize(void)
+void
+ProcArrayShmemRegister(void)
{
- Size size;
-
- /* Size of the ProcArray structure itself */
#define PROCARRAY_MAXPROCS (MaxBackends + max_prepared_xacts)
- size = offsetof(ProcArrayStruct, pgprocnos);
- size = add_size(size, mul_size(sizeof(int), PROCARRAY_MAXPROCS));
+ /* Create or attach to the ProcArray shared structure */
+ ProcArrayShmemDesc.size =
+ add_size(offsetof(ProcArrayStruct, pgprocnos),
+ mul_size(sizeof(int),
+ PROCARRAY_MAXPROCS));
+ ShmemRegisterStruct(&ProcArrayShmemDesc);
/*
* During Hot Standby processing we have a data structure called
@@ -403,64 +428,38 @@ ProcArrayShmemSize(void)
if (EnableHotStandby)
{
- size = add_size(size,
- mul_size(sizeof(TransactionId),
- TOTAL_MAX_CACHED_SUBXIDS));
- size = add_size(size,
- mul_size(sizeof(bool), TOTAL_MAX_CACHED_SUBXIDS));
+ KnownAssignedXidsShmemDesc.size =
+ mul_size(sizeof(TransactionId),
+ TOTAL_MAX_CACHED_SUBXIDS);
+ ShmemRegisterStruct(&KnownAssignedXidsShmemDesc);
+
+ KnownAssignedXidsValidShmemDesc.size =
+ mul_size(sizeof(bool), TOTAL_MAX_CACHED_SUBXIDS);
+ ShmemRegisterStruct(&KnownAssignedXidsValidShmemDesc);
}
-
- return size;
}
-/*
- * Initialize the shared PGPROC array during postmaster startup.
- */
-void
-ProcArrayShmemInit(void)
+static void
+ProcArrayShmemInit(void *arg)
{
- bool found;
-
- /* Create or attach to the ProcArray shared structure */
- procArray = (ProcArrayStruct *)
- ShmemInitStruct("Proc Array",
- add_size(offsetof(ProcArrayStruct, pgprocnos),
- mul_size(sizeof(int),
- PROCARRAY_MAXPROCS)),
- &found);
-
- if (!found)
- {
- /*
- * We're the first - initialize.
- */
- procArray->numProcs = 0;
- procArray->maxProcs = PROCARRAY_MAXPROCS;
- procArray->maxKnownAssignedXids = TOTAL_MAX_CACHED_SUBXIDS;
- procArray->numKnownAssignedXids = 0;
- procArray->tailKnownAssignedXids = 0;
- procArray->headKnownAssignedXids = 0;
- procArray->lastOverflowedXid = InvalidTransactionId;
- procArray->replication_slot_xmin = InvalidTransactionId;
- procArray->replication_slot_catalog_xmin = InvalidTransactionId;
- TransamVariables->xactCompletionCount = 1;
- }
+ procArray->numProcs = 0;
+ procArray->maxProcs = PROCARRAY_MAXPROCS;
+ procArray->maxKnownAssignedXids = TOTAL_MAX_CACHED_SUBXIDS;
+ procArray->numKnownAssignedXids = 0;
+ procArray->tailKnownAssignedXids = 0;
+ procArray->headKnownAssignedXids = 0;
+ procArray->lastOverflowedXid = InvalidTransactionId;
+ procArray->replication_slot_xmin = InvalidTransactionId;
+ procArray->replication_slot_catalog_xmin = InvalidTransactionId;
+ TransamVariables->xactCompletionCount = 1;
allProcs = ProcGlobal->allProcs;
+}
- /* Create or attach to the KnownAssignedXids arrays too, if needed */
- if (EnableHotStandby)
- {
- KnownAssignedXids = (TransactionId *)
- ShmemInitStruct("KnownAssignedXids",
- mul_size(sizeof(TransactionId),
- TOTAL_MAX_CACHED_SUBXIDS),
- &found);
- KnownAssignedXidsValid = (bool *)
- ShmemInitStruct("KnownAssignedXidsValid",
- mul_size(sizeof(bool), TOTAL_MAX_CACHED_SUBXIDS),
- &found);
- }
+static void
+ProcArrayShmemAttach(void *arg)
+{
+ allProcs = ProcGlobal->allProcs;
}
/*
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 8e56922dcea..5743f088324 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -102,7 +102,16 @@ struct ProcSignalHeader
#define BARRIER_CLEAR_BIT(flags, type) \
((flags) &= ~(((uint32) 1) << (uint32) (type)))
-NON_EXEC_STATIC ProcSignalHeader *ProcSignal = NULL;
+static void ProcSignalShmemInit(void *arg);
+
+static ShmemStructDesc ProcSignalShmemDesc = {
+ .name = "ProcSignal",
+ .size = 0, /* dynamic */
+ .init_fn = ProcSignalShmemInit,
+};
+
+#define ProcSignal ((ProcSignalHeader *) ProcSignalShmemDesc.ptr)
+
static ProcSignalSlot *MyProcSignalSlot = NULL;
static bool CheckProcSignal(ProcSignalReason reason);
@@ -110,51 +119,37 @@ static void CleanupProcSignalState(int status, Datum arg);
static void ResetProcSignalBarrierBits(uint32 flags);
/*
- * ProcSignalShmemSize
- * Compute space needed for ProcSignal's shared memory
+ * ProcSignalShmemRegister
+ * Register ProcSignal's shared memory needs at postmaster startup
*/
-Size
-ProcSignalShmemSize(void)
+void
+ProcSignalShmemRegister(void)
{
Size size;
size = mul_size(NumProcSignalSlots, sizeof(ProcSignalSlot));
size = add_size(size, offsetof(ProcSignalHeader, psh_slot));
- return size;
+
+ ProcSignalShmemDesc.size = size;
+ ShmemRegisterStruct(&ProcSignalShmemDesc);
}
-/*
- * ProcSignalShmemInit
- * Allocate and initialize ProcSignal's shared memory
- */
-void
-ProcSignalShmemInit(void)
+static void
+ProcSignalShmemInit(void *arg)
{
- Size size = ProcSignalShmemSize();
- bool found;
+ pg_atomic_init_u64(&ProcSignal->psh_barrierGeneration, 0);
- ProcSignal = (ProcSignalHeader *)
- ShmemInitStruct("ProcSignal", size, &found);
-
- /* If we're first, initialize. */
- if (!found)
+ for (int i = 0; i < NumProcSignalSlots; ++i)
{
- int i;
-
- pg_atomic_init_u64(&ProcSignal->psh_barrierGeneration, 0);
+ ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
- for (i = 0; i < NumProcSignalSlots; ++i)
- {
- ProcSignalSlot *slot = &ProcSignal->psh_slot[i];
-
- SpinLockInit(&slot->pss_mutex);
- pg_atomic_init_u32(&slot->pss_pid, 0);
- slot->pss_cancel_key_len = 0;
- MemSet(slot->pss_signalFlags, 0, sizeof(slot->pss_signalFlags));
- pg_atomic_init_u64(&slot->pss_barrierGeneration, PG_UINT64_MAX);
- pg_atomic_init_u32(&slot->pss_barrierCheckMask, 0);
- ConditionVariableInit(&slot->pss_barrierCV);
- }
+ SpinLockInit(&slot->pss_mutex);
+ pg_atomic_init_u32(&slot->pss_pid, 0);
+ slot->pss_cancel_key_len = 0;
+ MemSet(slot->pss_signalFlags, 0, sizeof(slot->pss_signalFlags));
+ pg_atomic_init_u64(&slot->pss_barrierGeneration, PG_UINT64_MAX);
+ pg_atomic_init_u32(&slot->pss_barrierCheckMask, 0);
+ ConditionVariableInit(&slot->pss_barrierCV);
}
}
diff --git a/src/backend/storage/ipc/shmem.c b/src/backend/storage/ipc/shmem.c
index 9f362ce8641..2ba6385ffc6 100644
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -19,6 +19,8 @@
* methods). The routines in this file are used for allocating and
* binding to shared memory data structures.
*
+ * FIXME: NOTES below are outdated
+ *
* NOTES:
* (a) There are three kinds of shared memory data structures
* available to POSTGRES: fixed-size structures, queues and hash
@@ -76,6 +78,16 @@
#include "storage/spin.h"
#include "utils/builtins.h"
+/* size constants for the shmem index table */
+ /* max size of data structure string name */
+#define SHMEM_INDEX_KEYSIZE (48)
+ /* estimated size of the shmem index table (not a hard limit) */
+#define SHMEM_INDEX_SIZE (64)
+
+/* these are in postmaster private memory */
+static ShmemStructDesc *registry[SHMEM_INDEX_SIZE];
+static int num_registrations = 0;
+
/*
* This is the first data structure stored in the shared memory segment, at
* the offset that PGShmemHeader->content_offset points to. Allocations by
@@ -95,6 +107,9 @@ typedef struct ShmemAllocatorData
static void *ShmemAllocRaw(Size size, Size *allocated_size);
+static void shmem_hash_init(void *arg);
+static void shmem_hash_attach(void *arg);
+
/* shared memory global variables */
static PGShmemHeader *ShmemSegHdr; /* shared mem segment header */
@@ -103,13 +118,134 @@ static void *ShmemEnd; /* end+1 address of shared memory */
static ShmemAllocatorData *ShmemAllocator;
slock_t *ShmemLock; /* points to ShmemAllocator->shmem_lock */
-static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+
+
+static ShmemHashDesc ShmemIndexHashDesc = {
+ .name = "ShmemIndex",
+ .init_size = SHMEM_INDEX_SIZE,
+ .max_size = SHMEM_INDEX_SIZE,
+};
+
+ /* primary index hashtable for shmem */
+#define ShmemIndex (ShmemIndexHashDesc.ptr)
+
/* To get reliable results for NUMA inquiry we need to "touch pages" once */
static bool firstNumaTouch = true;
Datum pg_numa_available(PG_FUNCTION_ARGS);
+
+void
+ShmemRegisterStruct(ShmemStructDesc *desc)
+{
+ elog(DEBUG2, "REGISTER: %s with size %zd", desc->name, desc->size);
+
+ registry[num_registrations++] = desc;
+}
+
+size_t
+ShmemRegisteredSize(void)
+{
+ size_t size;
+
+ size = 0;
+ for (int i = 0; i < num_registrations; i++)
+ {
+ size = add_size(size, registry[i]->size);
+ size = add_size(size, registry[i]->extra_size);
+ }
+
+ elog(DEBUG2, "SIZE: total %zd", size);
+
+ return size;
+}
+
+void
+ShmemInitRegistered(void)
+{
+ for (int i = 0; i < num_registrations; i++)
+ {
+ size_t allocated_size;
+ void *structPtr;
+ bool found;
+ ShmemIndexEnt *result;
+
+ elog(DEBUG2, "INIT [%d/%d]: %s", i, num_registrations, registry[i]->name);
+
+ /* look it up in the shmem index */
+ result = (ShmemIndexEnt *)
+ hash_search(ShmemIndex, registry[i]->name, HASH_ENTER_NULL, &found);
+ if (!result)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("could not create ShmemIndex entry for data structure \"%s\"",
+ registry[i]->name)));
+ }
+ if (found)
+ elog(ERROR, "shmem struct \"%s\" is already initialized", registry[i]->name);
+
+ /* allocate and initialize it */
+ structPtr = ShmemAllocRaw(registry[i]->size, &allocated_size);
+ if (structPtr == NULL)
+ {
+ /* out of memory; remove the failed ShmemIndex entry */
+ hash_search(ShmemIndex, registry[i]->name, HASH_REMOVE, NULL);
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("not enough shared memory for data structure"
+ " \"%s\" (%zu bytes requested)",
+ registry[i]->name, registry[i]->size)));
+ }
+ result->size = registry[i]->size;
+ result->allocated_size = allocated_size;
+ result->location = structPtr;
+
+ registry[i]->ptr = structPtr;
+ if (registry[i]->init_fn)
+ registry[i]->init_fn(registry[i]->init_fn_arg);
+ }
+}
+
+#ifdef EXEC_BACKEND
+void
+ShmemAttachRegistered(void)
+{
+ /* Must be initializing a (non-standalone) backend */
+ Assert(IsUnderPostmaster);
+ Assert(ShmemAllocator->index != NULL);
+
+ LWLockAcquire(ShmemIndexLock, LW_EXCLUSIVE);
+
+ for (int i = 0; i < num_registrations; i++)
+ {
+ bool found;
+ ShmemIndexEnt *result;
+
+ elog(LOG, "ATTACH [%d/%d]: %s", i, num_registrations, registry[i]->name);
+
+ /* look it up in the shmem index */
+ result = (ShmemIndexEnt *)
+ hash_search(ShmemIndex, registry[i]->name, HASH_FIND, &found);
+ if (!found)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("could not find ShmemIndex entry for data structure \"%s\"",
+ registry[i]->name)));
+ }
+
+ registry[i]->ptr = result->location;
+
+ if (registry[i]->attach_fn)
+ registry[i]->attach_fn(registry[i]->attach_fn_arg);
+ }
+
+ LWLockRelease(ShmemIndexLock);
+}
+#endif
+
/*
* InitShmemAllocator() --- set up basic pointers to shared memory.
*
@@ -292,6 +428,98 @@ InitShmemIndex(void)
HASH_ELEM | HASH_STRINGS);
}
+/*
+ * ShmemInitHash -- Create and initialize, or attach to, a
+ * shared memory hash table.
+ *
+ * We assume caller is doing some kind of synchronization
+ * so that two processes don't try to create/initialize the same
+ * table at once. (In practice, all creations are done in the postmaster
+ * process; child processes should always be attaching to existing tables.)
+ *
+ * max_size is the estimated maximum number of hashtable entries. This is
+ * not a hard limit, but the access efficiency will degrade if it is
+ * exceeded substantially (since it's used to compute directory size and
+ * the hash table buckets will get overfull).
+ *
+ * init_size is the number of hashtable entries to preallocate. For a table
+ * whose maximum size is certain, this should be equal to max_size; that
+ * ensures that no run-time out-of-shared-memory failures can occur.
+ *
+ * *infoP and hash_flags must specify at least the entry sizes and key
+ * comparison semantics (see hash_create()). Flag bits and values specific
+ * to shared-memory hash tables are added here, except that callers may
+ * choose to specify HASH_PARTITION and/or HASH_FIXED_SIZE.
+ *
+ * Note: before Postgres 9.0, this function returned NULL for some failure
+ * cases. Now, it always throws error instead, so callers need not check
+ * for NULL.
+ */
+void
+ShmemRegisterHash(ShmemHashDesc *desc, /* configuration */
+ HASHCTL *infoP, /* info about key and bucket size */
+ int hash_flags) /* info about infoP */
+{
+ /*
+ * Hash tables allocated in shared memory have a fixed directory; it can't
+ * grow or other backends wouldn't be able to find it. So, make sure we
+ * make it big enough to start with.
+ *
+ * The shared memory allocator must be specified too.
+ */
+ infoP->dsize = infoP->max_dsize = hash_select_dirsize(desc->max_size);
+ infoP->alloc = ShmemAllocNoError;
+ hash_flags |= HASH_SHARED_MEM | HASH_ALLOC | HASH_DIRSIZE;
+
+ /* look it up in the shmem index */
+ memset(&desc->base_desc, 0, sizeof(desc->base_desc));
+ desc->base_desc.name = desc->name;
+ desc->base_desc.size = hash_get_shared_size(infoP, hash_flags);
+ desc->base_desc.init_fn = shmem_hash_init;
+ desc->base_desc.init_fn_arg = desc;
+ desc->base_desc.attach_fn = shmem_hash_attach;
+ desc->base_desc.attach_fn_arg = desc;
+
+ desc->base_desc.extra_size = hash_estimate_size(desc->max_size, infoP->entrysize) - desc->base_desc.size;
+
+ desc->hash_flags = hash_flags;
+ desc->infoP = MemoryContextAlloc(TopMemoryContext, sizeof(HASHCTL));
+ memcpy(desc->infoP, infoP, sizeof(HASHCTL));
+
+ ShmemRegisterStruct(&desc->base_desc);
+}
+
+static void
+shmem_hash_init(void *arg)
+{
+ ShmemHashDesc *desc = (ShmemHashDesc *) arg;
+ int hash_flags = desc->hash_flags;
+
+ /* Pass location of hashtable header to hash_create */
+ desc->ptr = desc->base_desc.ptr;
+ desc->infoP->hctl = (HASHHDR *) desc->ptr;
+
+ desc->ptr = hash_create(desc->name, desc->init_size, desc->infoP, hash_flags);
+}
+
+static void
+shmem_hash_attach(void *arg)
+{
+ ShmemHashDesc *desc = (ShmemHashDesc *) arg;
+ int hash_flags = desc->hash_flags;
+
+ /*
+ * if it already exists, attach to it rather than allocate and initialize
+ * new space
+ */
+ hash_flags |= HASH_ATTACH;
+
+ /* Pass location of hashtable header to hash_create */
+ desc->infoP->hctl = (HASHHDR *) desc->ptr;
+
+ desc->ptr = hash_create(desc->name, desc->init_size, desc->infoP, hash_flags);
+}
+
/*
* ShmemInitHash -- Create and initialize, or attach to, a
* shared memory hash table.
diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c
index a7a7cc4f0a9..0fe0f256971 100644
--- a/src/backend/storage/ipc/sinvaladt.c
+++ b/src/backend/storage/ipc/sinvaladt.c
@@ -203,7 +203,16 @@ typedef struct SISeg
*/
#define NumProcStateSlots (MaxBackends + NUM_AUXILIARY_PROCS)
-static SISeg *shmInvalBuffer; /* pointer to the shared inval buffer */
+static void SharedInvalShmemInit(void *arg);
+
+static ShmemStructDesc SharedInvalShmemDesc = {
+ .name = "shmInvalBuffer",
+ .size = 0, /* dynamic */
+ .init_fn = SharedInvalShmemInit,
+};
+
+/* pointer to the shared inval buffer */
+#define shmInvalBuffer ((SISeg *) SharedInvalShmemDesc.ptr)
static LocalTransactionId nextLocalTransactionId;
@@ -212,10 +221,11 @@ static void CleanupInvalidationState(int status, Datum arg);
/*
- * SharedInvalShmemSize --- return shared-memory space needed
+ * SharedInvalShmemRegister
+ * Register shared memory needs for the SI message buffer
*/
-Size
-SharedInvalShmemSize(void)
+void
+SharedInvalShmemRegister(void)
{
Size size;
@@ -223,26 +233,17 @@ SharedInvalShmemSize(void)
size = add_size(size, mul_size(sizeof(ProcState), NumProcStateSlots)); /* procState */
size = add_size(size, mul_size(sizeof(int), NumProcStateSlots)); /* pgprocnos */
- return size;
+ /* Allocate space in shared memory */
+ SharedInvalShmemDesc.size = size;
+ ShmemRegisterStruct(&SharedInvalShmemDesc);
}
-/*
- * SharedInvalShmemInit
- * Create and initialize the SI message buffer
- */
-void
-SharedInvalShmemInit(void)
+static void
+SharedInvalShmemInit(void *arg)
{
int i;
- bool found;
-
- /* Allocate space in shared memory */
- shmInvalBuffer = (SISeg *)
- ShmemInitStruct("shmInvalBuffer", SharedInvalShmemSize(), &found);
- if (found)
- return;
- /* Clear message counters, save size of procState array, init spinlock */
+ /* Clear message counters, save size of procState array FIXME, init spinlock */
shmInvalBuffer->minMsgNum = 0;
shmInvalBuffer->maxMsgNum = 0;
shmInvalBuffer->nextThreshold = CLEANUP_MIN;
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index 8560a903bc8..96432c633bf 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -73,13 +73,33 @@ PGPROC *MyProc = NULL;
* relatively infrequently (only at backend startup or shutdown) and not for
* very long, so a spinlock is okay.
*/
-NON_EXEC_STATIC slock_t *ProcStructLock = NULL;
+#define ProcStructLock (&ProcGlobal->freeProcsLock)
+
+static void ProcGlobalShmemInit(void *arg);
+
+static ShmemStructDesc ProcGlobalShmemDesc = {
+ .name = "Proc Header",
+ .size = sizeof(PROC_HDR),
+ .init_fn = ProcGlobalShmemInit,
+};
+
+static ShmemStructDesc ProcGlobalAllProcsShmemDesc = {
+ .name = "PGPROC structures",
+ .size = 0, /* dynamic */
+};
+
+static ShmemStructDesc FastPathLockArrayShmemDesc = {
+ .name = "Fast-Path Lock Array",
+ .size = 0, /* dynamic */
+};
/* Pointers to shared-memory structures */
PROC_HDR *ProcGlobal = NULL;
NON_EXEC_STATIC PGPROC *AuxiliaryProcs = NULL;
PGPROC *PreparedXactProcs = NULL;
+static uint32 TotalProcs;
+
/* Is a deadlock check pending? */
static volatile sig_atomic_t got_deadlock_timeout;
@@ -89,24 +109,6 @@ static void AuxiliaryProcKill(int code, Datum arg);
static DeadLockState CheckDeadLock(void);
-/*
- * Report shared-memory space needed by PGPROC.
- */
-static Size
-PGProcShmemSize(void)
-{
- Size size = 0;
- Size TotalProcs =
- add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
-
- size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC)));
- size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids)));
- size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->subxidStates)));
- size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->statusFlags)));
-
- return size;
-}
-
/*
* Report shared-memory space needed by Fast-Path locks.
*/
@@ -114,8 +116,6 @@ static Size
FastPathLockShmemSize(void)
{
Size size = 0;
- Size TotalProcs =
- add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
Size fpLockBitsSize,
fpRelIdSize;
@@ -131,25 +131,6 @@ FastPathLockShmemSize(void)
return size;
}
-/*
- * Report shared-memory space needed by InitProcGlobal.
- */
-Size
-ProcGlobalShmemSize(void)
-{
- Size size = 0;
-
- /* ProcGlobal */
- size = add_size(size, sizeof(PROC_HDR));
- size = add_size(size, sizeof(slock_t));
-
- size = add_size(size, PGSemaphoreShmemSize(ProcGlobalSemas()));
- size = add_size(size, PGProcShmemSize());
- size = add_size(size, FastPathLockShmemSize());
-
- return size;
-}
-
/*
* Report number of semaphores needed by InitProcGlobal.
*/
@@ -184,35 +165,63 @@ ProcGlobalSemas(void)
* implementation typically requires us to create semaphores in the
* postmaster, not in backends.
*
- * Note: this is NOT called by individual backends under a postmaster,
+ * Note: this is NOT called by individual backends under a postmaster, XXX
* not even in the EXEC_BACKEND case. The ProcGlobal and AuxiliaryProcs
* pointers must be propagated specially for EXEC_BACKEND operation.
*/
void
-InitProcGlobal(void)
+ProcGlobalShmemRegister(void)
+{
+ Size size = 0;
+
+ /*
+ * Reserve all the PGPROC structures we'll need. There are
+ * six separate consumers: (1) normal backends, (2) autovacuum workers and
+ * special workers, (3) background workers, (4) walsenders, (5) auxiliary
+ * processes, and (6) prepared transactions. (For largely-historical
+ * reasons, we combine autovacuum and special workers into one category
+ * with a single freelist.) Each PGPROC structure is dedicated to exactly
+ * one of these purposes, and they do not move between groups.
+ */
+ TotalProcs =
+ add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts));
+
+ size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC)));
+
+ /* FIXME: the sizeofs look dangerous because ProcGlobal is not initialized yet */
+ size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids)));
+ size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->subxidStates)));
+ size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->statusFlags)));
+
+ ProcGlobalAllProcsShmemDesc.size = size;
+ ShmemRegisterStruct(&ProcGlobalAllProcsShmemDesc);
+
+ FastPathLockArrayShmemDesc.size = FastPathLockShmemSize();
+ ShmemRegisterStruct(&FastPathLockArrayShmemDesc);
+
+ /*
+ * Create the ProcGlobal shared structure last. Its init callback
+ * initializes the others too.
+ */
+ ShmemRegisterStruct(&ProcGlobalShmemDesc);
+}
+
+static void
+ProcGlobalShmemInit(void *arg)
{
+ char *ptr;
+ size_t requestSize;
PGPROC *procs;
int i,
j;
- bool found;
- uint32 TotalProcs = MaxBackends + NUM_AUXILIARY_PROCS + max_prepared_xacts;
-
/* Used for setup of per-backend fast-path slots. */
char *fpPtr,
*fpEndPtr PG_USED_FOR_ASSERTS_ONLY;
Size fpLockBitsSize,
fpRelIdSize;
- Size requestSize;
- char *ptr;
- /* Create the ProcGlobal shared structure */
- ProcGlobal = (PROC_HDR *)
- ShmemInitStruct("Proc Header", sizeof(PROC_HDR), &found);
- Assert(!found);
+ ProcGlobal = ProcGlobalShmemDesc.ptr;
- /*
- * Initialize the data structures.
- */
ProcGlobal->spins_per_delay = DEFAULT_SPINS_PER_DELAY;
dlist_init(&ProcGlobal->freeProcs);
dlist_init(&ProcGlobal->autovacFreeProcs);
@@ -223,23 +232,11 @@ InitProcGlobal(void)
ProcGlobal->checkpointerProc = INVALID_PROC_NUMBER;
pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PROC_NUMBER);
pg_atomic_init_u32(&ProcGlobal->clogGroupFirst, INVALID_PROC_NUMBER);
+ SpinLockInit(ProcStructLock);
- /*
- * Create and initialize all the PGPROC structures we'll need. There are
- * six separate consumers: (1) normal backends, (2) autovacuum workers and
- * special workers, (3) background workers, (4) walsenders, (5) auxiliary
- * processes, and (6) prepared transactions. (For largely-historical
- * reasons, we combine autovacuum and special workers into one category
- * with a single freelist.) Each PGPROC structure is dedicated to exactly
- * one of these purposes, and they do not move between groups.
- */
- requestSize = PGProcShmemSize();
-
- ptr = ShmemInitStruct("PGPROC structures",
- requestSize,
- &found);
-
- MemSet(ptr, 0, requestSize);
+ ptr = ProcGlobalAllProcsShmemDesc.ptr;
+ requestSize = ProcGlobalAllProcsShmemDesc.size;
+ memset(ptr, 0, requestSize);
procs = (PGPROC *) ptr;
ptr = ptr + TotalProcs * sizeof(PGPROC);
@@ -275,20 +272,13 @@ InitProcGlobal(void)
fpLockBitsSize = MAXALIGN(FastPathLockGroupsPerBackend * sizeof(uint64));
fpRelIdSize = MAXALIGN(FastPathLockSlotsPerBackend() * sizeof(Oid));
- requestSize = FastPathLockShmemSize();
-
- fpPtr = ShmemInitStruct("Fast-Path Lock Array",
- requestSize,
- &found);
-
- MemSet(fpPtr, 0, requestSize);
+ fpPtr = FastPathLockArrayShmemDesc.ptr;
+ requestSize = FastPathLockArrayShmemDesc.size;
+ memset(fpPtr, 0, requestSize);
/* For asserts checking we did not overflow. */
fpEndPtr = fpPtr + requestSize;
- /* Reserve space for semaphores. */
- PGReserveSemaphores(ProcGlobalSemas());
-
for (i = 0; i < TotalProcs; i++)
{
PGPROC *proc = &procs[i];
@@ -378,12 +368,6 @@ InitProcGlobal(void)
*/
AuxiliaryProcs = &procs[MaxBackends];
PreparedXactProcs = &procs[MaxBackends + NUM_AUXILIARY_PROCS];
-
- /* Create ProcStructLock spinlock, too */
- ProcStructLock = (slock_t *) ShmemInitStruct("ProcStructLock spinlock",
- sizeof(slock_t),
- &found);
- SpinLockInit(ProcStructLock);
}
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 02e9aaa6bca..eed188416ee 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -4117,6 +4117,8 @@ PostgresSingleUserMain(int argc, char *argv[],
* shared memory, determine the value of any runtime-computed GUCs that
* depend on the amount of shared memory required.
*/
+ RegisterShmemStructs();
+
InitializeShmemGUCs();
/*
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index 6fa91bfcdc0..49d476e9d5c 100644
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -15,7 +15,9 @@
#define TRANSAM_H
#include "access/xlogdefs.h"
-
+#ifndef FRONTEND
+#include "storage/shmem.h"
+#endif
/* ----------------
* Special transaction ID values
@@ -330,7 +332,10 @@ TransactionIdFollowsOrEquals(TransactionId id1, TransactionId id2)
extern bool TransactionStartedDuringRecovery(void);
/* in transam/varsup.c */
-extern PGDLLIMPORT TransamVariablesData *TransamVariables;
+#ifndef FRONTEND
+extern PGDLLIMPORT struct ShmemStructDesc TransamVariablesShmemDesc;
+#define TransamVariables ((TransamVariablesData *) TransamVariablesShmemDesc.ptr)
+#endif
/*
* prototypes for functions in transam/transam.c
@@ -345,8 +350,7 @@ extern TransactionId TransactionIdLatest(TransactionId mainxid,
extern XLogRecPtr TransactionIdGetCommitLSN(TransactionId xid);
/* in transam/varsup.c */
-extern Size VarsupShmemSize(void);
-extern void VarsupShmemInit(void);
+extern void VarsupShmemRegister(void);
extern FullTransactionId GetNewTransactionId(bool isSubXact);
extern void AdvanceNextFullTransactionIdPastXid(TransactionId xid);
extern FullTransactionId ReadNextFullTransactionId(void);
diff --git a/src/include/storage/dsm_registry.h b/src/include/storage/dsm_registry.h
index 506fae2c9ca..9a1b4d982af 100644
--- a/src/include/storage/dsm_registry.h
+++ b/src/include/storage/dsm_registry.h
@@ -22,7 +22,6 @@ extern dsa_area *GetNamedDSA(const char *name, bool *found);
extern dshash_table *GetNamedDSHash(const char *name,
const dshash_parameters *params,
bool *found);
-extern Size DSMRegistryShmemSize(void);
-extern void DSMRegistryShmemInit(void);
+extern void DSMRegistryShmemRegister(void);
#endif /* DSM_REGISTRY_H */
diff --git a/src/include/storage/ipc.h b/src/include/storage/ipc.h
index da32787ab51..8a3b71ad5d3 100644
--- a/src/include/storage/ipc.h
+++ b/src/include/storage/ipc.h
@@ -77,6 +77,7 @@ extern void check_on_shmem_exit_lists_are_empty(void);
/* ipci.c */
extern PGDLLIMPORT shmem_startup_hook_type shmem_startup_hook;
+extern void RegisterShmemStructs(void);
extern Size CalculateShmemSize(void);
extern void CreateSharedMemoryAndSemaphores(void);
#ifdef EXEC_BACKEND
diff --git a/src/include/storage/pmsignal.h b/src/include/storage/pmsignal.h
index 206fb78f8a5..7cdc4852334 100644
--- a/src/include/storage/pmsignal.h
+++ b/src/include/storage/pmsignal.h
@@ -66,8 +66,7 @@ extern PGDLLIMPORT volatile PMSignalData *PMSignalState;
/*
* prototypes for functions in pmsignal.c
*/
-extern Size PMSignalShmemSize(void);
-extern void PMSignalShmemInit(void);
+extern void PMSignalShmemRegister(void);
extern void SendPostmasterSignal(PMSignalReason reason);
extern bool CheckPostmasterSignal(PMSignalReason reason);
extern void SetQuitSignalReason(QuitSignalReason reason);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 679f0624f92..37023e1a93f 100644
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -418,6 +418,9 @@ typedef struct PROC_HDR
dlist_head bgworkerFreeProcs;
/* Head of list of walsender free PGPROC structures */
dlist_head walsenderFreeProcs;
+
+ slock_t freeProcsLock;
+
/* First pgproc waiting for group XID clear */
pg_atomic_uint32 procArrayGroupFirst;
/* First pgproc waiting for group transaction status update */
@@ -488,7 +491,7 @@ extern PGDLLIMPORT PGPROC *AuxiliaryProcs;
* Function Prototypes
*/
extern int ProcGlobalSemas(void);
-extern Size ProcGlobalShmemSize(void);
+extern void ProcGlobalShmemRegister(void);
extern void InitProcGlobal(void);
extern void InitProcess(void);
extern void InitProcessPhase2(void);
diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h
index 3a8593f87ba..41753c3a630 100644
--- a/src/include/storage/procarray.h
+++ b/src/include/storage/procarray.h
@@ -20,8 +20,7 @@
#include "utils/snapshot.h"
-extern Size ProcArrayShmemSize(void);
-extern void ProcArrayShmemInit(void);
+extern void ProcArrayShmemRegister(void);
extern void ProcArrayAdd(PGPROC *proc);
extern void ProcArrayRemove(PGPROC *proc, TransactionId latestXid);
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index e52b8eb7697..f2df1f30c5f 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -71,8 +71,7 @@ typedef enum
/*
* prototypes for functions in procsignal.c
*/
-extern Size ProcSignalShmemSize(void);
-extern void ProcSignalShmemInit(void);
+extern void ProcSignalShmemRegister(void);
extern void ProcSignalInit(const uint8 *cancel_key, int cancel_key_len);
extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
diff --git a/src/include/storage/shmem.h b/src/include/storage/shmem.h
index 89d45287c17..40e2fc17056 100644
--- a/src/include/storage/shmem.h
+++ b/src/include/storage/shmem.h
@@ -24,6 +24,53 @@
#include "storage/spin.h"
#include "utils/hsearch.h"
+typedef void (*ShmemInitCallback) (void *arg);
+typedef void (*ShmemAttachCallback) (void *arg);
+
+/*
+ * Descriptor for a named area or struct in shared memory
+ */
+typedef struct ShmemStructDesc
+{
+ /* Name of the shared memory area. Must be unique across the system */
+ const char *name;
+
+ size_t size;
+
+ size_t alignment;
+ ShmemInitCallback init_fn;
+ ShmemInitCallback attach_fn;
+ void *init_fn_arg;
+ void *attach_fn_arg;
+
+ /*
+ * Extra space to allocated in the shared memory segment, but it's not
+ * part of the struct itself. This is used for shared memory hash tables
+ * that can grow beyond the initial size when more buckets are allocated.
+ */
+ size_t extra_size;
+
+ /* Pointer to the shared memory area, when it's allocated. */
+ void *ptr;
+} ShmemStructDesc;
+
+/*
+ * Descriptor for shared memory hash table
+ */
+typedef struct ShmemHashDesc
+{
+ const char *name;
+
+ int hash_flags;
+
+ size_t init_size; /* initial number of entries */
+ size_t max_size; /* max number of entries */
+ HASHCTL *infoP;
+
+ HTAB *ptr;
+
+ ShmemStructDesc base_desc;
+} ShmemHashDesc;
/* shmem.c */
extern PGDLLIMPORT slock_t *ShmemLock;
@@ -34,9 +81,19 @@ extern void *ShmemAlloc(Size size);
extern void *ShmemAllocNoError(Size size);
extern bool ShmemAddrIsValid(const void *addr);
extern void InitShmemIndex(void);
+
+extern void ShmemRegisterHash(ShmemHashDesc *desc, HASHCTL *infoP, int hash_flags);
+extern void ShmemRegisterStruct(ShmemStructDesc *desc);
+
+/* Legacy functions */
extern HTAB *ShmemInitHash(const char *name, int64 init_size, int64 max_size,
HASHCTL *infoP, int hash_flags);
extern void *ShmemInitStruct(const char *name, Size size, bool *foundPtr);
+
+extern size_t ShmemRegisteredSize(void);
+extern void ShmemInitRegistered(void);
+extern void ShmemAttachRegistered(void);
+
extern Size add_size(Size s1, Size s2);
extern Size mul_size(Size s1, Size s2);
diff --git a/src/include/storage/sinvaladt.h b/src/include/storage/sinvaladt.h
index a1694500a85..4edba2936e6 100644
--- a/src/include/storage/sinvaladt.h
+++ b/src/include/storage/sinvaladt.h
@@ -28,8 +28,7 @@
/*
* prototypes for functions in sinvaladt.c
*/
-extern Size SharedInvalShmemSize(void);
-extern void SharedInvalShmemInit(void);
+extern void SharedInvalShmemRegister(void);
extern void SharedInvalBackendInit(bool sendOnly);
extern void SIInsertDataEntries(const SharedInvalidationMessage *data, int n);
--
2.47.3
