Here's a new version of the patch set in which I've attempted to address
the feedback in this thread. Note that 0001 is being tracked in a separate
thread [0], but it is basically a prerequisite for adding the documentation
for this feature, so that's why I've also added it here.
[0] https://postgr.es/m/20240112041430.GA3557928%40nathanxps13
--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com
>From 7cf22727a96757bf212ec106bd471bf55a6981b9 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nat...@postgresql.org>
Date: Thu, 11 Jan 2024 21:55:25 -0600
Subject: [PATCH v6 1/3] reorganize shared memory and lwlocks documentation
---
doc/src/sgml/xfunc.sgml | 182 +++++++++++++++++++++++++---------------
1 file changed, 114 insertions(+), 68 deletions(-)
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 89116ae74c..0ba52b41d4 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -3397,90 +3397,136 @@ CREATE FUNCTION make_array(anyelement) RETURNS anyarray
</sect2>
<sect2 id="xfunc-shared-addin">
- <title>Shared Memory and LWLocks</title>
+ <title>Shared Memory</title>
- <para>
- Add-ins can reserve LWLocks and an allocation of shared memory on server
- startup. The add-in's shared library must be preloaded by specifying
- it in
- <xref linkend="guc-shared-preload-libraries"/><indexterm><primary>shared_preload_libraries</primary></indexterm>.
- The shared library should register a <literal>shmem_request_hook</literal>
- in its <function>_PG_init</function> function. This
- <literal>shmem_request_hook</literal> can reserve LWLocks or shared memory.
- Shared memory is reserved by calling:
+ <sect3 id="xfunc-shared-addin-at-startup">
+ <title>Requesting Shared Memory at Startup</title>
+
+ <para>
+ Add-ins can reserve shared memory on server startup. To do so, the
+ add-in's shared library must be preloaded by specifying it in
+ <xref linkend="guc-shared-preload-libraries"/><indexterm><primary>shared_preload_libraries</primary></indexterm>.
+ The shared library should also register a
+ <literal>shmem_request_hook</literal> in its
+ <function>_PG_init</function> function. This
+ <literal>shmem_request_hook</literal> can reserve shared memory by
+ calling:
<programlisting>
void RequestAddinShmemSpace(int size)
</programlisting>
- from your <literal>shmem_request_hook</literal>.
- </para>
- <para>
- LWLocks are reserved by calling:
+ Each backend sould obtain a pointer to the reserved shared memory by
+ calling:
+<programlisting>
+void *ShmemInitStruct(const char *name, Size size, bool *foundPtr)
+</programlisting>
+ If this function sets <literal>foundPtr</literal> to
+ <literal>false</literal>, the caller should proceed to initialize the
+ contents of the reserved shared memory. If <literal>foundPtr</literal>
+ is set to <literal>true</literal>, the shared memory was already
+ initialized by another backend, and the caller need not initialize
+ further.
+ </para>
+
+ <para>
+ To avoid race conditions, each backend should use the LWLock
+ <function>AddinShmemInitLock</function> when initializing its allocation
+ of shared memory, as shown here:
+<programlisting>
+static mystruct *ptr = NULL;
+bool found;
+
+LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
+ptr = ShmemInitStruct("my struct name", size, &found);
+if (!found)
+{
+ ... initialize contents of shared memory ...
+ ptr->locks = GetNamedLWLockTranche("my tranche name");
+}
+LWLockRelease(AddinShmemInitLock);
+</programlisting>
+ <literal>shmem_startup_hook</literal> provides a convenient place for the
+ initialization code, but it is not strictly required that all such code
+ be placed in this hook. Each backend will execute the registered
+ <literal>shmem_startup_hook</literal> shortly after it attaches to shared
+ memory. Note that add-ins should still acquire
+ <function>AddinShmemInitLock</function> within this hook, as shown in the
+ example above.
+ </para>
+
+ <para>
+ An example of a <literal>shmem_request_hook</literal> and
+ <literal>shmem_startup_hook</literal> can be found in
+ <filename>contrib/pg_stat_statements/pg_stat_statements.c</filename> in
+ the <productname>PostgreSQL</productname> source tree.
+ </para>
+ </sect3>
+ </sect2>
+
+ <sect2 id="xfunc-addin-lwlocks">
+ <title>LWLocks</title>
+
+ <sect3 id="xfunc-addin-lwlocks-at-startup">
+ <title>Requesting LWLocks at Startup</title>
+
+ <para>
+ Add-ins can reserve LWLocks on server startup. Like with shared memory,
+ the add-in's shared library must be preloaded by specifying it in
+ <xref linkend="guc-shared-preload-libraries"/><indexterm><primary>shared_preload_libraries</primary></indexterm>,
+ and the shared library should register a
+ <literal>shmem_request_hook</literal> in its
+ <function>_PG_init</function> function. This
+ <literal>shmem_request_hook</literal> can reserve LWLocks by calling:
<programlisting>
void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks)
</programlisting>
- from your <literal>shmem_request_hook</literal>. This will ensure that an array of
- <literal>num_lwlocks</literal> LWLocks is available under the name
- <literal>tranche_name</literal>. Use <function>GetNamedLWLockTranche</function>
- to get a pointer to this array.
- </para>
- <para>
- An example of a <literal>shmem_request_hook</literal> can be found in
- <filename>contrib/pg_stat_statements/pg_stat_statements.c</filename> in the
- <productname>PostgreSQL</productname> source tree.
- </para>
- <para>
- There is another, more flexible method of obtaining LWLocks. First,
- allocate a <literal>tranche_id</literal> from a shared counter by
- calling:
+ This ensures that an array of <literal>num_lwlocks</literal> LWLocks is
+ available under the name <literal>tranche_name</literal>. A pointer to
+ this array can be obtained by calling:
<programlisting>
-int LWLockNewTrancheId(void)
+LWLockPadded *GetNamedLWLockTranche(const char *tranche_name)
</programlisting>
- Next, each individual process using the <literal>tranche_id</literal>
- should associate it with a <literal>tranche_name</literal> by calling:
+ </para>
+ </sect3>
+
+ <sect3 id="xfunc-addin-lwlocks-after-startup">
+ <title>Requesting LWLocks After Startup</title>
+
+ <para>
+ There is another, more flexible method of obtaining LWLocks that can be
+ done after server startup and outside a
+ <literal>shmem_request_hook</literal>. To do so, first allocate a
+ <literal>tranche_id</literal> by calling:
<programlisting>
-void LWLockRegisterTranche(int tranche_id, const char *tranche_name)
+int LWLockNewTrancheId(void)
</programlisting>
- It is also required to call <function>LWLockInitialize</function> once
- per LWLock, passing the <literal>tranche_id</literal> as argument:
+ Next, initialize each LWLock, passing the new
+ <literal>tranche_id</literal> as an argument:
<programlisting>
void LWLockInitialize(LWLock *lock, int tranche_id)
</programlisting>
- A complete usage example of <function>LWLockNewTrancheId</function>,
- <function>LWLockInitialize</function> and
- <function>LWLockRegisterTranche</function> can be found in
- <filename>contrib/pg_prewarm/autoprewarm.c</filename> in the
- <productname>PostgreSQL</productname> source tree.
- </para>
- <para>
- To avoid possible race-conditions, each backend should use the LWLock
- <function>AddinShmemInitLock</function> when connecting to and initializing
- its allocation of shared memory, as shown here:
-<programlisting>
-static mystruct *ptr = NULL;
+ Similar to shared memory, each backend should ensure that only one
+ process allocates a new <literal>tranche_id</literal> and initializes
+ each new LWLock. One way to do this is to only call these functions in
+ your shared memory initialization code with the
+ <function>AddinShmemInitLock</function> held exclusively.
+ </para>
-if (!ptr)
-{
- bool found;
-
- LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
- ptr = ShmemInitStruct("my struct name", size, &found);
- if (!found)
- {
- initialize contents of shmem area;
- acquire any requested LWLocks using:
- ptr->locks = GetNamedLWLockTranche("my tranche name");
- }
- LWLockRelease(AddinShmemInitLock);
-}
+ <para>
+ Finally, each backend using the <literal>tranche_id</literal> should
+ associate it with a <literal>tranche_name</literal> by calling:
+<programlisting>
+void LWLockRegisterTranche(int tranche_id, const char *tranche_name)
</programlisting>
- </para>
- <para>
- It is convenient to use <literal>shmem_startup_hook</literal> which allows
- placing all the code responsible for initializing shared memory in one
- place. When using <literal>shmem_startup_hook</literal> the extension
- still needs to acquire <function>AddinShmemInitLock</function> in order to
- work properly on all the supported platforms.
- </para>
+ </para>
+
+ <para>
+ A complete usage example of <function>LWLockNewTrancheId</function>,
+ <function>LWLockInitialize</function>, and
+ <function>LWLockRegisterTranche</function> can be found in
+ <filename>contrib/pg_prewarm/autoprewarm.c</filename> in the
+ <productname>PostgreSQL</productname> source tree.
+ </para>
+ </sect3>
</sect2>
<sect2 id="xfunc-addin-wait-events">
--
2.25.1
>From b635798c1d015d9304c3273fd7c522d140d4fd31 Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nat...@postgresql.org>
Date: Wed, 11 Oct 2023 22:07:26 -0500
Subject: [PATCH v6 2/3] add dsm registry
---
doc/src/sgml/xfunc.sgml | 49 ++++-
src/backend/storage/ipc/Makefile | 1 +
src/backend/storage/ipc/dsm_registry.c | 174 ++++++++++++++++++
src/backend/storage/ipc/ipci.c | 3 +
src/backend/storage/ipc/meson.build | 1 +
src/backend/storage/lmgr/lwlock.c | 4 +
src/backend/storage/lmgr/lwlocknames.txt | 1 +
.../utils/activity/wait_event_names.txt | 3 +
src/include/storage/dsm_registry.h | 23 +++
src/include/storage/lwlock.h | 2 +
src/test/modules/Makefile | 1 +
src/test/modules/meson.build | 1 +
src/test/modules/test_dsm_registry/.gitignore | 4 +
src/test/modules/test_dsm_registry/Makefile | 23 +++
.../expected/test_dsm_registry.out | 14 ++
.../modules/test_dsm_registry/meson.build | 33 ++++
.../sql/test_dsm_registry.sql | 4 +
.../test_dsm_registry--1.0.sql | 10 +
.../test_dsm_registry/test_dsm_registry.c | 76 ++++++++
.../test_dsm_registry.control | 4 +
src/tools/pgindent/typedefs.list | 3 +
21 files changed, 431 insertions(+), 3 deletions(-)
create mode 100644 src/backend/storage/ipc/dsm_registry.c
create mode 100644 src/include/storage/dsm_registry.h
create mode 100644 src/test/modules/test_dsm_registry/.gitignore
create mode 100644 src/test/modules/test_dsm_registry/Makefile
create mode 100644 src/test/modules/test_dsm_registry/expected/test_dsm_registry.out
create mode 100644 src/test/modules/test_dsm_registry/meson.build
create mode 100644 src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql
create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql
create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry.c
create mode 100644 src/test/modules/test_dsm_registry/test_dsm_registry.control
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 0ba52b41d4..9d305000f9 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -3460,6 +3460,45 @@ LWLockRelease(AddinShmemInitLock);
the <productname>PostgreSQL</productname> source tree.
</para>
</sect3>
+
+ <sect3 id="xfunc-shared-addin-after-startup">
+ <title>Requesting Shared Memory After Startup</title>
+
+ <para>
+ There is another, more flexible method of reserving shared memory that
+ can be done after server startup and outside a
+ <literal>shmem_request_hook</literal>. To do so, each backend that will
+ use the shared memory should obtain a pointer to it by calling:
+<programlisting>
+void *GetNamedDSMSegment(const char *name, size_t size,
+ void (*init_callback) (void *ptr),
+ bool *found)
+</programlisting>
+ If a dynamic shared memory segment with the given name does not yet
+ exist, this function will allocate it and initialize it with the provided
+ <function>init_callback</function> callback function. If the segment has
+ already been allocated and initialized by another backend, this function
+ simply attaches the existing dynamic shared memory segment to the current
+ backend.
+ </para>
+
+ <para>
+ Unlike shared memory reserved at server startup, there is no need to
+ acquire <function>AddinShmemInitLock</function> or otherwise take action
+ to avoid race conditions when reserving shared memory with
+ <function>GetNamedDSMSegment</function>. This function ensures that only
+ one backend allocates and initializes the segment and that all other
+ backends receive a pointer to the fully allocated and initialized
+ segment.
+ </para>
+
+ <para>
+ A complete usage example of <function>GetNamedDSMSegment</function> can
+ be found in
+ <filename>src/test/modules/test_dsm_registry/test_dsm_registry.c</filename>
+ in the <productname>PostgreSQL</productname> source tree.
+ </para>
+ </sect3>
</sect2>
<sect2 id="xfunc-addin-lwlocks">
@@ -3469,8 +3508,9 @@ LWLockRelease(AddinShmemInitLock);
<title>Requesting LWLocks at Startup</title>
<para>
- Add-ins can reserve LWLocks on server startup. Like with shared memory,
- the add-in's shared library must be preloaded by specifying it in
+ Add-ins can reserve LWLocks on server startup. Like with shared memory
+ reserved at server startup, the add-in's shared library must be preloaded
+ by specifying it in
<xref linkend="guc-shared-preload-libraries"/><indexterm><primary>shared_preload_libraries</primary></indexterm>,
and the shared library should register a
<literal>shmem_request_hook</literal> in its
@@ -3508,7 +3548,10 @@ void LWLockInitialize(LWLock *lock, int tranche_id)
process allocates a new <literal>tranche_id</literal> and initializes
each new LWLock. One way to do this is to only call these functions in
your shared memory initialization code with the
- <function>AddinShmemInitLock</function> held exclusively.
+ <function>AddinShmemInitLock</function> held exclusively. If using
+ <function>GetNamedDSMSegment</function>, calling these functions in the
+ <function>init_callback</function> callback function is sufficient to
+ avoid race conditions.
</para>
<para>
diff --git a/src/backend/storage/ipc/Makefile b/src/backend/storage/ipc/Makefile
index 6d5b921038..d8a1653eb6 100644
--- a/src/backend/storage/ipc/Makefile
+++ b/src/backend/storage/ipc/Makefile
@@ -12,6 +12,7 @@ OBJS = \
barrier.o \
dsm.o \
dsm_impl.o \
+ dsm_registry.o \
ipc.o \
ipci.o \
latch.o \
diff --git a/src/backend/storage/ipc/dsm_registry.c b/src/backend/storage/ipc/dsm_registry.c
new file mode 100644
index 0000000000..7deaa0d6c5
--- /dev/null
+++ b/src/backend/storage/ipc/dsm_registry.c
@@ -0,0 +1,174 @@
+/*-------------------------------------------------------------------------
+ *
+ * dsm_registry.c
+ *
+ * Functions for interfacing with the dynamic shared memory registry. This
+ * provides a way for libraries to use shared memory without needing to
+ * request it at startup time via a shmem_request_hook.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/storage/ipc/dsm_registry.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "lib/dshash.h"
+#include "storage/dsm_registry.h"
+#include "storage/lwlock.h"
+#include "storage/shmem.h"
+#include "utils/memutils.h"
+
+typedef struct DSMRegistryCtxStruct
+{
+ dsa_handle dsah;
+ dshash_table_handle dshh;
+} DSMRegistryCtxStruct;
+
+static DSMRegistryCtxStruct *DSMRegistryCtx;
+
+typedef struct DSMRegistryEntry
+{
+ char name[64];
+ dsm_handle handle;
+} DSMRegistryEntry;
+
+static const dshash_parameters dsh_params = {
+ offsetof(DSMRegistryEntry, handle),
+ sizeof(DSMRegistryEntry),
+ dshash_memcmp,
+ dshash_memhash,
+ LWTRANCHE_DSM_REGISTRY_HASH
+};
+
+static dsa_area *dsm_registry_dsa;
+static dshash_table *dsm_registry_table;
+
+Size
+DSMRegistryShmemSize(void)
+{
+ return MAXALIGN(sizeof(DSMRegistryCtxStruct));
+}
+
+void
+DSMRegistryShmemInit(void)
+{
+ bool found;
+
+ DSMRegistryCtx = (DSMRegistryCtxStruct *)
+ ShmemInitStruct("DSM Registry Data",
+ DSMRegistryShmemSize(),
+ &found);
+
+ if (!found)
+ {
+ DSMRegistryCtx->dsah = DSA_HANDLE_INVALID;
+ DSMRegistryCtx->dshh = DSHASH_HANDLE_INVALID;
+ }
+}
+
+/*
+ * Initialize or attach to the dynamic shared hash table that stores the DSM
+ * registry entries, if not already done. This must be called before accessing
+ * the table.
+ */
+static void
+init_dsm_registry(void)
+{
+ /* Quick exit if we already did this. */
+ if (dsm_registry_table)
+ return;
+
+ /* Otherwise, use a lock to ensure only one process creates the table. */
+ LWLockAcquire(DSMRegistryLock, LW_EXCLUSIVE);
+
+ if (DSMRegistryCtx->dshh == DSHASH_HANDLE_INVALID)
+ {
+ /* Initialize dynamic shared hash table for registry. */
+ dsm_registry_dsa = dsa_create(LWTRANCHE_DSM_REGISTRY_DSA);
+ dsa_pin(dsm_registry_dsa);
+ dsa_pin_mapping(dsm_registry_dsa);
+ dsm_registry_table = dshash_create(dsm_registry_dsa, &dsh_params, 0);
+
+ /* Store handles in shared memory for other backends to use. */
+ DSMRegistryCtx->dsah = dsa_get_handle(dsm_registry_dsa);
+ DSMRegistryCtx->dshh = dshash_get_hash_table_handle(dsm_registry_table);
+ }
+ else
+ {
+ /* Attach to existing dynamic shared hash table. */
+ dsm_registry_dsa = dsa_attach(DSMRegistryCtx->dsah);
+ dsa_pin_mapping(dsm_registry_dsa);
+ dsm_registry_table = dshash_attach(dsm_registry_dsa, &dsh_params,
+ DSMRegistryCtx->dshh, 0);
+ }
+
+ LWLockRelease(DSMRegistryLock);
+}
+
+/*
+ * Initialize or attach a named DSM segment.
+ *
+ * This routine returns the address of the segment. init_callback is called to
+ * initialize the segment when it is first created.
+ */
+void *
+GetNamedDSMSegment(const char *name, size_t size,
+ void (*init_callback) (void *ptr), bool *found)
+{
+ DSMRegistryEntry *entry;
+ MemoryContext oldcontext;
+ char name_padded[offsetof(DSMRegistryEntry, handle)] = {0};
+ void *ret;
+
+ Assert(name);
+ Assert(size);
+ Assert(found);
+
+ if (strlen(name) >= offsetof(DSMRegistryEntry, handle))
+ elog(ERROR, "DSM segment name too long");
+
+ /* Be sure any local memory allocated by DSM/DSA routines is persistent. */
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+
+ /* Connect to the registry. */
+ init_dsm_registry();
+
+ strcpy(name_padded, name);
+ entry = dshash_find_or_insert(dsm_registry_table, name_padded, found);
+ if (!(*found))
+ {
+ /* Initialize the segment. */
+ dsm_segment *seg = dsm_create(size, 0);
+
+ dsm_pin_segment(seg);
+ dsm_pin_mapping(seg);
+ entry->handle = dsm_segment_handle(seg);
+ ret = dsm_segment_address(seg);
+
+ if (init_callback)
+ (*init_callback) (ret);
+ }
+ else if (!dsm_find_mapping(entry->handle))
+ {
+ /* Attach to existing segment. */
+ dsm_segment *seg = dsm_attach(entry->handle);
+
+ dsm_pin_mapping(seg);
+ ret = dsm_segment_address(seg);
+ }
+ else
+ {
+ /* Return address of an already-attached segment. */
+ ret = dsm_segment_address(dsm_find_mapping(entry->handle));
+ }
+
+ dshash_release_lock(dsm_registry_table, entry);
+ MemoryContextSwitchTo(oldcontext);
+
+ return ret;
+}
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index e5119ed55d..fbc62b1563 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -40,6 +40,7 @@
#include "replication/walsender.h"
#include "storage/bufmgr.h"
#include "storage/dsm.h"
+#include "storage/dsm_registry.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
@@ -115,6 +116,7 @@ CalculateShmemSize(int *num_semaphores)
size = add_size(size, hash_estimate_size(SHMEM_INDEX_SIZE,
sizeof(ShmemIndexEnt)));
size = add_size(size, dsm_estimate_size());
+ size = add_size(size, DSMRegistryShmemSize());
size = add_size(size, BufferShmemSize());
size = add_size(size, LockShmemSize());
size = add_size(size, PredicateLockShmemSize());
@@ -289,6 +291,7 @@ CreateOrAttachShmemStructs(void)
InitShmemIndex();
dsm_shmem_init();
+ DSMRegistryShmemInit();
/*
* Set up xlog, clog, and buffers
diff --git a/src/backend/storage/ipc/meson.build b/src/backend/storage/ipc/meson.build
index 08bdc718b8..5a936171f7 100644
--- a/src/backend/storage/ipc/meson.build
+++ b/src/backend/storage/ipc/meson.build
@@ -4,6 +4,7 @@ backend_sources += files(
'barrier.c',
'dsm.c',
'dsm_impl.c',
+ 'dsm_registry.c',
'ipc.c',
'ipci.c',
'latch.c',
diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c
index b4b989ac56..2f2de5a562 100644
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -190,6 +190,10 @@ static const char *const BuiltinTrancheNames[] = {
"LogicalRepLauncherDSA",
/* LWTRANCHE_LAUNCHER_HASH: */
"LogicalRepLauncherHash",
+ /* LWTRANCHE_DSM_REGISTRY_DSA: */
+ "DSMRegistryDSA",
+ /* LWTRANCHE_DSM_REGISTRY_HASH: */
+ "DSMRegistryHash",
};
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
index d621f5507f..ef8542de46 100644
--- a/src/backend/storage/lmgr/lwlocknames.txt
+++ b/src/backend/storage/lmgr/lwlocknames.txt
@@ -55,3 +55,4 @@ WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
WaitEventExtensionLock 48
WALSummarizerLock 49
+DSMRegistryLock 50
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index f625473ad4..6bcb1cca0c 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -329,6 +329,7 @@ WrapLimitsVacuum "Waiting to update limits on transaction id and multixact consu
NotifyQueueTail "Waiting to update limit on <command>NOTIFY</command> message storage."
WaitEventExtension "Waiting to read or update custom wait events information for extensions."
WALSummarizer "Waiting to read or update WAL summarization state."
+DSMRegistry "Waiting to read or update the dynamic shared memory registry."
#
# END OF PREDEFINED LWLOCKS (DO NOT CHANGE THIS LINE)
@@ -367,6 +368,8 @@ PgStatsHash "Waiting for stats shared memory hash table access."
PgStatsData "Waiting for shared memory stats data access."
LogicalRepLauncherDSA "Waiting to access logical replication launcher's dynamic shared memory allocator."
LogicalRepLauncherHash "Waiting to access logical replication launcher's shared hash table."
+DSMRegistryDSA "Waiting to access dynamic shared memory registry's dynamic shared memory allocator."
+DSMRegistryHash "Waiting to access dynamic shared memory registry's shared hash table."
#
# Wait Events - Lock
diff --git a/src/include/storage/dsm_registry.h b/src/include/storage/dsm_registry.h
new file mode 100644
index 0000000000..8e8a23ba60
--- /dev/null
+++ b/src/include/storage/dsm_registry.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * dsm_registry.h
+ *
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/dsm_registry.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DSM_REGISTRY_H
+#define DSM_REGISTRY_H
+
+extern void *GetNamedDSMSegment(const char *name, size_t size,
+ void (*init_callback) (void *ptr),
+ bool *found);
+
+extern Size DSMRegistryShmemSize(void);
+extern void DSMRegistryShmemInit(void);
+
+#endif /* DSM_REGISTRY_H */
diff --git a/src/include/storage/lwlock.h b/src/include/storage/lwlock.h
index 167ae34208..50a65e046d 100644
--- a/src/include/storage/lwlock.h
+++ b/src/include/storage/lwlock.h
@@ -207,6 +207,8 @@ typedef enum BuiltinTrancheIds
LWTRANCHE_PGSTATS_DATA,
LWTRANCHE_LAUNCHER_DSA,
LWTRANCHE_LAUNCHER_HASH,
+ LWTRANCHE_DSM_REGISTRY_DSA,
+ LWTRANCHE_DSM_REGISTRY_HASH,
LWTRANCHE_FIRST_USER_DEFINED,
} BuiltinTrancheIds;
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 5d33fa6a9a..f656032589 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -18,6 +18,7 @@ SUBDIRS = \
test_custom_rmgrs \
test_ddl_deparse \
test_dsa \
+ test_dsm_registry \
test_extensions \
test_ginpostinglist \
test_integerset \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 00ff1d77d1..2c3b8d73bc 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -15,6 +15,7 @@ subdir('test_copy_callbacks')
subdir('test_custom_rmgrs')
subdir('test_ddl_deparse')
subdir('test_dsa')
+subdir('test_dsm_registry')
subdir('test_extensions')
subdir('test_ginpostinglist')
subdir('test_integerset')
diff --git a/src/test/modules/test_dsm_registry/.gitignore b/src/test/modules/test_dsm_registry/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/test_dsm_registry/Makefile b/src/test/modules/test_dsm_registry/Makefile
new file mode 100644
index 0000000000..b13e99a354
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/Makefile
@@ -0,0 +1,23 @@
+# src/test/modules/test_dsm_registry/Makefile
+
+MODULE_big = test_dsm_registry
+OBJS = \
+ $(WIN32RES) \
+ test_dsm_registry.o
+PGFILEDESC = "test_dsm_registry - test code for the DSM registry"
+
+EXTENSION = test_dsm_registry
+DATA = test_dsm_registry--1.0.sql
+
+REGRESS = test_dsm_registry
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_dsm_registry
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out b/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out
new file mode 100644
index 0000000000..8ffbd343a0
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/expected/test_dsm_registry.out
@@ -0,0 +1,14 @@
+CREATE EXTENSION test_dsm_registry;
+SELECT set_val_in_shmem(1236);
+ set_val_in_shmem
+------------------
+
+(1 row)
+
+\c
+SELECT get_val_in_shmem();
+ get_val_in_shmem
+------------------
+ 1236
+(1 row)
+
diff --git a/src/test/modules/test_dsm_registry/meson.build b/src/test/modules/test_dsm_registry/meson.build
new file mode 100644
index 0000000000..a4045fea37
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+test_dsm_registry_sources = files(
+ 'test_dsm_registry.c',
+)
+
+if host_system == 'windows'
+ test_dsm_registry_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'test_dsm_registry',
+ '--FILEDESC', 'test_dsm_registry - test code for the DSM registry',])
+endif
+
+test_dsm_registry = shared_module('test_dsm_registry',
+ test_dsm_registry_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += test_dsm_registry
+
+test_install_data += files(
+ 'test_dsm_registry.control',
+ 'test_dsm_registry--1.0.sql',
+)
+
+tests += {
+ 'name': 'test_dsm_registry',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'test_dsm_registry',
+ ],
+ },
+}
diff --git a/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql b/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql
new file mode 100644
index 0000000000..b3351be0a1
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/sql/test_dsm_registry.sql
@@ -0,0 +1,4 @@
+CREATE EXTENSION test_dsm_registry;
+SELECT set_val_in_shmem(1236);
+\c
+SELECT get_val_in_shmem();
diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql b/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql
new file mode 100644
index 0000000000..8c55b0919b
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql
@@ -0,0 +1,10 @@
+/* src/test/modules/test_dsm_registry/test_dsm_registry--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION test_dsm_registry" to load this file. \quit
+
+CREATE FUNCTION set_val_in_shmem(val INT) RETURNS VOID
+ AS 'MODULE_PATHNAME' LANGUAGE C;
+
+CREATE FUNCTION get_val_in_shmem() RETURNS INT
+ AS 'MODULE_PATHNAME' LANGUAGE C;
diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry.c b/src/test/modules/test_dsm_registry/test_dsm_registry.c
new file mode 100644
index 0000000000..b46cd6c620
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/test_dsm_registry.c
@@ -0,0 +1,76 @@
+/*--------------------------------------------------------------------------
+ *
+ * test_dsm_registry.c
+ * Test the DSM registry
+ *
+ * Copyright (c) 2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/test/modules/test_dsm_registry/test_dsm_registry.c
+ *
+ * -------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "storage/dsm_registry.h"
+#include "storage/lwlock.h"
+
+PG_MODULE_MAGIC;
+
+typedef struct TestDSMRegistryStruct
+{
+ int val;
+ LWLock lck;
+} TestDSMRegistryStruct;
+
+static TestDSMRegistryStruct *tdr_state;
+
+static void
+tdr_init_shmem(void *ptr)
+{
+ TestDSMRegistryStruct *state = (TestDSMRegistryStruct *) ptr;
+
+ LWLockInitialize(&state->lck, LWLockNewTrancheId());
+ state->val = 0;
+}
+
+static void
+tdr_attach_shmem(void)
+{
+ bool found;
+
+ tdr_state = GetNamedDSMSegment("test_dsm_registry",
+ sizeof(TestDSMRegistryStruct),
+ tdr_init_shmem,
+ &found);
+ LWLockRegisterTranche(tdr_state->lck.tranche, "test_dsm_registry");
+}
+
+PG_FUNCTION_INFO_V1(set_val_in_shmem);
+Datum
+set_val_in_shmem(PG_FUNCTION_ARGS)
+{
+ tdr_attach_shmem();
+
+ LWLockAcquire(&tdr_state->lck, LW_EXCLUSIVE);
+ tdr_state->val = PG_GETARG_UINT32(0);
+ LWLockRelease(&tdr_state->lck);
+
+ PG_RETURN_VOID();
+}
+
+PG_FUNCTION_INFO_V1(get_val_in_shmem);
+Datum
+get_val_in_shmem(PG_FUNCTION_ARGS)
+{
+ int ret;
+
+ tdr_attach_shmem();
+
+ LWLockAcquire(&tdr_state->lck, LW_SHARED);
+ ret = tdr_state->val;
+ LWLockRelease(&tdr_state->lck);
+
+ PG_RETURN_UINT32(ret);
+}
diff --git a/src/test/modules/test_dsm_registry/test_dsm_registry.control b/src/test/modules/test_dsm_registry/test_dsm_registry.control
new file mode 100644
index 0000000000..813f099889
--- /dev/null
+++ b/src/test/modules/test_dsm_registry/test_dsm_registry.control
@@ -0,0 +1,4 @@
+comment = 'Test code for the DSM registry'
+default_version = '1.0'
+module_pathname = '$libdir/test_dsm_registry'
+relocatable = true
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index f582eb59e7..b7e73736e9 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -610,6 +610,8 @@ DropSubscriptionStmt
DropTableSpaceStmt
DropUserMappingStmt
DropdbStmt
+DSMRegistryCtxStruct
+DSMRegistryEntry
DumpComponents
DumpId
DumpOptions
@@ -2799,6 +2801,7 @@ Tcl_NotifierProcs
Tcl_Obj
Tcl_Time
TempNamespaceStatus
+TestDSMRegistryStruct
TestDecodingData
TestDecodingTxnData
TestSpec
--
2.25.1
>From 26436c7da5e546cf112e9f8ecc90d59cc2e1bedf Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nat...@postgresql.org>
Date: Tue, 26 Dec 2023 22:25:45 -0600
Subject: [PATCH v6 3/3] use dsm registry for pg_prewarm
---
contrib/pg_prewarm/autoprewarm.c | 46 +++++++++++---------------------
1 file changed, 15 insertions(+), 31 deletions(-)
diff --git a/contrib/pg_prewarm/autoprewarm.c b/contrib/pg_prewarm/autoprewarm.c
index 9085a409db..9ea6c2252a 100644
--- a/contrib/pg_prewarm/autoprewarm.c
+++ b/contrib/pg_prewarm/autoprewarm.c
@@ -32,12 +32,12 @@
#include "access/xact.h"
#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
-#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
#include "storage/buf_internals.h"
#include "storage/dsm.h"
+#include "storage/dsm_registry.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/latch.h"
@@ -95,8 +95,6 @@ static void apw_start_database_worker(void);
static bool apw_init_shmem(void);
static void apw_detach_shmem(int code, Datum arg);
static int apw_compare_blockinfo(const void *p, const void *q);
-static void autoprewarm_shmem_request(void);
-static shmem_request_hook_type prev_shmem_request_hook = NULL;
/* Pointer to shared-memory state. */
static AutoPrewarmSharedState *apw_state = NULL;
@@ -140,26 +138,11 @@ _PG_init(void)
MarkGUCPrefixReserved("pg_prewarm");
- prev_shmem_request_hook = shmem_request_hook;
- shmem_request_hook = autoprewarm_shmem_request;
-
/* Register autoprewarm worker, if enabled. */
if (autoprewarm)
apw_start_leader_worker();
}
-/*
- * Requests any additional shared memory required for autoprewarm.
- */
-static void
-autoprewarm_shmem_request(void)
-{
- if (prev_shmem_request_hook)
- prev_shmem_request_hook();
-
- RequestAddinShmemSpace(MAXALIGN(sizeof(AutoPrewarmSharedState)));
-}
-
/*
* Main entry point for the leader autoprewarm process. Per-database workers
* have a separate entry point.
@@ -767,6 +750,16 @@ autoprewarm_dump_now(PG_FUNCTION_ARGS)
PG_RETURN_INT64((int64) num_blocks);
}
+static void
+apw_init_state(void *ptr)
+{
+ AutoPrewarmSharedState *state = (AutoPrewarmSharedState *) ptr;
+
+ LWLockInitialize(&state->lock, LWLockNewTrancheId());
+ state->bgworker_pid = InvalidPid;
+ state->pid_using_dumpfile = InvalidPid;
+}
+
/*
* Allocate and initialize autoprewarm related shared memory, if not already
* done, and set up backend-local pointer to that state. Returns true if an
@@ -777,19 +770,10 @@ apw_init_shmem(void)
{
bool found;
- LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
- apw_state = ShmemInitStruct("autoprewarm",
- sizeof(AutoPrewarmSharedState),
- &found);
- if (!found)
- {
- /* First time through ... */
- LWLockInitialize(&apw_state->lock, LWLockNewTrancheId());
- apw_state->bgworker_pid = InvalidPid;
- apw_state->pid_using_dumpfile = InvalidPid;
- }
- LWLockRelease(AddinShmemInitLock);
-
+ apw_state = GetNamedDSMSegment("autoprewarm",
+ sizeof(AutoPrewarmSharedState),
+ apw_init_state,
+ &found);
LWLockRegisterTranche(apw_state->lock.tranche, "autoprewarm");
return found;
--
2.25.1
