Re: [PATCHES] New shared memory hooks proposal (was Re:
I updated the style of your patch, and added a little to your comment
block about how to use this capability. I don't think any additional
documentation is necessary.
Thanks.
---
Marc Munro wrote:
-- Start of PGP signed section.
The attached patch provides add-ins with the means to register for
shared memory and LWLocks. This greatly improves the ease with which
shared memory may be used from add-ins, while keeping the accounting and
management for that shared memory separate.
Specifically it adds named add-in shared memory contexts. From these,
memory can be allocated without affecting the memory available in other
contexts.
Usage is as follows:
from add-in functions called from preload_libraries, you may call
RegisterAddinContext(const * name, size_t size)
to register a new (logical) shared memory segment.
and
RegisterAddinLWLock(LWLockid *lock_ptr);
to request that a LWLock be allocated, placed into *lock_ptr.
The actual creation of the shared memory segment and lwlocks is
performed later as part of shared memory initialisation.
To allocate shared memory from a named context you would use
ShmemAllocFromContext(size_t size, const char *name);
To reset a shared memory context back to its original unused state (from
which new allocations may be performed), you may use
ShmemResetContext(const char *name);
This works for me (for Veil) and make check runs fine.
I have not included any documentation updates in the patch as I'm not
sure where such API changes should be documented.
All comments, questions and suggestions are welcomed.
__
Marc
[ Attachment, skipping... ]
-- End of PGP section, PGP failed!
--
Bruce Momjian [EMAIL PROTECTED]
EnterpriseDBhttp://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Index: src/backend/storage/ipc/ipci.c
===
RCS file: /cvsroot/pgsql/src/backend/storage/ipc/ipci.c,v
retrieving revision 1.86
diff -c -c -r1.86 ipci.c
*** src/backend/storage/ipc/ipci.c 15 Jul 2006 15:47:17 - 1.86
--- src/backend/storage/ipc/ipci.c 1 Aug 2006 19:01:09 -
***
*** 57,62
--- 57,63
{
PGShmemHeader *seghdr;
Size size;
+ Size size_b4addins;
int numSemas;
/*
***
*** 93,98
--- 94,108
/* might as well round it off to a multiple of a typical page size */
size = add_size(size, 8192 - (size % 8192));
+ /*
+ * The shared memory for add-ins is treated as a separate
+ * segment, but in reality it is not.
+ */
+ size_b4addins = size;
+ size = add_size(size, AddinShmemSize());
+ /* round it off again */
+ size = add_size(size, 8192 - (size % 8192));
+
elog(DEBUG3, invoking IpcMemoryCreate(size=%lu),
(unsigned long) size);
***
*** 101,106
--- 111,126
*/
seghdr = PGSharedMemoryCreate(size, makePrivate, port);
+ /*
+ * Modify hdr to show segment size before add-ins
+ */
+ seghdr-totalsize = size_b4addins;
+
+ /*
+ * Set up segment header sections in each Addin context
+ */
+ InitAddinContexts((void *) ((char *) seghdr + size_b4addins));
+
InitShmemAccess(seghdr);
/*
Index: src/backend/storage/ipc/shmem.c
===
RCS file: /cvsroot/pgsql/src/backend/storage/ipc/shmem.c,v
retrieving revision 1.94
diff -c -c -r1.94 shmem.c
*** src/backend/storage/ipc/shmem.c 22 Jul 2006 23:04:39 - 1.94
--- src/backend/storage/ipc/shmem.c 1 Aug 2006 19:01:09 -
***
*** 61,66
--- 61,75
* cannot be redistributed to other tables. We could build a simple
* hash bucket garbage collector if need be. Right now, it seems
* unnecessary.
+ *
+ * (e) Add-ins can request their own logical shared memory segments
+ * by calling RegisterAddinContext() from the preload-libraries hook.
+ * Each call establishes a uniquely named add-in shared memopry
+ * context which will be set up as part of postgres intialisation.
+ * Memory can be allocated from these contexts using
+ * ShmemAllocFromContext(), and can be reset to its initial condition
+ * using ShmemResetContext(). Also, RegisterAddinLWLock(LWLockid *lock_ptr)
+ * can be used to request that a LWLock be allocated, placed into *lock_ptr.
*/
#include postgres.h
***
*** 86,91
--- 95,113
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+ /* Structures and globals for managing add-in shared memory contexts */
+ typedef struct context
+ {
+ char *name;
+ Sizesize;
+ PGShmemHeader *seg_hdr;
+ struct context *next;
+ } ContextNode;
+
+ static ContextNode *addin_contexts = NULL;
+ static Size addin_contexts_size = 0;
+
+
/*
* InitShmemAccess() --- set
Re: [PATCHES] New shared memory hooks proposal (was Re:
On Tue, 2006-07-18 at 16:36 -0700, Marc Munro wrote: The attached patch provides add-ins with the means to register for shared memory and LWLocks. This greatly improves the ease with which shared memory may be used from add-ins, while blah blah blah I have tried to be patient but this is my first patch and the suspense is killing me. I'd like some idea of what is likely to happen to this patch. If it's a non-starter I'd like to know. Ditto if it needs work. I realise everyone is busy right now, so I'm also prepared to be told to be patient; I'd just like to know it hasn't been overlooked and forgotten in the sudden rush of more interesting patches. Just to re-iterate, this patch enables Veil http://pgfoundry.org/projects/veil/ to be a good postgres citizen, removing its motivation to steal shared memory. __ Marc signature.asc Description: This is a digitally signed message part
Re: [PATCHES] New shared memory hooks proposal (was Re:
Marc Munro [EMAIL PROTECTED] writes: The attached patch provides add-ins with the means to register for shared memory and LWLocks. This greatly improves the ease with which shared memory may be used from add-ins, while blah blah blah I have tried to be patient but this is my first patch and the suspense is killing me. Sorry about that, but we're at the end of the development cycle and there's a long list of patches awaiting review :-(. Yours seems unlikely to affect any other work, so it's not high on the priority list to get pushed in. I'd like some idea of what is likely to happen to this patch. We'll look at it, and you will have an opportunity to fix anything anyone doesn't like. Most likely this will happen in early August. If you have schedule constraints (like you're on vacation in August) you should let us know. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] New shared memory hooks proposal (was Re: pre_load_libraries)
The attached patch provides add-ins with the means to register for
shared memory and LWLocks. This greatly improves the ease with which
shared memory may be used from add-ins, while keeping the accounting and
management for that shared memory separate.
Specifically it adds named add-in shared memory contexts. From these,
memory can be allocated without affecting the memory available in other
contexts.
Usage is as follows:
from add-in functions called from preload_libraries, you may call
RegisterAddinContext(const * name, size_t size)
to register a new (logical) shared memory segment.
and
RegisterAddinLWLock(LWLockid *lock_ptr);
to request that a LWLock be allocated, placed into *lock_ptr.
The actual creation of the shared memory segment and lwlocks is
performed later as part of shared memory initialisation.
To allocate shared memory from a named context you would use
ShmemAllocFromContext(size_t size, const char *name);
To reset a shared memory context back to its original unused state (from
which new allocations may be performed), you may use
ShmemResetContext(const char *name);
This works for me (for Veil) and make check runs fine.
I have not included any documentation updates in the patch as I'm not
sure where such API changes should be documented.
All comments, questions and suggestions are welcomed.
__
Marc
*** ./src/backend/storage/ipc/ipci.c Sat Jul 15 08:47:17 2006
--- ./new_src/backend/storage/ipc/ipci.c Tue Jul 18 15:26:22 2006
***
*** 57,62
--- 57,63
{
PGShmemHeader *seghdr;
Size size;
+ Size size_b4addins;
int numSemas;
/*
***
*** 93,98
--- 94,107
/* might as well round it off to a multiple of a typical page size */
size = add_size(size, 8192 - (size % 8192));
+ /* The shared memory for add-ins is treated as a separate
+ * segment but in reality is not
+ */
+ size_b4addins = size;
+ size = add_size(size, AddinShmemSize());
+ /* round it off again */
+ size = add_size(size, 8192 - (size % 8192));
+
elog(DEBUG3, invoking IpcMemoryCreate(size=%lu),
(unsigned long) size);
***
*** 101,106
--- 110,125
*/
seghdr = PGSharedMemoryCreate(size, makePrivate, port);
+ /*
+ * Modify hdr to show segment size before add-ins
+ */
+ seghdr-totalsize = size_b4addins;
+
+ /*
+ * Set up segment header sections in each Addin context
+ */
+ InitAddinContexts((void *) ((char *) seghdr + size_b4addins));
+
InitShmemAccess(seghdr);
/*
*** ./src/backend/storage/ipc/shmem.c Fri Jul 14 07:52:22 2006
--- ./new_src/backend/storage/ipc/shmem.c Tue Jul 18 15:26:22 2006
***
*** 61,66
--- 61,75
* cannot be redistributed to other tables. We could build a simple
* hash bucket garbage collector if need be. Right now, it seems
* unnecessary.
+ *
+ * (e) Add-ins can request their own logical shared memory segments
+ * by calling RegisterAddinContext() from the preload-libraries hook.
+ * Each call establishes a uniquely named add-in shared memopry
+ * context which will be set up as part of postgres intialisation.
+ * Memory can be allocated from these contexts using
+ * ShmemAllocFromContext(), and can be reset to its initial condition
+ * using ShmemResetContext().
+ *
*/
#include postgres.h
***
*** 86,91
--- 95,112
static HTAB *ShmemIndex = NULL; /* primary index hashtable for shmem */
+ /* Structures and globals for managing add-in shared memory contexts */
+ typedef struct context {
+ char *name;
+ Sizesize;
+ PGShmemHeader *seg_hdr;
+ struct context *next;
+ } ContextNode;
+
+ static ContextNode *addin_contexts = NULL;
+ static Size addin_contexts_size = 0;
+
+
/*
* InitShmemAccess() --- set up basic pointers to shared memory.
***
*** 140,146
}
/*
! * ShmemAlloc -- allocate max-aligned chunk from shared memory
*
* Assumes ShmemLock and ShmemSegHdr are initialized.
*
--- 161,255
}
/*
! * RegisterAddinContext -- Register the requirement for a named shared
! * memory context.
! */
! void
! RegisterAddinContext(const char *name, Size size)
! {
! char *newstr = malloc(strlen(name) + 1);
! ContextNode *node = malloc(sizeof(ContextNode));
! strcpy(newstr, name);
! node-name = newstr;
! // Round up to typical page size
! node-size = add_size(size, 8192 - (size % 8192));
! node-next = addin_contexts;
! addin_contexts = node;
!
! addin_contexts_size = add_size(addin_contexts_size, node-size);
! }
!
!
! /*
! * ContextFromName -- Return the ContextNode for the given named
! *context, or NULL if not found.
! */
! static ContextNode *
! ContextFromName(const char *name)
! {
! ContextNode *context = addin_contexts;
!
! while (context) {
! if (strcmp(name, context-name) == 0) {
! return context;
!
