On Sun, Jun 20, 2004 at 08:49:22PM -0400, Tom Lane wrote:
> Of these the one that I think most urgently needs attention is inval.c;
> that is complicated code already and figuring out what should happen
> for subtrans commit/abort is not trivial. The others look relatively
> straightforward to fix, if tedious.
A patch for this is attached. It _seems_ to work ok; conceptually it
seems ok too. I have done some testing which tends to indicate that it
is right, but I'm not sure of all the implications this code has so I
don't know how to test it exhaustively. Of course, regression tests
pass, but this doesn't actually prove anything.
--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"�Que diferencia tiene para los muertos, los hu�rfanos, y aquellos que han
perdido su hogar, si la loca destrucci�n ha sido realizada bajo el nombre
del totalitarismo o del santo nombre de la libertad y la democracia?" (Gandhi)
Index: src/include/storage/sinval.h
===================================================================
RCS file: /home/alvherre/cvs/pgsql-server/src/include/storage/sinval.h,v
retrieving revision 1.35
diff -c -w -b -B -c -r1.35 sinval.h
*** sinval.h 2 Jun 2004 21:29:29 -0000 1.35
--- sinval.h 22 Jun 2004 04:52:54 -0000
***************
*** 20,32 ****
/*
! * We currently support two types of shared-invalidation messages: one that
* invalidates an entry in a catcache, and one that invalidates a relcache
* entry. More types could be added if needed. The message type is
* identified by the first "int16" field of the message struct. Zero or
* positive means a catcache inval message (and also serves as the catcache
! * ID field). -1 means a relcache inval message. Other negative values
! * are available to identify other inval message types.
*
* Relcache invalidation messages usually also cause invalidation of entries
* in the smgr's relation cache. This means they must carry both logical
--- 20,33 ----
/*
! * We currently support three types of shared-invalidation messages: one that
* invalidates an entry in a catcache, and one that invalidates a relcache
* entry. More types could be added if needed. The message type is
* identified by the first "int16" field of the message struct. Zero or
* positive means a catcache inval message (and also serves as the catcache
! * ID field). -1 means a relcache inval message. -2 means a subtransaction
! * boundary message. Other negative values are available to identify other
! * inval message types.
*
* Relcache invalidation messages usually also cause invalidation of entries
* in the smgr's relation cache. This means they must carry both logical
***************
*** 53,58 ****
--- 54,64 ----
* and so that negative cache entries can be recognized with good accuracy.
* (Of course this assumes that all the backends are using identical hashing
* code, but that should be OK.)
+ *
+ * A subtransaction boundary is not really a cache invalidation message;
+ * rather it's an implementation artifact for nested transactions. The
+ * cleanup code for subtransaction abort looks for this message as a boundary
+ * to know when to stop processing messages.
*/
typedef struct
***************
*** 79,89 ****
--- 85,104 ----
*/
} SharedInvalRelcacheMsg;
+ #define SUBXACTBOUNDARYMSG_ID (-2)
+
+ typedef struct
+ {
+ int16 id; /* type field --- must be
first */
+ TransactionId xid; /* transaction id */
+ } SubxactBoundaryMsg;
+
typedef union
{
int16 id; /* type field --- must be
first */
SharedInvalCatcacheMsg cc;
SharedInvalRelcacheMsg rc;
+ SubxactBoundaryMsg sb;
} SharedInvalidationMessage;
Index: src/backend/utils/cache/inval.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql-server/src/backend/utils/cache/inval.c,v
retrieving revision 1.62
diff -c -w -b -B -c -r1.62 inval.c
*** inval.c 18 Jun 2004 06:13:52 -0000 1.62
--- inval.c 23 Jun 2004 00:04:35 -0000
***************
*** 66,73 ****
* manipulating the init file is in relcache.c, but we keep track of the
* need for it here.
*
! * All the request lists are kept in TopTransactionContext memory, since
* they need not live beyond the end of the current transaction.
*
*
* Portions Copyright (c) 1996-2003, PostgreSQL Global Development Group
--- 66,75 ----
* manipulating the init file is in relcache.c, but we keep track of the
* need for it here.
*
! * All the request lists are kept in CommitContext memory, since
* they need not live beyond the end of the current transaction.
+ * Also, this makes it easy to free messages created in an aborting
+ * subtransaction.
*
*
* Portions Copyright (c) 1996-2003, PostgreSQL Global Development Group
***************
*** 80,85 ****
--- 82,88 ----
*/
#include "postgres.h"
+ #include "access/xact.h"
#include "catalog/catalog.h"
#include "miscadmin.h"
#include "storage/sinval.h"
***************
*** 119,127 ****
* other backends or rolled back from local cache when we commit
* or abort the transaction.
*
! * The relcache-file-invalidated flag can just be a simple boolean,
! * since we only act on it at transaction commit; we don't care which
! * command of the transaction set it.
*----------------
*/
--- 122,140 ----
* other backends or rolled back from local cache when we commit
* or abort the transaction.
*
! * Note that the second list will carry a subtransaction boundary message
! * for each running subtransaction. It will be used to determine which
! * messages should be dropped from the list if the subtransaction aborts.
! *
! * The relcache-file-invalidated flag is a TransactionId which shows
! * what level of the transaction tree is requesting a invalidation.
! * To register an invalidation, the transaction saves its own TransactionId
! * in RelcacheInitFileInval, unless the value was already set to
! * a valid TransactionId. If it aborts and the value is its TransactionId,
! * it resets the value to InvalidTransactionId. If it commits, it changes
! * the value to its parent's TransactionId. This way the value is propagated
! * up to the topmost transaction, which will delete the file if a valid
! * TransactionId is detected.
*----------------
*/
***************
*** 131,137 ****
/* head of previous-commands event list */
static InvalidationListHeader PriorCmdInvalidMsgs;
! static bool RelcacheInitFileInval; /* init file must be invalidated? */
/*
* Dynamically-registered callback functions. Current implementation
--- 144,151 ----
/* head of previous-commands event list */
static InvalidationListHeader PriorCmdInvalidMsgs;
! /* init file must be invalidated? */
! static TransactionId RelcacheInitFileInval = InvalidTransactionId;
/*
* Dynamically-registered callback functions. Current implementation
***************
*** 176,182 ****
/* First time through; create initial chunk */
#define FIRSTCHUNKSIZE 16
chunk = (InvalidationChunk *)
! MemoryContextAlloc(TopTransactionContext,
sizeof(InvalidationChunk) +
(FIRSTCHUNKSIZE - 1)
*sizeof(SharedInvalidationMessage));
chunk->nitems = 0;
--- 190,196 ----
/* First time through; create initial chunk */
#define FIRSTCHUNKSIZE 16
chunk = (InvalidationChunk *)
! MemoryContextAlloc(CommitContext,
sizeof(InvalidationChunk) +
(FIRSTCHUNKSIZE - 1) *
sizeof(SharedInvalidationMessage));
chunk->nitems = 0;
***************
*** 190,196 ****
int chunksize = 2 * chunk->maxitems;
chunk = (InvalidationChunk *)
! MemoryContextAlloc(TopTransactionContext,
sizeof(InvalidationChunk) +
(chunksize - 1)
*sizeof(SharedInvalidationMessage));
chunk->nitems = 0;
--- 204,210 ----
int chunksize = 2 * chunk->maxitems;
chunk = (InvalidationChunk *)
! MemoryContextAlloc(CommitContext,
sizeof(InvalidationChunk) +
(chunksize - 1) *
sizeof(SharedInvalidationMessage));
chunk->nitems = 0;
***************
*** 208,214 ****
*
* NOTE: when we are about to commit or abort a transaction, it's
* not really necessary to pfree the lists explicitly, since they will
! * go away anyway when TopTransactionContext is destroyed.
*/
static void
FreeInvalidationMessageList(InvalidationChunk **listHdr)
--- 222,228 ----
*
* NOTE: when we are about to commit or abort a transaction, it's
* not really necessary to pfree the lists explicitly, since they will
! * go away anyway when CommitContext is destroyed.
*/
static void
FreeInvalidationMessageList(InvalidationChunk **listHdr)
***************
*** 405,412 ****
* If the relation being invalidated is one of those cached in the
* relcache init file, mark that we need to zap that file at commit.
*/
! if (RelationIdIsInInitFile(relId))
! RelcacheInitFileInval = true;
}
/*
--- 420,427 ----
* If the relation being invalidated is one of those cached in the
* relcache init file, mark that we need to zap that file at commit.
*/
! if (RelationIdIsInInitFile(relId) && RelcacheInitFileInval ==
InvalidTransactionId)
! RelcacheInitFileInval = GetCurrentTransactionId();
}
/*
***************
*** 467,472 ****
--- 482,489 ----
smgrclosenode(msg->rc.physId);
}
}
+ else if (msg->id == SUBXACTBOUNDARYMSG_ID)
+ ; /* do nothing */
else
elog(FATAL, "unrecognized SI message id: %d", msg->id);
}
***************
*** 619,624 ****
--- 636,662 ----
}
/*
+ * AtSubXactStart_Inval
+ * Insert subtransaction boundary messages to the lists.
+ *
+ * Note that we cheat and append the message to the prior cmd list. We can do
+ * this because we know it will be in the same relative position (last
+ * of prior cmds, first of current cmd), and it allows us to save searching the
+ * current cmd's list for the message in case of subtransaction abort.
+ */
+ void
+ AtSubStart_Inval(void)
+ {
+ SharedInvalidationMessage msg;
+
+ msg.sb.id = SUBXACTBOUNDARYMSG_ID;
+ msg.sb.xid = GetCurrentTransactionId();
+
+ AddInvalidationMessage(&(PriorCmdInvalidMsgs.cclist), &msg);
+ AddInvalidationMessage(&(PriorCmdInvalidMsgs.rclist), &msg);
+ }
+
+ /*
* AtEOXactInvalidationMessages
* Process queued-up invalidation messages at end of transaction.
*
***************
*** 636,642 ****
* the caches yet.
*
* In any case, reset the various lists to empty. We need not physically
! * free memory here, since TopTransactionContext is about to be emptied
* anyway.
*
* Note:
--- 674,680 ----
* the caches yet.
*
* In any case, reset the various lists to empty. We need not physically
! * free memory here, since CommitContext is about to be emptied
* anyway.
*
* Note:
***************
*** 652,658 ****
* and after we send the SI messages. However, we need not do
* anything unless we committed.
*/
! if (RelcacheInitFileInval)
RelationCacheInitFileInvalidate(true);
AppendInvalidationMessages(&PriorCmdInvalidMsgs,
--- 690,696 ----
* and after we send the SI messages. However, we need not do
* anything unless we committed.
*/
! if (RelcacheInitFileInval != InvalidTransactionId)
RelationCacheInitFileInvalidate(true);
AppendInvalidationMessages(&PriorCmdInvalidMsgs,
***************
*** 661,667 ****
ProcessInvalidationMessages(&PriorCmdInvalidMsgs,
SendSharedInvalidMessage);
! if (RelcacheInitFileInval)
RelationCacheInitFileInvalidate(false);
}
else
--- 699,705 ----
ProcessInvalidationMessages(&PriorCmdInvalidMsgs,
SendSharedInvalidMessage);
! if (RelcacheInitFileInval != InvalidTransactionId)
RelationCacheInitFileInvalidate(false);
}
else
***************
*** 670,682 ****
LocalExecuteInvalidationMessage);
}
! RelcacheInitFileInval = false;
DiscardInvalidationMessages(&PriorCmdInvalidMsgs, false);
DiscardInvalidationMessages(&CurrentCmdInvalidMsgs, false);
}
/*
* CommandEndInvalidationMessages
* Process queued-up invalidation messages at end of one command
* in a transaction.
--- 708,849 ----
LocalExecuteInvalidationMessage);
}
! RelcacheInitFileInval = InvalidTransactionId;
DiscardInvalidationMessages(&PriorCmdInvalidMsgs, false);
DiscardInvalidationMessages(&CurrentCmdInvalidMsgs, false);
}
/*
+ * Propagate the invalidate-init-file flag.
+ */
+ void
+ AtSubCommit_Inval(void)
+ {
+ if (RelcacheInitFileInval == GetCurrentTransactionId())
+ RelcacheInitFileInval = GetParentTransactionId();
+ }
+
+ /*
+ * AtSubAbort_Inval
+ * Process the invalidation list during subtransaction abort.
+ *
+ * During subtransaction abort, we have to discard the messages from
+ * the current command, and locally process the messages from prior
+ * commands that were inserted after the subtransaction boundary message.
+ */
+ void
+ AtSubAbort_Inval(void)
+ {
+ InvalidationChunk *chunk,
+ *savechunk;
+ TransactionId myXid = GetCurrentTransactionId();
+
+ /*
+ * Drop the invalidate-init-file flag, if I set it up.
+ */
+ if (RelcacheInitFileInval == myXid)
+ RelcacheInitFileInval = InvalidTransactionId;
+
+ /*
+ * We can discard the current command's messages because they haven't
+ * been executed. We don't need to free the messages because the
+ * CommitContext is going away soon.
+ *
+ * Note that we cheated in AtSubStart_Inval by inserting the boundary
+ * message in the PriorCmd list. If we hadn't done that, the message
+ * could still be in CurrentCmdInvalidMsgs!!!
+ */
+ DiscardInvalidationMessages(&CurrentCmdInvalidMsgs, false);
+
+ /*
+ * Sadly we have to break the chunk abstraction here.
+ * We need to walk it backwards, sort of -- starting from the
+ * last item on the first chunk, to the first item of same, until
+ * the subtransaction boundary message is found. If it isn't,
+ * jump to the next chunk and repeat.
+ *
+ * While walking, process the messages. When the boundary message
+ * is found, use its position as "last message" in the chunk, and
+ * save the chunk as the head of the message list. There's no need
+ * to free the dropped chunks, because they'll go away with the
+ * CommitContext.
+ *
+ * Start with the catcache message list.
+ */
+ chunk = PriorCmdInvalidMsgs.cclist;
+ savechunk = NULL;
+ while (chunk != NULL)
+ {
+ int i;
+
+ for (i = chunk->nitems - 1; i >= 0; i--)
+ {
+ if (chunk->msgs[i].id == SUBXACTBOUNDARYMSG_ID &&
+ ((SubxactBoundaryMsg *)
&(chunk->msgs[i]))->xid == myXid)
+ {
+ /*
+ * Drop the already processed messages. If this is
the first
+ * message in the chunk, use the next chunk as the
head of
+ * the list. Else, use this chunk, but be sure to
adjust the
+ * number of items.
+ */
+ if (i == 0)
+ savechunk = chunk->next;
+ else
+ {
+ chunk->nitems = i;
+ savechunk = chunk;
+ }
+ goto donecatcache;
+ }
+ LocalExecuteInvalidationMessage(&(chunk->msgs[i]));
+ }
+ chunk = chunk->next;
+ /*
+ * The next chunk can't be null because this means we
+ * failed to find the boundary message.
+ */
+ Assert(chunk != NULL);
+ }
+ donecatcache:
+ PriorCmdInvalidMsgs.cclist = savechunk;
+
+ /* do the same for the Relcache message list */
+ chunk = PriorCmdInvalidMsgs.rclist;
+ while (chunk != NULL)
+ {
+ int i;
+
+ for (i = chunk->nitems - 1; i >= 0; i--)
+ {
+ if (chunk->msgs[i].id == SUBXACTBOUNDARYMSG_ID &&
+ ((SubxactBoundaryMsg *)
&(chunk->msgs[i]))->xid == myXid)
+ {
+ /* Same as above. This code could be made a macro. */
+ if (i == 0)
+ savechunk = chunk->next;
+ else
+ {
+ chunk->nitems = i;
+ savechunk = chunk;
+ }
+ goto donerelcache;
+ }
+ LocalExecuteInvalidationMessage(&(chunk->msgs[i]));
+ }
+ chunk = chunk->next;
+ /*
+ * The next chunk can't be null because this means we
+ * failed to find the boundary message.
+ */
+ Assert(chunk != NULL);
+ }
+ donerelcache:
+ PriorCmdInvalidMsgs.rclist = savechunk;
+ }
+
+ /*
* CommandEndInvalidationMessages
* Process queued-up invalidation messages at end of one command
* in a transaction.
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings
