Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Hiroyuki Yamada
On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: * There is a window beween gathering lock information in GetRunningTransactionLocks() and writing WAL in LogAccessExclusiveLocks(). * In current lock redo algorithm, locks are released when the transaction holding the lock

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Hiroyuki Yamada wrote: The implementation assumes that transactions write COMMIT/ABORT WAL at the end of them, while it does not seem to write ABORT WAL in immediate shutdown. So, 1. acquire ACCESS EXCLUSIVE lock in table A in xact 1 2. execute immediate shutdown of the active node 3.

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 19:55 +0900, Hiroyuki Yamada wrote: Well, I've read some more and have a question. The implementation assumes that transactions write COMMIT/ABORT WAL at the end of them, while it does not seem to write ABORT WAL in immediate shutdown. So, 1. acquire ACCESS EXCLUSIVE

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: Hmm, looking at the code, I think Simon threw that baby with the bathwater when he removed support for starting standby from a shutdown checkpoint. Hmm, I think that code was just for starting points only. It would not have been

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: Hmm, looking at the code, I think Simon threw that baby with the bathwater when he removed support for starting standby from a shutdown checkpoint. Hmm, I think that code was just for starting points only. It

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 15:18 +0200, Heikki Linnakangas wrote: That removed piece of code was executed in the standby whenever we saw a shutdown checkpoint. It calls ProcArrayApplyRecoveryInfo(), which calls ExpireOldKnownAssignedTransactionIds() and StandbyReleaseOldLocks() to clean up

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:38 +, Simon Riggs wrote: I see now that in the presence of prepared transactions, we would fail to clean up failed transations with XID the oldest prepared transaction Good! I just spotted that also, just prior to posting my fix, so rewriting it again now.

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 09:10 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: Investigating how that could come about, it looks like there is some fairly strange stuff going on here. StandbyRecoverPreparedTransactions() is never called at all. I told you so:

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Hiroyuki Yamada
That fixes or explains all known issues, from me. Are there any other things you know about that I haven't responded to? Do you think we have addressed every issue, except deferred items? I will be looking to commit to CVS later today; waiting on any objections. Is following problem reported

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 18:08 +0900, Hiroyuki Yamada wrote: That fixes or explains all known issues, from me. Are there any other things you know about that I haven't responded to? Do you think we have addressed every issue, except deferred items? I will be looking to commit to CVS later

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Hiroyuki Yamada
On Wed, 2009-12-16 at 18:08 +0900, Hiroyuki Yamada wrote: That fixes or explains all known issues, from me. Are there any other things you know about that I haven't responded to? Do you think we have addressed every issue, except deferred items? I will be looking to commit to CVS later

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Heikki Linnakangas
Hiroyuki Yamada wrote: By the way, reading LogStandbySnapshot() and GetRunningTransactionLocks() raised following questions. * There is a window beween gathering lock information in GetRunningTransactionLocks() and writing WAL in LogAccessExclusiveLocks(). * In current lock redo

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: Sorry for annoying you. Not at all! Good to get fresh eyes on this. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: * There is a window beween gathering lock information in GetRunningTransactionLocks() and writing WAL in LogAccessExclusiveLocks(). * In current lock redo algorithm, locks are released when the transaction holding the lock

[HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Simon Riggs
I get this when testing prepared transactions, which looks like it is unrelated to Hot Standby. 2009-12-15 17:28:08 GMT[10385]LOG: archive recovery complete 2009-12-15 17:28:08 GMT[10428]LOG: checkpoint starting: end-of-recovery immediate wait 2009-12-15 17:28:08 GMT[10428]DEBUG: creating and

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Simon Riggs
On Tue, 2009-12-15 at 18:49 +, Simon Riggs wrote: This looks like the code is setting the parent to be zero. [Sorry, that comment is completely off, misread the line number.] The assertion is failing because the parent entry for that subxid had already been set. Investigating how that

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Heikki Linnakangas
Simon Riggs wrote: Investigating how that could come about, it looks like there is some fairly strange stuff going on here. StandbyRecoverPreparedTransactions() is never called at all. I told you so: http://archives.postgresql.org/message-id/4b260b5e.3010...@enterprisedb.com