Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2008-03-13 Thread Simon Riggs
On Wed, 2008-03-12 at 20:13 -0400, Bruce Momjian wrote: > Is this a TODO? Tom's reply was: The general topic, yes. The caveats still apply. > > Nonsense. Main transaction exit also takes an exclusive lock, and is > > far more likely to be exercised in typical workloads than a > > subtransaction

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2008-03-12 Thread Bruce Momjian
Is this a TODO? Tom's reply was: > Nonsense. Main transaction exit also takes an exclusive lock, and is > far more likely to be exercised in typical workloads than a > subtransaction abort. > > In any case: there has still not been any evidence presented by anyone > that optimizing XidCacheRem

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Simon Riggs
On Tue, 2007-09-11 at 09:58 -0400, Tom Lane wrote: > Can we let go of this for 8.3, please? OK, we've moved forward, so its a good place to break. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 1: if pos

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > As discussed in the other thread "Final Thoughts for 8.3 on LWLocking > and Scalability", XidCacheRemoveRunningXids() is now the only holder of > an X lock during normal processing, Nonsense. Main transaction exit also takes an exclusive lock, and is far

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2007-09-11 Thread Simon Riggs
On Wed, 2006-09-13 at 21:45 -0400, Tom Lane wrote: > Anyway, given that there's this one nonobvious gotcha, there might be > others. My recommendation is that we take this off the open-items list > for 8.2 and revisit it in the 8.3 cycle when there's more time. Well, its still 8.3 just... As di

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> --- and because the entries are surely added in increasing XID order, >> such an array could be binary-searched. > If they're only added if they write to disk then isn't it possible to add them > out of order? St

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > --- and because the entries are surely added in increasing XID order, > such an array could be binary-searched. If they're only added if they write to disk then isn't it possible to add them out of order? Start a child transaction, start a child of that on

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I think Theo's problem is probably somewhere else, too --- apparently >> it's not so much that TransactionIdIsCurrentTransactionId takes a long >> time as that something is calling it lots of times with no check for >> interrupt. > Co

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Alvaro Herrera
Tom Lane wrote: > I wrote: > > I see a bug though, which is that RecordSubTransactionAbort() calls > > GetCurrentTransactionId() before having verified that it needs to do > > anything. This means that we'll generate and then discard an XID > > uselessly in a failed subxact that didn't touch disk.

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
I wrote: > I see a bug though, which is that RecordSubTransactionAbort() calls > GetCurrentTransactionId() before having verified that it needs to do > anything. This means that we'll generate and then discard an XID > uselessly in a failed subxact that didn't touch disk. Well, it would be a bug

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
I wrote: > Yeah, I was just looking at that. Removing useless entries from the > child-xid list would presumably help him. Considering we're not even > formally in beta yet, I'm probably being too conservative to recommend > we not touch it. Actually ... wait a minute. We do not assign an XID t

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> Anyway, given that there's this one nonobvious gotcha, there might be >> others. My recommendation is that we take this off the open-items list >> for 8.2 and revisit it in the 8.3 cycle when there's more time. >

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-14 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Anyway, given that there's this one nonobvious gotcha, there might be > others. My recommendation is that we take this off the open-items list > for 8.2 and revisit it in the 8.3 cycle when there's more time. I wonder if Theo's recent reported problem with

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-13 Thread Tom Lane
I wrote: > ... it seems like in the > case where RecordSubTransactionCommit detects that the subxact has not > stored its XID anywhere, we could immediately remove the XID from > the PGPROC array, just as if it had aborted. This would avoid chewing > subxid slots for cases such as exception blocks

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-03 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > OTOH I think we only need to store live Xids and those finished that > wrote a WAL record; we can drop subaborted and subcommitted if they > didn't. While reviewing this thread, I see Alvaro already had the idea I just came to...

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-09-03 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > I added a subxid array to Snapshot and running subxids are gathered from > PGPROC->subxids cache. There are two overflowed case; any of PGPROC->subxids > are overflowed or the number of total subxids exceeds pre-allocated buffers. > If overflowed, we c

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-28 Thread ITAGAKI Takahiro
Bruce Momjian <[EMAIL PROTECTED]> wrote: > Is there anything to do for 8.2 here? I'm working on Tom's idea. It is not a feature and does not change the on-disk-structures, so I hope it meet the 8.2 deadline... Tom Lane <[EMAIL PROTECTED]> wrote: > I'm wondering about doing something similar to w

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-25 Thread Bruce Momjian
Is there anything to do for 8.2 here? --- ITAGAKI Takahiro wrote: > This is an additional information. > > I wrote: > > If we want to resolve the probmen fundamentally, we might have to > > improve SubTrans using a better b

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread ITAGAKI Takahiro
This is an additional information. I wrote: > If we want to resolve the probmen fundamentally, we might have to > improve SubTrans using a better buffer management algorithm or so. The above is maybe wrong. I checked each lwlock of pg_subtrans's buffers. All lwlocks are uniformly acquired and I c

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I was thinking at what time was the most appropiate to insert or remove > an Xid from the cache. We can do without any excl-locking because 1) we > already assume the storing of an Xid to be atomic, and 2) no one can be > interested in querying for an X

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> I'm wondering about doing something similar to what > >> TransactionIdIsInProgress does, ie, make use of the PGPROC lists > >> of live subtransactions. > > > Well, that sounds awfully more expensive than setting

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I'm wondering about doing something similar to what >> TransactionIdIsInProgress does, ie, make use of the PGPROC lists >> of live subtransactions. > Well, that sounds awfully more expensive than setting > local-to-my-database Xmins a

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Alvaro Herrera
Tom Lane wrote: > I'm wondering about doing something similar to what > TransactionIdIsInProgress does, ie, make use of the PGPROC lists > of live subtransactions. Suppose that GetSnapshotData copies not > only top xids but live subxids into the snapshot, and adds a flag > indicating whether the

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-07 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > The invokers of SubTrans module are two SubTransGetTopmostTransaction() > in HeapTupleSatisfiesSnapshot(). When I disabled the calls, CSStorm did > not occur. SubTransGetTopmostTransaction returns the argument without > change when we don't use SAVEPOI

Re: [HACKERS] CSStorm occurred again by postgreSQL8.2

2006-08-06 Thread ITAGAKI Takahiro
Tom Lane <[EMAIL PROTECTED]> wrote: > > It does not solve, even if it increases the number of NUM_SUBTRANS_BUFFERS. > > The problem was only postponed. > > Can you provide a reproducible test case for this? This is the reproducible test case: - Occurs on both 8.1.4 and HEAD. - On smp machine. I