Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2008-03-12 Thread Bruce Momjian
Added to TODO: * Expire published xmin for read-only and idle transactions http://archives.postgresql.org/pgsql-hackers/2007-09/msg00343.php --- Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: As a fallout

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Alvaro Herrera
As a fallout of this work that I haven't seen made explicit, a session opening a transaction and then sitting around doing nothing will not cause as many problems as it used to -- for example it won't cause VACUUM to be unable to clean up dead rows. Is this correct? Nowadays this is no longer

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: As a fallout of this work that I haven't seen made explicit, a session opening a transaction and then sitting around doing nothing will not cause as many problems as it used to -- for example it won't cause VACUUM to be unable to clean up dead rows. Is

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: If you issue BEGIN, then SELECT, then sit, you'll be publishing an xmin but not an xid, so at that point you become a problem for VACUUM. However, internally you don't have any live snapshots (if you're in READ COMMITTED mode), so eventually we could have

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: If you issue BEGIN, then SELECT, then sit, you'll be publishing an xmin but not an xid, so at that point you become a problem for VACUUM. However, internally you don't have any live snapshots (if you're in READ

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Josh Berkus
Greg, Aren't there some things that depend on the idea that even READ COMMITTED transactions still have a serializable snapshot lying around for them to use? No, that would be REPEATABLE READ. That's one of the areas where we need to test HOT; does RR and SERIALIZABLE still work correctly

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-08 Thread Gregory Stark
Josh Berkus [EMAIL PROTECTED] writes: No, that would be REPEATABLE READ. That's one of the areas where we need to test HOT; does RR and SERIALIZABLE still work correctly with HOT? I'm a little confused by your question. REPEATABLE READ is an isolation level we don't directly support in

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Florian G. Pflug
Simon Riggs wrote: On Fri, 2007-09-07 at 06:36 +0200, Florian G. Pflug wrote: Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: - I actually think with just a little bit of more work, we can go even further, and get rid of the ReadNewTransactionId() call completely during

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Simon Riggs
On Fri, 2007-09-07 at 06:36 +0200, Florian G. Pflug wrote: Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: So I believe you're right, and we can skip taking the lock in the no xid case Sounds good. - I actually think with just a little bit of more work, we can go even

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Why would it? The idea was to remember the largest committed xid, and that won't go away just because the proc array is rather empty xid-wise. I hadn't fully absorbed this idea last night, but now that I have, I'm starting to think it's a good one.

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Florian G. Pflug
Tom Lane wrote: I've spent the past hour or so trying to consolidate the comments in GetSnapshotData and related places into a single chunk of text to be added to src/backend/access/transam/README. Attached is what I have so far --- this incorporates the idea of not taking ProcArrayLock to exit

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Tom Lane wrote: Here is another variant of the risk scenario: 1. Xact A is running (in Read Committed mode). 2. Xact C's GetSnapshotData reads next transaction ID into xmax, then is swapped out before it can acquire ProcArrayLock. 3. Xact B gets

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Tom Lane
Here's some revised text for the README file, based on using Florian's idea of a global latestCompletedXid variable. As I worked through it I realized that in this design, XidGenLock gates entry of new XIDs into the ProcArray while ProcArrayLock gates their removal. Which is an interesting sort

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Florian G. Pflug
Tom Lane wrote: Here's some revised text for the README file, based on using Florian's idea of a global latestCompletedXid variable. As I worked through it I realized that in this design, XidGenLock gates entry of new XIDs into the ProcArray while ProcArrayLock gates their removal. Which is

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-07 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: I noticed two rather cosmetic issues .) latestCompletedXid sounds as it might refer to the *last* completed xid, but it actually refers to the largest / highest completed xid. So maybe we should call it highestCompletedXid or largestCompletedXid.

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-06 Thread Tom Lane
Florian G. Pflug [EMAIL PROTECTED] writes: Tom Lane wrote: The lazy-XID patch, as committed, doesn't help that situation at all, I think the comment is correct in principle - If we remove the oldest xmin without locking, then two concurrent OldestXmin calculations will get two different

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-06 Thread Florian G. Pflug
Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: So I believe you're right, and we can skip taking the lock in the no xid case - I actually think with just a little bit of more work, we can go even further, and get rid of the ReadNewTransactionId() call completely during snapshotting.

Re: [HACKERS] Low hanging fruit in lazy-XID-assignment patch?

2007-09-06 Thread Florian G. Pflug
Tom Lane wrote: Simon was complaining a bit ago that we still have problems with excessive contention for the ProcArrayLock, and that much of this stems from the need for transaction exit to take that lock exclusively. The lazy-XID patch, as committed, doesn't help that situation at all, saying