Re: [HACKERS] lazy snapshots?

2010-11-04 Thread Robert Haas
On Wed, Oct 20, 2010 at 8:24 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Oct 20, 2010 at 10:07 PM, Tom Lane wrote: >>> I'm less than convinced by the hypothesis that most transactions would >>> avoid taking snapshots in this regime, anyway.  It would only hold up >>> if there's little l

Re: [HACKERS] lazy snapshots?

2010-11-04 Thread Robert Haas
On Thu, Nov 4, 2010 at 2:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Oct 20, 2010 at 8:11 PM, Robert Haas wrote: I'm imagining that the kernel of a snapshot is just a WAL position, ie the end of WAL as of the time you take the snapshot (easy to get in O(1) time).  Visi

Re: [HACKERS] lazy snapshots?

2010-11-04 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 20, 2010 at 8:11 PM, Robert Haas wrote: >>> I'm imagining that the kernel of a >>> snapshot is just a WAL position, ie the end of WAL as of the time you >>> take the snapshot (easy to get in O(1) time).  Visibility tests then >>> reduce to "did this transaction c

Re: [HACKERS] lazy snapshots?

2010-11-04 Thread Robert Haas
On Wed, Oct 20, 2010 at 8:11 PM, Robert Haas wrote: >> so it's >> not exploiting that resource at all.  I'm imagining that the kernel of a >> snapshot is just a WAL position, ie the end of WAL as of the time you >> take the snapshot (easy to get in O(1) time).  Visibility tests then >> reduce to "

Re: [HACKERS] lazy snapshots?

2010-10-20 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 20, 2010 at 10:07 PM, Tom Lane wrote: >> I'm less than convinced by the hypothesis that most transactions would >> avoid taking snapshots in this regime, anyway.  It would only hold up >> if there's little locality of reference in terms of which tuples are >> get

Re: [HACKERS] lazy snapshots?

2010-10-20 Thread Robert Haas
On Wed, Oct 20, 2010 at 10:07 PM, Tom Lane wrote: > Robert Haas writes: >> It's necessary to convince ourselves not only that this has some >> performance benefit but that it's actually correct.  It's easy to see >> that, if we never take a snapshot, all the tuple visibility decisions >> we make

Re: [HACKERS] lazy snapshots?

2010-10-20 Thread Tom Lane
Robert Haas writes: > It's necessary to convince ourselves not only that this has some > performance benefit but that it's actually correct. It's easy to see > that, if we never take a snapshot, all the tuple visibility decisions > we make will be exactly identical to the ones that we would have

[HACKERS] lazy snapshots?

2010-10-20 Thread Robert Haas
I had the following idea for an optimization. Feel free to tell me I'm nuts. Would it be possible to postpone the operation of taking a snapshot until we encounter an in-doubt tuple - that is, a tuple whose XMIN or XMAX is committed but not all-visible? It seems to me that there are many transac

Re: [HACKERS] Lazy Snapshots

2009-08-22 Thread Gokulakannan Somasundaram
> The maintenance costs and update contention for such a datastructure > would render this completely impractical, even if consulting it were > free. > Thanks for the reply. a) Only one transaction would be updating its commit status. Its multiple readers Vs Single Writer for the position of a pa

Re: [HACKERS] Lazy Snapshots

2009-08-22 Thread Tom Lane
Gokulakannan Somasundaram writes: >I have given some thought in this direction. I am just providing my idea. > a) Have a structure similar to commit log, which should also store, > transaction id at which the transaction got committed. Every transaction, > after committing should update the t

Re: [HACKERS] Lazy Snapshots

2009-08-21 Thread Gokulakannan Somasundaram
Hi, I have given some thought in this direction. I am just providing my idea. a) Have a structure similar to commit log, which should also store, transaction id at which the transaction got committed. Every transaction, after committing should update the transaction id at which the commit has h

Re: [HACKERS] Lazy Snapshots

2009-08-21 Thread si...@2ndquadrant.com
On 18 August 2009 at 16:18 Tom Lane wrote: > Simon, this concept is completely broken, as far as I can tell. Thanks for the precise example. Yes, I agree it is not going to work when there could be more than one row version in the relation. Which doesn't leave useful wriggle room, so not pu

Re: [HACKERS] Lazy Snapshots

2009-08-21 Thread si...@2ndquadrant.com
On 19 August 2009 at 00:09 Josh Berkus wrote:   > When we discussed Hot Standby at pgCon 2008, we discussed the > possibility of stopping the replications stream whenever a VACUUM came > through until any queries currently running on the slave completed.  Did > that approach turn out to be comple

Re: [HACKERS] Lazy Snapshots

2009-08-18 Thread Josh Berkus
Simon, > Another way of doing this is to utilize lazy snapshots: do not take a > snapshot when a statement starts and only take one at the point that we > need one. No other changes to the MVCC mechanisms are proposed. I like your general thinking here, even if this specific idea proves unworkabl

Re: [HACKERS] Lazy Snapshots

2009-08-18 Thread Tom Lane
"si...@2ndquadrant.com" writes: > Currently, we take eager snapshots, meaning we take a snapshot at the start of > each statement whether or not it is necessary. Snapshots exist to disambiguate > the running state of recent transactions, so if a statement never sees data > written by recent transa

Re: [HACKERS] Lazy Snapshots

2009-08-18 Thread Heikki Linnakangas
si...@2ndquadrant.com wrote: > * Scan begins, no snapshot set. Ahead of scan a row that would have been > visible > at start of scan is deleted, commits and removed by VACUUM/HOT. The scan has > no > evidence that a removal has taken place, never sees contention and thus never > takes a snapshot.

[HACKERS] Lazy Snapshots

2009-08-18 Thread si...@2ndquadrant.com
One of the problems with Hot Standby is that a long running query on the standby can conflict with VACUUMed rows on the primary, causing queries to be cancelled. I've been looking at this problem for about a year now from various angles. Jeff Jane's recent thoughts on procarray scalability hav