[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-28 Thread Simon Riggs
On Sat, 2010-03-27 at 22:39 +, Greg Stark wrote: On Sat, Mar 27, 2010 at 7:36 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: If we're pruning an index entry to a heap tuple that has been HOT pruned wouldn't the HOT pruning record have

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Simon Riggs
On Fri, 2010-03-26 at 16:16 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: When replaying the deletion record, the standby could look at all the heap tuples whose index pointers are being removed, to see which

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Greg Stark
On Sat, Mar 27, 2010 at 10:10 AM, Simon Riggs si...@2ndquadrant.com wrote: It appears that in practice many of the index items point to heap items that are LP_DEAD. So for the purposes of accessing a heap tuple's xmin, then we're both right. To the current purpose the tuple has been removed,

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Simon Riggs
On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: On Sat, Mar 27, 2010 at 10:10 AM, Simon Riggs si...@2ndquadrant.com wrote: It appears that in practice many of the index items point to heap items that are LP_DEAD. So for the purposes of accessing a heap tuple's xmin, then we're both

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Greg Stark
On Sat, Mar 27, 2010 at 7:36 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: If we're pruning an index entry to a heap tuple that has been HOT pruned wouldn't the HOT pruning record have already conflicted with any queries that could see

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-26 Thread Simon Riggs
On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: When replaying the deletion record, the standby could look at all the heap tuples whose index pointers are being removed, to see which one was newest. Long after coding this, I now realise this really is a dumb-ass idea. There is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-26 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: When replaying the deletion record, the standby could look at all the heap tuples whose index pointers are being removed, to see which one was newest. Long after coding this, I now realise

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Simon Riggs
On Sun, 2010-01-31 at 16:53 -0500, Tom Lane wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: IIRC it was Greg Stark who suggested last time this was discussed that we could calculate the exact value for latestRemovedXid in the standby. When replaying the deletion

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Heikki Linnakangas
Simon Riggs wrote: XLOG_BTREE_DELETE records would not carry latestRemovedXid, this would be calculated by inspection of heap tuples. You might still want to keep the conservative latestRemovedXid value in the WAL record to avoid the extra work when latestRemovedXid alone allows. -- Heikki

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Simon Riggs
On Tue, 2010-03-09 at 11:20 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: XLOG_BTREE_DELETE records would not carry latestRemovedXid, this would be calculated by inspection of heap tuples. You might still want to keep the conservative latestRemovedXid value in the WAL record to

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-01-29 at 14:04 -0500, Tom Lane wrote: WTF? Simon, this seems to be getting way way beyond anything the community has agreed to. Particularly, inventing GUCs is not something to be doing without consensus. If you or anybody else would

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of the change; the parameter is really to ensure we can test during beta. Well, I was waiting to see

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of the change; the parameter is really to ensure we can test during beta. Well, I was

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 20:48 +0100, Stefan Kaltenbrunner wrote: Simon Riggs wrote: On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of the change;

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: At the moment a btree delete record will cancel every request 1. no matter how long they have been running 2. no matter if they haven't accessed the index being cleaned (they might later, is the thinking...) That seems seriously horrid. What is the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 15:41 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: At the moment a btree delete record will cancel every request 1. no matter how long they have been running 2. no matter if they haven't accessed the index being cleaned (they might later, is the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Sun, 2010-01-31 at 20:48 +0100, Stefan Kaltenbrunner wrote: Simon Riggs wrote: On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 22:05 +0100, Stefan Kaltenbrunner wrote: hmm makes sense from the HS side - but I think to make a case for a GUC it would help a lot to see actual numbers(as in benchmarks) showing how much of a hit it is to the master. Agreed, though my approach to benchmarking was to

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Heikki Linnakangas
Simon Riggs wrote: On Sun, 2010-01-31 at 22:05 +0100, Stefan Kaltenbrunner wrote: hmm makes sense from the HS side - but I think to make a case for a GUC it would help a lot to see actual numbers(as in benchmarks) showing how much of a hit it is to the master. Agreed, though my approach

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: IIRC it was Greg Stark who suggested last time this was discussed that we could calculate the exact value for latestRemovedXid in the standby. When replaying the deletion record, the standby could look at all the heap tuples whose

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: IIRC it was Greg Stark who suggested last time this was discussed that we could calculate the exact value for latestRemovedXid in the standby. When replaying the deletion record, the standby could look at all the heap tuples whose

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Should I revoke this change? Please. We can always put it back later if nothing better gets implemented. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 17:10 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Should I revoke this change? Please. Will do, in morning. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-29 Thread Simon Riggs
On Fri, 2010-01-29 at 14:04 -0500, Tom Lane wrote: sri...@postgresql.org (Simon Riggs) writes: Log Message: --- Augment WAL records for btree delete with GetOldestXmin() to reduce false positives during Hot Standby conflict processing. Simple patch to enhance conflict