Re: [HACKERS] Multixid hindsight design

On Mon, Nov 9, 2015 at 9:57 PM, Craig Ringer wrote: > On 10 November 2015 at 02:26, Robert Haas wrote: >>> I'd like to see, say, python and the 'unittest' module added, and >>> to see acceptance of tests using psycopg2 to stream and decode WAL >>>

Re: [HACKERS] Multixid hindsight design

On Mon, Nov 9, 2015 at 2:02 AM, Craig Ringer wrote: > On 25 June 2015 at 00:52, Robert Haas wrote: >> I agree that we can do much better at testing than we traditionally >> have done, and I think pretty much everyone in the room for the >> developer

Re: [HACKERS] Multixid hindsight design

Robert Haas writes: > On Mon, Nov 9, 2015 at 2:02 AM, Craig Ringer wrote: >> Something that'd really help with that, IMO, would be weakening the >> requirement that everything be C or be core Perl. Instead require that >> configure detect whether or

Re: [HACKERS] Multixid hindsight design

On 10 November 2015 at 02:26, Robert Haas wrote: >> I'd like to see, say, python and the 'unittest' module added, and >> to see acceptance of tests using psycopg2 to stream and decode WAL >> from a slot. > > I actually kind of hate this sort of thing. For what it's worth,

Re: [HACKERS] Multixid hindsight design

On Fri, Jun 5, 2015 at 10:46 AM, Robert Haas robertmh...@gmail.com wrote: It would be a great deal nicer if we didn't have to keep ANY of the transactional data for a tuple around once it's all-visible. Heikki defined ephemeral as only needed when xmin or xmax is in-progress, but if we

Re: [HACKERS] Multixid hindsight design

On 24 June 2015 at 14:57, Robert Haas robertmh...@gmail.com wrote: On Fri, Jun 5, 2015 at 10:46 AM, Robert Haas robertmh...@gmail.com wrote: It would be a great deal nicer if we didn't have to keep ANY of the transactional data for a tuple around once it's all-visible. Heikki defined

Re: [HACKERS] Multixid hindsight design

On Wed, Jun 24, 2015 at 11:30 AM, Simon Riggs si...@2ndquadrant.com wrote: Though TED sounds nice, the way to avoid another round of on-disk bugs is by using a new kind of testing, not simply by moving the bits around. I agree that we can do much better at testing than we traditionally have

Re: [HACKERS] Multixid hindsight design

On 24 June 2015 at 16:30, Simon Riggs si...@2ndquadrant.com wrote: Though TED sounds nice, the way to avoid another round of on-disk bugs is by using a new kind of testing, not simply by moving the bits around. It might be argued that we are increasing the diagnostic/forensic capabilities

Re: [HACKERS] Multixid hindsight design

On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like pg_subtrans, and pg_multixact before 9.3. It doesn't

Re: [HACKERS] Multixid hindsight design

On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like

Re: [HACKERS] Multixid hindsight design

On 11 May 2015 at 22:20, Heikki Linnakangas hlinn...@iki.fi wrote: I'd like to discuss how we should've implemented the infamous 9.3 multixid/row-locking stuff, and perhaps still should in 9.6. Hindsight is always 20/20 - I'll readily admit that I didn't understand the problems until well

Re: [HACKERS] Multixid hindsight design

On 5 June 2015 at 11:02, Andres Freund and...@anarazel.de wrote: On 2015-06-05 10:45:09 +0100, Simon Riggs wrote: On 1 June 2015 at 20:53, Thomas Munro thomas.mu...@enterprisedb.com wrote: On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of

Re: [HACKERS] Multixid hindsight design

Simon Riggs si...@2ndquadrant.com writes: On 11 May 2015 at 22:20, Heikki Linnakangas hlinn...@iki.fi wrote: So the lesson here is that having a permanent pg_multixact is not nice, and we should get rid of it. Here's how to do that: An alternate proposal: 1. Store only the Locking xids in

Re: [HACKERS] Multixid hindsight design

On Fri, Jun 5, 2015 at 6:17 AM, Simon Riggs si...@2ndquadrant.com wrote: I think we should think back to exactly what we are trying to store, why and for how long. A clear definition of what we are trying to achieve is essential to solving the problem. In my understanding we need to store *

Re: [HACKERS] Multixid hindsight design

On Fri, Jun 5, 2015 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: That way if we need to make Offsets SLRU persistent it won't bloat. We then leave the Members SLRU as non-persistent, just as it was 9.3 I don't think you can do that, because it supposes that locking XIDs need not be

Re: [HACKERS] Multixid hindsight design

On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like pg_subtrans, and pg_multixact before 9.3. It doesn't need to be WAL-logged, and we are free to change its on-disk layout even in a

Re: [HACKERS] Multixid hindsight design

On Mon, Jun 1, 2015 at 8:53 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: What about prepared transactions? They can lock rows FOR SHARE that survive server restarts. And they can make update chains that are still uncommitted after a restart. I think we should think separately about

Re: [HACKERS] Multixid hindsight design

On 05/12/2015 01:51 AM, Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: So the lesson here is that having a permanent pg_multixact is not nice, and we should get rid of it. Here's how to do that: That would be cool, but ... Looking at the tuple header, the CID and CTID fields are

Re: [HACKERS] Multixid hindsight design

On Mon, May 11, 2015 at 5:20 PM, Heikki Linnakangas hlinn...@iki.fi wrote: The main problem with the infamous multixid changes was that it made pg_multixact a permanent, critical, piece of data. Without it, you cannot decipher whether some rows have been deleted or not. The 9.3 changes

Re: [HACKERS] Multixid hindsight design

Heikki Linnakangas hlinn...@iki.fi writes: So the lesson here is that having a permanent pg_multixact is not nice, and we should get rid of it. Here's how to do that: That would be cool, but ... Looking at the tuple header, the CID and CTID fields are only needed, when either xmin or xmax