Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-03-03 Thread Andres Freund
On 2015-02-25 14:04:55 -0800, Peter Geoghegan wrote: On Wed, Feb 25, 2015 at 3:26 AM, Andres Freund and...@2ndquadrant.com wrote: I'm pretty sure this will entirely fail if you have a transaction that's large enough to spill to disk. Calling ReorderBufferIterTXNNext() will reuse the memory

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-03-03 Thread Peter Geoghegan
On Tue, Mar 3, 2015 at 3:13 AM, Andres Freund and...@2ndquadrant.com wrote: toast_save_datum() is called with a heap_insert() call before heap insertion for the tuple proper. We're relying on the assumption that if there is no immediate super deletion record, things are fine. We cannot

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-25 Thread Peter Geoghegan
On Wed, Feb 25, 2015 at 3:26 AM, Andres Freund and...@2ndquadrant.com wrote: I'm pretty sure this will entirely fail if you have a transaction that's large enough to spill to disk. Calling ReorderBufferIterTXNNext() will reuse the memory for the in memory changes. It's also borked because it

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-25 Thread Jim Nasby
On 2/19/15 4:11 AM, Andres Freund wrote: Thoughts? Can't say that I've given conflict resolution for multi-master systems a great deal of thought before now, so I might be quite off the mark here. I don't think conflict resolution actually plays a role here. This is about consistency inside a

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-25 Thread Andres Freund
Hi, Well, that implies that we'd actually know that we'd succeed when WAL logging the speculative heap tuple's insertion. We literally have no way of knowing if that's the case at that point, though - that's just the nature of value locking scheme #2's optimistic approach. Attached

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-23 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 3:44 PM, Peter Geoghegan p...@heroku.com wrote: they'd only see a REORDER_BUFFER_CHANGE_INSERT when that was the definitive outcome of an UPSERT, or alternatively a REORDER_BUFFER_CHANGE_UPDATE when that was the definitive outcome. No need for output plugins to consider

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-20 Thread Peter Geoghegan
On Thu, Feb 19, 2015 at 2:11 AM, Andres Freund and...@2ndquadrant.com wrote: On 2015-02-18 16:35:14 -0800, Peter Geoghegan wrote: Andres pointed out that the INSERT ... ON CONFLICT UPDATE patch doesn't work well with logical decoding. Just to make that clear: I didn't actually test it, but it

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-20 Thread Andres Freund
On 2015-02-20 15:44:12 -0800, Peter Geoghegan wrote: On Thu, Feb 19, 2015 at 2:11 AM, Andres Freund and...@2ndquadrant.com wrote: Yes. It'd be easiest if the only the final insert/update were actually WAL logged as full actions. Well, that implies that we'd actually know that we'd succeed

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-20 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 3:52 PM, Andres Freund and...@2ndquadrant.com wrote: On 2015-02-20 15:44:12 -0800, Peter Geoghegan wrote: On Thu, Feb 19, 2015 at 2:11 AM, Andres Freund and...@2ndquadrant.com wrote: Yes. It'd be easiest if the only the final insert/update were actually WAL logged

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-19 Thread Andres Freund
Hi, On 2015-02-18 16:35:14 -0800, Peter Geoghegan wrote: Andres pointed out that the INSERT ... ON CONFLICT UPDATE patch doesn't work well with logical decoding. Just to make that clear: I didn't actually test it, but it ddidn't look good. I guess that the best way of fixing this is exposing

[HACKERS] INSERT ... ON CONFLICT UPDATE and logical decoding

2015-02-18 Thread Peter Geoghegan
Andres pointed out that the INSERT ... ON CONFLICT UPDATE patch doesn't work well with logical decoding. Basically, as things stand, there is no means of determining that an INSERT had an ON CONFLICT UPDATE clause from logical decoding. It isn't obvious that this matters - after all, the relevant