Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Peter Geoghegan
On Tue, Sep 27, 2016 at 2:13 PM, Tom Lane wrote: > I can see the value of processing unique indexes before non-unique ones. > I'm pretty suspicious of trying to prioritize primary keys first, though, > because (a) it's not clear why bother, and (b) PG is a tad squishy about > whether an index is a

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Tom Lane
Heikki Linnakangas writes: > On 09/24/2016 02:34 PM, Peter Geoghegan wrote: >> I go on to explain how this patch represents a partial solution to >> that [1]. That's what I mean by "useful practical consequences". As I >> say in [1], I think we could even get a full solution, if we applied >> this

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Heikki Linnakangas
On 09/27/2016 11:38 AM, Peter Geoghegan wrote: On Tue, Sep 27, 2016 at 9:25 AM, Heikki Linnakangas wrote: I didn't think very hard about it, but yeah, think so.. Do you think it's worth a backpatch? Or, too early to tell? Too early to tell.. - Heikki -- Sent via pgsql-hackers mailing li

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Peter Geoghegan
On Tue, Sep 27, 2016 at 9:25 AM, Heikki Linnakangas wrote: > I didn't think very hard about it, but yeah, think so.. Do you think it's worth a backpatch? Or, too early to tell? -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Heikki Linnakangas
On 09/27/2016 11:22 AM, Peter Geoghegan wrote: On Tue, Sep 27, 2016 at 9:10 AM, Heikki Linnakangas wrote: Hmm, yeah, that'd be nice to fix. I'd like to see a patch specifically to fix that. I'm not convinced that we need all the complications of this patch, to get that fixed. (In particular, in

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Peter Geoghegan
On Tue, Sep 27, 2016 at 9:10 AM, Heikki Linnakangas wrote: > Hmm, yeah, that'd be nice to fix. I'd like to see a patch specifically to > fix that. I'm not convinced that we need all the complications of this > patch, to get that fixed. (In particular, indexam's still wouldn't need to > care about

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-27 Thread Heikki Linnakangas
On 09/24/2016 02:34 PM, Peter Geoghegan wrote: On Tue, Sep 20, 2016 at 8:55 AM, Heikki Linnakangas wrote: Likewise, in ExecInsertIndexTuples(), this makes the deferred-index case work slightly differently from speculative insertion. It's not a big difference, but it again forces you to keep one

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-24 Thread Peter Geoghegan
On Tue, Sep 20, 2016 at 8:55 AM, Heikki Linnakangas wrote: > Thanks for working on this, and sorry for disappearing and dropping this on > the floor earlier. This doesn't apply anymore, thanks to 9c810a2e. Shouldn't > be hard to fix. Thanks for looking at it again. > I was in support of this ear

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-20 Thread Heikki Linnakangas
On 03/17/2016 01:43 AM, Peter Geoghegan wrote: I attach a revision, that makes all the changes that Heikki suggested, except one. As already noted several times, following this suggestion would have added a bug. Alvaro preferred my original approach here in any case. I refer to my original approa

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-04-07 Thread Robert Haas
On Tue, Apr 5, 2016 at 9:47 PM, Robert Haas wrote: >> I do not pursue something like this without good reason. I'm >> optimistic that the patch will be accepted if it is carefully >> considered. > > This patch has attracted no reviewers. Any volunteers? Since nobody has emerged to review this, I

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-04-05 Thread Robert Haas
On Wed, Mar 16, 2016 at 7:43 PM, Peter Geoghegan wrote: > On Wed, Mar 16, 2016 at 11:25 AM, Robert Haas wrote: >> Sure, and if everybody does that, then there will be 40 patches that >> get updated in the last 2 days if the CommitFest, and that will be >> impossible. Come on. You're demanding a

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-19 Thread Peter Geoghegan
On Wed, Mar 16, 2016 at 11:25 AM, Robert Haas wrote: > Sure, and if everybody does that, then there will be 40 patches that > get updated in the last 2 days if the CommitFest, and that will be > impossible. Come on. You're demanding a degree of preferential > treatment which is unsupportable. I

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-19 Thread Robert Haas
On Mon, Mar 14, 2016 at 8:17 PM, Peter Geoghegan wrote: > On Mon, Mar 14, 2016 at 5:04 PM, Robert Haas wrote: >> There hasn't been a new version of this patch in 9 months, you're >> clearly not in a hurry to produce one, and nobody else seems to feel >> strongly that this is something that needs

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-14 Thread Peter Geoghegan
On Mon, Mar 14, 2016 at 5:21 PM, Andres Freund wrote: > So? You're not the only one. I don't see why we shouldn't move this to > 'returned with feedback' until there's a new version. I don't see any point in that; I intend to get a revision in to the ongoing CF. But fine. -- Peter Geoghegan -

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-14 Thread Andres Freund
On 2016-03-14 17:17:02 -0700, Peter Geoghegan wrote: > On Mon, Mar 14, 2016 at 5:04 PM, Robert Haas wrote: > > There hasn't been a new version of this patch in 9 months, you're > > clearly not in a hurry to produce one, and nobody else seems to feel > > strongly that this is something that needs t

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-14 Thread Peter Geoghegan
On Mon, Mar 14, 2016 at 5:04 PM, Robert Haas wrote: > There hasn't been a new version of this patch in 9 months, you're > clearly not in a hurry to produce one, and nobody else seems to feel > strongly that this is something that needs to be done at all. I think > we could just let this go and be

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-14 Thread Robert Haas
On Sat, Mar 12, 2016 at 5:53 PM, Peter Geoghegan wrote: > On Sat, Mar 12, 2016 at 2:43 PM, Michael Paquier > wrote: >> Only one version of this patch has been sent at the beginning of this >> thread, and Heikki has clearly expressed his disagreement about at >> least a portion of it at the beginn

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-12 Thread Peter Geoghegan
On Sat, Mar 12, 2016 at 2:53 PM, Peter Geoghegan wrote: > I said "basically uncontroversial", not "uncontroversial". That is a > perfectly accurate characterization of the patch, and if you disagree > than I suggest you re-read the thread. In particular, note that Alvaro eventually sided with me

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-12 Thread Peter Geoghegan
On Sat, Mar 12, 2016 at 2:43 PM, Michael Paquier wrote: > Only one version of this patch has been sent at the beginning of this > thread, and Heikki has clearly expressed his disagreement about at > least a portion of it at the beginning of this thread, so I find it > hard to define it as an "unco

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-12 Thread Michael Paquier
On Sat, Mar 12, 2016 at 11:24 PM, Peter Geoghegan wrote: > On Fri, Mar 11, 2016 at 5:26 AM, Robert Haas wrote: >> This patch was reviewed during CF 2016-01 and has not been updated for >> CF 2016-03. I think we should mark it Returned with Feedback. > > I have a full plate at the moment, Robert,

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-12 Thread Peter Geoghegan
On Fri, Mar 11, 2016 at 5:26 AM, Robert Haas wrote: > This patch was reviewed during CF 2016-01 and has not been updated for > CF 2016-03. I think we should mark it Returned with Feedback. I have a full plate at the moment, Robert, both as a reviewer and as a patch author. This patch is basicall

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-03-11 Thread Robert Haas
On Mon, Jan 18, 2016 at 3:14 PM, Peter Geoghegan wrote: > On Mon, Jan 18, 2016 at 11:56 AM, Alvaro Herrera > wrote: >>> That's because I believe this is quite broken, as already pointed out. >> >> I think I like your approach better. > > That makes things far simpler, then. > >>> Your premise her

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Peter Geoghegan
On Mon, Jan 18, 2016 at 11:56 AM, Alvaro Herrera wrote: >> That's because I believe this is quite broken, as already pointed out. > > I think I like your approach better. That makes things far simpler, then. >> Your premise here is that what Heikki said in passing months ago is >> incontrovertib

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Mon, Jan 18, 2016 at 8:36 AM, Alvaro Herrera > wrote: > > If you refuse to post an updated version of the patch until Heikki > > weighs in some more, and given that Heikki has (for the purposes of this > > patch) completely vanished, I think we should mark this rejected

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Peter Geoghegan
On Mon, Jan 18, 2016 at 8:36 AM, Alvaro Herrera wrote: > If you refuse to post an updated version of the patch until Heikki > weighs in some more, and given that Heikki has (for the purposes of this > patch) completely vanished, I think we should mark this rejected. I don't refuse. I just don't w

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-01-18 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Fri, Dec 18, 2015 at 12:55 PM, Robert Haas wrote: > > I only see one patch version on the thread. > > I'm not going to post a revision until I thrash out the tiny issues > with Heikki. He kind of trailed off. So maybe that kills it > immediately, which is a shame. If

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-19 Thread Peter Geoghegan
On Sat, Dec 19, 2015 at 3:26 PM, Michael Paquier wrote: > +1. There are folks around doing tests using 9.5 now, it is not > correct to impact the effort they have been putting on it until now. This is a total misrepresentation of what I've said. -- Peter Geoghegan -- Sent via pgsql-hackers m

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-19 Thread Michael Paquier
On Sat, Dec 19, 2015 at 1:58 AM, Robert Haas wrote: > No, it's far too late to be pushing this into 9.5. We are at RC1 now > and hoping to cut a final release right after Christmas. I think it's > quite wrong to argue that these changes have no risk of destabilizing > 9.5. Nobody is exempt from

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-18 Thread Peter Geoghegan
On Fri, Dec 18, 2015 at 12:55 PM, Robert Haas wrote: > On Fri, Dec 18, 2015 at 2:04 PM, Peter Geoghegan wrote: >> It isn't true that Heikki was not basically in favor of this. This >> should have been committed as part of the original patch, really. > > Maybe he wasn't against the whole thing, bu

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-18 Thread Robert Haas
On Fri, Dec 18, 2015 at 2:04 PM, Peter Geoghegan wrote: > It isn't true that Heikki was not basically in favor of this. This > should have been committed as part of the original patch, really. Maybe he wasn't against the whole thing, but he's posted two messages to this thread and they can't be r

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-18 Thread Peter Geoghegan
On Fri, Dec 18, 2015 at 8:58 AM, Robert Haas wrote: > No, it's far too late to be pushing this into 9.5. We are at RC1 now > and hoping to cut a final release right after Christmas. I think it's > quite wrong to argue that these changes have no risk of destabilizing > 9.5. Nobody is exempt from

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-18 Thread Robert Haas
On Thu, Dec 17, 2015 at 2:55 AM, Peter Geoghegan wrote: > On Wed, Dec 16, 2015 at 11:44 PM, Peter Geoghegan wrote: >>> In any case, at this point 9.5 is really aimed to be stabilized, so >>> targeting only master is a far saner approach IMO for this patch. >>> Pushing that in 9.5 a couple of mont

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-17 Thread Michael Paquier
On Fri, Dec 18, 2015 at 4:31 PM, Peter Geoghegan wrote: > On Thu, Dec 17, 2015 at 11:06 PM, Michael Paquier > wrote: >> Now per the two points listed in >> the first sentence in this paragraph, perhaps this opinion is fine as >> moot :) I didn't get much involved in the development of this code >

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-17 Thread Peter Geoghegan
On Thu, Dec 17, 2015 at 11:06 PM, Michael Paquier wrote: >> I should add: I think that the chances of this patch destabilizing the >> code are very slim, once it receives the proper review. Certainly, I >> foresee no possible downside to not inserting the doomed IndexTuple, >> since it's guarantee

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-17 Thread Michael Paquier
On Thu, Dec 17, 2015 at 4:55 PM, Peter Geoghegan wrote: > On Wed, Dec 16, 2015 at 11:44 PM, Peter Geoghegan wrote: >>> In any case, at this point 9.5 is really aimed to be stabilized, so >>> targeting only master is a far saner approach IMO for this patch. >>> Pushing that in 9.5 a couple of mont

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-16 Thread Peter Geoghegan
On Wed, Dec 16, 2015 at 11:44 PM, Peter Geoghegan wrote: >> In any case, at this point 9.5 is really aimed to be stabilized, so >> targeting only master is a far saner approach IMO for this patch. >> Pushing that in 9.5 a couple of months back may have given enough >> reason to do so... But well l

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-16 Thread Peter Geoghegan
On Wed, Dec 16, 2015 at 11:20 PM, Michael Paquier wrote: > (A couple of months later) > This is not an actual fix, but an optimization, no? > UNIQUE_CHECK_SPECULATIVE is just used to optimize a couple of code > paths in the case of a insert conflicting during btree insertion.. > > In any case, at

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-12-16 Thread Michael Paquier
On Sun, Oct 4, 2015 at 2:07 AM, Peter Geoghegan wrote: > On Sat, Oct 3, 2015 at 6:38 AM, Andres Freund wrote: >> I'm not arguing against any of this - but I don't think this needs to be >> on the 9.5 open items list. I plan to remove from there. > > Obviously I don't think that this is a critical

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-10-03 Thread Peter Geoghegan
On Sat, Oct 3, 2015 at 6:38 AM, Andres Freund wrote: > I'm not arguing against any of this - but I don't think this needs to be > on the 9.5 open items list. I plan to remove from there. Obviously I don't think that this is a critical fix. I do think that it would be nice to keep the branches in

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-10-03 Thread Andres Freund
Hi, On 2015-06-10 16:19:27 -0700, Peter Geoghegan wrote: > Currently, speculative insertion (the INSERT ... ON CONFLICT DO UPDATE > executor/storage infrastructure) uses checkUnique == > UNIQUE_CHECK_PARTIAL for unique indexes, which is a constant > originally only used by deferred unique constrai

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-19 Thread Peter Geoghegan
On Thu, Jul 2, 2015 at 2:58 PM, Peter Geoghegan wrote: > As with row locking, with insertion, if there is a conflict, any > outcome (UPDATE or INSERT) is then possible. Where are we on this? It would be nice to get this out of the way before a 9.5 beta is put out. Thanks -- Peter Geoghegan --

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-02 Thread Peter Geoghegan
On Thu, Jul 2, 2015 at 10:49 AM, Peter Geoghegan wrote: > Well, waiting means getting a ShareLock on the other session's XID. > You can't do that without first releasing your locks, unless you're > okay with unprincipled deadlocks. Besides, if the other session wins the race and inserts a physica

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-02 Thread Peter Geoghegan
On Thu, Jul 2, 2015 at 1:30 AM, Heikki Linnakangas wrote: >> Sure, if a speculative inserter detects a conflict, it still has to >> wait. But not in the aminsert call, and not until it cleans up its >> physical insertion (by super-deleting). Clearly a >> CHECK_UNIQUE_SPECULATIVE would have to be m

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-02 Thread Heikki Linnakangas
On 07/01/2015 09:19 PM, Peter Geoghegan wrote: On Wed, Jul 1, 2015 at 10:45 AM, Peter Geoghegan wrote: On Tue, Jun 30, 2015 at 11:19 AM, Heikki Linnakangas wrote: I agree it would be cleaner to have a separate CHECK_UNIQUE_XXX code for speculative insertions. You've defined CHECK_UNIQUE_SPECU

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-01 Thread Peter Geoghegan
On Wed, Jul 1, 2015 at 10:45 AM, Peter Geoghegan wrote: > On Tue, Jun 30, 2015 at 11:19 AM, Heikki Linnakangas wrote: >> I agree it would be cleaner to have a separate CHECK_UNIQUE_XXX code for >> speculative insertions. You've defined CHECK_UNIQUE_SPECULATIVE as "like >> CHECK_UNIQUE_PARTIAL, bu

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-01 Thread Peter Geoghegan
On Wed, Jul 1, 2015 at 10:45 AM, Peter Geoghegan wrote: > You can construct a theoretical case where lock starvation occurs with > unique constraint enforcement. I think it helps with nbtree here that > someone will reliably *not* see a conflict when concurrently > inserting, because unique index

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-07-01 Thread Peter Geoghegan
On Tue, Jun 30, 2015 at 11:19 AM, Heikki Linnakangas wrote: > I agree it would be cleaner to have a separate CHECK_UNIQUE_XXX code for > speculative insertions. You've defined CHECK_UNIQUE_SPECULATIVE as "like > CHECK_UNIQUE_PARTIAL, but you don't have to insert the index tuple if > there's a conf

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2015-06-30 Thread Heikki Linnakangas
On 06/11/2015 02:19 AM, Peter Geoghegan wrote: Currently, speculative insertion (the INSERT ... ON CONFLICT DO UPDATE executor/storage infrastructure) uses checkUnique == UNIQUE_CHECK_PARTIAL for unique indexes, which is a constant originally only used by deferred unique constraints. It occurred

[HACKERS] Refactoring speculative insertion with unique indexes a little

2015-06-10 Thread Peter Geoghegan
Currently, speculative insertion (the INSERT ... ON CONFLICT DO UPDATE executor/storage infrastructure) uses checkUnique == UNIQUE_CHECK_PARTIAL for unique indexes, which is a constant originally only used by deferred unique constraints. It occurred to me that this has a number of disadvantages: *