Re: [HACKERS] ask for review of MERGE

2010-11-12 Thread Bruce Momjian
Kevin Grittner wrote: Robert Haas robertmh...@gmail.com wrote: rhaas=# create table concurrent (x integer primary key); NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index concurrent_pkey for table concurrent CREATE TABLE rhaas=# insert into x values (1); rhaas=# begin;

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Simon Riggs
On Mon, 2010-10-25 at 16:58 -0400, Robert Haas wrote: On Mon, Oct 25, 2010 at 4:10 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas robertmh...@gmail.com wrote: Now, as Greg says, that might be what some people want, but it's certainly monumentally

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Robert Haas
On Tue, Oct 26, 2010 at 8:10 AM, Simon Riggs si...@2ndquadrant.com wrote: I agree with your analysis. Let me review... [review] Sounds like we're on the same page. Two options for resolution are 1) Throw SERIALIZABLE error 2) Implement something similar to EvalPlanQual As you say, we

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Simon Riggs
On Tue, 2010-10-26 at 16:08 -0400, Robert Haas wrote: On Tue, Oct 26, 2010 at 8:10 AM, Simon Riggs si...@2ndquadrant.com wrote: I agree with your analysis. Let me review... [review] Sounds like we're on the same page. Two options for resolution are 1) Throw SERIALIZABLE error

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas wrote: What we're talking about is what happens when there are concurrent table modifications in progress; and the answer is that you might get serialization anomalies. But we have serialization anomalies without MERGE, too - see the discussions around Kevin Grittner's SSI

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Greg Stark
On Sun, Oct 24, 2010 at 10:43 AM, Robert Haas robertmh...@gmail.com wrote: But let's back up and talk about MVCC for a minute.  Suppose we have three source tuples, (1), (2), and (3); and the target table contains tuples (1) and (2), of which only (1) is visible to our MVCC snapshot; suppose

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Greg Stark gsst...@mit.edu wrote: Robert Haas robertmh...@gmail.com wrote: But let's back up and talk about MVCC for a minute. Suppose we have three source tuples, (1), (2), and (3); and the target table contains tuples (1) and (2), of which only (1) is visible to our MVCC snapshot; suppose

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 1:42 PM, Greg Stark gsst...@mit.edu wrote: On Sun, Oct 24, 2010 at 10:43 AM, Robert Haas robertmh...@gmail.com wrote: But let's back up and talk about MVCC for a minute.  Suppose we have three source tuples, (1), (2), and (3); and the target table contains tuples (1)

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: rhaas=# create table concurrent (x integer primary key); NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index concurrent_pkey for table concurrent CREATE TABLE rhaas=# insert into x values (1); rhaas=# begin; BEGIN rhaas=# insert into

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 3:15 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: rhaas=# create table concurrent (x integer primary key); NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index concurrent_pkey for table concurrent CREATE

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Greg Stark
On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas robertmh...@gmail.com wrote: Now, as Greg says, that might be what some people want, but it's certainly monumentally unserializable. To be clear when I said it's what people want what I meant was that in the common cases it's doing exactly what

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: I would have thought that the INSERT would have created an in doubt tuple which would block the UPDATE. This is just standard MVCC - readers don't block writers, nor writers readers. Sure, but I

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 4:10 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas robertmh...@gmail.com wrote: Now, as Greg says, that might be what some people want, but it's certainly monumentally unserializable. To be clear when I said it's what people want

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sat, Oct 23, 2010 at 4:03 PM, Josh Berkus j...@agliodbs.com wrote: I think that such a lock would also be useful for improving the FK deadlock issues we have. I don't see how. I think the problem you're referring to occurs when different plans update rows in different orders and the

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Martijn van Oosterhout
On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: I think the blocker with MERGE has always been that the standard is *so* general that it could apply to conditions that *aren't* covered by a unique constraint or exclusion constriant. Personally, I'm fine saying that those cases will

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 5:50 AM, Martijn van Oosterhout klep...@svana.org wrote: On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: I think the blocker with MERGE has always been that the standard is *so* general that it could apply to conditions that *aren't* covered by a unique

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: I am also wondering exactly what the semantics are supposed to be under concurrency. We can't assume that it's OK to treat WHEN NOT MATCHED as WHEN MATCHED if a unique-key violation is encountered. The join condition could be something else entirely. I guess we could scan

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 12:15 PM, Greg Smith g...@2ndquadrant.com wrote: Robert Haas wrote: I am also wondering exactly what the semantics are supposed to be under concurrency.  We can't assume that it's OK to treat WHEN NOT MATCHED as WHEN MATCHED if a unique-key violation is encountered.  

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: Well, there's no guarantee that any index at all exists on the target table, so any solution based on index locks can't be fully general. Sure, but in the most common use case I think we're targeting at first, no indexes means there's also no unique constraints either.

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout klep...@svana.org wrote: Can we please not get MERGE hung up on trying to make it atomic. The standard requires no such thing and there are plenty of uses of MERGE that don't involve high concurrency updates of the same row by different

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:39 PM, Greg Smith g...@2ndquadrant.com wrote: Sure, but in the most common use case I think we're targeting at first, no indexes means there's also no unique constraints either.  I don't think people can reasonable expect to MERGE or anything else to guarantee they

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 7:11 PM, Greg Stark gsst...@mit.edu wrote: On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout klep...@svana.org wrote: Can we please not get MERGE hung up on trying to make it atomic. The standard requires no such thing and there are plenty of uses of MERGE that

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Marko Tiikkaja
On 2010-10-23 8:34 AM +0300, Greg Smith wrote: While the performance doesn't need to be great in V1, there needs to be at least some minimal protection against concurrency issues before this is commit quality. What's been bothering me is that so far there has not been an agreement on whether

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Greg Smith
Marko Tiikkaja wrote: What's been bothering me is that so far there has not been an agreement on whether we need to protect against concurrency issues or not. In fact, there has been almost no discussion about the concurrency issues which AIUI have been the biggest single reason we don't

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: Marko Tiikkaja wrote: What's been bothering me is that so far there has not been an agreement on whether we need to protect against concurrency issues or not. In fact, there has been almost no discussion about the concurrency issues which AIUI have

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Boxuan Zhai
This did seem to find a bug in the implementation after running for a while: TRAP: FailedAssertion(!(epqstate-origslot != ((void *)0)), File: execMain.c, Line: 1732) Line number there is relative to what you can see at

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Greg Stark
On Sat, Oct 23, 2010 at 9:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: PostgreSQL doesn't have a good way to lock access to a key value that doesn't exist yet--what other databases call key range locking...Improvements to the index implementation are needed to allow this feature. This seems to

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Josh Berkus
So the trick for MERGE on equality would be to refactor the btree api so that you could find the matching leaf page and *keep* that page locked. Then do an update against the conflicting row found there if any without ever releasing that page lock. Someone else can come along and delete the row

Re: [HACKERS] ask for review of MERGE

2010-10-22 Thread Stefan Kaltenbrunner
On 10/21/2010 08:36 PM, Greg Smith wrote: Robert Haas wrote: I think the right way to write UPSERT is something along the lines of: MERGE INTO Stock t USING (VALUES (10, 1)) s(item_id, balance) ON s.item_id = t.item_id ... [...] Here's what the query plan looks like on a MATCH: Merge

Re: [HACKERS] ask for review of MERGE

2010-10-22 Thread Greg Smith
There are now two branches of MERGE code in review progress available. http://github.com/greg2ndQuadrant/postgres/tree/merge-unstable has the bit-rotted version that doesn't quite work against HEAD yet, while http://github.com/greg2ndQuadrant/postgres/tree/merge is what I'm still testing

Re: [HACKERS] ask for review of MERGE

2010-10-21 Thread Greg Smith
Robert Haas wrote: I think the right way to write UPSERT is something along the lines of: MERGE INTO Stock t USING (VALUES (10, 1)) s(item_id, balance) ON s.item_id = t.item_id ... That led in the right direction, after a bit more fiddling I was finally able to get something that does

Re: [HACKERS] ask for review of MERGE

2010-10-19 Thread Boxuan Zhai
Hi, I considered the empty source table situation again. I think it is correct to upsert nothing in this case. Back to the original logic of MERGE command, it is main purpose is to add the supplementary data from the source table into the target table. Thus, an empty source table means no input

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Robert Haas
I think that MERGE is supposed to trigger one rule for each row in the source data. So: On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith g...@2ndquadrant.com wrote: MERGE INTO Stock t  USING (SELECT * FROM Stock WHERE item_id=10) AS s  ON s.item_id=t.item_id  WHEN MATCHED THEN UPDATE SET

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Boxuan Zhai
On Mon, Oct 18, 2010 at 9:54 PM, Robert Haas robertmh...@gmail.com wrote: I think that MERGE is supposed to trigger one rule for each row in the source data. So: On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith g...@2ndquadrant.com wrote: MERGE INTO Stock t USING (SELECT * FROM Stock WHERE

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Robert Haas
On Mon, Oct 18, 2010 at 10:09 AM, Boxuan Zhai bxzhai2...@gmail.com wrote: On Mon, Oct 18, 2010 at 9:54 PM, Robert Haas robertmh...@gmail.com wrote: I think that MERGE is supposed to trigger one rule for each row in the source data.  So: On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith

Re: [HACKERS] ask for review of MERGE

2010-10-14 Thread Robert Haas
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith g...@2ndquadrant.com wrote: Starting looking at the latest MERGE patch from Boxuan here tonight. The regression tests pass for me here, good starting sign. I expect to move onto trying to break it more creatively next, then onto performance tests.

Re: [HACKERS] ask for review of MERGE

2010-10-14 Thread Greg Smith
Robert Haas wrote: Greg, are you still working on a review of this patch? Yes, just had more distractions while coming to speed up on this area than I'd hoped. I'll get a second round of looking at this done by the weekend. -- Greg Smith, 2ndQuadrant US g...@2ndquadrant.com Baltimore,

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Greg Smith
Starting looking at the latest MERGE patch from Boxuan here tonight. The regression tests pass for me here, good starting sign. I expect to move onto trying to break it more creatively next, then onto performance tests. Nothing much more exciting than that to report so far. It had suffered

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Josh Kupershmidt
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith g...@2ndquadrant.com wrote: The rest of the compiler warnings I saw didn't look related to his code, maybe stuff my picky Ubuntu compiler is noticing that was done recently to HEAD. I haven't checked HEAD without this patch yet to confirm, and am

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Robert Haas
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith g...@2ndquadrant.com wrote: One compiler warning I noticed that needs to get resolved: src/backend/commands/explain.c: explain.c: In function ‘ExplainMergeActions’: explain.c:1528: warning: comparison of distinct pointer types lacks a cast That

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Boxuan Zhai
On Wed, Sep 29, 2010 at 9:16 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith g...@2ndquadrant.com wrote: One compiler warning I noticed that needs to get resolved: src/backend/commands/explain.c: explain.c: In function ‘ExplainMergeActions’:

Re: [HACKERS] ask for review of MERGE

2010-09-24 Thread Greg Smith
Finding time for a review as large as this one is a bit tough, but I've managed to set aside a couple of days for it over the next week. I'm delivering a large project tonight and intend to start in on the review work tomorrow onced that's cleared up. If you're ever not sure who is working

Re: [HACKERS] ask for review of MERGE

2010-09-23 Thread Thom Brown
On 23 September 2010 11:31, Boxuan Zhai bxzhai2...@gmail.com wrote: Dear All, I have just generate a new patch of MERGE command. One main change in this edition is the removal of RASIE ERROR action from MEREG, because its semantics is not well defined yet. I also rewrote the regress test file

Re: [HACKERS] ask for review of MERGE

2010-09-23 Thread Marko Tiikkaja
On 2010-09-23 1:31 PM +0300, Boxuan Zhai wrote: I have just generate a new patch of MERGE command. I haven't followed the discussion very closely, but this part in the regression tests caught my attention: +-- we now have a duplicate key in Buy, so when we join to +-- Stock we will generate