Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-16 Thread Robert Haas
On Sun, Jan 15, 2012 at 11:03 AM, Hitoshi Harada umi.tan...@gmail.com wrote: On Sat, Jan 14, 2012 at 6:48 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada umi.tan...@gmail.com wrote: The patch looks ok, though I wonder if we could have a way to

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-15 Thread Hitoshi Harada
On Sat, Jan 14, 2012 at 6:48 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada umi.tan...@gmail.com wrote: The patch looks ok, though I wonder if we could have a way to release the lock on namespace much before the end of transaction. Well, that wold

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Hitoshi Harada
On Fri, Jan 13, 2012 at 2:05 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas robertmh...@gmail.com wrote: I have plans to try to improve this, but it's one of those things that I care about more than the people who write the checks do, so it hasn't

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Hitoshi Harada
On Sat, Jan 14, 2012 at 2:25 AM, Hitoshi Harada umi.tan...@gmail.com wrote: On Fri, Jan 13, 2012 at 2:05 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas robertmh...@gmail.com wrote: I have plans to try to improve this, but it's one of those things

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-14 Thread Robert Haas
On Sat, Jan 14, 2012 at 5:25 AM, Hitoshi Harada umi.tan...@gmail.com wrote: The patch looks ok, though I wonder if we could have a way to release the lock on namespace much before the end of transaction. Well, that wold kind of miss the point, wouldn't it? I mean, the race is that the process

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2012-01-13 Thread Robert Haas
On Tue, Nov 29, 2011 at 10:10 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 29, 2011 at 3:37 AM, Daniel Farina dan...@heroku.com wrote: Hmm, just to prod this thread: has any fix for this been committed? After Nikhil confirmed that this bug could cause pg_dump to not be able to

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-29 Thread Daniel Farina
On Mon, Nov 14, 2011 at 12:07 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Nov 14, 2011 at 2:26 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: Well, it looks to me like there are three different places that

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-29 Thread Robert Haas
On Tue, Nov 29, 2011 at 3:37 AM, Daniel Farina dan...@heroku.com wrote: Hmm, just to prod this thread: has any fix for this been committed? After Nikhil confirmed that this bug could cause pg_dump to not be able to operate without direct catalog surgery I have encountered this bug (and treated

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Nikhil Sontakke
If all you need to do is lock a schema, you can just call LockDatabaseObject(NamespaceRelationId, namespace_oid, 0, AccessShareLock); there's no need to fake up an objectaddress just to take a lock. But I think that's not really all you need to do, because somebody could drop the namespace

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 11:48 AM, Nikhil Sontakke nikhil.sonta...@enterprisedb.com wrote: If all you need to do is lock a schema, you can just call LockDatabaseObject(NamespaceRelationId, namespace_oid, 0, AccessShareLock); there's no need to fake up an objectaddress just to take a lock.  But

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Nikhil Sontakke
So it's probably going to take a while to get this completely nailed down, but we can keep chipping away at it. Agreed. So are you planning to commit this change? Or we want some more objects to be fixed? Last I looked at this, we will need locking to be done while creating tables, views,

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 12:48 PM, Nikhil Sontakke nikkh...@gmail.com wrote: So it's probably going to take a while to get this completely nailed down, but we can keep chipping away at it. Agreed. So are you planning to commit this change? Or we want some more objects to be fixed? Last I

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: Well, it looks to me like there are three different places that we need to nail down: RangeVarGetAndCheckCreationNamespace() is used for relations (except that a few places call RangeVarGetCreationNamespace directly,

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-14 Thread Robert Haas
On Mon, Nov 14, 2011 at 2:26 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun nov 14 15:56:43 -0300 2011: Well, it looks to me like there are three different places that we need to nail down: RangeVarGetAndCheckCreationNamespace() is used for

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 7:00 AM, Nikhil Sontakke nikkh...@gmail.com wrote: PFA, a patch against git head. We take the AccessShareLock lock on the schema in DefineRelation now. Note that we do not want to interlock with other concurrent creations in the schema. We only want to interlock with

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
Hi, Ok, understood. PFA, a patch against git head. We take the AccessShareLock lock on the schema in DefineRelation now. Note that we do not want to interlock with other concurrent creations in the schema. We only want to interlock with deletion activity. So even performance wise this should

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Tom Lane
Nikhil Sontakke nikkh...@gmail.com writes: PFA, a patch against git head. We take the AccessShareLock lock on the schema in DefineRelation now. Um ... why would we do this only for tables, and not for creations of other sorts of objects that belong to schemas? Also, if we are going to believe

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
Um ... why would we do this only for tables, and not for creations of other sorts of objects that belong to schemas? Right, we need to do it for other objects like functions etc. too. Also, if we are going to believe that this is a serious problem, what of ALTER ... SET SCHEMA? I admit,

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Daniel Farina
On Wed, Nov 9, 2011 at 1:56 AM, Nikhil Sontakke nikkh...@gmail.com wrote: Hi, Consider the following sequence of events: s1 # CREATE SCHEMA test_schema; s1 # CREATE TABLE test_schema.c1(x int); Now open another session s2 and via gdb issue a breakpoint on heap_create_with_catalog() which

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-10 Thread Nikhil Sontakke
Continuing in gdb, also completes the creation of c2 table without any errors. We are now left with a dangling entry in pg_class along with all the corresponding data files in our data directory. The problem becomes worse if c2 was created using a TABLESPACE. Now dropping of that

[HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
Hi, Consider the following sequence of events: s1 # CREATE SCHEMA test_schema; s1 # CREATE TABLE test_schema.c1(x int); Now open another session s2 and via gdb issue a breakpoint on heap_create_with_catalog() which is called by DefineRelation(). s2 # CREATE TABLE test_schema.c2(y int); The

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Robert Haas
On Wed, Nov 9, 2011 at 4:56 AM, Nikhil Sontakke nikkh...@gmail.com wrote: Consider the following sequence of events: s1 # CREATE SCHEMA test_schema; s1 # CREATE TABLE test_schema.c1(x int); Now open another session s2 and via gdb issue a breakpoint on heap_create_with_catalog() which is

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
We definitely need some interlocking to handle this. For lack of better APIs, we could do a LockDatabaseObject() call in AccessShareLock mode on the namespace and release the same on completion of the creation of the object. Thoughts? In general, we've been reluctant to add locking

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Robert Haas
On Wed, Nov 9, 2011 at 9:51 AM, Nikhil Sontakke nikkh...@gmail.com wrote: Yeah thanks, that does the object locking. For pre-9.1 versions, we will need a similar solution. I encountered the issue on 8.3.x.. I don't think we should back-patch a fix of this type. There is a lot of cruftiness of

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-09 Thread Nikhil Sontakke
Yeah thanks, that does the object locking. For pre-9.1 versions, we will need a similar solution. I encountered the issue on 8.3.x.. I don't think we should back-patch a fix of this type. There is a lot of cruftiness of this type scattered throughout the code base, and if we start