Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Ashutosh Bapat]

On Wed, Jun 14, 2017 at 10:21 AM, Ashutosh Bapat
 wrote:
> On Tue, Jun 13, 2017 at 7:14 PM, Rahila Syed  wrote:
>> I reviewed and tested
>> 0001-Dependency-between-partitioned-table-and-partition_v1.patch
>> It applies cleanly on master and make check passes.
>>
>> Following are few comments:
>>
>>>/*
>>> * Drop the dependency created by StoreCatalogInheritance1 (CREATE TABLE
>>> * INHERITS/ALTER TABLE INHERIT -- refclassid will be RelationRelationId)
>>> or
>>> * heap_create_with_catalog (CREATE TABLE OF/ALTER TABLE OF -- refclassid
>>> will
>>> * be TypeRelationId).  There's no convenient way to do this, so go
>>> trawling
>>> * through pg_depend.
>>> */
>>>static void
>>>drop_parent_dependency(Oid relid, Oid refclassid, Oid refobjid,
>>DependencyType deptype)
>>
>> This is not directly related to this patch but still would like to mention.
>> drop_parent_dependency() is being used to drop the dependency created
>> by CREATE TABLE PARTITION OF command also, so probably the comment
>> needs to be modified to reflect that.
>>
>
> The comment is fine for dependency created by CREATE TABLE PARTITION
> OF since that too goes through StoreCatalogInheritance1(). But it's
> not correct for CREATE TABLE ... OF  since that does
> not go through StoreCatalogInheritance1().
>
I missed "or" part of the prologue while writing this. Now that I have
read it carefully, the prologue looks good to me.
-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Ashutosh Bapat]

On Tue, Jun 13, 2017 at 9:23 PM, Robert Haas  wrote:
> On Tue, Jun 13, 2017 at 9:44 AM, Rahila Syed  wrote:
>> I have added tests to the
>> 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please
>> find attached the v2 patch.
>
> Thanks.  Committed.

Thanks.

> I don't think the 0002 patch is an improvement -
> sure, it keeps those things in sync, but it also makes the code harder
> to read.  On balance I think it's a negative.
>

I don't think the code is hard to read, but I agree that the macro
name TABLE_COMPOSITE_TYPE_DEPENDENCY isn't conveying the real sense.
But that's not a topic for this thread. I will start a separate a
thread.

-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Ashutosh Bapat]

On Tue, Jun 13, 2017 at 7:14 PM, Rahila Syed  wrote:
> I reviewed and tested
> 0001-Dependency-between-partitioned-table-and-partition_v1.patch
> It applies cleanly on master and make check passes.
>
> Following are few comments:
>
>>/*
>> * Drop the dependency created by StoreCatalogInheritance1 (CREATE TABLE
>> * INHERITS/ALTER TABLE INHERIT -- refclassid will be RelationRelationId)
>> or
>> * heap_create_with_catalog (CREATE TABLE OF/ALTER TABLE OF -- refclassid
>> will
>> * be TypeRelationId).  There's no convenient way to do this, so go
>> trawling
>> * through pg_depend.
>> */
>>static void
>>drop_parent_dependency(Oid relid, Oid refclassid, Oid refobjid,
>DependencyType deptype)
>
> This is not directly related to this patch but still would like to mention.
> drop_parent_dependency() is being used to drop the dependency created
> by CREATE TABLE PARTITION OF command also, so probably the comment
> needs to be modified to reflect that.
>

The comment is fine for dependency created by CREATE TABLE PARTITION
OF since that too goes through StoreCatalogInheritance1(). But it's
not correct for CREATE TABLE ... OF  since that does
not go through StoreCatalogInheritance1().

>>+/*
>>+ * Partition tables are expected to be dropped when the parent partitioned
>>+ * table gets dropped. Hence for partitioning we use AUTO dependency.
>>+ * Otherwise, for regular inheritance use NORMAL dependency.
> A minor cosmetic correction:
> + Hence for declarative partitioning we use AUTO dependency
> + * Otherwise, for regular inheritance we use NORMAL dependency.
>
> I have added tests to the
> 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please
> find attached the v2 patch.
>
>
>
> On Tue, Jun 13, 2017 at 10:25 AM, Amit Langote
>  wrote:
>>
>> Hi Ashutosh,
>>
>> On 2017/06/12 20:56, Ashutosh Bapat wrote:
>> > Hi,
>> > If we detach a partition and drop the corresponding partitioned table,
>> > it drops the once-partition now-standalone table as well.
>>
>> Thanks for the report.  Looks like 8b4d582d279d78 missed the bit about
>> drop_parent_dependency() that you describe in your analysis below.
>>
>> > The reason for this is as follows
>> > An AUTO dependency is recorded between a partitioned table and
>> > partition. In
>> > case of inheritance we record a NORMAL dependency between parent
>> > and child. While
>> > detaching a partition, we call RemoveInheritance(), which should
>> > have taken
>> > care of removing this dependency. But it removes the dependencies
>> > which are
>> > marked as NORMAL. When we changed the dependency for partitioned
>> > case from
>> > NORMAL to AUTO by updating StoreCatalogInheritance1(), this function
>> > was not
>> > updated. This caused the dependency to linger behind even after
>> > detaching the
>> > partition, thus causing the now standalone table (which was once a
>> > partition)
>> > to be dropped when the parent partitioned table is dropped. This
>> > patch fixes
>> > RemoveInheritance() to choose appropriate dependency.
>> >
>> > Attached patch 0001 to fix this.
>>
>> Looks good to me.  Perhaps, the example in your email could be added as a
>> test case.
>>
>> > I see a similar issue-in-baking in case we change the type of
>> > dependency recorded between a table and the composite type associated
>> > with using CREATE TABLE ... OF command. 0002 patch addresses the
>> > potential issue by using a macro while creating and dropping the
>> > dependency in corresponding functions. There might be more such
>> > places, but I haven't searched for those.
>>
>> Might be a good idea too.
>>
>> Adding this to the open items list.
>>
>> Thanks,
>> Amit
>>
>>
>>
>> --
>> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-hackers
>
>



-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Robert Haas]

On Tue, Jun 13, 2017 at 9:44 AM, Rahila Syed  wrote:
> I have added tests to the
> 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please
> find attached the v2 patch.

Thanks.  Committed.  I don't think the 0002 patch is an improvement -
sure, it keeps those things in sync, but it also makes the code harder
to read.  On balance I think it's a negative.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Rahila Syed]

I reviewed and tested
0001-Dependency-between-partitioned-table-and-partition_v1.patch
It applies cleanly on master and make check passes.

Following are few comments:

>/*
> * Drop the dependency created by StoreCatalogInheritance1 (CREATE TABLE
> * INHERITS/ALTER TABLE INHERIT -- refclassid will be RelationRelationId)
or
> * heap_create_with_catalog (CREATE TABLE OF/ALTER TABLE OF -- refclassid
will
> * be TypeRelationId).  There's no convenient way to do this, so go
trawling
> * through pg_depend.
> */
>static void
>drop_parent_dependency(Oid relid, Oid refclassid, Oid refobjid,
   DependencyType deptype)

This is not directly related to this patch but still would like to mention.
drop_parent_dependency() is being used to drop the dependency created
by CREATE TABLE PARTITION OF command also, so probably the comment
needs to be modified to reflect that.

>+/*
>+ * Partition tables are expected to be dropped when the parent partitioned
>+ * table gets dropped. Hence for partitioning we use AUTO dependency.
>+ * Otherwise, for regular inheritance use NORMAL dependency.
A minor cosmetic correction:
+ Hence for *declarative* partitioning we use AUTO dependency
+ * Otherwise, for regular inheritance *we* use NORMAL dependency.

I have added tests to the
0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please
find attached the v2 patch.



On Tue, Jun 13, 2017 at 10:25 AM, Amit Langote <
langote_amit...@lab.ntt.co.jp> wrote:

> Hi Ashutosh,
>
> On 2017/06/12 20:56, Ashutosh Bapat wrote:
> > Hi,
> > If we detach a partition and drop the corresponding partitioned table,
> > it drops the once-partition now-standalone table as well.
>
> Thanks for the report.  Looks like 8b4d582d279d78 missed the bit about
> drop_parent_dependency() that you describe in your analysis below.
>
> > The reason for this is as follows
> > An AUTO dependency is recorded between a partitioned table and
> partition. In
> > case of inheritance we record a NORMAL dependency between parent
> > and child. While
> > detaching a partition, we call RemoveInheritance(), which should
> have taken
> > care of removing this dependency. But it removes the dependencies
> which are
> > marked as NORMAL. When we changed the dependency for partitioned
> case from
> > NORMAL to AUTO by updating StoreCatalogInheritance1(), this function
> was not
> > updated. This caused the dependency to linger behind even after
> > detaching the
> > partition, thus causing the now standalone table (which was once a
> > partition)
> > to be dropped when the parent partitioned table is dropped. This
> patch fixes
> > RemoveInheritance() to choose appropriate dependency.
> >
> > Attached patch 0001 to fix this.
>
> Looks good to me.  Perhaps, the example in your email could be added as a
> test case.
>
> > I see a similar issue-in-baking in case we change the type of
> > dependency recorded between a table and the composite type associated
> > with using CREATE TABLE ... OF command. 0002 patch addresses the
> > potential issue by using a macro while creating and dropping the
> > dependency in corresponding functions. There might be more such
> > places, but I haven't searched for those.
>
> Might be a good idea too.
>
> Adding this to the open items list.
>
> Thanks,
> Amit
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b61fda9..9aef67b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -283,6 +283,14 @@ struct DropRelationCallbackState
 #define		ATT_COMPOSITE_TYPE		0x0010
 #define		ATT_FOREIGN_TABLE		0x0020
 
+/*
+ * Partition tables are expected to be dropped when the parent partitioned
+ * table gets dropped. Hence for partitioning we use AUTO dependency.
+ * Otherwise, for regular inheritance use NORMAL dependency.
+ */
+#define child_dependency_type(child_is_partition)	\
+	((child_is_partition) ? DEPENDENCY_AUTO : DEPENDENCY_NORMAL)
+
 static void truncate_check_rel(Relation rel);
 static List *MergeAttributes(List *schema, List *supers, char relpersistence,
 bool is_partition, List **supOids, List **supconstr,
@@ -439,7 +447,8 @@ static void ATExecEnableDisableRule(Relation rel, char *rulename,
 static void ATPrepAddInherit(Relation child_rel);
 static ObjectAddress ATExecAddInherit(Relation child_rel, RangeVar *parent, LOCKMODE lockmode);
 static ObjectAddress ATExecDropInherit(Relation rel, RangeVar *parent, LOCKMODE lockmode);
-static void drop_parent_dependency(Oid relid, Oid refclassid, Oid refobjid);
+static void drop_parent_dependency(Oid relid, Oid refclassid, Oid refobjid,
+	   DependencyType deptype);
 static ObjectAddress ATExecAddOf(Relation rel, const TypeName *ofTypename, LOCKMODE lockmode);
 static void 

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

[Amit Langote]

Hi Ashutosh,

On 2017/06/12 20:56, Ashutosh Bapat wrote:
> Hi,
> If we detach a partition and drop the corresponding partitioned table,
> it drops the once-partition now-standalone table as well.

Thanks for the report.  Looks like 8b4d582d279d78 missed the bit about
drop_parent_dependency() that you describe in your analysis below.

> The reason for this is as follows
> An AUTO dependency is recorded between a partitioned table and partition. 
> In
> case of inheritance we record a NORMAL dependency between parent
> and child. While
> detaching a partition, we call RemoveInheritance(), which should have 
> taken
> care of removing this dependency. But it removes the dependencies which 
> are
> marked as NORMAL. When we changed the dependency for partitioned case from
> NORMAL to AUTO by updating StoreCatalogInheritance1(), this function was 
> not
> updated. This caused the dependency to linger behind even after
> detaching the
> partition, thus causing the now standalone table (which was once a
> partition)
> to be dropped when the parent partitioned table is dropped. This patch 
> fixes
> RemoveInheritance() to choose appropriate dependency.
> 
> Attached patch 0001 to fix this.

Looks good to me.  Perhaps, the example in your email could be added as a
test case.

> I see a similar issue-in-baking in case we change the type of
> dependency recorded between a table and the composite type associated
> with using CREATE TABLE ... OF command. 0002 patch addresses the
> potential issue by using a macro while creating and dropping the
> dependency in corresponding functions. There might be more such
> places, but I haven't searched for those.

Might be a good idea too.

Adding this to the open items list.

Thanks,
Amit



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Dropping partitioned table drops a previously detached partition

[Ashutosh Bapat]

Hi,
If we detach a partition and drop the corresponding partitioned table,
it drops the once-partition now-standalone table as well.

create table prt1 (a int, b int) partition by range(a);
create table prt1_p1 partition of prt1 for values from (0) to (100);
select oid, relname, relpartbound, relispartition, relkind from
pg_class where relname like 'prt%';
  oid  | relname
---+-
 16453 | prt1
 16456 | prt1_p1
(2 rows)

select * from pg_depend where refobjid = 'prt1'::regclass and objid =
'prt1_p1'::regclass;
 classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
-+---+--++--+-+-
1259 | 16456 |0 |   1259 |16453 |   0 | a
(1 row)

alter table prt1 detach partition prt1_p1;

-- so the dependency exists even after detach
select * from pg_depend where refobjid = 'prt1'::regclass and objid =
'prt1_p1'::regclass;
 classid | objid | objsubid | refclassid | refobjid | refobjsubid | deptype
-+---+--++--+-+-
1259 | 16456 |0 |   1259 |16453 |   0 | a
(1 row)

drop table prt1;

-- Oops, we deleted the once-partition now-standalone table as well
select oid, relname, relpartbound, relispartition, relkind from
pg_class where relname like 'prt%';
 oid | relname | relpartbound | relispartition | relkind
-+-+--++-
(0 rows)

The reason for this is as follows
An AUTO dependency is recorded between a partitioned table and partition. In
case of inheritance we record a NORMAL dependency between parent
and child. While
detaching a partition, we call RemoveInheritance(), which should have taken
care of removing this dependency. But it removes the dependencies which are
marked as NORMAL. When we changed the dependency for partitioned case from
NORMAL to AUTO by updating StoreCatalogInheritance1(), this function was not
updated. This caused the dependency to linger behind even after
detaching the
partition, thus causing the now standalone table (which was once a
partition)
to be dropped when the parent partitioned table is dropped. This patch fixes
RemoveInheritance() to choose appropriate dependency.

Attached patch 0001 to fix this.

I see a similar issue-in-baking in case we change the type of
dependency recorded between a table and the composite type associated
with using CREATE TABLE ... OF command. 0002 patch addresses the
potential issue by using a macro while creating and dropping the
dependency in corresponding functions. There might be more such
places, but I haven't searched for those.

-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
From 97a647c487761782340d7ebca82b6d3200d79142 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat 
Date: Mon, 12 Jun 2017 16:32:53 +0530
Subject: [PATCH 1/2] Dependency between partitioned table and partition.

An AUTO dependency is recorded between a partitioned table and partition. In
case of inheritance we record a NORMAL dependency between parent and child. While
detaching a partition, we call RemoveInheritance(), which should have taken
care of removing this dependency. But it removes the dependencies which are
marked as NORMAL. When we changed the dependency for partitioned case from
NORMAL to AUTO by updating StoreCatalogInheritance1(), this function was not
updated. This caused the dependency to linger behind even after detaching the
partition, thus causing the now standalone table (which was once a partition)
to be dropped when the parent partitioned table is dropped. This patch fixes
RemoveInheritance() to choose appropriate dependency.

Ashutosh Bapat.
---
 src/backend/commands/tablecmds.c |   35 +--
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b61fda9..9aef67b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -283,6 +283,14 @@ struct DropRelationCallbackState
 #define		ATT_COMPOSITE_TYPE		0x0010
 #define		ATT_FOREIGN_TABLE		0x0020
 
+/*
+ * Partition tables are expected to be dropped when the parent partitioned
+ * table gets dropped. Hence for partitioning we use AUTO dependency.
+ * Otherwise, for regular inheritance use NORMAL dependency.
+ */
+#define child_dependency_type(child_is_partition)	\
+	((child_is_partition) ? DEPENDENCY_AUTO : DEPENDENCY_NORMAL)
+
 static void truncate_check_rel(Relation rel);
 static List *MergeAttributes(List *schema, List *supers, char relpersistence,
 bool is_partition, List **supOids, List **supconstr,
@@ -439,7 +447,8 @@ static void ATExecEnableDisableRule(Relation rel, char *rulename,
 static void ATPrepAddInherit(Relation child_rel);
 static ObjectAddress ATExecAddInherit(Relation child_rel, RangeVar *parent, LOCKMODE