Re: [HACKERS] Declarative partitioning - another take

2016-12-07 Thread Amit Langote
Hi Erik, On Thu, Dec 8, 2016 at 1:19 AM, Erik Rijkers wrote: > On 2016-12-07 12:42, Amit Langote wrote: > >> 0001-Catalog-and-DDL-for-partitioned-tables-20.patch >> 0002-psql-and-pg_dump-support-for-partitioned-tables-20.patch >> 0003-Catalog-and-DDL-for-partitions-20.patch >> 0004-psql-and-pg_du

Re: [HACKERS] Declarative partitioning - another take

2016-12-07 Thread Erik Rijkers
On 2016-12-07 12:42, Amit Langote wrote: 0001-Catalog-and-DDL-for-partitioned-tables-20.patch 0002-psql-and-pg_dump-support-for-partitioned-tables-20.patch 0003-Catalog-and-DDL-for-partitions-20.patch 0004-psql-and-pg_dump-support-for-partitions-20.patch 0005-Teach-a-few-places-to-use-partition-

Re: [HACKERS] Declarative partitioning - another take

2016-12-07 Thread Robert Haas
On Wed, Dec 7, 2016 at 6:42 AM, Amit Langote wrote: > On 2016/12/07 13:38, Robert Haas wrote: >> On Wed, Nov 30, 2016 at 10:56 AM, Amit Langote >> wrote: >>> The latest patch I posted earlier today has this implementation. >> >> I decided to try out these patches today with #define >> CLOBBER_CA

Re: [HACKERS] Declarative partitioning - another take

2016-12-06 Thread Robert Haas
On Wed, Nov 30, 2016 at 10:56 AM, Amit Langote wrote: > The latest patch I posted earlier today has this implementation. I decided to try out these patches today with #define CLOBBER_CACHE_ALWAYS 1 in pg_config_manual.h, which found a couple of problems: 1. RelationClearRelation() wasn't preserv

Re: [HACKERS] Declarative partitioning - another take

2016-11-30 Thread Amit Langote
On Thu, Dec 1, 2016 at 12:48 AM, Robert Haas wrote: > On Fri, Nov 25, 2016 at 5:49 AM, Amit Langote wrote: >> On 2016/11/25 11:44, Robert Haas wrote: >>> On Thu, Nov 24, 2016 at 6:13 AM, Amit Langote wrote: Also, it does nothing to help the undesirable situation that one can insert a row

Re: [HACKERS] Declarative partitioning - another take

2016-11-30 Thread Robert Haas
On Tue, Nov 29, 2016 at 6:24 AM, Amit Langote wrote: > # All times in seconds (on my modestly-powerful development VM) > # > # nrows = 10,000,000 generated using: > # > # INSERT INTO $tab > # SELECT '$last'::date - ((s.id % $maxsecs + 1)::bigint || 's')::interval, > # (random() * 5000)::int

Re: [HACKERS] Declarative partitioning - another take

2016-11-30 Thread Robert Haas
On Fri, Nov 25, 2016 at 5:49 AM, Amit Langote wrote: > On 2016/11/25 11:44, Robert Haas wrote: >> On Thu, Nov 24, 2016 at 6:13 AM, Amit Langote wrote: >>> Also, it does nothing to help the undesirable situation that one can >>> insert a row with a null partition key (expression) into any of the ra

Re: [HACKERS] Declarative partitioning - another take

2016-11-29 Thread Amit Langote
On 2016/11/17 20:27, Amit Langote wrote: > On 2016/11/16 4:21, Robert Haas wrote: >> Have you done any performance testing on the tuple routing code? >> Suppose we insert a million (or 10 million) tuples into an >> unpartitioned table, a table with 10 partitions, a table with 100 >> partitions, a t

Re: [HACKERS] Declarative partitioning - another take

2016-11-28 Thread Ashutosh Bapat
Here are some comments on 0003 patch. 1. In ALTER TABLE documentation we should refer to CREATE TABLE documentation for partition_bound_spec syntax, similar ADD table_constraint note. 2. I think, "of the target table" is missing after all the ... constraints + match. Also, it must have all t

Re: [HACKERS] Declarative partitioning - another take

2016-11-25 Thread Amit Langote
On 2016/11/25 11:44, Robert Haas wrote: > On Thu, Nov 24, 2016 at 6:13 AM, Amit Langote wrote: >> Also, it does nothing to help the undesirable situation that one can >> insert a row with a null partition key (expression) into any of the range >> partitions if targeted directly, because of how Exec

Re: [HACKERS] Declarative partitioning - another take

2016-11-25 Thread Amit Langote
On 2016/11/25 13:51, Ashutosh Bapat wrote: >> >> I assume you meant "...right after the column name"? >> >> I will modify the grammar to allow that way then, so that the following >> will work: >> >> create table p1 partition of p ( >> a primary key >> ) for values in (1); >> > > That seems t

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Ashutosh Bapat
> > I assume you meant "...right after the column name"? > > I will modify the grammar to allow that way then, so that the following > will work: > > create table p1 partition of p ( > a primary key > ) for values in (1); > That seems to be non-intuitive as well. The way it's written it looks

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Alvaro Herrera
Amit Langote wrote: > On 2016/11/25 4:36, Alvaro Herrera wrote: > > I think CREATE TABLE OF is pretty much a corner case. I agree that > > allowing the constraint right after the constraint name is more > > intuitive. > > I assume you meant "...right after the column name"? Eh, right. -- Álv

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Robert Haas
On Thu, Nov 24, 2016 at 6:13 AM, Amit Langote wrote: > On 2016/11/23 4:50, Robert Haas wrote: >> On Tue, Nov 22, 2016 at 4:15 AM, Amit Langote >> wrote: The easiest thing to do might be to just enforce that all of the partition key columns have to be not-null when the range-partitioned

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Amit Langote
On 2016/11/25 4:36, Alvaro Herrera wrote: > Amit Langote wrote: >> On 2016/11/24 15:10, Ashutosh Bapat wrote: >>> On Thu, Nov 24, 2016 at 11:34 AM, Amit Langote wrote: > You have to specify column constraints using the keywords WITH OPTIONS, like below: create table p1 partitio

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Alvaro Herrera
Amit Langote wrote: > On 2016/11/24 15:10, Ashutosh Bapat wrote: > > On Thu, Nov 24, 2016 at 11:34 AM, Amit Langote wrote: > >> You have to specify column constraints using the keywords WITH OPTIONS, > >> like below: > >> > >> create table p1 partition of p ( > >> a with options primary key >

Re: [HACKERS] Declarative partitioning - another take

2016-11-24 Thread Amit Langote
On 2016/11/23 4:50, Robert Haas wrote: > On Tue, Nov 22, 2016 at 4:15 AM, Amit Langote > wrote: >>> The easiest thing to do might be to just enforce that all of the >>> partition key columns have to be not-null when the range-partitioned >>> table is defined, and reject any attempt to DROP NOT NUL

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Ashutosh Bapat
On Thu, Nov 24, 2016 at 12:02 PM, Amit Langote wrote: > On 2016/11/24 15:10, Ashutosh Bapat wrote: >> On Thu, Nov 24, 2016 at 11:34 AM, Amit Langote wrote: >>> On 2016/11/24 14:35, Ashutosh Bapat wrote: IIUC, it should allow "create table t1_p1 partition of t1 (a primary key) ...", (a pr

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Amit Langote
On 2016/11/24 15:10, Ashutosh Bapat wrote: > On Thu, Nov 24, 2016 at 11:34 AM, Amit Langote wrote: >> On 2016/11/24 14:35, Ashutosh Bapat wrote: >>> IIUC, it should allow "create table t1_p1 partition of t1 (a primary >>> key) ...", (a primary key) is nothing but "column_name >>> column_constraint"

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Ashutosh Bapat
On Thu, Nov 24, 2016 at 11:34 AM, Amit Langote wrote: > > Hi Ashutosh, > > On 2016/11/24 14:35, Ashutosh Bapat wrote: >> I am trying to create a partitioned table with primary keys on the >> partitions. Here's the corresponding syntax as per documentation >> CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Amit Langote
Hi Ashutosh, On 2016/11/24 14:35, Ashutosh Bapat wrote: > I am trying to create a partitioned table with primary keys on the > partitions. Here's the corresponding syntax as per documentation > CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ > IF NOT EXISTS ] table_name >

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Ashutosh Bapat
Unsurprisingly ALTER TABLE worked alter table t1_p1 add primary key(a); ALTER TABLE postgres=# \d+ t1_p1 Table "public.t1_p1" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description +-+---+--+--

Re: [HACKERS] Declarative partitioning - another take

2016-11-23 Thread Ashutosh Bapat
I am trying to create a partitioned table with primary keys on the partitions. Here's the corresponding syntax as per documentation CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] table_name PARTITION OF parent_table [ ( { column_name WITH OPTIONS [ colum

Re: [HACKERS] Declarative partitioning - another take

2016-11-22 Thread Robert Haas
On Tue, Nov 22, 2016 at 4:15 AM, Amit Langote wrote: >> The easiest thing to do might be to just enforce that all of the >> partition key columns have to be not-null when the range-partitioned >> table is defined, and reject any attempt to DROP NOT NULL on them >> later. That's probably better th

Re: [HACKERS] Declarative partitioning - another take

2016-11-22 Thread Rushabh Lathia
Hi Amit, I was just reading through your patches and here are some quick review comments for 0001-Catalog-and-DDL-for-partitioned-tables-17.patch. Review comments for 0001-Catalog-and-DDL-for-partitioned-tables-17.patch: 1) @@ -1102,9 +1104,10 @@ heap_create_with_catalog(const char *relname,

Re: [HACKERS] Declarative partitioning - another take

2016-11-18 Thread Robert Haas
On Fri, Nov 18, 2016 at 5:59 AM, Amit Langote wrote: > Oh but wait, that means I can insert rows with NULLs in the range > partition key if I choose to insert it directly into the partition, > whereas I have been thinking all this while that there could never be > NULLs in the partition key of a r

Re: [HACKERS] Declarative partitioning - another take

2016-11-18 Thread Robert Haas
On Thu, Nov 17, 2016 at 8:18 PM, Amit Langote wrote: > The reason NULLs in an input row are caught and rejected (with the current > message) before control reaches range_partition_for_tuple() is because > it's not clear to me whether the range bound comparison logic in > partition_rbound_datum_cmp

Re: [HACKERS] Declarative partitioning - another take

2016-11-17 Thread Amit Langote
On 2016/11/18 1:43, Robert Haas wrote: > On Thu, Nov 17, 2016 at 6:27 AM, Amit Langote wrote: >> OK, I will share the performance results soon. > > Thanks. > >>> Also, in 0006: >>> >>> - I doubt that PartitionTreeNodeData's header comment will survive >>> contact with pgindent. >> >> Fixed by add

Re: [HACKERS] Declarative partitioning - another take

2016-11-17 Thread Robert Haas
On Thu, Nov 17, 2016 at 6:27 AM, Amit Langote wrote: > Meanwhile, here are updated patch that address most of the following comments. OK, I have re-reviewed 0005 and it looks basically fine to me, modulo a few minor nitpicks. "This is called, *iff*" shouldn't have a comma there, and I think the e

Re: [HACKERS] Declarative partitioning - another take

2016-11-17 Thread Robert Haas
On Thu, Nov 17, 2016 at 6:27 AM, Amit Langote wrote: > OK, I will share the performance results soon. Thanks. >> Also, in 0006: >> >> - I doubt that PartitionTreeNodeData's header comment will survive >> contact with pgindent. > > Fixed by adding "/* " at the top of the comment. OK. Hopefu

Re: [HACKERS] Declarative partitioning - another take

2016-11-15 Thread Robert Haas
On Tue, Nov 15, 2016 at 5:30 AM, Amit Langote wrote: > On 2016/11/11 19:30, Amit Langote wrote: >> >> Attached updated patches. > > Here is the latest version of the patches with some fixes along with those > mentioned below (mostly in 0003): > > - Fixed the logic to skip the attach partition vali

Re: [HACKERS] Declarative partitioning - another take

2016-11-15 Thread Amit Langote
On 2016/11/11 20:49, Ashutosh Bapat wrote: > I have not looked at the latest set of patches, but in the version > that I have we create one composite type for every partition. This > means that if there are thousand partitions, there will be thousand > identical entries in pg_type. Since all the pa

Re: [HACKERS] Declarative partitioning - another take

2016-11-13 Thread Amit Langote
On 2016/11/04 0:49, Robert Haas wrote: > On Thu, Nov 3, 2016 at 7:46 AM, wrote: >> El 2016-10-28 07:53, Amit Langote escribió: >>> @@ -6267,6 +6416,12 @@ ATAddForeignKeyConstraint(AlteredTableInfo *tab, >>> Relation rel, >>> * Validity checks (permission checks wait till we have the colu

Re: [HACKERS] Declarative partitioning - another take

2016-11-13 Thread Amit Langote
I forgot to quote your comments in the email I sent on Friday [1], with new patches that do take care of the following comments. On 2016/11/11 4:04, Robert Haas wrote: > On Thu, Nov 10, 2016 at 7:40 AM, Amit Langote >> >> OK, "partition key" and "partitioning method" it is then. Source code >> c

Re: [HACKERS] Declarative partitioning - another take

2016-11-11 Thread Ashutosh Bapat
I have not looked at the latest set of patches, but in the version that I have we create one composite type for every partition. This means that if there are thousand partitions, there will be thousand identical entries in pg_type. Since all the partitions share the same definition (by syntax), it

Re: [HACKERS] Declarative partitioning - another take

2016-11-10 Thread Robert Haas
On Wed, Nov 9, 2016 at 9:58 PM, Amit Langote wrote: >> With all patches applied, "make check" fails with a bunch of diffs >> that look like this: >> >> Check constraints: >> - "pt1chk2" CHECK (c2 <> ''::text) >> "pt1chk3" CHECK (c2 <> ''::text) > > Hm, I can't seem to reproduce this on

Re: [HACKERS] Declarative partitioning - another take

2016-11-10 Thread Robert Haas
On Thu, Nov 10, 2016 at 7:40 AM, Amit Langote wrote: >> I think "partitioning key" is a bit awkward and actually prefer >> "partiton key". But "partition method" sounds funny so I would go >> with "partitioning method". > > OK, "partition key" and "partitioning method" it is then. Source code >

Re: [HACKERS] Declarative partitioning - another take

2016-11-09 Thread Amit Langote
On 2016/11/10 2:00, Robert Haas wrote: > In this latest patch set: > > src/backend/parser/parse_utilcmd.c:3194: indent with spaces. > +*rdatum; This one I will fix. > > With all patches applied, "make check" fails with a bunch of diffs > that look like this:

Re: [HACKERS] Declarative partitioning - another take

2016-11-09 Thread Robert Haas
On Wed, Nov 9, 2016 at 12:00 PM, Robert Haas wrote: > In this latest patch set: > > src/backend/parser/parse_utilcmd.c:3194: indent with spaces. > +*rdatum; > > With all patches applied, "make check" fails with a bunch of diffs > that look like this: > > Check

Re: [HACKERS] Declarative partitioning - another take

2016-11-09 Thread Robert Haas
In this latest patch set: src/backend/parser/parse_utilcmd.c:3194: indent with spaces. +*rdatum; With all patches applied, "make check" fails with a bunch of diffs that look like this: Check constraints: - "pt1chk2" CHECK (c2 <> ''::text) "pt1chk3"

Re: [HACKERS] Declarative partitioning - another take

2016-11-07 Thread Amit Langote
Hi Jaime, On 2016/11/08 2:15, Jaime Casanova wrote: > On 28 October 2016 at 02:53, Amit Langote > wrote: > > I started to review the functionality of this patch, so i applied all > 9 patches. After that i found this warning, which i guess is because > it needs a cast. Thanks a ton for reviewi

Re: [HACKERS] Declarative partitioning - another take

2016-11-07 Thread Amit Langote
Hi Jaime, On 2016/11/08 2:24, Jaime Casanova wrote: > On 7 November 2016 at 12:15, Jaime Casanova > wrote: >> On 28 October 2016 at 02:53, Amit Langote >> wrote: >>> >>> Please find attached the latest version of the patches >> >> Hi, >> >> I started to review the functionality of this patch,

Re: [HACKERS] Declarative partitioning - another take

2016-11-07 Thread Jaime Casanova
On 7 November 2016 at 12:15, Jaime Casanova wrote: > On 28 October 2016 at 02:53, Amit Langote > wrote: >> >> Please find attached the latest version of the patches > > Hi, > > I started to review the functionality of this patch, so i applied all > 9 patches. After that i found this warning, whi

Re: [HACKERS] Declarative partitioning - another take

2016-11-07 Thread Jaime Casanova
On 28 October 2016 at 02:53, Amit Langote wrote: > > Please find attached the latest version of the patches Hi, I started to review the functionality of this patch, so i applied all 9 patches. After that i found this warning, which i guess is because it needs a cast. After that, i tried a case

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread Robert Haas
Apologies if I've made some of these comments before and/or missed comments you've made on these topics. The size of this patch set is so large that it's hard to keep track of everything. Re-reviewing 0001: + indicate which table columns are used as partition key. For example, s/are used

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread Robert Haas
On Wed, Nov 2, 2016 at 3:41 AM, Amit Langote wrote: > As for which parts of the system need to know about these implicit > partition constraints to *enforce* them for data integrity, we could say > that it's really just one site - ExecConstraints() called from > ExecInsert()/ExecUpdate(). Well, t

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread Robert Haas
On Wed, Nov 2, 2016 at 6:47 AM, Amit Langote wrote: > OK, I changed things so that DROP TABLE a_partition no longer complains > about requiring to detach first. Much like how index_drop() locks the > parent table ('parent' in a different sense, of course) and later > invalidates its relcache, hea

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread Robert Haas
On Tue, Nov 1, 2016 at 2:36 PM, Corey Huinker wrote: > On Tue, Nov 1, 2016 at 2:24 PM, Robert Haas wrote: >> Well, I'm not sure we've exactly reached consensus here, and you're >> making me feel like I just kicked a puppy. > > It was hyperbole. I hope you found it as funny to read as I did to wri

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread Robert Haas
On Thu, Nov 3, 2016 at 7:46 AM, wrote: > El 2016-10-28 07:53, Amit Langote escribió: >> @@ -6267,6 +6416,12 @@ ATAddForeignKeyConstraint(AlteredTableInfo *tab, >> Relation rel, >> * Validity checks (permission checks wait till we have the column >> * numbers) >> */ >> +

Re: [HACKERS] Declarative partitioning - another take

2016-11-03 Thread alvherre
El 2016-10-28 07:53, Amit Langote escribió: @@ -6267,6 +6416,12 @@ ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel, * Validity checks (permission checks wait till we have the column * numbers) */ + if (pkrel->rd_rel->relkind == RELKIND_PARTITIONED_

Re: [HACKERS] Declarative partitioning - another take

2016-11-02 Thread Amit Langote
On 2016/11/02 2:44, Robert Haas wrote: > On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote > wrote: >>> Insisting that you can't drop a child without detaching it first seems >>> wrong to me. If I already made this comment and you responded to it, >>> please point me back to whatever you said. Howev

Re: [HACKERS] Declarative partitioning - another take

2016-11-02 Thread Amit Langote
On 2016/11/02 16:41, Amit Langote wrote: > Having said all that, I am open to switching to the catalogued partition > constraints if the arguments I make above in favor of this patch are not > all that sound. One problem I didn't immediately see a solution for if we go with the catalogued partitio

Re: [HACKERS] Declarative partitioning - another take

2016-11-02 Thread Amit Langote
On 2016/11/02 2:34, Robert Haas wrote: > On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote > wrote: >> [ new patches ] > > Reviewing 0006: Thanks for the review! > This patch seems scary. I sort of assumed from the title -- "Teach a > few places to use partition check quals." -- that this was an o

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Amit Langote
On 2016/11/02 2:53, Robert Haas wrote: > On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote > wrote: >> [ new patches ] > > Reviewing 0005: > > Your proposed commit messages says this: > >> If relation is the target table (UPDATE and DELETE), flattening is >> done regardless (scared to modify inher

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Corey Huinker
On Tue, Nov 1, 2016 at 2:24 PM, Robert Haas wrote: > Well, I'm not sure we've exactly reached consensus here, and you're > making me feel like I just kicked a puppy. > It was hyperbole. I hope you found it as funny to read as I did to write. This is a great feature and I'm not going to make "per

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Tue, Nov 1, 2016 at 2:11 PM, Corey Huinker wrote: > On Tue, Nov 1, 2016 at 2:01 PM, Robert Haas wrote: >> Yeah. That syntax has some big advantages, though. If we define that >> partition as START ('2014-01-01') INCLUSIVE END ('2014-12-31') >> INCLUSIVE, there's no way for the system to tell

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Francisco Olarte
Robert: On Tue, Nov 1, 2016 at 7:09 PM, Robert Haas wrote: > In defense of Corey's position, that's not so easy. First, \0 doesn't > work; our strings can't include null bytes. Second, the minimum legal > character depends on the collation in use. It's not so easy to figure > out what the "nex

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Corey Huinker
> > OTOH I've seen a lot of people bitten by [2014-01-01,2014-12-31] on > TIMESTAMP intervals. > No argument there. > Everybody remembers december has 31 days, but when we have to do > MONTHLY partitions if you use closed intervals someone always miskeys > the number of days, or forgets wheter a

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Francisco Olarte
On Tue, Nov 1, 2016 at 7:01 PM, Robert Haas wrote: > In the end, keywords are not the defining issue here; the issue is > whether all of this complexity around inclusive and exclusive bounds > carries its weight, and whether we want to be committed to that. > > Any other opinions out there? If it

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Corey Huinker
On Tue, Nov 1, 2016 at 2:01 PM, Robert Haas wrote: > Yeah. That syntax has some big advantages, though. If we define that > partition as START ('2014-01-01') INCLUSIVE END ('2014-12-31') > INCLUSIVE, there's no way for the system to tell that the there's no > gap between the that ending bound a

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Tue, Nov 1, 2016 at 2:05 PM, Francisco Olarte wrote: >> /me raises hand. We have tables with a taxonomy in them where the even data >> splits don't fall on single letter boundaries, and often the single string >> values have more rows than entire letters. In those situations, being able >> to

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Francisco Olarte
On Tue, Nov 1, 2016 at 6:49 PM, Corey Huinker wrote: > > On Tue, Nov 1, 2016 at 12:57 PM, Robert Haas wrote: >> For strings and numeric types that are not integers, there is in >> theory a loss of power. If you want a partition that allows very >> value starting with 'a' plus the string 'b' but

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Tue, Nov 1, 2016 at 1:49 PM, Corey Huinker wrote: > Exactly. This is especially true for date ranges. There's a lot of cognitive > dissonance in defining the "2014" partition as < '2015-01-01', as was the > case in Oracle waterfall-style partitioning. That was my reasoning for > pushing for ran

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote wrote: > [ new patches ] Reviewing 0005: Your proposed commit messages says this: > If relation is the target table (UPDATE and DELETE), flattening is > done regardless (scared to modify inheritance_planner() yet). In the immortal words of Frank H

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Corey Huinker
On Tue, Nov 1, 2016 at 12:57 PM, Robert Haas wrote: > For strings and numeric types that are not integers, there is in > theory a loss of power. If you want a partition that allows very > value starting with 'a' plus the string 'b' but not anything after > that, you are out of luck. START ('a')

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote wrote: >> Insisting that you can't drop a child without detaching it first seems >> wrong to me. If I already made this comment and you responded to it, >> please point me back to whatever you said. However, my feeling is >> this is flat wrong and ab

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote wrote: > [ new patches ] Reviewing 0006: This patch seems scary. I sort of assumed from the title -- "Teach a few places to use partition check quals." -- that this was an optional thing, some kind of optimization from which we could reap further ad

Re: [HACKERS] Declarative partitioning - another take

2016-11-01 Thread Robert Haas
On Fri, Oct 28, 2016 at 3:53 AM, Amit Langote wrote: >> 4. I'm somewhat wondering if we ought to just legislate that the lower >> bound is always inclusive and the upper bound is always exclusive. >> The decision to support both inclusive and exclusive partition bounds >> is responsible for an eno

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Langote
On 2016/10/27 3:13, Robert Haas wrote: > On Tue, Oct 25, 2016 at 9:06 PM, Amit Langote > wrote: >> I will include these changes in the next version of patches I will post >> soon in reply to [1]. >> [1] >> https://www.postgresql.org/message-id/CA%2BTgmoYJcUTcN7vVgg54GHtffH11JJWYZnfF4KiRxjV-iaACQg%

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Robert Haas
On Tue, Oct 25, 2016 at 10:00 PM, Amit Langote wrote: > I am not (or no longer) sure how that argument affects INSERT on > partitioned tables with tuple-routing though. Are partitions at all > levels *implicitly specified to be affected* when we say INSERT INTO > root_partitioned_table? I'd say

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Robert Haas
On Tue, Oct 25, 2016 at 9:06 PM, Amit Langote wrote: > I will include these changes in the next version of patches I will post > soon in reply to [1]. > [1] > https://www.postgresql.org/message-id/CA%2BTgmoYJcUTcN7vVgg54GHtffH11JJWYZnfF4KiRxjV-iaACQg%40mail.gmail.com How soon? Tempus fugit, and

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Robert Haas
On Tue, Oct 25, 2016 at 2:58 AM, Amit Kapila wrote: > a. Policy on table_name No, because queries against the parent will apply the policy to children. See today's commit 162477a63d3c0fd1c31197717140a88077a8d0aa. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQ

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Robert Haas
On Wed, Oct 26, 2016 at 6:12 AM, Amit Kapila wrote: > On Wed, Oct 26, 2016 at 3:04 PM, Amit Langote > wrote: >> On 2016/10/26 17:57, Amit Kapila wrote: >>> @@ -123,6 +123,9 @@ typedef struct RelationData >>> { >>> .. >>> MemoryContext rd_partkeycxt; /* private memory cxt for the below */ >>>

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Kapila
On Wed, Oct 26, 2016 at 3:04 PM, Amit Langote wrote: > On 2016/10/26 17:57, Amit Kapila wrote: >> @@ -123,6 +123,9 @@ typedef struct RelationData >> { >> .. >> MemoryContext rd_partkeycxt; /* private memory cxt for the below */ >> struct PartitionKeyData *rd_partkey; /* partition key, or NULL

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Langote
On 2016/10/26 17:57, Amit Kapila wrote: > @@ -123,6 +123,9 @@ typedef struct RelationData > { > .. > MemoryContext rd_partkeycxt; /* private memory cxt for the below */ > struct PartitionKeyData *rd_partkey; /* partition key, or NULL */ > + MemoryContext rd_pdcxt; /* private context for partdes

Re: [HACKERS] Declarative partitioning - another take

2016-10-26 Thread Amit Kapila
On Wed, Oct 5, 2016 at 7:20 AM, Robert Haas wrote: > On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote > wrote: >> [ latest patch set ] > > Reviewing 0003: > > > 5. I wonder how well this handles multi-column partition keys. You've > just got one Datum flag and one is-finite flag per partition, but I

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Langote
On 2016/10/26 12:09, Amit Kapila wrote: > On Wed, Oct 26, 2016 at 8:27 AM, Amit Langote > wrote: >> On 2016/10/26 11:41, Amit Kapila wrote: >>> On Wed, Oct 26, 2016 at 6:36 AM, Amit Langote >>> wrote: Sure, CopyTo() can be be taught to scan leaf partitions when a partitioned table

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
On Wed, Oct 26, 2016 at 8:27 AM, Amit Langote wrote: > On 2016/10/26 11:41, Amit Kapila wrote: >> On Wed, Oct 26, 2016 at 6:36 AM, Amit Langote >> wrote: 1. @@ -1775,6 +1775,12 @@ BeginCopyTo(ParseState *pstate, { .. + else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Langote
On 2016/10/26 11:41, Amit Kapila wrote: > On Wed, Oct 26, 2016 at 6:36 AM, Amit Langote > wrote: >>> 1. >>> @@ -1775,6 +1775,12 @@ BeginCopyTo(ParseState *pstate, >>> { >>> .. >>> + else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) >>> + ereport(ERROR, >>> + (errcode(ERRCODE_WRONG_OBJECT

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
On Wed, Oct 26, 2016 at 6:36 AM, Amit Langote wrote: >> 1. >> @@ -1775,6 +1775,12 @@ BeginCopyTo(ParseState *pstate, >> { >> .. >> + else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE) >> + ereport(ERROR, >> + (errcode(ERRCODE_WRONG_OBJECT_TYPE), >> + errmsg("cannot copy from partitioned t

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Langote
On 2016/10/25 15:58, Amit Kapila wrote: > On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote > wrote: >> On 2016/10/05 2:12, Robert Haas wrote: >>> Hmm, do we ever fire triggers on the parent for operations on a child >>> table? Note this thread, which seems possibly relevant: >>> >>> https://www.post

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Langote
Thanks for the review! On 2016/10/25 20:32, Amit Kapila wrote: > On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote > wrote: >> On 2016/10/05 2:12, Robert Haas wrote: > >> Attached revised patches. > > Few assorted review comments for 0001-Catalog*: > > > 1. > @@ -1775,6 +1775,12 @@ BeginCopyTo(P

Re: [HACKERS] Declarative partitioning - another take

2016-10-25 Thread Amit Kapila
On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wrote: > On 2016/10/05 2:12, Robert Haas wrote: > Attached revised patches. Few assorted review comments for 0001-Catalog*: 1. @@ -1775,6 +1775,12 @@ BeginCopyTo(ParseState *pstate, { .. + else if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)

Re: [HACKERS] Declarative partitioning - another take

2016-10-24 Thread Amit Kapila
On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wrote: > On 2016/10/05 2:12, Robert Haas wrote: >> On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote >> wrote: Even if we leave the empty relfilenode around for now -- in the long run I think it should die -- I think we should prohibit the creati

Re: [HACKERS] Declarative partitioning - another take

2016-10-07 Thread Amit Langote
On 2016/10/07 18:27, Ashutosh Bapat wrote: > It's allowed to specify an non-default opclass in partition by clause, > but I do not see any testcase testing the same. Can you please add > one. OK, I will add some tests related to that. Thanks, Amit -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Declarative partitioning - another take

2016-10-07 Thread Ashutosh Bapat
It's allowed to specify an non-default opclass in partition by clause, but I do not see any testcase testing the same. Can you please add one. On Fri, Oct 7, 2016 at 2:50 PM, Amit Langote wrote: > On 2016/10/07 17:33, Rajkumar Raghuwanshi wrote: >> On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wr

Re: [HACKERS] Declarative partitioning - another take

2016-10-07 Thread Rajkumar Raghuwanshi
On Thu, Oct 6, 2016 at 12:44 PM, Amit Langote wrote: > > Attached revised patches. Also, includes a fix for an issue reported by > Rajkumar Raghuwanshi [1] which turned out to be a bug in one of the later > patches. I will now move on to addressing the comments on patch 0003. > > Thanks a lot f

Re: [HACKERS] Declarative partitioning - another take

2016-10-05 Thread Amit Langote
Hi, On 2016/10/05 16:57, Rajkumar Raghuwanshi wrote: > I observed, when creating foreign table with range partition, data is not > inserting into specified partition range. below are steps to reproduce. > > [ ... ] > > postgres=# INSERT INTO test_range (a) values (5),(25),(15); > INSERT 0 3 >

Re: [HACKERS] Declarative partitioning - another take

2016-10-05 Thread Rajkumar Raghuwanshi
On Tue, Oct 4, 2016 at 1:32 PM, Amit Langote wrote: Attached updated patches. > > Thanks, > Amit > Hi, I observed, when creating foreign table with range partition, data is not inserting into specified partition range. below are steps to reproduce. CREATE EXTENSION postgres_fdw; CREATE SERVER

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Petr Jelinek
On 05/10/16 03:50, Robert Haas wrote: > On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote > wrote: >> [ latest patch set ] > > Reviewing 0003: > > + with matching types and no more. Also, it must have all the matching > + constraints as the target table. That includes both NOT > NULL > +

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote wrote: > [ latest patch set ] Reviewing 0003: + This form attaches an existing table (partitioned or otherwise) as (which might itself be partitioned) + partition of the target table. Partition bound specification must + correspond w

Re: [HACKERS] Declarative partitioning - another take

2016-10-04 Thread Robert Haas
On Tue, Oct 4, 2016 at 4:02 AM, Amit Langote wrote: >> Even if we leave the empty relfilenode around for now -- in the long >> run I think it should die -- I think we should prohibit the creation >> of subsidiary object on the parent which is only sensible if it has >> rows - e.g. indexes. It mak

Re: [HACKERS] Declarative partitioning - another take

2016-10-02 Thread Amit Langote
On 2016/10/03 13:26, Michael Paquier wrote: > On Fri, Sep 30, 2016 at 9:10 AM, Robert Haas wrote: >> On Thu, Sep 29, 2016 at 8:09 AM, Amit Langote >> wrote: >>> I removed DEPENDENCY_IGNORE. Does the following look good or am I still >>> missing something? >> >> You missed your commit message, bu

Re: [HACKERS] Declarative partitioning - another take

2016-10-02 Thread Michael Paquier
On Fri, Sep 30, 2016 at 9:10 AM, Robert Haas wrote: > On Thu, Sep 29, 2016 at 8:09 AM, Amit Langote > wrote: >> I removed DEPENDENCY_IGNORE. Does the following look good or am I still >> missing something? > > You missed your commit message, but otherwise looks fine. I have moved this patch to

Re: [HACKERS] Declarative partitioning - another take

2016-09-29 Thread Robert Haas
On Thu, Sep 29, 2016 at 8:09 AM, Amit Langote wrote: > I removed DEPENDENCY_IGNORE. Does the following look good or am I still > missing something? You missed your commit message, but otherwise looks fine. >> Also, I think this should be rephrased a bit to be more clear about >> how the partiti

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Amit Langote
On 2016/09/27 18:09, Ashutosh Bapat wrote: >>> I tried to debug the problem somewhat. In set_append_rel_pathlist(), >>> it finds that at least one child has a parameterized path as the >>> cheapest path, so it doesn't create an unparameterized path for append >>> rel. At the same time there is a pa

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Ashutosh Bapat
On Tue, Sep 27, 2016 at 2:46 PM, Amit Langote wrote: > On 2016/09/27 15:44, Ashutosh Bapat wrote: >>> By the way, I fixed one thinko in your patch as follows: >>> >>> -result->oids[i] = oids[mapping[i]]; >>> +result->oids[mapping[i]] = oids[i]; >> >> While I can not spot any proble

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Amit Langote
On 2016/09/27 15:44, Ashutosh Bapat wrote: >> By the way, I fixed one thinko in your patch as follows: >> >> -result->oids[i] = oids[mapping[i]]; >> +result->oids[mapping[i]] = oids[i]; > > While I can not spot any problem with this logic, when I make that > change and run partitio

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Ashutosh Bapat
>> >> Please check if you are able to reproduce these errors in your >> repository. I made sure that I cleaned up all partition-wise join code >> before testing this, but ... . > > Thanks for the test case. I can reproduce the same. > >> I tried to debug the problem somewhat. In set_append_rel_pat

Re: [HACKERS] Declarative partitioning - another take

2016-09-27 Thread Amit Langote
Sorry about the delay in replying. On 2016/09/15 21:58, Ashutosh Bapat wrote: > Hi Amit, > > It looks like there is some problem while creating paramterized paths > for multi-level partitioned tables. Here's a longish testcase > > [ ... ] > > Please check if you are able to reproduce these err

<    1   2   3   4   5   6   >