Re: [HACKERS] [POC] hash partitioning

2017-11-09 Thread amul sul
On Fri, Nov 10, 2017 at 4:41 AM, Robert Haas wrote: > On Wed, Nov 1, 2017 at 6:16 AM, amul sul wrote: >> Fixed in the 0003 patch. > > I have committed this patch set with the attached adjustments. > Thanks a lot for your support & a ton of thanks to all reviewer. Regards, Amul -- Sent via pg

Re: [HACKERS] [POC] hash partitioning

2017-11-09 Thread Robert Haas
On Wed, Nov 1, 2017 at 6:16 AM, amul sul wrote: > Fixed in the 0003 patch. I have committed this patch set with the attached adjustments. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company hash-adjustments.patch Description: Binary data -- Sent via pg

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 1:52 PM, Ashutosh Bapat wrote: > Right now partition keys are immutable but we don't have much code > written with that assumption. All the code usually keeps a lock on the > parent till the time they use the information in the partition key. In > a distant future, which may

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Thu, Nov 2, 2017 at 1:45 PM, amul sul wrote: > Yes, you are correct, column involved in the partitioning are immutable. > > I was just worried about any change in the partition key column that > might change selected hash function. Any such change, even if it were allowed, would have to take A

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Ashutosh Bapat
On Thu, Nov 2, 2017 at 1:35 PM, Robert Haas wrote: > On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: >> Although partition constraints become more simple, there isn't any >> performance >> gain with 0005 patch. Also I am little skeptic about logic in 0005 where we >> copied extended hash functio

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread amul sul
On Thu, Nov 2, 2017 at 1:35 PM, Robert Haas wrote: > On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: >> Although partition constraints become more simple, there isn't any >> performance >> gain with 0005 patch. Also I am little skeptic about logic in 0005 where we >> copied extended hash functio

Re: [HACKERS] [POC] hash partitioning

2017-11-02 Thread Robert Haas
On Wed, Nov 1, 2017 at 3:46 PM, amul sul wrote: > Although partition constraints become more simple, there isn't any performance > gain with 0005 patch. Also I am little skeptic about logic in 0005 where we > copied extended hash function info from the partition key, what if parent is > changed wh

Re: [HACKERS] [POC] hash partitioning

2017-10-30 Thread amul sul
On Tue, Oct 31, 2017 at 9:54 AM, Robert Haas wrote: > On Mon, Oct 30, 2017 at 5:52 PM, amul sul wrote: >> Actually, int4[] is also inappropriate type as we have started using a 64bit >> hash function. We need something int8[] which is not available, so that I >> have used ANYARRAYOID in the atta

Re: [HACKERS] [POC] hash partitioning

2017-10-30 Thread Robert Haas
On Mon, Oct 30, 2017 at 5:52 PM, amul sul wrote: > Actually, int4[] is also inappropriate type as we have started using a 64bit > hash function. We need something int8[] which is not available, so that I > have used ANYARRAYOID in the attached patch(0004). I don't know why you think int8[] is no

Re: [HACKERS] [POC] hash partitioning

2017-10-29 Thread Robert Haas
On Tue, Oct 24, 2017 at 1:21 PM, amul sul wrote: > Updated patch attached. This patch needs a rebase. It appears that satisfies_hash_func is declared incorrectly in pg_proc.h. ProcedureCreate seems to think that provariadic should be ANYOID if the type of the last element is ANYOID, ANYELEMENTO

Re: [HACKERS] [POC] hash partitioning

2017-10-24 Thread amul sul
On Tue, Oct 24, 2017 at 5:00 PM, Andres Freund wrote: > On 2017-10-24 12:43:12 +0530, amul sul wrote: >> I tried to get suggested SMHasher[1] test result for the hash_combine >> for 32-bit and 64-bit version. >> >> SMHasher works on hash keys of the form {0}, {0,1}, {0,1,2}... up to >> N=255, usin

Re: [HACKERS] [POC] hash partitioning

2017-10-24 Thread Andres Freund
On 2017-10-24 12:43:12 +0530, amul sul wrote: > I tried to get suggested SMHasher[1] test result for the hash_combine > for 32-bit and 64-bit version. > > SMHasher works on hash keys of the form {0}, {0,1}, {0,1,2}... up to > N=255, using 256-N as the seed, for the hash_combine testing we > needed

Re: [HACKERS] [POC] hash partitioning

2017-10-24 Thread amul sul
On Fri, Oct 13, 2017 at 3:00 AM, Andres Freund wrote: > On 2017-10-12 17:27:52 -0400, Robert Haas wrote: >> On Thu, Oct 12, 2017 at 4:20 PM, Andres Freund wrote: >> >> In other words, it's not utterly fixed in stone --- we invented >> >> --load-via-partition-root primarily to cope with circumstan

Re: [HACKERS] [POC] hash partitioning

2017-10-16 Thread Ashutosh Bapat
On Mon, Oct 16, 2017 at 2:36 PM, Ashutosh Bapat wrote: > > Probably we should move changes to partition_bounds_copy() in 0003 to > 0001, whose name suggests so. > We can't do this, hash partition strategy is introduced by 0002. Sorry for the noise. -- Best Wishes, Ashutosh Bapat EnterpriseDB C

Re: [HACKERS] [POC] hash partitioning

2017-10-16 Thread Ashutosh Bapat
On Tue, Oct 10, 2017 at 4:37 PM, amul sul wrote: > On Tue, Oct 10, 2017 at 3:42 PM, Ashutosh Bapat > wrote: >> On Tue, Oct 10, 2017 at 3:32 PM, amul sul wrote: >> +hash_part? true : key->parttypbyval[j], +key

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Andres Freund
On 2017-10-12 17:27:52 -0400, Robert Haas wrote: > On Thu, Oct 12, 2017 at 4:20 PM, Andres Freund wrote: > >> In other words, it's not utterly fixed in stone --- we invented > >> --load-via-partition-root primarily to cope with circumstances that > >> could change hash values --- but we sure don't

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Robert Haas
On Thu, Oct 12, 2017 at 4:20 PM, Andres Freund wrote: >> In other words, it's not utterly fixed in stone --- we invented >> --load-via-partition-root primarily to cope with circumstances that >> could change hash values --- but we sure don't want to be changing it >> with any regularity, or for a

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Andres Freund
On 2017-10-12 16:06:11 -0400, Robert Haas wrote: > On Thu, Oct 12, 2017 at 3:43 PM, Andres Freund wrote: > > Are we going to rely on the the combine function to stay the same > > forever after? > > If we change them, it will be a pg_upgrade compatibility break for > anyone using hash-partitioned

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Robert Haas
On Thu, Oct 12, 2017 at 3:43 PM, Andres Freund wrote: > Are we going to rely on the the combine function to stay the same > forever after? If we change them, it will be a pg_upgrade compatibility break for anyone using hash-partitioned tables with more than one partitioning column. Dump and relo

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Andres Freund
On 2017-10-12 10:05:26 -0400, Robert Haas wrote: > On Thu, Oct 12, 2017 at 9:08 AM, amul sul wrote: > > How about combining high 32 bits and the low 32 bits separately as shown > > below? > > > > static inline uint64 > > hash_combine64(uint64 a, uint64 b) > > { > > return (((uint64) hash_comb

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread Robert Haas
On Thu, Oct 12, 2017 at 9:08 AM, amul sul wrote: > How about combining high 32 bits and the low 32 bits separately as shown > below? > > static inline uint64 > hash_combine64(uint64 a, uint64 b) > { > return (((uint64) hash_combine((uint32) a >> 32, (uint32) b >> 32) << 32) > | ha

Re: [HACKERS] [POC] hash partitioning

2017-10-12 Thread amul sul
On Thu, Oct 12, 2017 at 6:31 AM, Robert Haas wrote: > On Tue, Oct 10, 2017 at 7:07 AM, amul sul wrote: >> How about the attached patch(0003)? >> Also, the dim variable is renamed to natts. > > I'm not sure I believe this comment: > > +/* > + * We arrange the partitions in the asce

Re: [HACKERS] [POC] hash partitioning

2017-10-11 Thread Amit Langote
On 2017/09/30 1:53, Robert Haas wrote: > On Thu, Sep 28, 2017 at 1:54 AM, Amit Langote > wrote: >> I looked into how satisfies_hash_partition() works and came up with an >> idea that I think will make constraint exclusion work. What if we emitted >> the hash partition constraint in the following

Re: [HACKERS] [POC] hash partitioning

2017-10-11 Thread Robert Haas
On Tue, Oct 10, 2017 at 7:07 AM, amul sul wrote: > How about the attached patch(0003)? > Also, the dim variable is renamed to natts. I'm not sure I believe this comment: +/* + * We arrange the partitions in the ascending order of their modulus + * and remainders. Also ev

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread amul sul
On Tue, Oct 10, 2017 at 3:42 PM, Ashutosh Bapat wrote: > On Tue, Oct 10, 2017 at 3:32 PM, amul sul wrote: > >>> +hash_part? true : key->parttypbyval[j], >>> +key->parttyplen[j]); >>> parttyplen is the length of partition key

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread Ashutosh Bapat
On Tue, Oct 10, 2017 at 3:40 PM, amul sul wrote: >> >> natts represents the number of attributes, but for the hash partition bound >> we >> are not dealing with the attribute so that I have used short-form of >> dimension, >> thoughts? > > Okay, I think the dimension(dim) is also unfit here. An

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread Ashutosh Bapat
On Tue, Oct 10, 2017 at 3:32 PM, amul sul wrote: >> +hash_part? true : key->parttypbyval[j], >> +key->parttyplen[j]); >> parttyplen is the length of partition key attribute, whereas what you want >> here >> is the length of

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread amul sul
On Tue, Oct 10, 2017 at 3:32 PM, amul sul wrote: > On Mon, Oct 9, 2017 at 5:51 PM, Ashutosh Bapat > wrote: >> On Mon, Oct 9, 2017 at 4:44 PM, amul sul wrote: >> > > Thanks Ashutosh for your review, please find my comment inline. > >> >>> 0002 few changes in partition-wise join code to support >>

Re: [HACKERS] [POC] hash partitioning

2017-10-10 Thread amul sul
On Mon, Oct 9, 2017 at 5:51 PM, Ashutosh Bapat wrote: > On Mon, Oct 9, 2017 at 4:44 PM, amul sul wrote: > Thanks Ashutosh for your review, please find my comment inline. > >> 0002 few changes in partition-wise join code to support >> hash-partitioned table as well & regression tests. > > +s

Re: [HACKERS] [POC] hash partitioning

2017-10-09 Thread Ashutosh Bapat
On Mon, Oct 9, 2017 at 4:44 PM, amul sul wrote: > 0002 few changes in partition-wise join code to support > hash-partitioned table as well & regression tests. +switch (key->strategy) +{ +case PARTITION_STRATEGY_HASH: +/* + * Indexes array is same as the g

Re: [HACKERS] [POC] hash partitioning

2017-10-09 Thread amul sul
On Sat, Oct 7, 2017 at 5:22 PM, amul sul wrote: > On Fri, Oct 6, 2017 at 5:35 PM, Jesper Pedersen > wrote: >> Hi Amul, >> >> Could you rebase on latest master ? >> > > Sure will post that soon, but before that, I need to test hash partitioning > with recent partition-wise join commit (f49842d1ee)

Re: [HACKERS] [POC] hash partitioning

2017-10-07 Thread amul sul
On Fri, Oct 6, 2017 at 5:35 PM, Jesper Pedersen wrote: > Hi Amul, > > Could you rebase on latest master ? > Sure will post that soon, but before that, I need to test hash partitioning with recent partition-wise join commit (f49842d1ee), thanks. Regards, Amul -- Sent via pgsql-hackers mailing

Re: [HACKERS] [POC] hash partitioning

2017-10-06 Thread Jesper Pedersen
Hi Amul, On 09/28/2017 05:56 AM, amul sul wrote: It does not really do the partition pruning via constraint exclusion and I don't think anyone is going to use the remainder in the where condition to fetch data and hash partitioning is not meant for that. But I am sure that we could solve this p

Re: [HACKERS] [POC] hash partitioning

2017-09-29 Thread Robert Haas
On Thu, Sep 28, 2017 at 1:54 AM, Amit Langote wrote: > I looked into how satisfies_hash_partition() works and came up with an > idea that I think will make constraint exclusion work. What if we emitted > the hash partition constraint in the following form instead: > > hash_partition_mod(hash_part

Re: [HACKERS] [POC] hash partitioning

2017-09-28 Thread amul sul
On Thu, Sep 28, 2017 at 11:24 AM, Amit Langote wrote: > On 2017/09/27 22:41, Jesper Pedersen wrote: >> On 09/27/2017 03:05 AM, amul sul wrote: > Attached rebased patch, thanks. > > While reading through the patch I thought it would be better to keep MODULUS and REMAINDER in c

Re: [HACKERS] [POC] hash partitioning

2017-09-27 Thread Amit Langote
On 2017/09/27 22:41, Jesper Pedersen wrote: > On 09/27/2017 03:05 AM, amul sul wrote: Attached rebased patch, thanks. >>> While reading through the patch I thought it would be better to keep >>> MODULUS and REMAINDER in caps, if CREATE TABLE was in caps too in order to >>> highlight

Re: [HACKERS] [POC] hash partitioning

2017-09-27 Thread Jesper Pedersen
On 09/27/2017 03:05 AM, amul sul wrote: Attached rebased patch, thanks. While reading through the patch I thought it would be better to keep MODULUS and REMAINDER in caps, if CREATE TABLE was in caps too in order to highlight that these are "keywords" for hash partition. Also updated some of

Re: [HACKERS] [POC] hash partitioning

2017-09-27 Thread amul sul
On Mon, Sep 18, 2017 at 8:55 PM, Jesper Pedersen wrote: > On 09/15/2017 02:30 AM, amul sul wrote: > >> Attached rebased patch, thanks. >> >> > While reading through the patch I thought it would be better to keep > MODULUS and REMAINDER in caps, if CREATE TABLE was in caps too in order to > highli

Re: [HACKERS] [POC] hash partitioning

2017-09-18 Thread Jesper Pedersen
On 09/15/2017 02:30 AM, amul sul wrote: Attached rebased patch, thanks. While reading through the patch I thought it would be better to keep MODULUS and REMAINDER in caps, if CREATE TABLE was in caps too in order to highlight that these are "keywords" for hash partition. Also updated some

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread amul sul
On Fri, Sep 15, 2017 at 4:30 AM, Thom Brown wrote: > On 14 September 2017 at 09:58, amul sul wrote: > > On Wed, Sep 13, 2017 at 7:43 PM, Jesper Pedersen > > wrote: > >> > >> Hi Amul, > >> > >> On 09/08/2017 08:40 AM, amul sul wrote: > >>> > >>> Rebased 0002 against this commit & renamed to 0001

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Thom Brown
On 14 September 2017 at 09:58, amul sul wrote: > On Wed, Sep 13, 2017 at 7:43 PM, Jesper Pedersen > wrote: >> >> Hi Amul, >> >> On 09/08/2017 08:40 AM, amul sul wrote: >>> >>> Rebased 0002 against this commit & renamed to 0001, PFA. >>> >> >> This patch needs a rebase. >> > > Thanks for your note

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Robert Haas
On Thu, Sep 14, 2017 at 2:05 PM, Jesper Pedersen wrote: > However, it is a little bit difficult to follow the dependencies between > different partition patches, so I may not always provide sane feedback, as > seen in [1]. > > [1] > https://www.postgresql.org/message-id/579077fd-8f07-aff7-39bc-b92

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
On 09/14/2017 01:52 PM, Robert Haas wrote: On Thu, Sep 14, 2017 at 1:07 PM, Jesper Pedersen wrote: Yeah, it would be nice to have a syntax like ) PARTITION BY HASH (col) WITH (AUTO_CREATE = 64); But then there also needs to be a way to create the 64 associated indexes too for everything to be

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Robert Haas
On Thu, Sep 14, 2017 at 1:07 PM, Jesper Pedersen wrote: > Yeah, it would be nice to have a syntax like > > ) PARTITION BY HASH (col) WITH (AUTO_CREATE = 64); > > But then there also needs to be a way to create the 64 associated indexes > too for everything to be easy. Well, for that, there's this

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
On 09/14/2017 12:56 PM, Robert Haas wrote: On Thu, Sep 14, 2017 at 12:54 PM, David Fetter wrote: Should we be pointing the gun away from people's feet by making hash partitions that cover the space automagically when the partitioning scheme[1] is specified? In other words, do we have a good re

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Robert Haas
On Thu, Sep 14, 2017 at 12:54 PM, David Fetter wrote: > Should we be pointing the gun away from people's feet by making hash > partitions that cover the space automagically when the partitioning > scheme[1] is specified? In other words, do we have a good reason to have > only some of the hash par

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread David Fetter
On Mon, Sep 11, 2017 at 07:43:29AM -0400, Robert Haas wrote: > On Mon, Sep 11, 2017 at 4:17 AM, Ashutosh Bapat > wrote: > >> Rebased 0002 against this commit & renamed to 0001, PFA. > > > > Given that we have default partition support now, I am wondering > > whether hash partitioned tables also sh

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
Hi, On 09/14/2017 12:05 PM, Robert Haas wrote: On Thu, Sep 14, 2017 at 11:39 AM, Jesper Pedersen wrote: When I do CREATE TABLE mytab ( a integer NOT NULL, b integer NOT NULL, c integer, d integer ) PARTITION BY HASH (b); and create 64 partitions; CREATE TABLE mytab_p00 PARTITION

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Robert Haas
On Thu, Sep 14, 2017 at 11:39 AM, Jesper Pedersen wrote: > When I do > > CREATE TABLE mytab ( > a integer NOT NULL, > b integer NOT NULL, > c integer, > d integer > ) PARTITION BY HASH (b); > > and create 64 partitions; > > CREATE TABLE mytab_p00 PARTITION OF mytab FOR VALUES WITH (MODULUS

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread Jesper Pedersen
Hi Amul, On 09/14/2017 04:58 AM, amul sul wrote: On Wed, Sep 13, 2017 at 7:43 PM, Jesper Pedersen This patch needs a rebase. Thanks for your note. ​ ​ Attached is the patch rebased on the latest master head. Also added error on ​creating ​​default partition ​for the hash partitioned table​,

Re: [HACKERS] [POC] hash partitioning

2017-09-14 Thread amul sul
On Wed, Sep 13, 2017 at 7:43 PM, Jesper Pedersen wrote: > Hi Amul, > > On 09/08/2017 08:40 AM, amul sul wrote: > >> Rebased 0002 against this commit & renamed to 0001, PFA. >> >> > This patch needs a rebase. > > Thanks for your note. ​ ​ Attached is the patch rebased on the latest master head. Al

Re: [HACKERS] [POC] hash partitioning

2017-09-13 Thread Jesper Pedersen
Hi Amul, On 09/08/2017 08:40 AM, amul sul wrote: Rebased 0002 against this commit & renamed to 0001, PFA. This patch needs a rebase. Best regards, Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread amul sul
On Mon, Sep 11, 2017 at 5:30 PM, Alvaro Herrera wrote: > Robert Haas wrote: > > On Mon, Sep 11, 2017 at 4:17 AM, Ashutosh Bapat > > wrote: > > >> Rebased 0002 against this commit & renamed to 0001, PFA. > > > > > > Given that we have default partition support now, I am wondering > > > whether ha

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Robert Haas
On Mon, Sep 11, 2017 at 8:00 AM, Alvaro Herrera wrote: > How difficult/tedious/troublesome would be to install the missing > partitions if you set hash partitioning with a default partition and > only later on notice that some partitions are missing? I think if the > answer is that you need to ex

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Sep 11, 2017 at 4:17 AM, Ashutosh Bapat > wrote: > >> Rebased 0002 against this commit & renamed to 0001, PFA. > > > > Given that we have default partition support now, I am wondering > > whether hash partitioned tables also should have default partitions. > > The way

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Robert Haas
On Mon, Sep 11, 2017 at 4:17 AM, Ashutosh Bapat wrote: >> Rebased 0002 against this commit & renamed to 0001, PFA. > > Given that we have default partition support now, I am wondering > whether hash partitioned tables also should have default partitions. > The way we have structured hash partition

Re: [HACKERS] [POC] hash partitioning

2017-09-11 Thread Ashutosh Bapat
On Fri, Sep 8, 2017 at 6:10 PM, amul sul wrote: > On Fri, Sep 8, 2017 at 6:45 AM, Robert Haas wrote: >> >> On Mon, Sep 4, 2017 at 6:38 AM, amul sul wrote: >> > I've updated patch to use an extended hash function (Commit # >> > 81c5e46c490e2426db243eada186995da5bb0ba7) for the partitioning. >> >>

Re: [HACKERS] [POC] hash partitioning

2017-09-08 Thread amul sul
On Fri, Sep 8, 2017 at 6:45 AM, Robert Haas wrote: > On Mon, Sep 4, 2017 at 6:38 AM, amul sul wrote: > > I've updated patch to use an extended hash function (Commit # > > 81c5e46c490e2426db243eada186995da5bb0ba7) for the partitioning. > > Committed 0001 after noticing that Jeevan Ladhe also foun

Re: [HACKERS] [POC] hash partitioning

2017-09-07 Thread Robert Haas
On Mon, Sep 4, 2017 at 6:38 AM, amul sul wrote: > I've updated patch to use an extended hash function (Commit # > 81c5e46c490e2426db243eada186995da5bb0ba7) for the partitioning. Committed 0001 after noticing that Jeevan Ladhe also found that change convenient for default partitioning. I made a f

Re: [HACKERS] [POC] hash partitioning

2017-09-05 Thread Rajkumar Raghuwanshi
On Mon, Sep 4, 2017 at 4:08 PM, amul sul wrote: > I've updated patch to use an extended hash function (​Commit # > 81c5e46c490e2426db243eada186995da5bb0ba7) for the partitioning. > > I have done some testing with these patches, everything looks fine, attaching sql and out file for reference. Tha

Re: [HACKERS] [POC] hash partitioning

2017-09-04 Thread amul sul
I've updated patch to use an extended hash function (​Commit # 81c5e46c490e2426db243eada186995da5bb0ba7) for the partitioning. Regards, Amul On Thu, Jul 27, 2017 at 5:11 PM, amul sul wrote: > Attaching newer patches rebased against the latest master head. Thanks ! > > Regards, > Amul > 0001-

Re: [HACKERS] [POC] hash partitioning

2017-08-29 Thread yangjie
Hi, This is my patch, before I forgot to add attachments, and the following address is also discussed. https://www.postgresql.org/message-id/2017082612390093777512%40highgo.com

Re: [HACKERS] [POC] hash partitioning

2017-08-28 Thread yang...@highgo.com
Hello Looking at your hash partitioning syntax, I implemented a hash partition in a more concise way, with no need to determine the number of sub-tables, and dynamically add partitions. Description The hash partition's implement is on the basis of the original range / list partition,and using

Re: [HACKERS] [POC] hash partitioning

2017-08-28 Thread Yugo Nagata
Hi young, On Mon, 28 Aug 2017 15:33:46 +0800 "yang...@highgo.com" wrote: > Hello > > Looking at your hash partitioning syntax, I implemented a hash partition in a > more concise way, with no need to determine the number of sub-tables, and > dynamically add partitions. I think it is great wor

Re: [HACKERS] [POC] hash partitioning

2017-07-27 Thread amul sul
Attaching newer patches rebased against the latest master head. Thanks ! Regards, Amul 0001-Cleanup_v6.patch Description: Binary data 0002-hash-partitioning_another_design-v16.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] [POC] hash partitioning

2017-07-05 Thread amul sul
On Wed, Jul 5, 2017 at 4:50 PM, Dilip Kumar wrote: > On Mon, Jul 3, 2017 at 4:39 PM, amul sul wrote: >> Thanks to catching this, fixed in the attached version. > > Few comments on the latest version. > Thanks for your review, please find my comment inline: > 0001 looks fine, for 0002 I have som

Re: [HACKERS] [POC] hash partitioning

2017-07-05 Thread Dilip Kumar
On Mon, Jul 3, 2017 at 4:39 PM, amul sul wrote: > Thanks to catching this, fixed in the attached version. Few comments on the latest version. 0001 looks fine, for 0002 I have some comments. 1. + hbounds = (PartitionHashBound * *) palloc(nparts * + sizeof(PartitionHashBound *)); /s/(PartitionH

Re: [HACKERS] [POC] hash partitioning

2017-07-03 Thread amul sul
On Fri, Jun 23, 2017 at 11:19 AM, Yugo Nagata wrote: > On Fri, 23 Jun 2017 13:41:15 +0900 > Yugo Nagata wrote: > >> On Tue, 6 Jun 2017 13:03:58 +0530 >> amul sul wrote: >> >> >> > Updated patch attached. >> >> I looked into the latest patch (v13) and have some comments >> althogh they might be t

Re: [HACKERS] [POC] hash partitioning

2017-07-03 Thread amul sul
On Fri, Jun 23, 2017 at 10:11 AM, Yugo Nagata wrote: > On Tue, 6 Jun 2017 13:03:58 +0530 > amul sul wrote: > > >> Updated patch attached. > > I looked into the latest patch (v13) and have some comments > althogh they might be trivial. > Thanks for your review. > First, I couldn't apply this patc

Re: [HACKERS] [POC] hash partitioning

2017-06-22 Thread Yugo Nagata
On Fri, 23 Jun 2017 13:41:15 +0900 Yugo Nagata wrote: > On Tue, 6 Jun 2017 13:03:58 +0530 > amul sul wrote: > > > > Updated patch attached. > > I looked into the latest patch (v13) and have some comments > althogh they might be trivial. One more comment: + if (spec->remainder < 0) +

Re: [HACKERS] [POC] hash partitioning

2017-06-22 Thread Yugo Nagata
On Tue, 6 Jun 2017 13:03:58 +0530 amul sul wrote: > Updated patch attached. I looked into the latest patch (v13) and have some comments althogh they might be trivial. First, I couldn't apply this patch to the latest HEAD due to a documentation fix and pgintend updates. It needes rebase. $ git

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
On Tue, Jun 6, 2017 at 2:41 PM, Amit Langote wrote: > Consider an example using the partition hierarchy: > > root (a int, b char, c int) partition by range (a) > > -> level1 from (1) to (10) partition by list (b) > > -> level2 in ('a') parition by range (c) > > -> leaf from (1) to (

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Amit Langote
On 2017/06/06 17:50, Dilip Kumar wrote: > On Tue, Jun 6, 2017 at 1:03 PM, amul sul wrote: >> May I ask you, how you sure about 8 is an unfit value for t1 relation? >> And what if the value other than 8, for e.g. 7? > > Well, First I created t1 as a leaf relation like below, and I tested > insert

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
On Tue, Jun 6, 2017 at 1:03 PM, amul sul wrote: > May I ask you, how you sure about 8 is an unfit value for t1 relation? > And what if the value other than 8, for e.g. 7? Well, First I created t1 as a leaf relation like below, and I tested insert into t1 with value 8 and it was violating the part

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread amul sul
Hi Dilip, Thanks for review. On Sat, Jun 3, 2017 at 6:54 PM, Dilip Kumar wrote: > On Thu, May 25, 2017 at 9:59 AM, amul sul wrote: >> On Mon, May 22, 2017 at 2:23 PM, amul sul wrote: >>> >>> Updated patch attached. Thanks a lot for review. >>> >> Minor fix in the document, PFA. > > Patch need

Re: [HACKERS] [POC] hash partitioning

2017-06-03 Thread Dilip Kumar
On Thu, May 25, 2017 at 9:59 AM, amul sul wrote: > On Mon, May 22, 2017 at 2:23 PM, amul sul wrote: >> >> Updated patch attached. Thanks a lot for review. >> > Minor fix in the document, PFA. Patch need rebase --- Function header is not consistent with other neighbouring functions (some fun

Re: [HACKERS] [POC] hash partitioning

2017-05-25 Thread Robert Haas
On Mon, May 22, 2017 at 1:49 AM, Ashutosh Bapat wrote: > The prologue is arranged as one paragraph (with a new line) per > member. Within each paragraph explanation for each partitioning > strategy starts on its own line. One paragraph per member is more > readable than separate paragraphs for eac

Re: [HACKERS] [POC] hash partitioning

2017-05-24 Thread amul sul
On Mon, May 22, 2017 at 2:23 PM, amul sul wrote: > > Updated patch attached. Thanks a lot for review. > Minor fix in the document, PFA. Regards, Amul 0002-hash-partitioning_another_design-v12.patch Description: Binary data 0001-Cleanup_v4.patch Description: Binary data -- Sent via pgsql-hac

Re: [HACKERS] [POC] hash partitioning

2017-05-22 Thread amul sul
On Fri, May 19, 2017 at 10:35 PM, Robert Haas wrote: > On Fri, May 19, 2017 at 5:32 AM, amul sul wrote: >> Updated patch attached. 0001-patch rebased against latest head. >> 0002-patch also incorporates code comments and error message changes >> as per Robert's & your suggestions. Thanks ! > > -

Re: [HACKERS] [POC] hash partitioning

2017-05-21 Thread Ashutosh Bapat
On Fri, May 19, 2017 at 10:35 PM, Robert Haas wrote: > > + * For range and list partitioned tables, datums is an array of datum-tuples > + * with key->partnatts datums each. > + * For hash partitioned tables, it is an array of datum-tuples with 2 datums, > + * modulus and remainder, corresponding

Re: [HACKERS] [POC] hash partitioning

2017-05-19 Thread Robert Haas
On Fri, May 19, 2017 at 5:32 AM, amul sul wrote: > Updated patch attached. 0001-patch rebased against latest head. > 0002-patch also incorporates code comments and error message changes > as per Robert's & your suggestions. Thanks ! -if (spec->strategy != PARTITION_STRATEGY_LIST)

Re: [HACKERS] [POC] hash partitioning

2017-05-19 Thread amul sul
On Wed, May 17, 2017 at 6:54 PM, Ashutosh Bapat wrote: [...] > > Comments on the tests > +#ifdef USE_ASSERT_CHECKING > +{ > +/* > + * Hash partition bound stores modulus and remainder at > + * b1->datums[i][0] and b1->datums[i][1] position respectively.

Re: [HACKERS] [POC] hash partitioning

2017-05-18 Thread Amit Langote
On 2017/05/19 1:09, Dilip Kumar wrote: > On Wed, May 17, 2017 at 2:07 PM, amul sul wrote: >>> I would suggest "non-zero positive", since that's what we are using in >>> the documentation. >>> >> >> Understood, Fixed in the attached version. > > Why non-zero positive? We do support zero for the r

Re: [HACKERS] [POC] hash partitioning

2017-05-18 Thread Dilip Kumar
On Wed, May 17, 2017 at 2:07 PM, amul sul wrote: >> I would suggest "non-zero positive", since that's what we are using in >> the documentation. >> > > Understood, Fixed in the attached version. Why non-zero positive? We do support zero for the remainder right? -- Regards, Dilip Kumar Enterpri

Re: [HACKERS] [POC] hash partitioning

2017-05-17 Thread Ashutosh Bapat
On Wed, May 17, 2017 at 11:51 PM, Robert Haas wrote: > On Wed, May 17, 2017 at 1:41 AM, Ashutosh Bapat > wrote: >>> Fixed in the attached version; used "hash partition remainder must be >>> greater than or equal to 0" instead. >> >> I would suggest "non-zero positive", since that's what we are u

Re: [HACKERS] [POC] hash partitioning

2017-05-17 Thread Robert Haas
On Wed, May 17, 2017 at 1:41 AM, Ashutosh Bapat wrote: >> Fixed in the attached version; used "hash partition remainder must be >> greater than or equal to 0" instead. > > I would suggest "non-zero positive", since that's what we are using in > the documentation. Well, that's not very good termi

Re: [HACKERS] [POC] hash partitioning

2017-05-17 Thread Ashutosh Bapat
On Wed, May 17, 2017 at 2:07 PM, amul sul wrote: > >> In partition_bounds_equal(), please add comments explaining why is it safe to >> check just the indexes? May be we should add code under assertion to make >> sure >> that the datums are equal as well. > > Added assert in the attached version.

Re: [HACKERS] [POC] hash partitioning

2017-05-17 Thread amul sul
On Wed, May 17, 2017 at 11:11 AM, Ashutosh Bapat wrote: > On Wed, May 17, 2017 at 12:04 AM, amul sul wrote: >> On Tue, May 16, 2017 at 10:00 PM, Dilip Kumar wrote: >>> On Tue, May 16, 2017 at 4:22 PM, amul sul wrote: v6 patch has bug in partition oid mapping and indexing, fixed in the

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Ashutosh Bapat
On Wed, May 17, 2017 at 12:04 AM, amul sul wrote: > On Tue, May 16, 2017 at 10:00 PM, Dilip Kumar wrote: >> On Tue, May 16, 2017 at 4:22 PM, amul sul wrote: >>> v6 patch has bug in partition oid mapping and indexing, fixed in the >>> attached version. >>> >>> Now partition oids will be arranged

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Ashutosh Bapat
On Wed, May 17, 2017 at 9:38 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, May 16, 2017 at 6:50 PM, Peter Eisentraut >> wrote: >>> On 5/15/17 23:45, Ashutosh Bapat wrote: +1. We should throw an error and add a line in documentation that collation should not be specified for h

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Tom Lane
Ashutosh Bapat writes: > On Tue, May 16, 2017 at 6:50 PM, Peter Eisentraut > wrote: >> On 5/15/17 23:45, Ashutosh Bapat wrote: >>> +1. We should throw an error and add a line in documentation that >>> collation should not be specified for hash partitioned table. >> Why is it even allowed in the

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Ashutosh Bapat
On Tue, May 16, 2017 at 6:50 PM, Peter Eisentraut wrote: > On 5/15/17 23:45, Ashutosh Bapat wrote: >> +1. We should throw an error and add a line in documentation that >> collation should not be specified for hash partitioned table. > > Why is it even allowed in the parser then? That grammar is c

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread amul sul
On Tue, May 16, 2017 at 10:00 PM, Dilip Kumar wrote: > On Tue, May 16, 2017 at 4:22 PM, amul sul wrote: >> v6 patch has bug in partition oid mapping and indexing, fixed in the >> attached version. >> >> Now partition oids will be arranged in the ascending order of hash >> partition bound (i.e. m

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Robert Haas
On Tue, May 16, 2017 at 3:19 AM, Ashutosh Bapat wrote: > While earlier, I thought the same, I am wondering whether this is > true. Don't different collations deem different strings equal e.g one > collation may deem 'aa' and 'AA' as same but other may not. No, that's not allowed. This has been d

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Dilip Kumar
On Tue, May 16, 2017 at 4:22 PM, amul sul wrote: > v6 patch has bug in partition oid mapping and indexing, fixed in the > attached version. > > Now partition oids will be arranged in the ascending order of hash > partition bound (i.e. modulus and remainder sorting order) Thanks for the update pa

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Peter Eisentraut
On 5/16/17 03:19, Ashutosh Bapat wrote: > On Tue, May 16, 2017 at 10:03 AM, amul sul wrote: >> On Mon, May 15, 2017 at 9:13 PM, Robert Haas wrote: > Collation is only relevant for ordering, not equality. > > While earlier, I thought the same, I am wondering whether this is > true. Don't diff

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Peter Eisentraut
On 5/15/17 23:45, Ashutosh Bapat wrote: > +1. We should throw an error and add a line in documentation that > collation should not be specified for hash partitioned table. Why is it even allowed in the parser then? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Developme

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread amul sul
On Tue, May 16, 2017 at 3:30 PM, amul sul wrote: > On Tue, May 16, 2017 at 1:02 PM, Ashutosh Bapat > wrote: > [...] +if (key->strategy == PARTITION_STRATEGY_HASH) +{ +ndatums = nparts; +hbounds = (PartitionHashBound **) palloc(npar

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread amul sul
On Tue, May 16, 2017 at 1:17 PM, Ashutosh Bapat wrote: > Hi, > Here's patch with some cosmetic fixes to 0002, to be applied on top of 0002. > Thank you, included in v6 patch. Regards, Amul -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread amul sul
On Tue, May 16, 2017 at 1:02 PM, Ashutosh Bapat wrote: [...] >>> >>> +if (key->strategy == PARTITION_STRATEGY_HASH) >>> +{ >>> +ndatums = nparts; >>> +hbounds = (PartitionHashBound **) palloc(nparts * >>> + >>> sizeof(PartitionHashBound *)); >>> +

  1   2   >