Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Apr 6, 2020 at 8:43 PM Ashutosh Bapat wrote: > On Fri, 3 Apr 2020 at 20:45, Etsuro Fujita wrote: > But it will be good to have following addition I suggested in my patches to > make sure nparts is set to 0 for an unpartitioned relation as per the comment > on nparts in RelOptInfo. > @@

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Sat, Apr 11, 2020 at 1:00 AM Tom Lane wrote: > Ashutosh Bapat writes: > > On Fri, Apr 10, 2020 at 9:14 PM Tom Lane wrote: > >> I see no patch here ... > > > Sorry. Here it is > > LGTM, will push in a moment. Thanks for taking care of this, Tom! Thanks for the patch, Ashutosh! Thanks for

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Ashutosh Bapat writes: > On Fri, Apr 10, 2020 at 9:14 PM Tom Lane wrote: >> I see no patch here ... > Sorry. Here it is LGTM, will push in a moment. regards, tom lane

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Apr 10, 2020 at 9:14 PM Tom Lane wrote: > > Ashutosh Bapat writes: > > On Fri, 10 Apr 2020 at 20:44, Jeff Janes wrote: > >> In that case, we really should add the PG_USED_FOR_ASSERTS_ONLY to make > >> the compiler happy. > > > Attaching my patch again. It doesn't need

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Ashutosh Bapat writes: > On Fri, 10 Apr 2020 at 20:44, Jeff Janes wrote: >> In that case, we really should add the PG_USED_FOR_ASSERTS_ONLY to make >> the compiler happy. > Attaching my patch again. It doesn't need PG_USED_FOR_ASSERTS_ONLY as well. > Kuntal has confirmed that this fixes the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, 10 Apr 2020 at 20:44, Jeff Janes wrote: > On Thu, Apr 9, 2020 at 10:04 AM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> On Thu, Apr 9, 2020 at 12:03 PM Etsuro Fujita >> wrote: >> > >> > On Thu, Apr 9, 2020 at 2:36 PM Tom Lane wrote: >> > > Etsuro Fujita writes: >> > > >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Apr 9, 2020 at 10:04 AM Ashutosh Bapat wrote: > On Thu, Apr 9, 2020 at 12:03 PM Etsuro Fujita > wrote: > > > > On Thu, Apr 9, 2020 at 2:36 PM Tom Lane wrote: > > > Etsuro Fujita writes: > > > > Yeah, partition_bounds_merge() is currently called only from > > > >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Apr 09, 2020 at 07:34:01PM +0530, Ashutosh Bapat wrote: On Thu, Apr 9, 2020 at 12:03 PM Etsuro Fujita wrote: On Thu, Apr 9, 2020 at 2:36 PM Tom Lane wrote: > Etsuro Fujita writes: > > Yeah, partition_bounds_merge() is currently called only from > > try_partitionwise_join(), which

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Apr 9, 2020 at 12:03 PM Etsuro Fujita wrote: > > On Thu, Apr 9, 2020 at 2:36 PM Tom Lane wrote: > > Etsuro Fujita writes: > > > Yeah, partition_bounds_merge() is currently called only from > > > try_partitionwise_join(), which guarantees that the strategies are the > > > same. > > > If

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Apr 9, 2020 at 2:36 PM Tom Lane wrote: > Etsuro Fujita writes: > > Yeah, partition_bounds_merge() is currently called only from > > try_partitionwise_join(), which guarantees that the strategies are the > > same. > If there's only one caller and there's not likely to ever be more, >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Etsuro Fujita writes: > Yeah, partition_bounds_merge() is currently called only from > try_partitionwise_join(), which guarantees that the strategies are the > same. The assertion cost would be cheap, but not zero, so I still > think it would be better to remove the assertion from >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi, On Thu, Apr 9, 2020 at 12:06 AM Kuntal Ghosh wrote: > Both of your patches fix the problem. I don't have much exposure in > this area to comment on whether we should keep/remove the assertion > from the code. But, here is my opinion: > > The code structure looks like following: >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hello Ashutosh, Fujita, On Wed, Apr 8, 2020 at 3:49 PM Ashutosh Bapat wrote: > On Wed, 8 Apr 2020 at 15:42, Etsuro Fujita wrote: >> On Wed, Apr 8, 2020 at 4:30 PM Kuntal Ghosh >> wrote: >> > I'm getting the following warning during compilation. >> > >> > partbounds.c: In function

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, 8 Apr 2020 at 15:42, Etsuro Fujita wrote: > Hi Kuntal, > > On Wed, Apr 8, 2020 at 4:30 PM Kuntal Ghosh > wrote: > > I'm getting the following warning during compilation. > > > > partbounds.c: In function ‘partition_bounds_merge’: > > partbounds.c:1024:21: warning: unused variable

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Thanks Kuntal for the report. Let me know if this patch works for you. On Wed, 8 Apr 2020 at 13:00, Kuntal Ghosh wrote: > Hi, > > On Wed, Apr 8, 2020 at 7:07 AM Etsuro Fujita > wrote: > > > > Pushed after modifying some comments further, based on the suggestions > > of Ashutosh. > I'm getting

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Kuntal, On Wed, Apr 8, 2020 at 4:30 PM Kuntal Ghosh wrote: > I'm getting the following warning during compilation. > > partbounds.c: In function ‘partition_bounds_merge’: > partbounds.c:1024:21: warning: unused variable ‘inner_binfo’ > [-Wunused-variable] > PartitionBoundInfo inner_binfo =

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi, On Wed, Apr 8, 2020 at 7:07 AM Etsuro Fujita wrote: > > Pushed after modifying some comments further, based on the suggestions > of Ashutosh. I'm getting the following warning during compilation. partbounds.c: In function ‘partition_bounds_merge’: partbounds.c:1024:21: warning: unused

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Thanks a lot Fujita-san. Thanks Dmitry, Rajkumar, Amul, Mark, Robert, Antonin, Amit, Justin,Thomas and Tomas for all your help and review. On Wed, 8 Apr 2020 at 07:07, Etsuro Fujita wrote: > On Wed, Apr 8, 2020 at 2:24 AM Etsuro Fujita > wrote: > > On Wed, Apr 8, 2020 at 12:15 AM Tomas Vondra

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Apr 8, 2020 at 2:24 AM Etsuro Fujita wrote: > On Wed, Apr 8, 2020 at 12:15 AM Tomas Vondra > wrote: > > On Mon, Apr 06, 2020 at 05:28:52PM +0900, Etsuro Fujita wrote: > > >On Sat, Apr 4, 2020 at 12:15 AM Etsuro Fujita > > >wrote: > > >> Attached is the original patch (0001) and one

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Tomas, On Wed, Apr 8, 2020 at 12:15 AM Tomas Vondra wrote: > On Mon, Apr 06, 2020 at 05:28:52PM +0900, Etsuro Fujita wrote: > >On Sat, Apr 4, 2020 at 12:15 AM Etsuro Fujita > >wrote: > >> Attached is the original patch (0001) and one patch (0002) with > >> changes including those by Tomas

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Apr 06, 2020 at 05:28:52PM +0900, Etsuro Fujita wrote: On Sat, Apr 4, 2020 at 12:15 AM Etsuro Fujita wrote: Attached is the original patch (0001) and one patch (0002) with changes including those by Tomas and Ashutosh. I merged the patches into one and rebased it against HEAD.

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Here are some changes suggested on top of v34 as per my previous mail. These are mostly comment changes. On Mon, 6 Apr 2020 at 13:59, Etsuro Fujita wrote: > On Sat, Apr 4, 2020 at 12:15 AM Etsuro Fujita > wrote: > > Attached is the original patch (0001) and one patch (0002) with > > changes

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, 3 Apr 2020 at 20:45, Etsuro Fujita wrote: > Hi, > > On Thu, Apr 2, 2020 at 2:12 AM Ashutosh Bapat > wrote: > > On Thu, 26 Mar 2020 at 00:35, Tomas Vondra > wrote: > >> I've started reviewing the patch a couple days ago. I haven't done any > >> extensive testing, but I do have a bunch

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Tomas and Ashutosh, On Thu, Apr 2, 2020 at 1:51 AM Ashutosh Bapat wrote: > On Thu, 26 Mar 2020 at 05:47, Tomas Vondra > wrote: >> three more comments after eye-balling the code for a bit longer. >> >> 1) The patch probably needs to tweak config.sgml which says this about >> the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, 26 Mar 2020 at 00:35, Tomas Vondra wrote: > Hi, > > I've started reviewing the patch a couple days ago. I haven't done any > extensive testing, but I do have a bunch of initial comments that I can > share now. > > 1) I wonder if this needs to update src/backend/optimizer/README, which >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi, I've started reviewing the patch a couple days ago. I haven't done any extensive testing, but I do have a bunch of initial comments that I can share now. 1) I wonder if this needs to update src/backend/optimizer/README, which does have a section about partitionwise joins. It seems

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Mar 17, 2020 at 1:44 PM Etsuro Fujita wrote: > > > + /* > > + * If this segment of the join is empty, it means that this segment > > > > "partition of the join" looks consistent with other usages than "segment of > > the > > join". > > Actually, "segment" is used in the existing comments

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Ashutosh, On Wed, Mar 4, 2020 at 1:48 AM Ashutosh Bapat wrote: > I reviewed the patch. Except for the logic of matching the pairs of > partitions from already merged partitions, I think the code changes are good. > But there are several places where it needs further changes to comments. The

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Fujita-san, I reviewed the patch. Except for the logic of matching the pairs of partitions from already merged partitions, I think the code changes are good. But there are several places where it needs further changes to comments. The attached patch has those. I have described some of them

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Feb 6, 2020 at 3:55 AM Mark Dilger wrote: > The patches apply and pass all tests. A review of the patch vs. master looks > reasonable. Thanks for the review! > The partition_join.sql test has multiple levels of partitioning, but when > your patch extends that test with “advanced

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Feb 6, 2020 at 2:03 AM Mark Dilger wrote: > > On Feb 5, 2020, at 4:51 AM, Etsuro Fujita wrote: > >> On Tue, Jan 28, 2020 at 1:39 PM Mark Dilger > >> wrote: > >>> I have added tests checking correctness and showing some partition > >>> pruning limitations. Find three patches,

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Feb 5, 2020, at 4:51 AM, Etsuro Fujita wrote: > > The patches apply and pass all tests. A review of the patch vs. master looks reasonable. The partition_join.sql test has multiple levels of partitioning, but when your patch extends that test with “advanced partition-wise join”,

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Feb 5, 2020, at 4:51 AM, Etsuro Fujita wrote: > > On Wed, Jan 29, 2020 at 9:15 PM Etsuro Fujita wrote: >> On Tue, Jan 28, 2020 at 1:39 PM Mark Dilger >> wrote: >>> I have added tests checking correctness and showing some partition pruning >>> limitations. Find three patches,

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Mark, On Tue, Jan 28, 2020 at 1:39 PM Mark Dilger wrote: > There is stray whitespace in v30-0002: > > src/backend/partitioning/partbounds.c:4557: space before tab in indent. > + outer_null_unmerged = true; Good catch! > I have added tests checking correctness and

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Amul, On Tue, Dec 10, 2019 at 3:49 PM amul sul wrote: > On Mon, Dec 9, 2019 at 3:08 PM amul sul wrote: >> Attached is the rebase version atop the latest master head(2d0fdfaccec). Thanks for that! > I have been through your changes proposed in [1] -- the changes make sense to > me & > I

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Dec 9, 2019 at 3:08 PM amul sul wrote: > Attached is the rebase version atop the latest master head(2d0fdfaccec). > > Hi Fujita san, I have been through your changes proposed in [1] -- the changes make sense to me & I didn't see any unobvious behaviour in testing as well. Regards, Amul

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Nov 22, 2019 at 10:08 PM Etsuro Fujita wrote: > Done. I modified compare_range_partitions() as well to match its the > variable names with others. Attached is a new version of the patch. > > I reviewed the rest of the partbounds.c changes. Here are my review comments: > > * I don't

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Amul, On Fri, Nov 15, 2019 at 2:21 PM amul sul wrote: > Thank you Fujita san for the patch & the enhancements. I am fine with your > delta patch. OK, I'll merge the delta patch with the main one in the next version, if no objections from others. > I would like to share some thought for

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Oct 29, 2019 at 7:29 PM Etsuro Fujita wrote: > On Fri, Oct 25, 2019 at 6:59 PM amul sul wrote: > > On Wed, Oct 16, 2019 at 6:20 PM Etsuro Fujita > > wrote: > >> So I'd like to propose to introduce separate functions like > >> process_outer_partition() and process_inner_partition() in

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Amul, On Fri, Oct 25, 2019 at 6:59 PM amul sul wrote: > On Wed, Oct 16, 2019 at 6:20 PM Etsuro Fujita wrote: >> So I'd like to propose to introduce separate functions like >> process_outer_partition() and process_inner_partition() in the >> attached, instead of handle_missing_partition().

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Oct 16, 2019 at 6:20 PM Etsuro Fujita wrote: > On Wed, Sep 25, 2019 at 12:59 AM Etsuro Fujita > wrote: > > I will continue to review the rest of the patch. > > I've been reviewing the rest of the patch. Here are my review comments: > [] > So I'd like to propose to introduce

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Sep 25, 2019 at 12:59 AM Etsuro Fujita wrote: > I will continue to review the rest of the patch. I've been reviewing the rest of the patch. Here are my review comments: * map_and_merge_partitions() checks whether the two partitions from the outer and inner sides can be merged in two

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi, On Thu, Sep 5, 2019 at 1:24 PM amul sul wrote: > On Wed, Sep 4, 2019 at 2:40 AM Alvaro Herrera > wrote: >> CFbot complains that Fujita-san submitted a patch that doesn't apply, >> which makes sense since the necessary previous patch was only referred >> to without being resubmitted. I

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Sep 4, 2019 at 2:40 AM Alvaro Herrera wrote: > Fujita-san, amul, > > CFbot complains that Fujita-san submitted a patch that doesn't apply, > which makes sense since the necessary previous patch was only referred > to without being resubmitted. I suggest to always post all patches >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Fujita-san, amul, CFbot complains that Fujita-san submitted a patch that doesn't apply, which makes sense since the necessary previous patch was only referred to without being resubmitted. I suggest to always post all patches together with each resubmission so that it can be checked

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Fujita San, Please find my comments inline below: On Wed, Aug 28, 2019 at 3:52 PM Etsuro Fujita wrote: > On Fri, Aug 16, 2019 at 10:25 PM Etsuro Fujita > wrote: > > [... skipped ..] > > About the attached: > > * The attached patch modified try_partitionwise_join() so that we call >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Aug 16, 2019 at 10:25 PM Etsuro Fujita wrote: > It seems that I performed the above tests on an assertion-enabled > build. :( So I executed the tests one more time. Here are the > results. > > * 2-way self-join of pt: explain analyze select * from pt t0, pt t1 > where t0.a = t1.a; > -

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Thank you Fujita San for the enhancement, will have a look. Regards, Amul On Wed, Aug 28, 2019 at 3:52 PM Etsuro Fujita wrote: > On Fri, Aug 16, 2019 at 10:25 PM Etsuro Fujita > wrote: > > It seems that I performed the above tests on an assertion-enabled > > build. :( So I executed the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Jul 30, 2019 at 6:00 PM Etsuro Fujita wrote: > On Fri, Jul 19, 2019 at 10:44 PM Robert Haas wrote: > > On Thu, Jul 18, 2019 at 2:55 AM Etsuro Fujita > > wrote: > > > I.e., partition_bounds_merge() is performed for each pair of input > > > partitioned relations for a join relation in

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Amit-san, On Wed, Jul 31, 2019 at 2:47 PM Amit Langote wrote: > On Tue, Jul 30, 2019 at 6:00 PM Etsuro Fujita wrote: > > On Fri, Jul 19, 2019 at 10:44 PM Robert Haas wrote: > > > I don't know whether partition_bounds_merge() is well-implemented; I > > > haven't looked. > > > > My concern about

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Jul 30, 2019 at 6:00 PM Etsuro Fujita wrote: > On Fri, Jul 19, 2019 at 10:44 PM Robert Haas wrote: > > On Thu, Jul 18, 2019 at 2:55 AM Etsuro Fujita > > wrote: > > > I.e., partition_bounds_merge() is performed for each pair of input > > > partitioned relations for a join relation in

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Jul 19, 2019 at 10:44 PM Robert Haas wrote: > On Thu, Jul 18, 2019 at 2:55 AM Etsuro Fujita wrote: > > I.e., partition_bounds_merge() is performed for each pair of input > > partitioned relations for a join relation in try_partitionwise_join(). > > Since partition_bounds_merge() would

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Jul 19, 2019 at 8:09 PM amul sul wrote: > On Mon, Jul 8, 2019 at 5:03 PM Etsuro Fujita wrote: >> I started reviewing this. Here is my initial review comments: >> >> * 0001-Hash-partition-bound-equality-refactoring-v22.patch >> However, I don't think it's a good idea to do this >>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Jul 18, 2019 at 2:55 AM Etsuro Fujita wrote: > I.e., partition_bounds_merge() is performed for each pair of input > partitioned relations for a join relation in try_partitionwise_join(). > Since partition_bounds_merge() would need a lot of CPU cycles, I don't > think this is acceptable;

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Jul 8, 2019 at 5:03 PM Etsuro Fujita wrote: > On Wed, Jul 3, 2019 at 3:44 PM Etsuro Fujita > wrote: > > On Tue, Jul 2, 2019 at 1:47 PM amul sul wrote: > > > Attached version is rebase atop of the latest master head(c74d49d41c), > thanks. > > > > Thanks! Will review. > > I started

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Jul 8, 2019 at 8:33 PM Etsuro Fujita wrote: > I'll review the remaining parts (ie, > "0002-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi-v22.patch" > and "0003-Tests-for-0-1-1-1-and-1-0-partition-matching-v22.patch") > closely next. I've been reviewing the main patch

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Jul 2, 2019 at 1:47 PM amul sul wrote: > Attached version is rebase atop of the latest master head(c74d49d41c), thanks. Thanks! Will review. Best regards, Etsuro Fujita

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Jul 1, 2019 at 6:50 PM Thomas Munro wrote: > On Sat, May 18, 2019 at 12:20 AM Robert Haas wrote: > > On Tue, May 14, 2019 at 12:24 AM Amit Langote > > wrote: > > > He did mention that cases where the nullable side is provably empty can be > > > handled by simply returning the path of

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Sat, May 18, 2019 at 12:20 AM Robert Haas wrote: > On Tue, May 14, 2019 at 12:24 AM Amit Langote > wrote: > > He did mention that cases where the nullable side is provably empty can be > > handled by simply returning the path of the non-nullable side with > > suitable projection path added on

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, May 14, 2019 at 12:24 AM Amit Langote wrote: > He did mention that cases where the nullable side is provably empty can be > handled by simply returning the path of the non-nullable side with > suitable projection path added on top to emit NULLs for the columns of the > nullable-side. If

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, May 14, 2019 at 10:00 AM Amit Langote wrote: > On 2019/05/14 13:23, Amit Langote wrote: > > Tom > > strongly objected to that idea saying that such join paths are kind of > > silly [1], even outside the context of partitionwise join. He suggested > > that we abandon partitionwise join

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On 2019/05/14 13:23, Amit Langote wrote: > Tom > strongly objected to that idea saying that such join paths are kind of > silly [1], even outside the context of partitionwise join. He suggested > that we abandon partitionwise join in such cases, because having to build > a dummy base relation for

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Amul, Ashutosh, On 2019/04/24 20:26, amul sul wrote: > Attached version is rebase atop of the latest master head(fdc7efcc30), also > incorporates the Ashutosh's suggestion, thanks. Reading the commit message of 0002 and after testing to confirm, I understand that the patch doesn't handle

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Apr 24, 2019 at 4:56 PM amul sul wrote: > Attached version is rebase atop of the latest master head(fdc7efcc30), also > incorporates the Ashutosh's suggestion, thanks. > Thanks for rebase patch, patches applied cleanly on PG head. I did some crash testing with extra test case [0006

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Mar 11, 2019 at 10:40 AM amul sul wrote: > > All the places from where this handle_missing_partition() get called > have the following code to decide the value for missing_side_outer/_inner > which > I yet to understand. Do you think this has some flaw? > > /* > * For a

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Mar 11, 2019 at 8:29 AM Ashutosh Bapat wrote: > On Thu, Mar 7, 2019 at 8:20 PM amul sul wrote: > >> >> >> On Thu, Mar 7, 2019 at 1:02 PM amul sul wrote: >> >>> Thanks Rajkumar, >>> >>> I am looking into this. >>> >>> >> The crash happens when none of the if-else branch of >>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Mar 7, 2019 at 8:20 PM amul sul wrote: > > > On Thu, Mar 7, 2019 at 1:02 PM amul sul wrote: > >> Thanks Rajkumar, >> >> I am looking into this. >> >> > The crash happens when none of the if-else branch of > handle_missing_partition() > evaluates and returns merged_index unassigned. > >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Thanks Rajkumar, I am looking into this. Regards, Amul On Thu, Mar 7, 2019 at 11:54 AM Rajkumar Raghuwanshi < rajkumar.raghuwan...@enterprisedb.com> wrote: > > > On Tue, Mar 5, 2019 at 3:45 PM amul sul wrote: > >> Attached is the rebased atop of the latest master head(35bc0ec7c8). >> > thanks

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Mar 5, 2019 at 3:45 PM amul sul wrote: > Attached is the rebased atop of the latest master head(35bc0ec7c8). > thanks Amul, patches applied cleanly on PG head. While testing this I got a server crash with below test case. CREATE TABLE plt1 (a int, b int, c varchar) PARTITION BY

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

(2019/01/21 20:56), amul sul wrote: On Mon, Nov 26, 2018 at 9:33 PM Dmitry Dolgov <9erthali...@gmail.com > wrote: > On Mon, Nov 26, 2018 at 1:41 PM Dmitry Dolgov <9erthali...@gmail.com > wrote: > > Thanks, I've

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Nov 26, 2018 at 9:33 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Mon, Nov 26, 2018 at 1:41 PM Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > > Thanks, I've messed this up too - rebased a wrong branch, the correct one > > doesn't have this code already. Sorry for that, and

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Mon, Nov 26, 2018 at 9:03 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I've noticed, that this patch set is outdated, so here is the rebased version. This turned red on cfbot because I turned on -Werror: partbounds.c: In function ‘partition_bounds_merge’: partbounds.c:3619:43: error:

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Sep 13, 2018 at 1:45 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Fri, 31 Aug 2018 at 08:23, Ashutosh Bapat < > ashutosh.ba...@enterprisedb.com> wrote: > > > > On Thu, Aug 30, 2018 at 2:23 PM, Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > > > >> I won't be working on this

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Aug 30, 2018 at 2:23 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >> I won't be working on this actively in the next commitfest. I will be >> glad if somebody else wants to take this up. If there's nobody, >> probably we should mark this entry as "returned with feedback" in the >>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Wed, 29 Aug 2018 at 09:32, Ashutosh Bapat > wrote: > > > * Many functions carry some unrelated arguments just to pass them through, > > which obscures the purpose of a function. > > Can you please provide some examples? E.g this chain with partsupfunc & collations:

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Aug 23, 2018 at 4:01 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Fri, 27 Jul 2018 at 20:13, Robert Haas wrote: >> >> On Fri, Jul 27, 2018 at 3:17 AM, Ashutosh Bapat >> wrote: >> > Apart from the complexity there's also a possibility that this >> > skipping will reduce the

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Fri, 27 Jul 2018 at 20:13, Robert Haas wrote: > > On Fri, Jul 27, 2018 at 3:17 AM, Ashutosh Bapat > wrote: > > Apart from the complexity there's also a possibility that this > > skipping will reduce the efficiency actually in normal cases. Consider > > a case where A and B have exactly

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Jul 27, 2018 at 3:17 AM, Ashutosh Bapat wrote: > Apart from the complexity there's also a possibility that this > skipping will reduce the efficiency actually in normal cases. Consider > a case where A and B have exactly matching partitions. Current > partition matching algorithm compare

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Jul 26, 2018 at 8:37 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Mon, 23 Jul 2018 at 10:38, Ashutosh Bapat >> wrote: >> >> On Fri, Jul 20, 2018 at 3:13 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> > >> > It's of course wrong, it's going to be O(max(m, n)) as you said,

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Mon, 23 Jul 2018 at 10:38, Ashutosh Bapat > wrote: > > On Fri, Jul 20, 2018 at 3:13 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > It's of course wrong, it's going to be O(max(m, n)) as you said, but > > the point is still valid - if we have partitions A1, A2 from one side > > and

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Jul 20, 2018 at 3:13 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > It's of course wrong, it's going to be O(max(m, n)) as you said, but > the point is still valid - if we have partitions A1, A2 from one side > and B1, ..., BN on another side, we can skip necessary the > partitions

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Thu, 19 Jul 2018 at 21:04, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > * Just to clarify - the iterating through all the partitions, is it the > > > best > > > way of finding matching ranges? Correct me if I'm wrong, but from what > > > I see > > > in the comments for

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Tue, 17 Jul 2018 at 11:58, Ashutosh Bapat > wrote: > > On Sun, Jul 15, 2018 at 11:13 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >> On Thu, 28 Jun 2018 at 07:54, Amit Langote > >> wrote: > >> > >> On 2018/06/27 22:21, Ashutosh Bapat wrote: > >> > On Wed, Jun 27, 2018 at 12:28 PM,

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Sun, Jul 15, 2018 at 1:43 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Partitions: test11 FOR VALUES FROM (0) TO (100), > test12 FOR VALUES FROM (100) TO (200), > test13 FOR VALUES FROM (200) TO (300) > > Partitions: test21 FOR VALUES FROM (10) TO (110), >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On Thu, 28 Jun 2018 at 07:54, Amit Langote > wrote: > > On 2018/06/27 22:21, Ashutosh Bapat wrote: > > On Wed, Jun 27, 2018 at 12:28 PM, Amit Langote > >> Ah, okay. I thought of reporting this because I felt the errors may have > >> to do with changes to the related code in HEAD between May

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On 2018/06/27 22:21, Ashutosh Bapat wrote: > On Wed, Jun 27, 2018 at 12:28 PM, Amit Langote >> Ah, okay. I thought of reporting this because I felt the errors may have >> to do with changes to the related code in HEAD between May 14 when you >> last posted the patches and today that you may need

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Jun 27, 2018 at 12:28 PM, Amit Langote wrote: > On 2018/06/26 18:02, Ashutosh Bapat wrote: >> On Tue, Jun 26, 2018 at 2:27 PM, Amit Langote >> wrote: >>> Hi Ashutosh, >>> >>> On 2018/05/14 20:14, Ashutosh Bapat wrote: 0001-Hash-partition-bound-equality-refactoring.patch

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On 2018/06/26 18:02, Ashutosh Bapat wrote: > On Tue, Jun 26, 2018 at 2:27 PM, Amit Langote > wrote: >> Hi Ashutosh, >> >> On 2018/05/14 20:14, Ashutosh Bapat wrote: >>> 0001-Hash-partition-bound-equality-refactoring.patch >>> 0002-Targetlist-of-a-child-join-is-produced-by-translatin.patch >>>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Tue, Jun 26, 2018 at 2:27 PM, Amit Langote wrote: > Hi Ashutosh, > > On 2018/05/14 20:14, Ashutosh Bapat wrote: >> 0001-Hash-partition-bound-equality-refactoring.patch >> 0002-Targetlist-of-a-child-join-is-produced-by-translatin.patch >>

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Ashutosh, On 2018/05/14 20:14, Ashutosh Bapat wrote: > 0001-Hash-partition-bound-equality-refactoring.patch > 0002-Targetlist-of-a-child-join-is-produced-by-translatin.patch > 0003-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi.patch >

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On 3 April 2018 at 15:34, Ashutosh Bapat > wrote: > On Fri, Mar 30, 2018 at 7:36 PM, Ashutosh Bapat > wrote: >> >> I am working on commenting portions of the code to make it more clear >> and readable. Will update the patches

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Thanks for your review comments. They are helpful. On Thu, Mar 29, 2018 at 2:53 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > But actually you also mentioned > another topic that bothers me about this patch. Different parts of the > algorithm implementation (at least for functions that

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On 22 March 2018 at 14:18, Ashutosh Bapat > wrote: > On Thu, Mar 22, 2018 at 4:32 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >>> On 12 March 2018 at 06:00, Ashutosh Bapat >>> wrote: >>> Thanks for the note. Here are

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Thu, Mar 22, 2018 at 4:32 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On 12 March 2018 at 06:00, Ashutosh Bapat >> wrote: >> Thanks for the note. Here are rebased patches. > > Since I started to look at this patch, I can share few random notes

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On 12 March 2018 at 06:00, Ashutosh Bapat > wrote: > Thanks for the note. Here are rebased patches. Since I started to look at this patch, I can share few random notes (although it's not a complete review, I'm in the progress now), maybe it can be helpful. In

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

> On 26 February 2018 at 11:03, Ashutosh Bapat > wrote: > On Fri, Feb 23, 2018 at 7:35 PM, Robert Haas wrote: >> On Fri, Feb 16, 2018 at 12:14 AM, Ashutosh Bapat >> wrote: >>> Appreciate you taking time

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Feb 16, 2018 at 12:14 AM, Ashutosh Bapat wrote: > Appreciate you taking time for review. > > PFA updated version. Committed 0001. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

Hi Ashutosh. On 2018/02/09 14:27, Ashutosh Bapat wrote: > Here's updated patch set with those comments added. I looked at patches 0002 and 0003. In 0002: + * In case of hash partition we store modulus and remainder in datums array In case of hash partitioned table? + * which has the same

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Fri, Feb 9, 2018 at 11:26 AM, Amit Langote wrote: > On 2018/02/09 14:31, Ashutosh Bapat wrote: >>> I also noticed that a later patch adds partsupfunc to PartitionScheme, >>> which the pruning patch needs too. So, perhaps would be nice to take out >>> that

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On 2018/02/09 14:31, Ashutosh Bapat wrote: >> I also noticed that a later patch adds partsupfunc to PartitionScheme, >> which the pruning patch needs too. So, perhaps would be nice to take out >> that portion of the patch. That is, the changes to PartitionScheme struct >> definition and those to

  1   2   >