Re: A problem about partitionwise join

2024-08-12 Thread Ashutosh Bapat
On Mon, Aug 12, 2024 at 8:50 AM Richard Guo wrote: > > On Sat, Aug 10, 2024 at 6:22 AM Alexey Dvoichenkov > wrote: > > I haven't read the entire thread so I might be missing something, but > > one interesting consequence of this patch is that it kind of breaks > > the initial pruning of generic p

Re: A problem about partitionwise join

2024-08-11 Thread Richard Guo
On Mon, Mar 25, 2024 at 7:09 PM Ashutosh Bapat wrote: > I think we need some way to avoid two different ways of looking up partition > keys - if we can't teach the EC machinery to produce clauses with partition > keys (always), we need to teach EC to contain partition keys in case of outer > jo

Re: A problem about partitionwise join

2024-08-11 Thread Richard Guo
On Sat, Aug 10, 2024 at 6:22 AM Alexey Dvoichenkov wrote: > I haven't read the entire thread so I might be missing something, but > one interesting consequence of this patch is that it kind of breaks > the initial pruning of generic plans. Given a query such as SELECT > ... WHERE A.PK = B.PK AND A

Re: A problem about partitionwise join

2024-07-30 Thread Richard Guo
On Wed, May 8, 2024 at 5:01 PM Richard Guo wrote: > Below is what I got on my local machine. > > -- on master > > measurement | average | maximum | minimum | std_dev | > std_dev_as_perc_of_avg > ---+--+--+--+-+ > planning

Re: A problem about partitionwise join

2024-05-08 Thread Richard Guo
On Fri, May 3, 2024 at 9:31 PM Robert Haas wrote: > On Fri, May 3, 2024 at 7:47 AM Richard Guo wrote: > > I think one concern regarding performance cost is that the function > > exprs_known_equal() would be called O(N^2) times, where N is the number > > of partition key expressions. But I think

Re: A problem about partitionwise join

2024-05-03 Thread Robert Haas
On Fri, May 3, 2024 at 7:47 AM Richard Guo wrote: >> Does the additional logic added by this patch have a noticeable >> performance cost? > > I think one concern regarding performance cost is that the function > exprs_known_equal() would be called O(N^2) times, where N is the number > of partition

Re: A problem about partitionwise join

2024-05-03 Thread Richard Guo
On Wed, May 1, 2024 at 1:31 AM Robert Haas wrote: > I think it's slightly questionable whether this patch is worthwhile. > The case memorialized in the regression tests, t1.a = t2.a AND t1.a = > t2.b, is a very weird thing to do. The case mentioned in the original > email, foo.k1 = bar.k1 and foo

Re: A problem about partitionwise join

2024-04-30 Thread Robert Haas
On Wed, Feb 21, 2024 at 6:25 AM Richard Guo wrote: > This patch was returned due to 'lack of interest'. However, upon > verification, it appears that the reported issue still exists, and the > proposed fix in the thread remains valid. Hence, resurrect this patch > after rebasing it on master. I

Re: A problem about partitionwise join

2024-03-25 Thread Ashutosh Bapat
On Mon, Mar 25, 2024 at 9:01 AM Richard Guo wrote: > > create table p (k1 int, k2 int, val int) partition by range(k1, k2); > create table p_1 partition of p for values from (1,1) to (10,100); > create table p_2 partition of p for values from (10,100) to (20,200); > > set enable_partitionwise_joi

Re: A problem about partitionwise join

2024-03-24 Thread Richard Guo
On Tue, Mar 19, 2024 at 3:40 PM Ashutosh Bapat wrote: > On Tue, Mar 19, 2024 at 8:18 AM Richard Guo > wrote: > >> On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat < >> ashutosh.bapat@gmail.com> wrote: >> >>> Approach >>> >>> The equijoin condition between partition keys doesn't appear

Re: A problem about partitionwise join

2024-03-19 Thread Ashutosh Bapat
On Tue, Mar 19, 2024 at 8:18 AM Richard Guo wrote: > (Sorry it takes me some time to get back to this thread.) > > On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> I did a deeper review of the patch. Here are some comments >> > > Thank you for the review

Re: A problem about partitionwise join

2024-03-18 Thread Richard Guo
(Sorry it takes me some time to get back to this thread.) On Thu, Mar 7, 2024 at 7:13 PM Ashutosh Bapat wrote: > I did a deeper review of the patch. Here are some comments > Thank you for the review! > Approach > > The equijoin condition between partition keys doesn't appear in the j

Re: A problem about partitionwise join

2024-03-07 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 2:56 PM Ashutosh Bapat wrote: > On Wed, Feb 21, 2024 at 4:55 PM Richard Guo > wrote: > > > > > > On Tue, Aug 2, 2022 at 4:24 AM Jacob Champion > wrote: > >> > >> Once you think you've built up some community support and the patchset > >> is ready for review, you (or any

Re: A problem about partitionwise join

2024-02-22 Thread Ashutosh Bapat
On Wed, Feb 21, 2024 at 4:55 PM Richard Guo wrote: > > > On Tue, Aug 2, 2022 at 4:24 AM Jacob Champion wrote: >> >> Once you think you've built up some community support and the patchset >> is ready for review, you (or any interested party) can resurrect the >> patch entry by visiting >> >> h

Re: A problem about partitionwise join

2024-02-21 Thread Richard Guo
On Tue, Aug 2, 2022 at 4:24 AM Jacob Champion wrote: > Once you think you've built up some community support and the patchset > is ready for review, you (or any interested party) can resurrect the > patch entry by visiting > > https://commitfest.postgresql.org/38/2266/ > > and changing the st

Re: A problem about partitionwise join

2022-08-01 Thread Jacob Champion
As discussed in [1], we're taking this opportunity to return some patchsets that don't appear to be getting enough reviewer interest. This is not a rejection, since we don't necessarily think there's anything unacceptable about the entry, but it differs from a standard "Returned with Feedback" in

Re: A problem about partitionwise join

2022-04-25 Thread Richard Guo
On Mon, Nov 22, 2021 at 3:04 PM Richard Guo wrote: > > The suggested changes have already been included in v5 patch. Sorry for > the confusion. > > Verified that the patch still applies and works on latest master. So I'm > moving it to the next CF (which is Commitfest 2022-01). Please correct > m

Re: A problem about partitionwise join

2021-11-21 Thread Richard Guo
On Wed, Oct 6, 2021 at 1:19 AM Jaime Casanova wrote: > On Wed, Jul 21, 2021 at 04:44:53PM +0800, Richard Guo wrote: > > On Fri, Nov 27, 2020 at 8:05 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> > > wrote: > > > > > > > > In the example you gave earlier, the equi join on partition key was >

Re: A problem about partitionwise join

2021-10-05 Thread Jaime Casanova
On Wed, Jul 21, 2021 at 04:44:53PM +0800, Richard Guo wrote: > On Fri, Nov 27, 2020 at 8:05 PM Ashutosh Bapat > wrote: > > > > > In the example you gave earlier, the equi join on partition key was > > there but it was replaced by individual constant assignment clauses. > > So if we keep the origi

Re: A problem about partitionwise join

2021-07-21 Thread Richard Guo
On Fri, Nov 27, 2020 at 8:05 PM Ashutosh Bapat wrote: > On Tue, Nov 10, 2020 at 2:43 PM Richard Guo > wrote: > > Thanks Anastasia. I've rebased the patch with latest master. > > > > To recap, the problem we are fixing here is when generating join clauses > > from equivalence classes, we only sel

Re: A problem about partitionwise join

2021-03-09 Thread David Steele
On 11/27/20 7:05 AM, Ashutosh Bapat wrote: On Tue, Nov 10, 2020 at 2:43 PM Richard Guo wrote: To recap, the problem we are fixing here is when generating join clauses from equivalence classes, we only select the joinclause with the 'best score', or the first joinclause with a score of 3. This

Re: A problem about partitionwise join

2020-11-27 Thread Ashutosh Bapat
On Tue, Nov 10, 2020 at 2:43 PM Richard Guo wrote: > > > On Fri, Nov 6, 2020 at 11:26 PM Anastasia Lubennikova > wrote: >> >> Status update for a commitfest entry. >> >> According to CFbot this patch fails to apply. Richard, can you send an >> update, please? >> >> Also, I see that the thread w

Re: A problem about partitionwise join

2020-11-10 Thread Richard Guo
On Fri, Nov 6, 2020 at 11:26 PM Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > Status update for a commitfest entry. > > According to CFbot this patch fails to apply. Richard, can you send an > update, please? > > Also, I see that the thread was inactive for a while. > Are you goin

Re: A problem about partitionwise join

2020-11-06 Thread Anastasia Lubennikova
Status update for a commitfest entry. According to CFbot this patch fails to apply. Richard, can you send an update, please? Also, I see that the thread was inactive for a while. Are you going to continue this work? I think it would be helpful, if you could write a short recap about current sta

Re: A problem about partitionwise join

2020-04-09 Thread Ashutosh Bapat
> > I think it would not work for outer joins if we only check > exprs_known_equal() for equivalences. If the equi-join conditions > involving pairs of matching partition keys are outer join quals > mentioning nonnullable side rels, they would not exist in any EC > according to the current EC infra

Re: A problem about partitionwise join

2020-04-08 Thread Tom Lane
Richard Guo writes: > On Thu, Apr 9, 2020 at 1:07 AM Tom Lane wrote: >> I have hopes of being able to incorporate outer >> joins into the EC logic in a less squishy way in the future, by making >> the representation of Vars distinguish explicitly between >> value-before-outer-join and value-after

Re: A problem about partitionwise join

2020-04-08 Thread Richard Guo
On Thu, Apr 9, 2020 at 1:07 AM Tom Lane wrote: > Richard Guo writes: > > On Sun, Apr 5, 2020 at 4:38 AM Tom Lane wrote: > >> There is already something in equivclass.c that would almost do what > >> we want here: exprs_known_equal() would tell us whether the partkeys > >> can be found in the sa

Re: A problem about partitionwise join

2020-04-08 Thread Tom Lane
Richard Guo writes: > On Sun, Apr 5, 2020 at 4:38 AM Tom Lane wrote: >> There is already something in equivclass.c that would almost do what >> we want here: exprs_known_equal() would tell us whether the partkeys >> can be found in the same eclass, without having to generate data >> structures al

Re: A problem about partitionwise join

2020-04-07 Thread Richard Guo
On Sun, Apr 5, 2020 at 4:38 AM Tom Lane wrote: > Richard Guo writes: > > Rebased the patch with latest master and also addressed the test case > > failure reported by PostgreSQL Patch Tester. > > I looked this patch over, but I don't like it too much: it seems very > brute-force (and badly under

Re: A problem about partitionwise join

2020-04-04 Thread Tom Lane
Richard Guo writes: > Rebased the patch with latest master and also addressed the test case > failure reported by PostgreSQL Patch Tester. I looked this patch over, but I don't like it too much: it seems very brute-force (and badly under-commented). Creating all those extra RestrictInfos isn't t

Re: A problem about partitionwise join

2020-01-18 Thread Richard Guo
Rebased the patch with latest master and also addressed the test case failure reported by PostgreSQL Patch Tester. Thanks Richard On Fri, Nov 29, 2019 at 11:35 AM Etsuro Fujita wrote: > On Fri, Nov 29, 2019 at 12:08 PM Richard Guo wrote: > > On Fri, Nov 29, 2019 at 11:03 AM Michael Paquier >

Re: A problem about partitionwise join

2019-11-28 Thread Etsuro Fujita
On Fri, Nov 29, 2019 at 12:08 PM Richard Guo wrote: > On Fri, Nov 29, 2019 at 11:03 AM Michael Paquier wrote: >> On Tue, Nov 26, 2019 at 08:35:33PM +0900, Etsuro Fujita wrote: >> > I've just started reviewing this patch. One comment I have for now >> > is: this is categorized into Bug Fixes, but

Re: A problem about partitionwise join

2019-11-28 Thread Richard Guo
On Fri, Nov 29, 2019 at 11:03 AM Michael Paquier wrote: > On Tue, Nov 26, 2019 at 08:35:33PM +0900, Etsuro Fujita wrote: > > I've just started reviewing this patch. One comment I have for now > > is: this is categorized into Bug Fixes, but we have a workaround at > > least to the regression test

Re: A problem about partitionwise join

2019-11-28 Thread Michael Paquier
On Tue, Nov 26, 2019 at 08:35:33PM +0900, Etsuro Fujita wrote: > I've just started reviewing this patch. One comment I have for now > is: this is categorized into Bug Fixes, but we have a workaround at > least to the regression test case in the patch (ie, just reorder join > clauses), so this seem

Re: A problem about partitionwise join

2019-11-26 Thread Etsuro Fujita
Hi Richard, On Fri, Aug 30, 2019 at 3:08 AM Etsuro Fujita wrote: > On Thu, Aug 29, 2019 at 6:45 PM Richard Guo wrote: > > Attached is a patch as an attempt to address this issue. The idea is > > quite straightforward. When building partition info for joinrel, we > > generate any possible EC-deri

Re: A problem about partitionwise join

2019-09-20 Thread Dilip Kumar
On Fri, Sep 20, 2019 at 2:33 PM Richard Guo wrote: > > Hi Dilip, > > Thank you for reviewing this patch. > > On Fri, Sep 20, 2019 at 12:48 PM Dilip Kumar wrote: >> >> On Thu, Aug 29, 2019 at 3:15 PM Richard Guo wrote: >> > >> > >> > Attached is a patch as an attempt to address this issue. The id

Re: A problem about partitionwise join

2019-09-20 Thread Richard Guo
Hi Dilip, Thank you for reviewing this patch. On Fri, Sep 20, 2019 at 12:48 PM Dilip Kumar wrote: > On Thu, Aug 29, 2019 at 3:15 PM Richard Guo wrote: > > > > > > Attached is a patch as an attempt to address this issue. The idea is > > quite straightforward. When building partition info for jo

Re: A problem about partitionwise join

2019-09-19 Thread Dilip Kumar
On Thu, Aug 29, 2019 at 3:15 PM Richard Guo wrote: > > > Attached is a patch as an attempt to address this issue. The idea is > quite straightforward. When building partition info for joinrel, we > generate any possible EC-derived joinclauses of form 'outer_em = > inner_em', which will be used tog

Re: A problem about partitionwise join

2019-09-10 Thread Richard Guo
Hi Alvaro, Thank you for reviewing this patch. On Wed, Sep 11, 2019 at 4:48 AM Alvaro Herrera from 2ndQuadrant < alvhe...@alvh.no-ip.org> wrote: > So in this patch, the input restrictlist is modified to include the > clauses generated by generate_join_implied_equalities_for_all. That > doesn't

Re: A problem about partitionwise join

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
So in this patch, the input restrictlist is modified to include the clauses generated by generate_join_implied_equalities_for_all. That doesn't seem okay -- doesn't it affect downstream usage of the restrictlist in the caller of set_joinrel_size_estimates? I wonder if it's possible to do this by

Re: A problem about partitionwise join

2019-08-29 Thread Etsuro Fujita
On Fri, Aug 30, 2019 at 12:15 PM Richard Guo wrote: > On Fri, Aug 30, 2019 at 2:08 AM Etsuro Fujita wrote: >> On Thu, Aug 29, 2019 at 6:45 PM Richard Guo wrote: >> > Attached is a patch as an attempt to address this issue. The idea is >> > quite straightforward. When building partition info for

Re: A problem about partitionwise join

2019-08-29 Thread Richard Guo
On Fri, Aug 30, 2019 at 2:08 AM Etsuro Fujita wrote: > On Thu, Aug 29, 2019 at 6:45 PM Richard Guo wrote: > > On Wed, Aug 28, 2019 at 6:49 PM Etsuro Fujita > wrote: > >> On Tue, Aug 27, 2019 at 4:57 PM Richard Guo wrote: > >> > Check the query below as a more illustrative example: > >> > > >>

Re: A problem about partitionwise join

2019-08-29 Thread Etsuro Fujita
On Thu, Aug 29, 2019 at 6:45 PM Richard Guo wrote: > On Wed, Aug 28, 2019 at 6:49 PM Etsuro Fujita wrote: >> On Tue, Aug 27, 2019 at 4:57 PM Richard Guo wrote: >> > Check the query below as a more illustrative example: >> > >> > create table p (k int, val int) partition by range(k); >> > create

Re: A problem about partitionwise join

2019-08-29 Thread Richard Guo
On Wed, Aug 28, 2019 at 6:49 PM Etsuro Fujita wrote: > Hi, > > On Tue, Aug 27, 2019 at 4:57 PM Richard Guo wrote: > > Check the query below as a more illustrative example: > > > > create table p (k int, val int) partition by range(k); > > create table p_1 partition of p for values from (1) to (1

Re: A problem about partitionwise join

2019-08-28 Thread Etsuro Fujita
Hi, On Tue, Aug 27, 2019 at 4:57 PM Richard Guo wrote: > Check the query below as a more illustrative example: > > create table p (k int, val int) partition by range(k); > create table p_1 partition of p for values from (1) to (10); > create table p_2 partition of p for values from (10) to (100);

Re: A problem about partitionwise join

2019-08-27 Thread Richard Guo
On Tue, Aug 27, 2019 at 8:51 AM Amit Langote wrote: > Hi Richard, > > On Mon, Aug 26, 2019 at 6:33 PM Richard Guo wrote: > > > > Hi All, > > > > To generate partitionwise join, we need to make sure there exists an > > equi-join condition for each pair of partition keys, which is performed > > by

Re: A problem about partitionwise join

2019-08-26 Thread Amit Langote
Hi Richard, On Mon, Aug 26, 2019 at 6:33 PM Richard Guo wrote: > > Hi All, > > To generate partitionwise join, we need to make sure there exists an > equi-join condition for each pair of partition keys, which is performed > by have_partkey_equi_join(). This makes sense and works well. > > But if,

A problem about partitionwise join

2019-08-26 Thread Richard Guo
Hi All, To generate partitionwise join, we need to make sure there exists an equi-join condition for each pair of partition keys, which is performed by have_partkey_equi_join(). This makes sense and works well. But if, let's say, one certain pair of partition keys (foo.k = bar.k) has formed an eq