Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2022-03-31 Thread Greg Stark
On Mon, 22 Nov 2021 at 02:14, Andy Fan wrote: > > _I_ think there is no way to bypass that. I guess Tom has a bigger plan on > Var (not only for notnull), but no time to invest in them so far. If > that is the case, > personally I think we can go ahead with my method first and continue the >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-11-21 Thread Andy Fan
Hi Dmitry: On Wed, Nov 17, 2021 at 11:20 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Wed, Jul 07, 2021 at 01:20:24PM +1200, David Rowley wrote: > > On Wed, 7 Jul 2021 at 13:04, Andy Fan wrote: > > > Looking forward to watching this change closely, thank you both David and > > >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-11-17 Thread Dmitry Dolgov
> On Wed, Jul 07, 2021 at 01:20:24PM +1200, David Rowley wrote: > On Wed, 7 Jul 2021 at 13:04, Andy Fan wrote: > > Looking forward to watching this change closely, thank you both David and > > Tom! > > But I still don't understand what the faults my way have , do you mind > > telling the > >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-11-08 Thread Andy Fan
Unknown why people have so little interest in this, AFAICS, there are more great usages of UniqueKey rather than the 'marking-distinct-as-noop'. The most exciting usage for me is it is helpful for JoinRel's pathkey. Take an example of this: SELECT .. FROM t1 JOIN t2 ON t1.any_column =

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-30 Thread Andy Fan
On Tue, Jul 6, 2021 at 9:14 PM Tom Lane wrote: > > David Rowley writes: > > Tom, I'm wondering if you might get a chance to draw up a design for > > what you've got in mind with this? I assume adding a new field in > > Var, but I'm drawing a few blanks on how things might work for equal() > >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-18 Thread Andy Fan
Hi: patch v4 fixed the 2 bugs Zhihong reported. Any feedback is welcome. v4-0005-Support-UniqueKey-on-JoinRel.patch Description: Binary data v4-0002-Just-some-utils-functions.patch Description: Binary data v4-0004-Support-UniqueKey-on-BaseRel.patch Description: Binary data

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-15 Thread Andy Fan
Hi Zhihong, On Mon, Aug 16, 2021 at 12:35 AM Zhihong Yu wrote: > > > > On Sun, Aug 15, 2021 at 7:33 AM Andy Fan wrote: >> >> Hi: >> >> I have finished the parts for baserel, joinrel, subquery, distinctrel. I >> think >> the hardest ones have been verified. Here is the design overview. >> >>

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-15 Thread Zhihong Yu
On Sun, Aug 15, 2021 at 7:33 AM Andy Fan wrote: > Hi: > > I have finished the parts for baserel, joinrel, subquery, distinctrel. I > think > the hardest ones have been verified. Here is the design overview. > > 1. Use EC instead of expr to cover more UniqueKey cases. > 2. Redesign the UniqueKey

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-08-15 Thread Andy Fan
Hi: I have finished the parts for baserel, joinrel, subquery, distinctrel. I think the hardest ones have been verified. Here is the design overview. 1. Use EC instead of expr to cover more UniqueKey cases. 2. Redesign the UniqueKey as below: @@ -246,6 +246,7 @@ struct PlannerInfo List

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-15 Thread Andy Fan
On Tue, Jul 13, 2021 at 5:55 PM Andy Fan wrote: > > > 4. Cut the useless UniqueKey totally on the baserel stage based on > >root->distinct_pathkey. If we want to use it anywhere else, I think this > >design is OK as well. for example: group by UniqueKey. > > > > The intention of this is

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-13 Thread Andy Fan
> 4. Cut the useless UniqueKey totally on the baserel stage based on >root->distinct_pathkey. If we want to use it anywhere else, I think this >design is OK as well. for example: group by UniqueKey. > The intention of this is I want to cut off the useless UniqueKey ASAP. In the previous

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-07 Thread Ranier Vilela
>You can check the more details in the attached patch. Any feedback is welcome. I have tiny comments about your patch: 1. name of file is uniquekey.c? + * pathkeys.c + * Utilities for maintaining uniquekey. 2. Variable "PathKey *pathkey" at function: add_uniquekey_for_uniqueindex, can have

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-07 Thread Andy Fan
On Tue, Jul 6, 2021 at 5:34 PM David Rowley wrote: > > On Sun, 4 Jul 2021 at 02:08, Andy Fan wrote: > > I'd start to work on UniqueKey again, it would be great that we can target > > it > > to PG 15. The attached patch is just for the notnull_attrs. Since we can't > > say > > a column is

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread Andy Fan
> > > > For example: SELECT nullablecol FROM tab WHERE nullablecol = ; > > If the equality operator is strict then the nullablecol can be NULL in > the WHERE clause but not in the SELECT list. Tom's idea should allow > us to determine both of those things but your idea cannot tell them > apart,

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread David Rowley
On Wed, 7 Jul 2021 at 13:04, Andy Fan wrote: > Looking forward to watching this change closely, thank you both David and Tom! > But I still don't understand what the faults my way have , do you mind > telling the > details? The problem is that we don't need 6 different ways to determine if a

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread Andy Fan
On Tue, Jul 6, 2021 at 9:14 PM Tom Lane wrote: > David Rowley writes: > > Tom, I'm wondering if you might get a chance to draw up a design for > > what you've got in mind with this? I assume adding a new field in > > Var, but I'm drawing a few blanks on how things might work for equal() > >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread Tom Lane
David Rowley writes: > Tom, I'm wondering if you might get a chance to draw up a design for > what you've got in mind with this? I assume adding a new field in > Var, but I'm drawing a few blanks on how things might work for equal() > when one Var has the field set and another does not. As I

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-06 Thread David Rowley
On Sun, 4 Jul 2021 at 02:08, Andy Fan wrote: > I'd start to work on UniqueKey again, it would be great that we can target it > to PG 15. The attached patch is just for the notnull_attrs. Since we can't > say > a column is nullable or not without saying in which resultset, so I think >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-07-03 Thread Andy Fan
Hi: I'd start to work on UniqueKey again, it would be great that we can target it to PG 15. The attached patch is just for the notnull_attrs. Since we can't say a column is nullable or not without saying in which resultset, so I think attaching it to RelOptInfo is unavoidable. Here is how my

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-04-06 Thread Andy Fan
> > However the thing becomes complex with the below 2 cases. > > 1. SELECT * FROM t INNER JOIN j on t.nullable = q.b; > We know t.b will be not null **finally**. But the current plan may > something > like this: > > QUERY PLAN > -- > Merge

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-30 Thread Andy Fan
> > I assume we want to know if a Var is nullable with a function like. > is_var_notnullable(Var *var, Relids relids). If so, we can define the > data as below: > > struct RelOptInfo { > > Bitmapset** notnullattrs; > .. > }; > > After this we can implement the function as: > > bool >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-27 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to use RelOptInfo.notnullattrs. If someone

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-05 Thread Andy Fan
On Fri, Mar 5, 2021 at 4:16 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Fri, Mar 05, 2021 at 10:22:45AM +0800, Andy Fan wrote: > > > > I checked again and found I do miss the check on JoinExpr->quals. I > have > > > > fixed it in v3 patch. Thanks for the review! > > > > > > > > In the

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-05 Thread Dmitry Dolgov
> On Fri, Mar 05, 2021 at 10:22:45AM +0800, Andy Fan wrote: > > > I checked again and found I do miss the check on JoinExpr->quals. I have > > > fixed it in v3 patch. Thanks for the review! > > > > > > In the attached v3, commit 1 is the real patch, and commit 2 is just add > > > some logs to

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-04 Thread Andy Fan
On Fri, Mar 5, 2021 at 12:00 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Thu, Feb 18, 2021 at 08:58:13PM +0800, Andy Fan wrote: > > Thanks for continuing work on this patch! > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > > > > > On Fri, 12 Feb 2021 at 15:18, Andy Fan

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-04 Thread Dmitry Dolgov
> On Thu, Feb 18, 2021 at 08:58:13PM +0800, Andy Fan wrote: Thanks for continuing work on this patch! > On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > > > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > > wrote: > > >> The

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-18 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to use RelOptInfo.notnullattrs. If someone

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-16 Thread Andy Fan
On Tue, Feb 16, 2021 at 10:03 PM Andy Fan wrote: > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > >> On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: >> > >> > On Fri, Feb 12, 2021 at 9:02 AM David Rowley >> wrote: >> >> The reason I don't really like this is that it really depends

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-16 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to use RelOptInfo.notnullattrs. If someone

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-15 Thread David Rowley
On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley wrote: >> The reason I don't really like this is that it really depends where >> you want to use RelOptInfo.notnullattrs. If someone wants to use it >> to optimise something before the base quals are

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Ashutosh Bapat
On Thu, Feb 11, 2021 at 8:22 PM Tom Lane wrote: > > Ashutosh Bapat writes: > > Can this information be part of PathTarget structure and hence part of > > RelOptInfo::reltarget, so that it can be extended to join, group and > > other kinds of RelOptInfo in future? > > Why would that be better

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Andy Fan
On Thu, Feb 11, 2021 at 9:09 PM Ashutosh Bapat wrote: > Can this information be part of PathTarget structure and hence part of > RelOptInfo::reltarget, so that it can be extended to join, group and > other kinds of RelOptInfo in future? I think you want to expand this field in a more generic

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Andy Fan
Thank you all, friends! On Fri, Feb 12, 2021 at 9:02 AM David Rowley wrote: > On Wed, 10 Feb 2021 at 16:18, Andy Fan wrote: > > v1-0001-Introduce-notnullattrs-field-in-RelOptInfo-to-ind.patch > > > > Introduce notnullattrs field in RelOptInfo to indicate which attr are > not null > > in

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread David Rowley
On Wed, 10 Feb 2021 at 16:18, Andy Fan wrote: > v1-0001-Introduce-notnullattrs-field-in-RelOptInfo-to-ind.patch > > Introduce notnullattrs field in RelOptInfo to indicate which attr are not null > in current query. The not null is judged by checking pg_attribute and query's > restrictinfo. The

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Tom Lane
Ashutosh Bapat writes: > Can this information be part of PathTarget structure and hence part of > RelOptInfo::reltarget, so that it can be extended to join, group and > other kinds of RelOptInfo in future? Why would that be better than keeping it in RelOptInfo? regards,

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Ashutosh Bapat
Can this information be part of PathTarget structure and hence part of RelOptInfo::reltarget, so that it can be extended to join, group and other kinds of RelOptInfo in future? In fact, it might be easy to do that in this patch itself. On Wed, Feb 10, 2021 at 8:57 AM Andy Fan wrote: > > > On

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-09 Thread Andy Fan
On Wed, Feb 10, 2021 at 11:18 AM Andy Fan wrote: > Hi: > > This patch is the first patch in UniqueKey patch series[1], since I need > to revise > that series many times but the first one would be not that often, so I'd > like to > submit this one for review first so that I don't need to maintain

Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-09 Thread Andy Fan
Hi: This patch is the first patch in UniqueKey patch series[1], since I need to revise that series many times but the first one would be not that often, so I'd like to submit this one for review first so that I don't need to maintain it again and again.