Re: Parallelize correlated subqueries that execute within each worker

2023-06-06 Thread Richard Guo
On Mon, Jan 23, 2023 at 10:00 PM James Coleman wrote: > Which this patch we do in fact now see (as expected) rels with > non-empty lateral_relids showing up in generate_[useful_]gather_paths. > And the partial paths can now have non-empty required outer rels. I'm > not able to come up with a

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-06-05 Thread Richard Guo
On Mon, Jun 5, 2023 at 9:19 PM Etsuro Fujita wrote: > If the patch is intended for HEAD only, I also think it goes in the > right direction. But if it is intended for back branches as well, I > do not think so, because it would cause ABI breakage due to changes > made to the ForeignPath struct

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-06-04 Thread Richard Guo
On Fri, Jun 2, 2023 at 8:31 PM Nishant Sharma < nishant.sha...@enterprisedb.com> wrote: > *I only had a minor comment on below change:-* > > > > > > *- gating_clauses = get_gating_quals(root, scan_clauses);+ if > (best_path->pathtype == T_ForeignScan && IS_JOIN_REL(rel))+ > gating_clauses =

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-06-04 Thread Richard Guo
On Fri, Jun 2, 2023 at 11:30 AM Suraj Kharage < suraj.khar...@enterprisedb.com> wrote: > +1 for fixing this in the backend code rather than FDW code. > > Thanks, Richard, for working on this. The patch looks good to me at > a glance. > Thank you Suraj for the review! Thanks Richard

Re: Avoid unncessary always true test (src/backend/storage/buffer/bufmgr.c)

2023-06-04 Thread Richard Guo
On Sun, Jun 4, 2023 at 8:42 PM Ranier Vilela wrote: > Hi, > > Per Coverity. > > At function ExtendBufferedRelShared, has a always true test. > eb.rel was dereferenced one line above, so in > if (eb.rel) is always true. > > I think it's worth removing the test, because Coverity raises dozens of >

Tighten a little bit the computation of potential commutator pairs

2023-06-02 Thread Richard Guo
In make_outerjoininfo, I think we can additionally check a property that's needed to apply OJ identity 3: the lower OJ in the RHS cannot be a member of inner_join_rels because we do not try to commute with any of lower inner joins. --- a/src/backend/optimizer/plan/initsplan.c +++

Re: An inefficient query caused by unnecessary PlaceHolderVar

2023-05-31 Thread Richard Guo
On Wed, May 31, 2023 at 1:27 AM James Coleman wrote: > This looks good to me. Thanks for the review! > A few small tweaks suggested to comment wording: > > +-- lateral reference for simple Var can escape PlaceHolderVar if the > +-- referenced rel is under the same lowest nulling outer join >

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-05-31 Thread Richard Guo
On Wed, May 31, 2023 at 10:47 AM Richard Guo wrote: > On Tue, May 30, 2023 at 10:48 PM Markus Winand > wrote: > >> I found an error similar to others before ([1]) that is still persists as >> of head right now (0bcb3ca3b9). >> >> CREATE TABLE t ( >&g

Re: ERROR: wrong varnullingrels (b 3) (expected (b)) for Var 2/1

2023-05-30 Thread Richard Guo
On Tue, May 30, 2023 at 10:48 PM Markus Winand wrote: > I found an error similar to others before ([1]) that is still persists as > of head right now (0bcb3ca3b9). > > CREATE TABLE t ( > n INTEGER > ); > > SELECT * > FROM (VALUES (1)) t(c) > LEFT JOIN t ljl1 ON true > LEFT JOIN

Re: ERROR: no relation entry for relid 6

2023-05-30 Thread Richard Guo
On Tue, May 30, 2023 at 10:28 AM Richard Guo wrote: > I haven't thought through how to fix it, but I suspect that we may need > to do more checking before we decide to remove PHVs in > remove_rel_from_query. > Hmm, maybe we can additionally check if the PHV needs to be evaluated ab

Re: ERROR: no relation entry for relid 6

2023-05-29 Thread Richard Guo
On Sat, May 27, 2023 at 12:16 AM Tom Lane wrote: > Ah. I realized that we could make the problem testable by adding > assertions that a joinclause we're not removing doesn't contain > any surviving references to the target rel or join. That turns > out to fire (without the bug fix) in half a

Re: ERROR: no relation entry for relid 6

2023-05-26 Thread Richard Guo
On Fri, May 26, 2023 at 6:06 AM Tom Lane wrote: > 1. The test case you give upthread only needs to ignore commute_below_l, > that is it still passes without the lines > > +join_plus_commute = bms_add_members(join_plus_commute, > + > removable_sjinfo->commute_above_r); > > Based on what

Re: Wrong results due to missing quals

2023-05-25 Thread Richard Guo
On Thu, May 25, 2023 at 5:28 AM Tom Lane wrote: > I tried this and it seems to work all right: it fixes the example > you showed while not causing any new failures. (Doesn't address > the broken join-removal logic you showed in the other thread, > though.) > > While at it, I also changed

Wrong results due to missing quals

2023-05-24 Thread Richard Guo
Testing with SQLancer reports a wrong results issue on master and I reduced it to the repro query below. create table t (a int, b int); explain (costs off) select * from t t1 left join (t t2 left join t t3 full join t t4 on false on false) left join t t5 on t2.a = t5.a on t2.b = 1;

Re: ERROR: no relation entry for relid 6

2023-05-23 Thread Richard Guo
On Wed, May 24, 2023 at 2:48 AM Tom Lane wrote: > Richard Guo writes: > > Considering that clone clauses should always be outer-join clauses not > > filter clauses, I'm wondering if we can add an additional check for that > > in RINFO_IS_PUSHED_DOWN, somethin

Re: ERROR: no relation entry for relid 6

2023-05-23 Thread Richard Guo
On Tue, May 23, 2023 at 2:38 PM Richard Guo wrote: > I came across $subject on master and here is the query I'm running. > > create table t (a int unique, b int); > > explain (costs off) > select 1 from t t1 > left join t t2 on true >inner join t t3 on true &g

ERROR: no relation entry for relid 6

2023-05-23 Thread Richard Guo
I came across $subject on master and here is the query I'm running. create table t (a int unique, b int); explain (costs off) select 1 from t t1 left join t t2 on true inner join t t3 on true left join t t4 on t2.a = t4.a and t2.a = t3.a; ERROR: no relation entry for relid 6 I

Re: ERROR: wrong varnullingrels (b 5 7) (expected (b)) for Var 3/3

2023-05-19 Thread Richard Guo
On Fri, May 19, 2023 at 1:57 PM Michael Paquier wrote: > Thanks for the test case, issue reproduced here on HEAD and not v15. > This causes an assertion failure here: > #4 0x55a6f8faa776 in ExceptionalCondition > (conditionName=0x55a6f915ac60 "bms_equal(rel->relids, >

Re: Assert failure of the cross-check for nullingrels

2023-05-18 Thread Richard Guo
On Fri, May 19, 2023 at 12:33 AM Tom Lane wrote: > Bleah. The other solution I'd been poking at involved adding an > extra check for clone clauses, as attached (note this requires > 8a2523ff3). This survives your example, but I wonder if it might > reject all the clones in some cases. It

Re: Assert failure of the cross-check for nullingrels

2023-05-18 Thread Richard Guo
On Thu, May 18, 2023 at 3:42 AM Tom Lane wrote: > ... BTW, something I'd considered in an earlier attempt at fixing this > was to change clause_is_computable_at's API to pass the clause's > RestrictInfo not just the clause_relids, along the lines of > > @@ -541,9 +547,10 @@

Re: Assert failure of the cross-check for nullingrels

2023-05-18 Thread Richard Guo
On Thu, May 18, 2023 at 3:34 AM Tom Lane wrote: > After some poking at it I hit on what seems like a really simple > solution: we should be checking syn_righthand not min_righthand > to see whether a Var should be considered nullable by a given OJ. > Maybe that's still not quite right, but it

Re: Assert failure of the cross-check for nullingrels

2023-05-12 Thread Richard Guo
On Fri, Mar 17, 2023 at 11:05 AM Richard Guo wrote: > Here is an updated patch with comments and test case. I also change the > code to store 'group_clause_relids' directly in RestrictInfo. > BTW, I've added an open item for this issue. Thanks Richard

An inefficient query caused by unnecessary PlaceHolderVar

2023-05-12 Thread Richard Guo
I happened to notice that the query below can be inefficient. # explain (costs off) select * from int8_tbl a left join (int8_tbl b inner join lateral (select *, b.q2 as x from int8_tbl c) ss on b.q2 = ss.q1) on a.q1 = b.q1; QUERY PLAN

Re: Improve list manipulation in several places

2023-05-08 Thread Richard Guo
On Tue, May 9, 2023 at 1:48 AM Ranier Vilela wrote: > I think you missed list_nth_xid, It makes perfect sense to exist. > It seems that list_nth_xid is more about simplification. So maybe we should put it in 0001? Thanks Richard

Re: Improve list manipulation in several places

2023-05-08 Thread Richard Guo
On Tue, May 9, 2023 at 1:26 AM Alvaro Herrera wrote: > The problem I see is that each of these new functions has a single > caller, and the only one that looks like it could have a performance > advantage is list_copy_move_nth_to_head() (which is the weirdest of the > lot). I'm inclined not to

Re: Improve list manipulation in several places

2023-05-08 Thread Richard Guo
On Mon, May 8, 2023 at 11:22 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 23.04.23 08:42, Richard Guo wrote: > > Thanks for the suggestion. I've split the patch into two as attached. > > 0001 is just a minor simplification by replacing

Re: Introduce join_info_array for direct lookups of SpecialJoinInfo by ojrelid

2023-05-07 Thread Richard Guo
On Thu, May 4, 2023 at 4:07 PM Richard Guo wrote: > When working on the improper qual pushdown issue [1], there is a need in > the proposed fix to avoid scanning all the SpecialJoinInfos, since that > is too expensive. I think this might be a common requirement. In the > c

Re: Questionable coding in nth_value

2023-05-06 Thread Richard Guo
On Sat, May 6, 2023 at 4:44 PM Tatsuo Ishii wrote: > Currently Window function nth_value is coded as following: > > nth = DatumGetInt32(WinGetFuncArgCurrent(winobj, 1, )); > if (isnull) > PG_RETURN_NULL(); > const_offset =

Introduce join_info_array for direct lookups of SpecialJoinInfo by ojrelid

2023-05-04 Thread Richard Guo
When working on the improper qual pushdown issue [1], there is a need in the proposed fix to avoid scanning all the SpecialJoinInfos, since that is too expensive. I think this might be a common requirement. In the current codes there are several places where we need to scan all the

Re: [pg_rewind] use the passing callback instead of global function

2023-04-26 Thread Richard Guo
On Wed, Apr 26, 2023 at 9:51 AM Junwang Zhao wrote: > `local_traverse_files` and `libpq_traverse_files` both have a > callback parameter but instead use the global process_source_file > which is no good for function encapsulation. Nice catch. This should be a typo introduced by 37d2ff38.

Re: postgres_fdw: wrong results with self join + enable_nestloop off

2023-04-25 Thread Richard Guo
On Fri, Apr 14, 2023 at 8:51 PM Etsuro Fujita wrote: > I think that the root cause for this issue would be in the > create_scan_plan handling of pseudoconstant quals when creating a > foreign-join (or custom-join) plan. Yes exactly. In create_scan_plan, we are supposed to extract all the

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-24 Thread Richard Guo
On Sun, Apr 23, 2023 at 5:29 PM Richard Guo wrote: > On Sat, Apr 22, 2023 at 11:21 PM Tom Lane wrote: > >> Steinar Kaldager writes: >> > First-time potential contributor here. We recently had an incident due >> > to a sudden 1000x slowdown of a Postgres query

Re: Improving worst-case merge join performance with often-null foreign key

2023-04-23 Thread Richard Guo
On Sat, Apr 22, 2023 at 11:21 PM Tom Lane wrote: > Steinar Kaldager writes: > > First-time potential contributor here. We recently had an incident due > > to a sudden 1000x slowdown of a Postgres query (from ~10ms to ~10s) > > due to a join with a foreign key that was often null. We found that

Re: Improve list manipulation in several places

2023-04-23 Thread Richard Guo
On Fri, Apr 21, 2023 at 7:16 PM Ranier Vilela wrote: > +lcons_copy(void *datum, const List *list) > +lappend_copy(const List *list, void *datum) > list param pointer can be const here not? > Correct. Good point. V2 patch does that. Thanks Richard

Re: Improve list manipulation in several places

2023-04-23 Thread Richard Guo
On Sat, Apr 22, 2023 at 12:55 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 21.04.23 09:34, Richard Guo wrote: > > There was discussion in [1] about improvements to list manipulation in > > several places. But since the discussion is not

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-21 Thread Richard Guo
On Fri, Apr 21, 2023 at 5:43 AM David Rowley wrote: > On Thu, 20 Apr 2023 at 18:46, Richard Guo wrote: > > I searched the codes and found some other places where the manipulation > > of lists can be improved in a similar way. > I'd be happy to discuss our thought about L

Improve list manipulation in several places

2023-04-21 Thread Richard Guo
There was discussion in [1] about improvements to list manipulation in several places. But since the discussion is not related to the topic in that thread, fork a new thread here and attach a patch to show my thoughts. Some are just cosmetic changes by using macros. The others should have

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-20 Thread Richard Guo
On Thu, Apr 20, 2023 at 6:38 AM David Rowley wrote: > That function is pretty new and was exactly added so we didn't have to > write list_truncate(list_copy(...), n) anymore. That gets pretty > wasteful when the input List is long and we only need a small portion > of it. I searched the codes

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Tue, Apr 18, 2023 at 9:33 PM Tom Lane wrote: > Richard Guo writes: > > It seems that in this case the top_plan does not have any extParam, so > > the Gather node that is added atop the top_plan does not have a chance > > to get its initParam filled in set_param_reference

Support "Right Semi Join" plan shapes

2023-04-18 Thread Richard Guo
In thread [1] which discussed 'Right Anti Join', Tom once mentioned 'Right Semi Join'. After a preliminary investigation I think it is beneficial and can be implemented with very short change. With 'Right Semi Join', what we want to do is to just have the first match for each inner tuple. For

Re: Allowing parallel-safe initplans

2023-04-18 Thread Richard Guo
On Mon, Apr 17, 2023 at 11:04 PM Tom Lane wrote: > Richard Guo writes: > > So now it seems that the breakage of regression tests is more severe > > than being cosmetic. I wonder if we need to update the comments to > > indicate the potential wrong results issue if

Re: Incremental sort for access method with ordered scan support (amcanorderbyop)

2023-04-17 Thread Richard Guo
On Sun, Apr 16, 2023 at 1:20 AM Miroslav Bendik wrote: > Postgres allows incremental sort only for ordered indexes. Function > build_index_paths dont build partial order paths for access methods > with order support. My patch adds support for incremental ordering > with access method. I think

Re: Allowing parallel-safe initplans

2023-04-17 Thread Richard Guo
On Mon, Apr 17, 2023 at 10:57 AM Richard Guo wrote: > The initPlan has been moved from the Result node to the Gather node. As > a result, when doing tuple projection for the Result node, we'd get a > ParamExecData entry with NULL execPlan. So the initPlan does not get > chance to

Re: Allowing parallel-safe initplans

2023-04-16 Thread Richard Guo
On Thu, Apr 13, 2023 at 10:00 PM Tom Lane wrote: > Richard Guo writes: > > * For the diff in standard_planner, I was wondering why not move the > > initPlans up to the Gather node, just as we did before. So I tried that > > way but did not notice the breakage of regre

Re: Allowing parallel-safe initplans

2023-04-13 Thread Richard Guo
On Thu, Apr 13, 2023 at 12:43 AM Tom Lane wrote: > Pursuant to the discussion at [1], here's a patch that removes our > old restriction that a plan node having initPlans can't be marked > parallel-safe (dating to commit ab77a5a45). That was really a special > case of the fact that we couldn't

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-12 Thread Richard Guo
On Thu, Apr 13, 2023 at 10:39 AM Richard Guo wrote: > On Wed, Apr 12, 2023 at 7:13 PM David Rowley wrote: > >> There's already code to effectively handle <> operators. Just the >> PartClauseInfo.op_is_ne needs to be set to true. >> get_matching_list_bounds

Re: Unexpected (wrong?) result querying boolean partitioned table with NULL partition

2023-04-12 Thread Richard Guo
On Wed, Apr 12, 2023 at 7:13 PM David Rowley wrote: > There's already code to effectively handle <> operators. Just the > PartClauseInfo.op_is_ne needs to be set to true. > get_matching_list_bounds() then handles that by taking the inverse of > the partitions matching the equality operator. > >

Wrong results from Parallel Hash Full Join

2023-04-12 Thread Richard Guo
I came across $subject and reduced the repro query as below. create table a (i int); create table b (i int); insert into a values (1); insert into b values (2); update b set i = 2; set min_parallel_table_scan_size to 0; set parallel_tuple_cost to 0; set parallel_setup_cost to 0; # explain

Re: v12: ERROR: subplan "InitPlan 2 (returns $4)" was not initialized

2023-04-12 Thread Richard Guo
On Wed, Apr 12, 2023 at 3:59 AM Tom Lane wrote: > The v1 patch attached is enough to fix the immediate issue, > but there's another thing not to like, which is that we're also > discarding the costs associated with the initplans. That's > strictly cosmetic given that all the planning decisions

Re: Protecting allocator headers with Valgrind

2023-04-11 Thread Richard Guo
On Tue, Apr 11, 2023 at 9:28 PM David Rowley wrote: > Over on [1], Tom mentioned that we might want to rethink the decision > to not protect chunk headers with Valgrind. That thread fixed a bug > that was accessing array element -1, which effectively was reading the > MemoryChunk at the start

A minor adjustment to get_cheapest_path_for_pathkeys

2023-04-11 Thread Richard Guo
The check for parallel_safe should be even cheaper than cost comparison so I think it's better to do that first. The attached patch does this and also updates the comment to mention the requirement about being parallel-safe. Thanks Richard

Can we rely on the ordering of paths in pathlist?

2023-04-10 Thread Richard Guo
As the comment above add_path() says, 'The pathlist is kept sorted by total_cost, with cheaper paths at the front.' And it seems that get_cheapest_parallel_safe_total_inner() relies on this ordering (without being mentioned in the comments, which I think we should do). I'm wondering if this is

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-10 Thread Richard Guo
On Fri, Apr 7, 2023 at 3:28 PM Richard Guo wrote: > On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > >> On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: >> >>> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] >> >> Maybe

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-07 Thread Richard Guo
On Tue, Aug 2, 2022 at 3:13 PM Richard Guo wrote: > On Sun, Jul 31, 2022 at 12:07 AM Tom Lane wrote: > >> [ wanders away wondering if JOIN_RIGHT_SEMI should become a thing ... ] > > Maybe this is something we can do. Currently for the query below: > > # explain

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-06 Thread Richard Guo
On Thu, Apr 6, 2023 at 1:06 PM Tom Lane wrote: > This: > > > +#if 0 > > /* If any limit was set to zero, the user doesn't want a > > parallel scan. */ > > if (parallel_workers <= 0) > > return; > > +#endif > > seems like it adds a lot of new paths with a lot lower

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-04-06 Thread Richard Guo
On Thu, Apr 6, 2023 at 8:18 AM Thomas Munro wrote: > On Thu, Apr 6, 2023 at 9:11 AM Tom Lane wrote: > > Richard Guo writes: > > > Thanks for reminding. Attached is the rebased patch, with no other > > > changes. I think the patch is ready for commit. > >

Re: same query but different result on pg16devel and pg15.2

2023-04-04 Thread Richard Guo
On Tue, Apr 4, 2023 at 10:53 AM tender wang wrote: > Hi hackers, > I encounter a problem, as shown below: > > query: > select > ref_0.ps_suppkey as c0, > ref_1.c_acctbal as c1, > ref_2.o_totalprice as c2, > ref_2.o_orderpriority as c3, > ref_2.o_clerk as c4 > from >

Re: About the constant-TRUE clause in reconsider_outer_join_clauses

2023-03-26 Thread Richard Guo
On Sat, Mar 25, 2023 at 11:41 PM Tom Lane wrote: > Richard Guo writes: > > Should we instead mark the constant-TRUE clause with required_relids > > plus the OJ relid? > > I do not think it matters. Yeah, I agree that it makes no difference currently. One day

About the constant-TRUE clause in reconsider_outer_join_clauses

2023-03-25 Thread Richard Guo
I happened to notice a constant-TRUE clause with is_pushed_down being true while its required_relids not including the OJ being formed, which seems abnormal to me. It turns out that this clause comes from reconsider_outer_join_clauses(), as a dummy replacement if we've generated a derived clause.

Re: Comment in preptlist.c

2023-03-21 Thread Richard Guo
On Tue, Mar 21, 2023 at 5:41 PM Etsuro Fujita wrote: > While working on something else, I noticed $SUBJECT added by commit > 86dc90056: > > * For UPDATE and DELETE queries, the targetlist must also contain "junk" > * tlist entries needed to allow the executor to identify the rows to be > *

Re: Fix typo plgsql to plpgsql.

2023-03-20 Thread Richard Guo
On Tue, Mar 21, 2023 at 12:26 AM Zhang Mingli wrote: > Found several typos like plgsql, I think it should be plpgsql. > +1. I believe these are typos. And a grep search shows that all typos of this kind are addressed by the patch. Thanks Richard

Re: Missing rules for queryjumblefuncs.{funcs,switch}.c for maintainer-clean

2023-03-20 Thread Richard Guo
On Mon, Mar 20, 2023 at 3:49 PM Daniel Gustafsson wrote: > > On 20 Mar 2023, at 08:46, Michael Paquier wrote: > > How about removing the "fout/five" entirely here > > and make that simpler? I would propose: > > "For some classes of node types, you don't need all the support > > functions." > >

Re: Missing rules for queryjumblefuncs.{funcs,switch}.c for maintainer-clean

2023-03-20 Thread Richard Guo
On Mon, Mar 20, 2023 at 2:43 PM Michael Paquier wrote: > Nathan has reported to me offlist that maintainer-clean was not doing > its job for the files generated by gen_node_support.pl in > src/backend/nodes/ for the query jumbling. Attached is a patch to > take care of this issue. > > While on

Re: An oversight in ExecInitAgg for grouping sets

2023-03-20 Thread Richard Guo
On Mon, Jan 9, 2023 at 5:21 PM David Rowley wrote: > On Thu, 5 Jan 2023 at 20:06, Richard Guo wrote: > > I reviewed this patch and have some comments. > > Thanks for looking at this. I think I've fixed all the issues you > mentioned. > > One extra thing I noticed was

Re: A problem about ParamPathInfo for an AppendPath

2023-03-16 Thread Richard Guo
On Fri, Mar 17, 2023 at 6:15 AM Tom Lane wrote: > Pushed. I thought the comment needed to be completely rewritten not just > tweaked, and I felt it was probably reasonable to continue to exclude > dummy paths from getting the more expensive treatment. Yes agreed. Thanks for the changes and

Re: Assert failure of the cross-check for nullingrels

2023-03-16 Thread Richard Guo
On Mon, Mar 13, 2023 at 5:44 PM Richard Guo wrote: > On Mon, Mar 13, 2023 at 5:03 PM Richard Guo > wrote: > >> Back to the original issue, if a join has more than one quals, actually >> we treat them as a whole when we check if identity 3 applies as well as >> when we

Re: postgres_fdw: Useless if-test in GetConnection()

2023-03-15 Thread Richard Guo
On Wed, Mar 15, 2023 at 6:18 PM Etsuro Fujita wrote: > While working on something else, I noticed that the “if (entry->conn > == NULL)” test after doing disconnect_pg_server() when re-establishing > a given connection in GetConnection() is pointless, because the former > function ensures that

Re: Using each rel as both outer and inner for JOIN_ANTI

2023-03-15 Thread Richard Guo
On Wed, Mar 15, 2023 at 2:25 AM Gregory Stark (as CFM) wrote: > So what is the status of this patch? > > It looks like you received some feedback from Emre, Tom, Ronan, and > Alvaro but it also looks like you responded to most or all of that. > Are you still blocked waiting for feedback?

Re: Assert failure of the cross-check for nullingrels

2023-03-13 Thread Richard Guo
On Mon, Mar 13, 2023 at 5:03 PM Richard Guo wrote: > Back to the original issue, if a join has more than one quals, actually > we treat them as a whole when we check if identity 3 applies as well as > when we adjust them to be suitable for commutation according to identity > 3. So w

Re: Assert failure of the cross-check for nullingrels

2023-03-13 Thread Richard Guo
On Fri, Mar 10, 2023 at 4:13 PM Richard Guo wrote: > I wonder if we should consider syn_xxxhand rather than min_xxxhand in > clause_is_computable_at when we check if clause mentions any nullable > Vars. But I'm not sure about that. > No, considering syn_xxxhand is not right. Aft

Assert failure of the cross-check for nullingrels

2023-03-10 Thread Richard Guo
While looking into issue [1], I came across $subject on master. Below is how to reproduce it. DROP TABLE IF EXISTS t1,t2,t3,t4 CASCADE; CREATE TABLE t1 AS SELECT true AS x FROM generate_series(0,1) x; CREATE TABLE t2 AS SELECT true AS x FROM generate_series(0,1) x; CREATE TABLE t3 AS SELECT true

Re: Making empty Bitmapsets always be NULL

2023-03-01 Thread Richard Guo
On Thu, Mar 2, 2023 at 6:59 AM Tom Lane wrote: > Yeah. I split out those executor fixes as 0002; 0003 is the changes > to bitmapsets proper, and then 0004 removes now-dead code. +1 to all these patches. Some minor comments from me. *0003 It seems that the Bitmapset checked by

Re: Wrong query results caused by loss of join quals

2023-02-23 Thread Richard Guo
On Thu, Feb 23, 2023 at 4:50 AM Tom Lane wrote: > I thought about this and decided that it's not really a problem. > have_relevant_joinclause is just a heuristic, and I don't think we > need to prioritize forming a join if the only relevant clauses look > like this. We won't be able to use such

Re: Some revises in adding sorting path

2023-02-21 Thread Richard Guo
On Tue, Feb 14, 2023 at 10:53 AM David Rowley wrote: > I looked at the three patches and have some thoughts: Thanks for reviewing! > 0001: > > Does the newly added test have to be this complex? I think it might > be better to just come up with the most simple test you can that uses > an

Re: Some revises in adding sorting path

2023-02-21 Thread Richard Guo
On Thu, Feb 16, 2023 at 7:50 PM Etsuro Fujita wrote: > I'm not sure this is a good idea, because the epq_path will return at > most one tuple in an EPQ recheck. > > The reason why an extra Sort node is injected into the epq_path is > only label it with the correct sort order to use it as an

Re: Wrong query results caused by loss of join quals

2023-02-20 Thread Richard Guo
On Mon, Feb 20, 2023 at 4:56 AM Tom Lane wrote: > * I suspect the other use of get_common_eclass_indexes, in > have_relevant_eclass_joinclause, is broken as well. It seems have_relevant_joinclause is broken for the presented case. It does not get a change to call

Re: wrong query result due to wang plan

2023-02-20 Thread Richard Guo
On Mon, Feb 20, 2023 at 3:01 AM Tom Lane wrote: > We still have to fix process_implied_equality though, and in that > context we do have all the SpecialJoinInfos, so the strip-the-outer-joins > fix seems to work. See attached. Yeah, process_implied_equality is also broken for variable-free

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 5:50 PM Richard Guo wrote: > It seems we still need to check whether a variable-free qual comes from > somewhere that is below the nullable side of an outer join before we > decide that it can be evaluated at join domain level, just like we did > before.

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 5:50 PM Richard Guo wrote: > It seems we still need to check whether a variable-free qual comes from > somewhere that is below the nullable side of an outer join before we > decide that it can be evaluated at join domain level, just like we did > before.

Re: wrong query result due to wang plan

2023-02-16 Thread Richard Guo
On Thu, Feb 16, 2023 at 3:16 PM tender wang wrote: > select ref_1.r_comment as c0, subq_0.c1 as c1 from public.region as > sample_0 right join public.partsupp as sample_1 right join public.lineitem > as sample_2 on (cast(null as path) = cast(null as path)) on (cast(null as > "timestamp") <

Wrong query results caused by loss of join quals

2023-02-14 Thread Richard Guo
I came across $subject on HEAD and here is the query I'm using. create table t1 (a int, b int); create table t2 (a int, b int); create table t3 (a int, b int); insert into t1 values (1, 1); insert into t2 values (2, 200); insert into t3 values (3, 3); # select * from t1 left join t2 on true,

Re: Killing off removed rels properly

2023-02-13 Thread Richard Guo
On Sat, Feb 11, 2023 at 4:50 AM Tom Lane wrote: > I think it's time to clean this up by removing the rel from the > planner data structures altogether. The attached passes check-world, > and if it does trigger any problems I would say that's a clear > sign of bugs elsewhere. +1. The patch

Re: Making Vars outer-join aware

2023-02-12 Thread Richard Guo
On Mon, Feb 13, 2023 at 7:58 AM Justin Pryzby wrote: > The patch broke this query: > > select from pg_inherits inner join information_schema.element_types > right join (select from pg_constraint as sample_2) on true > on false, lateral (select scope_catalog, inhdetachpending from >

Re: Inconsistent nullingrels due to oversight in deconstruct_distribute_oj_quals

2023-02-10 Thread Richard Guo
On Fri, Feb 10, 2023 at 11:08 AM Richard Guo wrote: > On Thu, Feb 9, 2023 at 11:55 PM Tom Lane wrote: > >> Richard Guo writes: >> > This query would trigger the Assert() in search_indexed_tlist_for_var. >> > So I wonder that we should use othersj->syn_ri

Re: Inconsistent nullingrels due to oversight in deconstruct_distribute_oj_quals

2023-02-09 Thread Richard Guo
On Thu, Feb 9, 2023 at 11:55 PM Tom Lane wrote: > Richard Guo writes: > > This query would trigger the Assert() in search_indexed_tlist_for_var. > > So I wonder that we should use othersj->syn_righthand here. > > There are two such calls in deconstruct_distribute_oj_qua

Inconsistent nullingrels due to oversight in deconstruct_distribute_oj_quals

2023-02-09 Thread Richard Guo
When we try to generate qual variants with different nullingrels in deconstruct_distribute_oj_quals, we traverse all the JoinTreeItems and adjust qual nulling bits as we crawl up the join tree. For a SpecialJoinInfo which commutes with current sjinfo from below left, in the next level up it would

Re: A bug in make_outerjoininfo

2023-02-07 Thread Richard Guo
On Wed, Feb 8, 2023 at 1:55 PM Richard Guo wrote: > > On Wed, Feb 8, 2023 at 7:33 AM Tom Lane wrote: > >> We might want to see if we can devise a new example (or wait for >> Robins to break it ;-)) before expending a lot of effort on making >> the commute_xxx bi

Re: A bug in make_outerjoininfo

2023-02-07 Thread Richard Guo
On Wed, Feb 8, 2023 at 7:33 AM Tom Lane wrote: > We might want to see if we can devise a new example (or wait for > Robins to break it ;-)) before expending a lot of effort on making > the commute_xxx bits more precise. Here is an example that can trigger the same assertion as in bug #17781

Re: A problem in deconstruct_distribute_oj_quals

2023-02-07 Thread Richard Guo
On Tue, Feb 7, 2023 at 2:12 PM Tom Lane wrote: > Richard Guo writes: > > I noticed this code because I came across a problem with a query as > > below. > > > create table t (a int); > > > select t1.a from (t t1 left join t t2 on true) left join (t t3 left join

A bug in make_outerjoininfo

2023-02-06 Thread Richard Guo
In cases where we have any clauses between two outer joins, these clauses should be treated as degenerate clauses in the upper OJ, and they may prevent us from re-ordering the two outer joins. Previously we have the flag 'delay_upper_joins' to help avoid the re-ordering in such cases. In

A problem in deconstruct_distribute_oj_quals

2023-02-06 Thread Richard Guo
In deconstruct_distribute_oj_quals, when we've identified a commutable left join which provides join clause with flexible semantics, we try to generate multiple versions of the join clause. Here we have the logic that puts back any ojrelids that were removed from its min_righthand. /*

Re: pgsql: Remove over-optimistic Assert.

2023-02-02 Thread Richard Guo
On Thu, Feb 2, 2023 at 8:40 AM Tom Lane wrote: > Remove over-optimistic Assert. > > In commit 2489d76c4, I'd thought it'd be safe to assert that a > PlaceHolderVar appearing in a scan-level expression has empty > nullingrels. However this is not so, as when we determine that a > join relation

Fwd: pgsql: Remove over-optimistic Assert.

2023-02-01 Thread Richard Guo
Resend this email to -hackers. Sorry for the noise. Thanks Richard -- Forwarded message - From: Richard Guo Date: Thu, Feb 2, 2023 at 9:51 AM Subject: Re: pgsql: Remove over-optimistic Assert. To: Tom Lane Cc: , PostgreSQL-development < pgsql-hack...@postgresql.org>

Re: Making Vars outer-join aware

2023-01-30 Thread Richard Guo
On Tue, Jan 24, 2023 at 4:38 AM Tom Lane wrote: > Richard, are you planning to review this any more? I'm getting > a little antsy to get it committed. For such a large patch, > it's surprising it's had so few conflicts to date. Sorry for the delayed reply. I don't have any more review

Re: Check lateral references within PHVs for memoize cache keys

2023-01-30 Thread Richard Guo
On Tue, Jan 24, 2023 at 10:07 AM David Rowley wrote: > I'm surprised to see that it's only Memoize that ever makes use of > lateral_vars. I'd need a bit more time to process your patch, but one > additional thought I had was that I wonder if the following code is > still needed in nodeMemoize.c

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-18 Thread Richard Guo
On Tue, Jan 17, 2023 at 3:05 PM Tom Lane wrote: > Richard Guo writes: > > BTW, I wonder if we should have checked CoercionForm before > > fix_upper_expr_mutator steps into CoerceViaIO->arg to adjust the expr > > there. > > I will just quote what it says in

Re: Removing redundant grouping columns

2023-01-17 Thread Richard Guo
On Wed, Jan 18, 2023 at 6:51 AM Tom Lane wrote: > I wrote: > > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. > > And immediately sideswiped by da5800d5f. Yeah, I noticed this too yesterday. I reviewed through these two patches yesterday and I think they are in good

Re: Removing redundant grouping columns

2023-01-16 Thread Richard Guo
On Sun, Jan 15, 2023 at 5:23 AM Tom Lane wrote: > vignesh C writes: > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > > Yeah, sideswiped by 3c6fc5820 apparently. No substantive change needed. I looked through these two patches and they look good to me.

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2023-01-16 Thread Richard Guo
On Tue, Jan 17, 2023 at 11:39 AM David Rowley wrote: > On Tue, 17 Jan 2023 at 13:16, Dean Rasheed > wrote: > > I took a look at this, and I agree that the best solution is probably > > to have make_pathkeys_for_groupagg() ignore Aggrefs that contain > > volatile functions. > > Thanks for giving

Improve LATERAL join case in test memoize.sql

2023-01-16 Thread Richard Guo
I happened to notice we have the case in memoize.sql that tests for memoize node with LATERAL joins, which is -- Try with LATERAL joins SELECT explain_memoize(' SELECT COUNT(*),AVG(t2.unique1) FROM tenk1 t1, LATERAL (SELECT t2.unique1 FROM tenk1 t2 WHERE t1.twenty = t2.unique1) t2 WHERE

<    1   2   3   4   5   6   7   >