Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-20 Thread Andy Fan
Thanks for the detailed explanation. On Sat, Feb 19, 2022 at 2:27 AM Robert Haas wrote: > On Fri, Feb 18, 2022 at 12:56 AM Andy Fan > wrote: > > What do you think about moving on this feature? The items known by me > > are: 1). Make sure the estimation error can be fixe

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-20 Thread Andy Fan
does not look very reasonable to me. Do you think we can go further in direction for the issue here? and it would be super great that you can take a look at the commit 3 [1]. IIUC, It can solve the issue and is pretty straightforward. [1] https://www.postgresql.org/message-id/CAKU4AWrdeQZ8xvf%3DDVhndUs%3DRGn8oVoSJvYK3Yj7uWq2%3Ddt%3DMg%40mail.gmail.com -- Best Regards Andy Fan

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-03-01 Thread Andy Fan
've not looked at the subsequent ones. > > I agree with 0001 patch should be the first one to reach an agreement . -- Best Regards Andy Fan

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-03-01 Thread Andy Fan
> put, which seems certain not to be optimal. If we can have some agreement (after more discussion) the EC filter is acceptable on semantics level, I think we may have some chances to improve something at execution level. -- Best Regards Andy Fan

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-03-16 Thread Andy Fan
Hi: I just tested more cases for the estimation issue for this feature, and we can find **we get a more accurate/stable estimation than before**. Here is the test cases and result (by comparing the master version and patched version). create table ec_t110 as select i::int as a from generate_serie

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-12-05 Thread Andy Fan
r4dgOXxEg%40mail.gmail.com -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-12-05 Thread Andy Fan
Thank you Heikki for your attention. On Mon, Nov 30, 2020 at 11:20 PM Heikki Linnakangas wrote: > On 30/11/2020 16:30, Jesper Pedersen wrote: > > On 11/30/20 5:04 AM, Heikki Linnakangas wrote: > >> On 26/11/2020 16:58, Andy Fan wrote: > >>> This patch has st

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-12-05 Thread Andy Fan
valenceClasses, but any suggestion would be welcome. > But if the > motivation is only to remove this overspecification, I humbly suggest > that it ain't worth the trouble. > > regards, tom lane > [1] https://www.postgresql.org/message-id/CAKU4AWqy3Uv67%3DPR8RXG6LVoO-cMEwfW_LMwTxHdGrnu%2Bcf%2BdA%40mail.gmail.com -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-12-07 Thread Andy Fan
On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen wrote: > Hi, > > On 12/5/20 10:38 PM, Andy Fan wrote: > > Currently the UniqueKey is defined as a List of Expr, rather than > > EquivalenceClasses. > > A complete discussion until now can be found at [1] (The messa

initscan for MVCC snapshot

2020-12-07 Thread Andy Fan
Buffers, use a bulk-read access @@ -1210,6 +1212,7 @@ heap_beginscan(Relation relation, Snapshot snapshot, else scan->rs_base.rs_key = NULL; + scan->rs_nblocks = -1; initscan(scan, key, false); -- Best Regards Andy Fan

Re: initscan for MVCC snapshot

2020-12-10 Thread Andy Fan
On Mon, Dec 7, 2020 at 8:26 PM Andy Fan wrote: > Hi: > I see initscan calls RelationGetwNumberOfBlocks every time and rescan > calls > initscan as well. In my system, RelationGetNumberOfBlocks is expensive > (the reason > doesn't deserve a talk.. ), so in a nest loop

Re: initscan for MVCC snapshot

2020-12-10 Thread Andy Fan
On Thu, Dec 10, 2020 at 7:31 PM Andy Fan wrote: > > > On Mon, Dec 7, 2020 at 8:26 PM Andy Fan wrote: > >> Hi: >> I see initscan calls RelationGetwNumberOfBlocks every time and rescan >> calls >> initscan as well. In my system, RelationGetNumberOfBlocks is

Re: Cache relation sizes?

2020-12-16 Thread Andy Fan
clobber dirty updates -- do > you think that is on the right path? > Hi Thomas: Thank you for working on it. I spent one day studying the patch and I want to talk about one question for now. What is the purpose of calling smgrimmedsync to evict a DIRTY sr (what will happen if we remove it and the SR_SYNCING and SR_JUST_DIRTIED flags)? -- Best Regards Andy Fan

Re: Cache relation sizes?

2020-12-17 Thread Andy Fan
Hi Thomas, Thank you for your quick response. On Thu, Dec 17, 2020 at 3:05 PM Thomas Munro wrote: > Hi Andy, > > On Thu, Dec 17, 2020 at 7:29 PM Andy Fan wrote: > > I spent one day studying the patch and I want to talk about one question > for now. > > What i

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-12-25 Thread Andy Fan
ER TABLE DETACH > CONCURRENTLY cannot work as part of a multi-command ALTER TABLE, but > that's alreay enforced by the grammar anyway. > > DETACH CONCURRENTLY doesn't work if a default partition exists. It's > just too problematic a case; you would still need to have AEL on the > default partition. > > > I haven't yet experimented with queries running in a standby in tandem > with a detach. > > -- > Álvaro Herrera > -- Best Regards Andy Fan

Re: Cache relation sizes?

2020-12-27 Thread Andy Fan
On Thu, Dec 24, 2020 at 6:59 AM Thomas Munro wrote: > On Thu, Dec 17, 2020 at 10:22 PM Andy Fan > wrote: > > Let me try to understand your point. Suppose process 1 extends a file to > > 2 blocks from 1 block, and fsync is not called, then a). the lseek *may* > still >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Andy Fan
Thank you all, friends! On Fri, Feb 12, 2021 at 9:02 AM David Rowley wrote: > On Wed, 10 Feb 2021 at 16:18, Andy Fan wrote: > > v1-0001-Introduce-notnullattrs-field-in-RelOptInfo-to-ind.patch > > > > Introduce notnullattrs field in RelOptInfo to indicate which attr

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-11 Thread Andy Fan
A) is null even if the UDF is strict and A is not null? > In fact, it might be easy to do that in this patch itself. > Actually I can't think out the method:) > On Wed, Feb 10, 2021 at 8:57 AM Andy Fan wrote: > > > > > > On Wed, Feb 10, 2021 at 11:18 AM Andy

Re: How to get Relation tuples in C function

2021-02-13 Thread Andy Fan
be the difference between the SPI and "write a pure SQL UDF" and call it with DirectFunctionCall1? I just ran into a similar situation some days before. Currently I think DirectFunctionCall1 doesn't need to maintain a connection but SPI has to do that. -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-16 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-16 Thread Andy Fan
On Tue, Feb 16, 2021 at 10:03 PM Andy Fan wrote: > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > >> On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: >> > >> > On Fri, Feb 12, 2021 at 9:02 AM David Rowley >> wrote: >> >> The

Re: How to get Relation tuples in C function

2021-02-16 Thread Andy Fan
On Sun, Feb 14, 2021 at 7:56 PM Michael Paquier wrote: > On Sun, Feb 14, 2021 at 09:29:08AM +0800, Andy Fan wrote: > > Thank you tom for the reply. What would be the difference between the > > SPI and "write a pure SQL UDF" and call it with DirectFunctionCall1? I &g

Re: Implementing Incremental View Maintenance

2021-02-18 Thread Andy Fan
thout asking for a new rebase usually. -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-18 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-02-19 Thread Andy Fan
On Mon, Feb 8, 2021 at 3:43 PM Andy Fan wrote: > > > On Mon, Jan 25, 2021 at 10:21 AM Andy Fan > wrote: > >> >> >> On Sun, Jan 24, 2021 at 6:34 PM Andy Fan >> wrote: >> >>> Hi: >>> >>> I recently found a use case like

UniqueKey on Partitioned table.

2021-02-19 Thread Andy Fan
l.com -- Best Regards Andy Fan (https://www.aliyun.com/) v1-0001-Introduce-notnullattrs-field-in-RelOptInfo-to-ind.patch Description: Binary data v1-0002-UniqueKey-with-EquivalenceClass-for-single-rel-on.patch Description: Binary data

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-02-21 Thread Andy Fan
les and load the > data into PostgreSQL. This seemed to work okay apart from the > movie_info_idx table appeared to be missing. Many of the 113 join > order benchmark queries need this table. I followed the steps in [1] and changed something with the attached patch. At last I got 2367725 rows. But probably you are running into a different problem since no change is for movie_info_idx table. [1] https://github.com/gregrahn/join-order-benchmark -- Best Regards Andy Fan (https://www.aliyun.com/) 0001-fix.patch Description: Binary data

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-02-21 Thread Andy Fan
On Fri, Feb 19, 2021 at 6:03 PM Andy Fan wrote: > > > On Mon, Feb 8, 2021 at 3:43 PM Andy Fan wrote: > >> >> >> On Mon, Jan 25, 2021 at 10:21 AM Andy Fan >> wrote: >> >>> >>> >>> On Sun, Jan 24, 2021 at 6:34 PM Andy Fan >

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2021-02-22 Thread Andy Fan
e calls = rcpath->calls; > ... > > + /* estimate on the distinct number of parameter values */ > > + ndistinct = estimate_num_groups(root, rcpath->param_exprs, calls, > NULL, > > + &estinfo); > > Shouldn't this pass "tuples" and not "calls" ? > > -- > Justin > -- Best Regards Andy Fan (https://www.aliyun.com/)

pg_temp_%d namespace creation can invalidate all the cached plan in other backends

2021-02-22 Thread Andy Fan
called (for pg_temp_%d change only). I think pg_temp_%d is not meaningful for others, so I think the bypassing is OK. I still have not kicked off any coding so far, I want to know if it is a correct thing to do? -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: pg_temp_%d namespace creation can invalidate all the cached plan in other backends

2021-02-22 Thread Andy Fan
On Tue, Feb 23, 2021 at 12:07 PM Andy Fan wrote: > Planning is expensive and we use plancache to bypass its effect. I find the > $subject recently which is caused by we register NAMESPACEOID invalidation > message for pg_temp_%s as well as other normal namespaces. Is it a > must

Re: pg_temp_%d namespace creation can invalidate all the cached plan in other backends

2021-02-23 Thread Andy Fan
On Tue, Feb 23, 2021 at 1:50 PM Tom Lane wrote: > Andy Fan writes: > > Planning is expensive and we use plancache to bypass its effect. I find > the > > $subject recently which is caused by we register NAMESPACEOID > invalidation > > message for pg_temp_%s as well

Re: Make Append Cost aware of some run time partition prune case

2021-03-03 Thread Andy Fan
Hi Ryan: On Thu, Mar 4, 2021 at 8:14 AM Ryan Lambert wrote: > On Mon, Nov 9, 2020 at 5:44 PM Andy Fan wrote: > >> Currently the cost model of append path sums the cost/rows for all the >> subpaths, it usually works well until we run into the run-time partition >> p

Re: Make Append Cost aware of some run time partition prune case

2021-03-03 Thread Andy Fan
oduces extra complexity. c). at last, we can't estimate it well like partkey > $1, what would be a prune ratio for ). Something I don't handle so far are: 1). accumulate_append_subpath stuff. 2). MergeAppend. 3). Multi Partition key. -- Best Regards Andy Fan (https://www.aliyun.com/) v1-0001-adjust-cost-model-for-partition-prune-case.patch Description: Binary data

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-04 Thread Andy Fan
On Fri, Mar 5, 2021 at 12:00 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Thu, Feb 18, 2021 at 08:58:13PM +0800, Andy Fan wrote: > > Thanks for continuing work on this patch! > > > On Tue, Feb 16, 2021 at 12:01 PM David Rowley > wrote: > > > >

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-05 Thread Andy Fan
On Fri, Mar 5, 2021 at 4:16 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Fri, Mar 05, 2021 at 10:22:45AM +0800, Andy Fan wrote: > > > > I checked again and found I do miss the check on JoinExpr->quals. I > have > > > > fixed it in v3 patch. Tha

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-03-05 Thread Andy Fan
Hi Amit: Thanks for your review! On Thu, Mar 4, 2021 at 5:07 PM Amit Langote wrote: > Hi Andy, > > On Sun, Jan 24, 2021 at 7:34 PM Andy Fan wrote: > > I recently found a use case like this. SELECT * FROM p, q WHERE > p.partkey = > > q.colx AND (q.colx = $1 OR q.c

Re: Huge memory consumption on partitioned table with FKs

2021-03-07 Thread Andy Fan
this is passing, and I'd sure like to think that our test coverage > is not so thin that it'd fail to detect probing the wrong partition > for foreign key matches. But that's what it looks like this patch > will do. > > regards, tom lane > > > -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Andy Fan
On Mon, Mar 8, 2021 at 3:43 PM Andy Fan wrote: > > > On Fri, Mar 5, 2021 at 5:00 AM Tom Lane wrote: > >> Amit Langote writes: >> > Updated patch attached. >> >> This claim seems false on its face: >> >> > All child constraints of a given

Re: Huge memory consumption on partitioned table with FKs

2021-03-08 Thread Andy Fan
On Mon, Mar 8, 2021 at 8:42 PM Amit Langote wrote: > Hi Andy, > > On Mon, Mar 8, 2021 at 8:39 PM Andy Fan wrote: > > On Mon, Mar 8, 2021 at 3:43 PM Andy Fan > wrote: > >> My point below is a bit off-topic, but I want to share it here. Since > >> we implemen

Advance xmin aggressively on Read Commit isolation level

2020-11-06 Thread Andy Fan
h are older than xxx? If users want to access that, we can just raise errors. Oracle uses this strategy and the error code is ORA-01555. -- Best Regards Andy Fan

Re: Advance xmin aggressively on Read Commit isolation level

2020-11-06 Thread Andy Fan
On Fri, Nov 6, 2020 at 4:54 PM Thomas Munro wrote: > On Fri, Nov 6, 2020 at 9:48 PM Andy Fan wrote: > > I have 2 ideas about this. One is in the Read Committed level, we can > advance xmin > > aggressively. suppose it started at t1, and complete a query at t2. the >

Re: Allow "snapshot too old" error, to prevent bloat

2020-11-08 Thread Andy Fan
ng for this situation. Since most users will use Read Committed isolation level, so after a user completes a query, the next query will use a fresh new snapshot, so there is no need to block the oldest xmin because of this. will it be correct to advance the oldest xmin in this case? If yes, what would be the blocker in implementing this feature? -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-08 Thread Andy Fan
" > "MRU Cache" > "Parameterized Tuple Cache" (bit long) > "Parameterized Cache" > "Parameterized MRU Cache" > > I think "Tuple Cache" would be OK which means it is a cache for tuples. Telling MRU/LRU would be too internal for an end user and "Parameterized" looks redundant given that we have said "Cache Key" just below the node name. Just my $0.01. -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-08 Thread Andy Fan
t really see any obvious reason as to why > this is. I'm very much inclined to just pursue the v8 patch (separate > Result Cache node) and just drop the v9 idea altogether. > > David > -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-08 Thread Andy Fan
On Mon, Nov 9, 2020 at 10:07 AM David Rowley wrote: > On Mon, 9 Nov 2020 at 03:52, Andy Fan wrote: > > then I did a perf on the 2 version, Is it possible that you called > tts_minimal_clear twice in > > the v9 version? Both ExecClearTuple and ExecStoreMinimalTuple called &

Make Append Cost aware of some run time partition prune case

2020-11-09 Thread Andy Fan
2) + 1))) (12 rows) Any thoughts about this? Thanks [1] https://www.postgresql.org/message-id/CA%2BTgmoZHYoAL4HYwnGO25B8CxCB%2BvNMdf%2B7rbUzYykR4sU9yUA%40mail.gmail.com -- Best Regards Andy Fan v1-0001-Adjust-Append-Path-cost-model-for-runtime-partiti.patch Description: Binary data part_runtime_prune.sql Description: Binary data

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-09 Thread Andy Fan
to. So I think any hints on why v9 should be better at a noticeable level in theory should be very helpful. After that, I'd like to read the code or profile more carefully. -- Best Regards Andy Fan

Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
select * from (select * from cte1) c where a = 1; I know how we treat cte and subqueries differently currently, I just don't know why we can't treat cte as a subquery, so lots of subquery related technology can apply to it. Do we have any discussion about this? Thanks -- Best Regards Andy Fan

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
On Sat, Nov 14, 2020 at 2:14 PM Tom Lane wrote: > Andy Fan writes: > > Take the following example: > > > insert into cte1 select i, i from generate_series(1, 100)i; > > create index on cte1(a); > > > explain > > with cte1 as (select * from cte1) &g

Re: Have we tried to treat CTE as SubQuery in planner?

2020-11-13 Thread Andy Fan
On Sat, Nov 14, 2020 at 2:44 PM Jesse Zhang wrote: > Hi, > > On Fri, Nov 13, 2020 at 10:04 PM Andy Fan wrote: > > > > Hi: > > > > Take the following example: > > > > insert into cte1 select i, i from generate_series(1, 100)i; > > create

Different results between PostgreSQL and Oracle for "for update" statement

2020-11-19 Thread Andy Fan
UTPUT - Any thoughts on who is wrong? -- Best Regards Andy Fan

Re: Different results between PostgreSQL and Oracle for "for update" statement

2020-11-20 Thread Andy Fan
On Thu, Nov 19, 2020 at 11:49 PM Tom Lane wrote: > Andy Fan writes: > > create table su (a int, b int); > > insert into su values(1, 1); > > > - session 1: > > begin; > > update su set b = 2 where b = 1; > > > - sess 2: > > select * from su w

Re: Different results between PostgreSQL and Oracle for "for update" statement

2020-11-20 Thread Andy Fan
Hi Andreas: Thanks for your input. On Fri, Nov 20, 2020 at 9:37 PM Andreas Karlsson wrote: > On 11/20/20 9:57 AM, Andy Fan wrote: > > Thank you for your attention. Your suggestion would fix the issue. > However > > The difference will cause some risks when users move their ap

Re: Different results between PostgreSQL and Oracle for "for update" statement

2020-11-21 Thread Andy Fan
b int); begin; insert into t values(1,1); insert into t values(1, 1); commit; Oracle : t has 1 row, PG: t has 0 row (since the whole transaction is aborted). I don't mean we need to be the same as Oracle, but to support a customer who comes from Oracle, it would be good to know the difference. -- Best Regards Andy Fan

Re: Different results between PostgreSQL and Oracle for "for update" statement

2020-11-21 Thread Andy Fan
On Sat, Nov 21, 2020 at 11:27 PM Pavel Stehule wrote: > > > so 21. 11. 2020 v 9:59 odesílatel Andy Fan > napsal: > >> Thank all of you for your great insight! >> >> On Sat, Nov 21, 2020 at 9:04 AM Peter Geoghegan wrote: >> >>> On Fri, Nov

Re: Different results between PostgreSQL and Oracle for "for update" statement

2020-11-21 Thread Andy Fan
On Sun, Nov 22, 2020 at 5:56 AM Peter Geoghegan wrote: > On Sat, Nov 21, 2020 at 12:58 AM Andy Fan > wrote: > > I don't mean we need to be the same as Oracle, but to support a > > customer who comes from Oracle, it would be good to know the > > difference. > &

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-22 Thread Andy Fan
nlikely(TTS_SHOULDFREE(slot))) { VirtualTupleTableSlot *vslot = (VirtualTupleTableSlot *) slot; pfree(vslot->data); vslot->data = NULL; slot->tts_flags &= ~TTS_FLAG_SHOULDFREE; } slot->tts_nvalid = 0; slot->tts_flags |= TTS_FLAG_EMPTY; ItemPointerSetInvalid(&slot->tts_tid); } -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-22 Thread Andy Fan
On Sun, Nov 22, 2020 at 9:21 PM Andy Fan wrote: > Hi David: > > I did a review on the v8, it looks great to me. Here are some tiny > things noted, > just FYI. > > 1. modified src/include/utils/selfuncs.h > @@ -70,9 +70,9 @@ > * callers to provide further det

About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
es it doesn't work or is it something not worth doing? -- Best Regards Andy Fan

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Tue, Nov 24, 2020 at 11:11 PM Alvaro Herrera wrote: > On 2020-Nov-24, Andy Fan wrote: > > > then we modified the copy/read/out functions for this node. In > > _readFuncExpr, > > we probably add something like > > > [ ... ] > > > Then we will get a

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Wed, Nov 25, 2020 at 8:10 AM Andy Fan wrote: > > > On Tue, Nov 24, 2020 at 11:11 PM Alvaro Herrera > wrote: > >> On 2020-Nov-24, Andy Fan wrote: >> >> > then we modified the copy/read/out functions for this node. In >> > _read

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Wed, Nov 25, 2020 at 9:40 AM Tom Lane wrote: > Andy Fan writes: > > What I mean here is something like below. > > What exactly would be the value of that? > > There is work afoot, or at least on people's to-do lists, to mechanize > creation of the outfuncs/rea

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Wed, Nov 25, 2020 at 11:54 AM Tom Lane wrote: > Andy Fan writes: > > On Wed, Nov 25, 2020 at 9:40 AM Tom Lane wrote: > >> What exactly would be the value of that? > >> ... > > > I agree with this, but I don't think there is no value in my suggest

Re: [doc] plan invalidation when statistics are update

2020-11-24 Thread Andy Fan
t;have" and "updated" in the above "objects > used in the statement have updated"? > > I'm inclined to add "since the previous use of the prepared statement" into > also the second description, to make it clear. But if we do that, it's > better > to merge the above two description into one, as follows? > > whenever database objects used in the statement have undergone > - definitional (DDL) changes since the previous use of the prepared > + definitional (DDL) changes or the planner statistics of them have > + been updated since the previous use of the prepared > statement. Also, if the value of > changes > > > +1 for documenting this case since I just spent time reading code last week for it. and +1 for the above sentence to describe this case. -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-11-26 Thread Andy Fan
This patch has stopped moving for a while, any suggestion about how to move on is appreciated. -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-11-26 Thread Andy Fan
On Fri, Nov 27, 2020 at 8:10 AM David Rowley wrote: > Thanks for having another look at this. > > > On Sun, Nov 22, 2020 at 9:21 PM Andy Fan > wrote: > > add 2 more comments. > > > > 1. I'd suggest adding Assert(false); in RC_END_OF_SCAN case to

cost_sort vs cost_agg

2021-01-14 Thread Andy Fan
order of input data for the ordered column as well? c). store the Oids in SortPath and AggPath to avoid the double calculation during createPlan stage? or any better suggestion? Thanks -- Best Regards Andy Fan (https://www.aliyun.com/) v1-0001-Improve-the-cost_sort-v1.patch Description: Binary data

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2021-01-24 Thread Andy Fan
Hi Masahiko: On Fri, Jan 22, 2021 at 9:15 PM Masahiko Sawada wrote: > Hi Andy, > > On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote: > > > > > > > > On Mon, Dec 7, 2020 at 4:16 PM Jesper Pedersen < > jesper.peder...@redhat.com> wrote: > >> >

Extend more usecase for planning time partition pruning and init partition pruning.

2021-01-24 Thread Andy Fan
rn-and-extend-C.patch Just some existing refactoring and extending ChangeVarNodes to be able to change var->attno. - v1-0002-Build-some-implied-pruning-quals-to-extend-the-us.patch Do the real job. Thought? -- Best Regards Andy Fan (https://www.aliyun.com/) v1-0001-Make-some-static-functi

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-01-24 Thread Andy Fan
On Sun, Jan 24, 2021 at 6:34 PM Andy Fan wrote: > Hi: > > I recently found a use case like this. SELECT * FROM p, q WHERE > p.partkey = > q.colx AND (q.colx = $1 OR q.colx = $2); Then we can't do either planning > time > partition prune or init partition prune. Ev

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-02-07 Thread Andy Fan
On Mon, Jan 25, 2021 at 10:21 AM Andy Fan wrote: > > > On Sun, Jan 24, 2021 at 6:34 PM Andy Fan wrote: > >> Hi: >> >> I recently found a use case like this. SELECT * FROM p, q WHERE >> p.partkey = >> q.colx AND (q.colx = $1 OR q.colx = $2)

Re: cost_sort vs cost_agg

2021-02-08 Thread Andy Fan
Thank you Ashutosh. On Fri, Jan 15, 2021 at 7:18 PM Ashutosh Bapat wrote: > On Thu, Jan 14, 2021 at 7:12 PM Andy Fan wrote: > > > > Currently the cost_sort doesn't consider the number of columns to sort, > which > > means the cost of SELECT * FROM t ORDER

Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-09 Thread Andy Fan
far. Any thoughts? [1] https://www.postgresql.org/message-id/CAKU4AWr1BmbQB4F7j22G%2BNS4dNuem6dKaUf%2B1BK8me61uBgqqg%40mail.gmail.com -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-02-09 Thread Andy Fan
On Wed, Feb 10, 2021 at 11:18 AM Andy Fan wrote: > Hi: > > This patch is the first patch in UniqueKey patch series[1], since I need > to revise > that series many times but the first one would be not that often, so I'd > like to > submit this one for review first so tha

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2021-02-10 Thread Andy Fan
On Sun, Jan 24, 2021 at 6:26 PM Andy Fan wrote: > Hi Masahiko: > > On Fri, Jan 22, 2021 at 9:15 PM Masahiko Sawada > wrote: > >> Hi Andy, >> >> On Mon, Dec 7, 2020 at 9:15 PM Andy Fan wrote: >> > >> > >> > >> > On Mon, D

Re: Can I assume relation would not be invalid during from ExecutorRun to ExecutorEnd

2021-12-01 Thread Andy Fan
ion is open with some lock, then the content of the relation will never change until the RelationClose. It would take time to fill the gap, but I'd like to say "thank you!" first. -- Best Regards Andy Fan

Proposal for col LIKE $1 with generic Plan

2021-03-24 Thread Andy Fan
_or_null Index Cond: ((md5 >= $1::text) AND (text_less_than_or_null(md5, make_greater_string_fn($1))) Is this a right thing to do and a right method? Thanks -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Proposal for col LIKE $1 with generic Plan

2021-03-25 Thread Andy Fan
On Thu, Mar 25, 2021 at 10:15 AM Andy Fan wrote: > Thanks to the get_index_clause_from_support, we can use index for WHERE a > like > 'abc%' case. However this currently only works on custom plan. Think about > the > case where the planning takes lots of time, custom pl

Re: UniqueKey on Partitioned table.

2021-03-26 Thread Andy Fan
On Sat, Mar 27, 2021 at 3:07 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Sat, Feb 20, 2021 at 10:25:59AM +0800, Andy Fan wrote: > > > > The attached is a UnqiueKey with EquivalenceClass patch, I just complete > the > > single relation part and may have

Re: Extend more usecase for planning time partition pruning and init partition pruning.

2021-03-26 Thread Andy Fan
; [1] > https://www.postgresql.org/message-id/flat/CAKJS1f9fPdLKM6%3DSUZAGwucH3otbsPk6k0YT8-A1HgjFapL-zQ%40mail.gmail.com#024ad18e19bb9b6c022fb572edc8c992 > [2] > https://www.postgresql.org/message-id/flat/20190828234136.fk2ndqtld3onfrrp%40alap3.anarazel.de > [3] > https://www.postgresql.org/message-id/flat/30810.1449335...@sss.pgh.pa.us#906319f5e212fc3a6a682f16da079f04 > -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-27 Thread Andy Fan
On Tue, Feb 16, 2021 at 12:01 PM David Rowley wrote: > On Fri, 12 Feb 2021 at 15:18, Andy Fan wrote: > > > > On Fri, Feb 12, 2021 at 9:02 AM David Rowley > wrote: > >> The reason I don't really like this is that it really depends where > >> you want to

Re: UniqueKey on Partitioned table.

2021-03-29 Thread Andy Fan
On Tue, Mar 30, 2021 at 4:16 AM David Rowley wrote: > On Tue, 30 Mar 2021 at 02:27, Ashutosh Bapat > wrote: > > > > On Sat, Mar 27, 2021 at 11:44 AM Andy Fan > wrote: > > > > > > On Sat, Mar 27, 2021 at 3:07 AM Dmitry Dolgov <9erthali...@gmail.com>

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-03-30 Thread Andy Fan
s the length of rel->notnullattrs. 3). For some cases, the notnullattrs of a baserel is not changed in later stages, we can just reuse the same Bitmapset * in later stages. -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: What to call an executor node which lazily caches tuples in a hash table?

2021-03-30 Thread Andy Fan
the source code. When naming it, we may also think about some non native English speakers, so some too advanced words may make them uncomfortable. Actually when I read "Reactive", I googled to find what its meaning is. I knew reactive programming, but I do not truly understand "reactive hash". And Compared with HashJoin, Hash may mislead people the result may be spilled into disk as well. so I prefer "Cache" over "Hash". At last, I still want to vote for "Tuple(s) Cache", which sounds simple and enough. I was thinking if we need to put "Lazy" in the node name since we do build cache lazily, then I found we didn't call "Materialize" as "Lazy Materialize", so I think we can keep consistent. > I was hoping to commit the final patch pretty soon Very glad to see it, thanks for the great feature. -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: UniqueKey on Partitioned table.

2021-04-06 Thread Andy Fan
ead of bare expressions. > TBH, I haven't thought about this too hard, but I think when we build the UniqueKey, all the ECs have been built already. So can you think out an case we start with an EC with a single member at the beginning and have more members later for UniqueKey cases? -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-04-06 Thread Andy Fan
lution? > bool > is_var_notnullable(Var* var, Relids relids) > { > RelOptInfo *rel = find_rel_by_relids(reldis); > return bms_is_member(var->varattno, rel->notnullattrs[var->varno]); > } > > Probably we can make some hackers to reduce the notnullattrs's memory usage > overhead. > > -- Best Regards Andy Fan (https://www.aliyun.com/)

Cost model improvement for run-time partition prune

2021-04-07 Thread Andy Fan
ser case, looks the algorithm works great. I am planning to implement the full version recently. Any suggestion for the design/scope part? [1] https://www.postgresql.org/message-id/CAKU4AWpO4KegS6tw8UUnWA4GWr-Di%3DWBmuQnnyjxFGA0MhEHyA%40mail.gmail.com -- Best Regards Andy Fan (https://www.aliyun.com/)

Re: Wired if-statement in gen_partprune_steps_internal

2021-04-12 Thread Andy Fan
bms_add_members(relinfo->all_partrels, childrelinfo->relids); which would be more explicit to say add the child rt index to all_partrels. -- v1-0002-Split-gen_prune_steps_from_exprs-into-some-smalle.patch Just split the gen_prune_steps_from_opexps into some smaller chunks. The benefits are

Re: Difference for Binary format vs Text format for client-server communication

2020-07-26 Thread Andy Fan
On Sun, Jul 26, 2020 at 1:49 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-07-16 18:52, Andy Fan wrote: > > The reason I ask this is because I have a task to make numeric output > > similar to oracle. > > > > Oracle: >

Allows Extend Protocol support CURSOR_OPT_HOLD with prepared stmt.

2020-07-26 Thread Andy Fan
t it to true, server will create a holdable portal, or else nothing changed. Then let the user set it to true in the above case and reset it to false afterward. Is there any issue with this method? -- Best Regards Andy Fan

Re: Allows Extend Protocol support CURSOR_OPT_HOLD with prepared stmt.

2020-07-26 Thread Andy Fan
; > I forget to say in this case, the user has to drop the holdable portal explicitly. -- Best Regards Andy Fan

Re: Index Skip Scan (new UniqueKeys)

2020-08-02 Thread Andy Fan
ust add a EC checking for populate_baserel_uniquekeys. As for the DISTINCT/GROUP BY case, we should build the UniqueKeys from root->distinct_pathkeys and root->group_pathkeys where the EquivalenceClasses are already there. I am still not insisting on either Expr or EquivalenceClasses right now, if we need to change it to EquivalenceClasses, I'd see if we need to have more places to take care before doing that. -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-08-03 Thread Andy Fan
y access of unique_index->opfamily[c] – this array only has nkeycolumns > entries. > You are correct, I would include this in the next version patch, Thank you for this checking! -- Andy Fan Best Regards > > > > > *From:* Andy Fan > *Sent:* Sunday 19 July 2020 5:

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-05 Thread Andy Fan
lpful to check that a stale plan > still works. > > regards, tom lane > > > -- Best Regards Andy Fan

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-06 Thread Andy Fan
On Thu, Aug 6, 2020 at 12:02 PM Tom Lane wrote: > Andy Fan writes: > > On Thu, Aug 6, 2020 at 2:22 AM Tom Lane wrote: > >> In the longer term, it's annoying that we have no test methodology > >> for this other than "manually set a breakpoint here". >

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-06 Thread Andy Fan
On Thu, Aug 6, 2020 at 10:42 PM Tom Lane wrote: > Andy Fan writes: > > On Thu, Aug 6, 2020 at 12:02 PM Tom Lane wrote: > >> See my straw-man proposal downthread. > > > Thanks for your explanation, I checked it again and it looks a very clean > > method. The a

Re: FailedAssertion("pd_idx == pinfo->nparts", File: "execPartition.c", Line: 1689)

2020-08-06 Thread Andy Fan
On Fri, Aug 7, 2020 at 8:32 AM Tom Lane wrote: > Andy Fan writes: > > Attached is the v2 patch. > > Forgot to mention that I'd envisioned adding this as a src/test/modules/ > module; contrib/ is for things that we intend to expose to users, which > I think this isn&#

Can I test Extended Query in core test framework

2020-08-10 Thread Andy Fan
ied grep '\->prepare' and '\->execute' and get nothing. am I miss something? -- Best Regards Andy Fan

Re: Can I test Extended Query in core test framework

2020-08-11 Thread Andy Fan
to test extended queries. [1] https://www.postgresql.org/message-id/CAKU4AWqvwmo=nlpga_ohxb4f+u4ts1_3yry9m6xtjlt9dkh...@mail.gmail.com -- Best Regards Andy Fan

  1   2   3   4   5   6   7   >