some hints to understand the plsql cursor.

2019-02-27 Thread Andy Fan
actually I'm hacking pg for a function like : 1. define a select query. 2. client ask for some data. and server reply some data. server will do NOTHING if client doesn't ask any more.. 3. client ask some data more data with a batch and SERVER reply some data then. then do NOTHING. currently the

Re: When is the MessageContext released?

2019-02-27 Thread Andy Fan
Thanks you Andres for your time! this context is free with AllocSetReset rather than AllocSetDelete, that makes my breakpoint doesn't catch it. On Wed, Feb 27, 2019 at 2:15 PM Andres Freund wrote: > On 2019-02-27 14:08:47 +0800, Andy Fan wrote: > > Hi : > > I run a query like

Looks heap_create_with_catalog ignored the if_not_exists options

2019-03-01 Thread Andy Fan
for a createStmt, it will call transformCreateStmt, and then heap_create_with_catalog. but looks it just check the if_not_exists in transformCreateStmt. so there is a chance that when the transformCreateStmt is called, the table is not created, but before the heap_create_with_catalog is called,

Re: Looks heap_create_with_catalog ignored the if_not_exists options

2019-03-01 Thread Andy Fan
quier wrote: > On Fri, Mar 01, 2019 at 07:17:04PM +0800, Andy Fan wrote: > > for a createStmt, it will call transformCreateStmt, and then > > heap_create_with_catalog. > > but looks it just check the if_not_exists in transformCreateStmt. > > > > is it designed

When is the MessageContext released?

2019-02-26 Thread Andy Fan
Hi : I run a query like "select * from t" and set the break like this: break exec_simple_query break MemoryContextDelete commands p context->name c end I can see most of the MemoryContext is relased, but never MessageContext, when will it be released? /* * Create the memory context

any plan to support shared servers like Oracle in PG?

2019-03-05 Thread Andy Fan
currently there is one process per connection and it will not not very good for some short time connection.In oracle database, it support shared server which can serve more than 1 users at the same time. See https://docs.oracle.com/cd/B28359_01/server.111/b28310/manproc001.htm#ADMIN11166

Re: some hints to understand the plsql cursor.

2019-02-27 Thread Andy Fan
code definitely makes the debug much quicker. Thank you very much! On Wed, Feb 27, 2019 at 11:35 PM Dilip Kumar wrote: > On Wed, Feb 27, 2019 at 4:42 PM Andy Fan wrote: > > > > actually I'm hacking pg for a function like : > > 1. define a select query. > >

Re: Suggestions on message transfer among backends

2019-03-12 Thread Andy Fan
On Tue, Mar 12, 2019 at 1:59 PM Andrey Lepikhov wrote: > On 11/03/2019 18:36, Andy Fan wrote: > > Hi: > >I need some function which requires some message exchange among > > different back-ends (connections). > > specially I need a shared hash map and a message q

Re: Suggestions on message transfer among backends

2019-03-12 Thread Andy Fan
On Tue, Mar 12, 2019 at 2:36 PM Andy Fan wrote: > On Tue, Mar 12, 2019 at 1:59 PM Andrey Lepikhov > wrote: > >> On 11/03/2019 18:36, Andy Fan wrote: >> > Hi: >> >I need some function which requires some message exchange among >> > different back

Re: what makes the PL cursor life-cycle must be in the same transaction?

2019-03-10 Thread Andy Fan
DECLARE cur CURSOR with hold FOR SELECT * FROM t; the "with hold" is designed for this purpose. sorry for this interruption. On Sun, Mar 10, 2019 at 4:14 PM Andy Fan wrote: > for example: > begin; > declare cur cursor for select * from t; > insert into t2 values(

Re: I have some troubles to run test_shm_mq;

2019-03-11 Thread Andy Fan
Works, thank you Thomas! I have spent more than 2 hours on this. do you know which document I miss for this question? Thanks On Mon, Mar 11, 2019 at 4:05 PM Thomas Munro wrote: > On Mon, Mar 11, 2019 at 8:59 PM Andy Fan wrote: > > 4. CREATE EXTENSION test_shm_mq; ==> . co

I have some troubles to run test_shm_mq;

2019-03-11 Thread Andy Fan
My code is based on commit zhifan@zhifandeMacBook-Pro ~/g/polardb_clean> git log commit d06fe6ce2c79420fd19ac89ace81b66579f08493 Author: Tom Lane Date: Tue Nov 6 18:56:26 2018 -0500 what I did includes: 1. ./configure --enable-debug 2. make world // doesn't see the test_shm_mq on the

Re: I have some troubles to run test_shm_mq;

2019-03-11 Thread Andy Fan
and whenever I run a simple query "SELECT test_shm_mq(1024, 'a');" I see the following log 2019-03-11 16:33:17.800 CST [65021] LOG: background worker "test_shm_mq" (PID 65052) exited with exit code 1 does it indicates something wrong? On Mon, Mar 11, 2019 at 4:3

Re: I have some troubles to run test_shm_mq;

2019-03-11 Thread Andy Fan
Thanks for the clarification! On Mon, Mar 11, 2019 at 5:02 PM Thomas Munro wrote: > On Mon, Mar 11, 2019 at 9:35 PM Andy Fan wrote: > > and whenever I run a simple query "SELECT test_shm_mq(1024, 'a');" > > > > I see the following log > > > &

Suggestions on message transfer among backends

2019-03-11 Thread Andy Fan
Hi: I need some function which requires some message exchange among different back-ends (connections). specially I need a shared hash map and a message queue. Message queue: it should be many writers, 1 reader. Looks POSIX message queue should be OK, but postgre doesn't use it. is there

Re: Suggestions on message transfer among backends

2019-03-11 Thread Andy Fan
notes on the shared hash map: it needs multi writers and multi readers. On Mon, Mar 11, 2019 at 9:36 PM Andy Fan wrote: > Hi: > I need some function which requires some message exchange among > different back-ends (connections). > specially I need a shared hash map and a m

what makes the PL cursor life-cycle must be in the same transaction?

2019-03-10 Thread Andy Fan
for example: begin; declare cur cursor for select * from t; insert into t2 values(...); fetch next cur; commit; // after this, I can't fetch cur any more. My question are: 1. Is this must in principle? or it is easy to implement as this in PG? 2. Any bad thing would happen if I keep the

Re: any plan to support shared servers like Oracle in PG?

2019-03-07 Thread Andy Fan
thank you for this information! takes 2 days to read the discussion.. On Wed, Mar 6, 2019 at 3:13 AM legrand legrand wrote: > There already are solutions regarding this feature in Postgres > using "connection pooler" wording > > see > > pgpool:

Question about the holdable cursor

2019-04-18 Thread Andy Fan
when I fetch from holdable cursor, I found the fact is more complex than I expected. suppose we fetched 20 rows. 1). It will fill a PortalStore, the dest is not the client, it is the DestTupleStore, called ExecutePlan once and receiveSlot will be call 20 times. 2). the portal for client then

Re: Question about the holdable cursor

2019-04-18 Thread Andy Fan
On Thu, Apr 18, 2019 at 10:09 PM Tom Lane wrote: > Andy Fan writes: > > when I fetch from holdable cursor, I found the fact is more complex > than I > > expected. > > ... > > why the 3rd time is necessary and will the performance be bad due to this > > desi

How to include the header files effectively

2019-04-11 Thread Andy Fan
I find the dependency is complex among header files in PG. At the same time, I find the existing code still can use the header file very cleanly/alphabetically. so I probably missed some knowledge here. for example, when I want the LOCKTAG in .c file, which is defined in "storage/lock.h".

Re: why doesn't optimizer can pull up where a > ( ... )

2019-11-21 Thread Andy Fan
On Thu, Nov 21, 2019 at 6:12 PM Tomas Vondra wrote: > On Thu, Nov 21, 2019 at 08:30:51AM +0800, Andy Fan wrote: > >> > >> > >> Hm. That actually raises the stakes a great deal, because if that's > >> what you're expecting, it would requ

what is the purpose to use 3 function to compare cost add_path/set_cheapest/get_cheapest_fractional_path

2019-12-05 Thread Andy Fan
Hello Hackers: I'm reading the code of optimizer and get confused about the 3 functions. add_path/set_cheapest/get_cheapest_fractional_path add_(partial_)path: For every relations, optimizer will build path for it and add then call add_path to the rel->pathlist. during this stage, *it

Re: why doesn't optimizer can pull up where a > ( ... )

2019-11-20 Thread Andy Fan
On Wed, Nov 20, 2019 at 8:15 PM Andy Fan wrote: > Hi Hackers: > > First I found the following queries running bad on pg. > > select count(*) from part2 p1 where p_size > 40 and p_retailprice > > (select avg(p_retailprice) from part2 p2 where p2.p_brand=p

why doesn't optimizer can pull up where a > ( ... )

2019-11-20 Thread Andy Fan
Hi Hackers: First I found the following queries running bad on pg. select count(*) from part2 p1 where p_size > 40 and p_retailprice > (select avg(p_retailprice) from part2 p2 where p2.p_brand=p1.p_brand); the plan is QUERY PLAN

Re: Planner chose a much slower plan in hashjoin, using a large table as the inner table.

2019-11-28 Thread Andy Fan
On Fri, Nov 22, 2019 at 6:51 PM Jinbao Chen wrote: > Hi hackers, > > I have made a patch to fix the problem. > > Added the selection rate of the inner table non-empty bucket > > The planner will use big table as inner table in hash join > if small table have fewer unique values. But this plan is

Re: why doesn't optimizer can pull up where a > ( ... )

2019-11-20 Thread Andy Fan
> > > Hm. That actually raises the stakes a great deal, because if that's > what you're expecting, it would require planning out both the transformed > and untransformed versions of the query before you could make a cost > comparison. I don't know an official name, let's call it as "bloom

Re: function calls optimization

2019-11-20 Thread Andy Fan
On Thu, Oct 31, 2019 at 11:07 PM Tom Lane wrote: > > > Possibly this could be finessed by only trying to find duplicates of > functions that have high cost estimates. Not sure how high is high > enough. can we just add a flag on pg_proc to show if the cost is high or not, if user are not

Re: Planner chose a much slower plan in hashjoin, using a large table as the inner table.

2019-11-28 Thread Andy Fan
ate_series(1, 1)i ; > analyze t_small; > analyze t_big; > set max_parallel_workers_per_gather = 0; > > On Thu, Nov 28, 2019 at 5:46 PM Andy Fan wrote: > >> >> >> On Fri, Nov 22, 2019 at 6:51 PM Jinbao Chen wrote: >> >>> Hi hacke

Dynamic gathering the values for seq_page_cost/xxx_cost

2019-11-25 Thread Andy Fan
The optimizer cost model usually needs 2 inputs, one is used to represent data distribution and the other one is used to represent the capacity of the hardware, like cpu/io let's call this one as system stats. In Oracle database, the system stats can be gathered with

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-13 Thread Andy Fan
On Thu, Feb 13, 2020 at 5:39 PM Julien Rouhaud wrote: > On Tue, Feb 11, 2020 at 10:06:17PM +0530, Ashutosh Bapat wrote: > > On Tue, Feb 11, 2020 at 8:27 AM Andy Fan > wrote: > > > > > On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < > > &

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-10 Thread Andy Fan
On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > > >> >> [PATCH] Erase the distinctClause if the result is unique by >> definition >> > > I forgot to mention this in the last round of comments. Your patch was > actually removing distictClause from the

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-04 Thread Andy Fan
> > >> * There are some changes in existing regression cases that aren't >> visibly related to the stated purpose of the patch, eg it now >> notices that "select distinct max(unique2) from tenk1" doesn't >> require an explicit DISTINCT step. That's not wrong, but I wonder >> if maybe you should

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-02 Thread Andy Fan
Thank you Tom for the review! On Mon, Mar 2, 2020 at 4:46 AM Tom Lane wrote: > Andy Fan writes: > > Please see if you have any comments. Thanks > > The cfbot isn't at all happy with this. Its linux build is complaining > about a possibly-uninitialized variable, and then gi

Re: Trying to pull up EXPR SubLinks

2020-02-27 Thread Andy Fan
On Fri, Feb 28, 2020 at 2:35 PM Richard Guo wrote: > Hi All, > > Currently we will not consider EXPR_SUBLINK when pulling up sublinks and > this would cause performance issues for some queries with the form of: > 'a > (SELECT agg(b) from ...)' as described in [1]. > > So here is a patch as an

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-06 Thread Andy Fan
. But since the current patch not tied with this closely, so I would put this patch for review first. On Wed, Mar 4, 2020 at 9:13 PM Andy Fan wrote: > > >> >>> * There are some changes in existing regression cases that aren't >>> visibly related to the stated p

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-24 Thread Andy Fan
. Thanks On Mon, Feb 24, 2020 at 8:38 PM Andy Fan wrote: > > > On Wed, Feb 12, 2020 at 12:36 AM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> >> >> On Tue, Feb 11, 2020 at 8:27 AM Andy Fan >> wrote: >> >>

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-24 Thread Andy Fan
On Wed, Feb 12, 2020 at 12:36 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > > > On Tue, Feb 11, 2020 at 8:27 AM Andy Fan wrote: > >> >> >> On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < >> ashutosh.bapat@gmail.com> wrote: >&

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-02 Thread Andy Fan
On Tue, Mar 3, 2020 at 1:24 AM Andy Fan wrote: > Thank you Tom for the review! > > On Mon, Mar 2, 2020 at 4:46 AM Tom Lane wrote: > >> Andy Fan writes: >> > Please see if you have any comments. Thanks >> >> The cfbot isn't at all happy with this.

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-05 Thread Andy Fan
update the patch with considering the semi/anti join. Can anyone help to review this patch? Thanks On Fri, Jan 31, 2020 at 8:39 PM Andy Fan wrote: > Hi: > > I wrote a patch to erase the distinctClause if the result is unique by > definition, I find this because a user switch th

[PATCH] Erase the distinctClause if the result is unique by definition

2020-01-31 Thread Andy Fan
Hi: I wrote a patch to erase the distinctClause if the result is unique by definition, I find this because a user switch this code from oracle to PG and find the performance is bad due to this, so I adapt pg for this as well. This patch doesn't work for a well-written SQL, but some drawback

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-07 Thread Andy Fan
en we create a node for Unique/HashAggregate/Group, we can just create a dummy node? > 5. Have you tested this OUTER joins, which can render inner side nullable? > Yes, that part was missed in the test case. I just added them. On Thu, Feb 6, 2020 at 11:31 AM Andy Fan wrote: > >>

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Andy Fan
On Tue, Feb 11, 2020 at 3:56 PM Julien Rouhaud wrote: > On Tue, Feb 11, 2020 at 10:57:26AM +0800, Andy Fan wrote: > > On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < > > ashutosh.bapat@gmail.com> wrote: > > > > > I forgot to mention this in th

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Andy Fan
On Tue, Feb 11, 2020 at 3:56 PM Julien Rouhaud wrote: > > > > and if we prepare sql outside a transaction, and execute it in the > > transaction, the other session can't drop the constraint until the > > transaction is ended. > > And what if you create a view on top of a query containing a

Re: How to make a OpExpr check compatible among different versions

2020-01-13 Thread Andy Fan
On Mon, Jan 13, 2020 at 4:09 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2020-01-13 08:29, Andy Fan wrote: > > During one of my works for logical rewrite, I want to check if the expr > > is a given Expr. > > > > so the simplest way is: >

How to make a OpExpr check compatible among different versions

2020-01-12 Thread Andy Fan
During one of my works for logical rewrite, I want to check if the expr is a given Expr. so the simplest way is: if (expr->opno == 418 && nodeTag(linitial(expr->args)) == T_xxx && nodeTag(lsecond(expr->args)) == T_ ) { .. } if we write code like above, we may have issues if the oid

[PATCH] query rewrite for distinct stage under some cases

2020-01-19 Thread Andy Fan
Hi Hackers: This is a patch for unique elimination rewrite for distinct query. it will cost much for a big result set and some times it is not necessary. The basic idea is the unique node like in the following can be eliminated. 1. select distinct pk, ... from t; 2. select distinct

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-10 Thread Andy Fan
On Wed, Mar 11, 2020 at 6:49 AM David Rowley wrote: > On Wed, 11 Mar 2020 at 02:50, Ashutosh Bapat > wrote: > > > > On Tue, Mar 10, 2020 at 1:49 PM Andy Fan > wrote: > > > In my current implementation, it calculates the uniqueness for each > > > BaseRe

Re: Index Skip Scan

2020-03-10 Thread Andy Fan
> > > I think the UniqueKeys may need to be changed from using > EquivalenceClasses to use Exprs instead. > When I try to understand why UniqueKeys needs EquivalenceClasses, see your comments here. I feel that FuncExpr can't be used to as a UniquePath even we can create unique index on f(a) and

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-10 Thread Andy Fan
t looks at unique > indexes and group by / distinct clauses. > > I can do this after we have agreement on the UniquePath. For my cbbot failure, another strange thing is "A" appear ahead of "a" after the order by.. Still didn't find out why. [1] https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.83298 Regards Andy Fan

Re: Index Skip Scan

2020-03-11 Thread Andy Fan
On Tue, Mar 10, 2020 at 4:32 AM James Coleman wrote: > On Mon, Mar 9, 2020 at 3:56 PM Dmitry Dolgov <9erthali...@gmail.com> > wrote: > > > > Assuming we'll implement it in a way that we do not know about what kind > > of path type is that in create_distinct_path, then it can also work for > >

Questions about the CI process and proposal

2020-03-06 Thread Andy Fan
2433/ Regards Andy Fan.

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-10 Thread Andy Fan
find this in your patch or in the code. > > This is a proposal from David, so not in current patch/code :) Regards Andy Fan

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-12 Thread Andy Fan
Hi David: On Thu, Mar 12, 2020 at 3:51 PM David Rowley wrote: > On Wed, 11 Mar 2020 at 17:23, Andy Fan wrote: > > Now I am convinced that we should maintain UniquePath on RelOptInfo, > > I would see how to work with "Index Skip Scan" patch. > > I've attached a ve

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-12 Thread Andy Fan
On Fri, Mar 13, 2020 at 11:46 AM David Rowley wrote: > On Fri, 13 Mar 2020 at 14:47, Andy Fan wrote: > > 1. for pupulate_baserel_uniquekeys, we need handle the "pk = Const" > as well. > > (relation_has_unqiue_for has a similar logic) currently the follow

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-14 Thread Andy Fan
Hi David: Thanks for your time. > 1. Out of date comment in join.sql > > -- join removal is not possible when the GROUP BY contains a column that is > -- not in the join condition. (Note: as of 9.6, we notice that b.id is a > -- primary key and so drop b.c_id from the GROUP BY of the resulting

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-16 Thread Andy Fan
On Thu, Apr 16, 2020 at 8:36 PM Ashutosh Bapat wrote: > On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote: > > > (9 rows) > > > > With this feature: > > explain analyze select a, sum(c) from grp2 group by a; > >

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 4:58 PM Amit Langote wrote: > On Tue, Apr 14, 2020 at 5:29 PM Andy Fan wrote: > > On Tue, Apr 14, 2020 at 3:40 PM Amit Langote > wrote: > >> On Tue, Apr 14, 2020 at 4:13 PM Richard Guo > wrote: > >> > On Tue, Apr 14, 2020 at 2:44 PM

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 3:40 PM Amit Langote wrote: > On Tue, Apr 14, 2020 at 4:13 PM Richard Guo > wrote: > > On Tue, Apr 14, 2020 at 2:44 PM Amit Langote > wrote: > >> Maybe I am missing something obvious, but is it intentional that > >> enable_indexscan is checked by cost_index(), that is,

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 5:12 PM Andy Fan wrote: > > > On Tue, Apr 14, 2020 at 4:58 PM Amit Langote > wrote: > >> On Tue, Apr 14, 2020 at 5:29 PM Andy Fan >> wrote: >> > On Tue, Apr 14, 2020 at 3:40 PM Amit Langote >> wrote: >> >>

Re: WIP: Aggregation push-down

2020-04-21 Thread Andy Fan
more tests on your patch, the more powerful I feel it is! At the same time, I think the most difficult part to understand your design is you can accept any number of generic join clauses, so I guess more explanation on this part may be helpful. At the code level, I did som

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-15 Thread Andy Fan
strategy sounds awesome, but I didn't check the details so far. 5. more clearer commit message. 6. any more ? Any feedback is welcome, Thanks for you for your any ideas, suggestions, demo code! Best Regards Andy Fan v4-0001-Patch-Bypass-distinctClause-groupbyClause-if-the-.patch Descriptio

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-03-25 Thread Andy Fan
Because I replied the old thread, cfbot run a test based on the old patch on that thread. I have detached the old thread from commitfest. Reply this email again to wake up Mr. cfbot with the right information. v2-0001-Maintain-the-uniqueness-of-a-Query-from-bottom-to.patch Description:

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-02 Thread Andy Fan
The updated patch should fixed all the issues. See the comments below for more information. On Tue, Mar 31, 2020 at 9:44 AM David Rowley wrote: > On Sun, 29 Mar 2020 at 20:50, Andy Fan wrote: > > Some other changes made in the new patch: > > 1. Fixed bug for UniqueKey calcul

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-03 Thread Andy Fan
upposed in some past discussions.) > > Agreed for remove_useless_groupby_columns(), but we'd need it if we > wanted to detect functional dependencies in > check_functional_grouping() using unique indexes. > Thanks for the explanation. I will add the removal in the next version of this patch. Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-03-25 Thread Andy Fan
On Mon, Mar 23, 2020 at 6:21 PM Andy Fan wrote: > Greetings. > > This thread is a follow-up thread for [1], where I submit a patch for > erasing the > distinct node if we have known the data is unique for sure. But since the > implementation has changed a lot

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-25 Thread Andy Fan
I have started the new thread [1] to continue talking about this. Mr. cfbot is happy now. [1] https://www.postgresql.org/message-id/flat/CAKU4AWrwZMAL%3DuaFUDMf4WGOVkEL3ONbatqju9nSXTUucpp_pw%40mail.gmail.com Thanks >

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-03-29 Thread Andy Fan
3. Renamed the field "grantee" as "guarantee". Best Regards Andy Fan v3-0001-Maintain-UniqueKey-at-each-RelOptInfo-this-inform.patch Description: Binary data

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-03-31 Thread Andy Fan
Thanks David for your time, I will acknowledge every item you mentioned with the updated patch. Now I just talk about part of them. > 1. There seem to be some cases where joins are no longer being > detected as unique. This is evident in postgres_fdw.out. We shouldn't > be regressing any of

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-28 Thread Andy Fan
On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote: > On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote: > > V6 also includes: > > 1. Fix the comment misleading you mentioned above. > > 2. Fixed a concern case for `relation_has_uniquekeys_for` function. > > Over

Re: Pulling up sublink may break join-removal logic

2020-04-28 Thread Andy Fan
ith UniqueKey respect. search "In the past we have some limited ability to detect the unqiueness after join, so that's would be ok. Since we have such ability now, this may be another opportunity to improve the join_is_removable function" I'm checking it today and will have a feedback soon. [1] https://www.postgresql.org/message-id/CAKU4AWrGrs0Vk5OrZmS1gbTA2ijDH18NHKnXZTPZNuupn%2B%2Bing%40mail.gmail.com Best Regards Andy Fan

Re: Trying to pull up EXPR SubLinks

2020-04-24 Thread Andy Fan
an that, like. (1, 2, 1) and (1, 1, 2) is same for your suggestion, but they are not different in this path. and we also may be think about if we can get a lower cost if we add a new sort path. Best Regards Andy Fan

Re: Trying to pull up EXPR SubLinks

2020-04-24 Thread Andy Fan
On Fri, Apr 24, 2020 at 5:24 PM David Rowley wrote: > On Fri, 24 Apr 2020 at 15:26, Andy Fan wrote: > > > > Actually I have a different opinion to handle this issue, to execute the > > a > (select avg(a) from tinner where x = touer.x); The drawback of > current

Re: Trying to pull up EXPR SubLinks

2020-04-23 Thread Andy Fan
; But when we take this action to production case, how to cost this strategy is challenge since it can neither reduce the total_cost nor result in a new PathKey. I will check other place to see how this kind can be added. Best Regards Andy Fan test.sql Description: Binary data v1-0001-Add-a

Re: Subplan result caching

2020-04-26 Thread Andy Fan
nly if the datum really changed, will it still be semantic correctly. It's great to see someone working on this. > I'd like to have a try. Best Regards Andy Fan

Re: WIP: Aggregation push-down

2020-04-26 Thread Andy Fan
On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska wrote: > Andy Fan wrote: > > > The more tests on your patch, the more powerful I feel it is! > > Thanks for the appreciation. Given the poor progress it's increasingly hard > for me to find motivation to work on it. I'll try to

Re: Subplan result caching

2020-04-26 Thread Andy Fan
On Sun, Apr 26, 2020 at 5:49 PM David Rowley wrote: > On Sun, 26 Apr 2020 at 19:08, Andy Fan wrote: > > If we want to handle this case as well, one of the changes would > > be it needs to cache multi records for one input parameter, or return > > one row each time but retur

Re: WIP: Aggregation push-down

2020-04-21 Thread Andy Fan
precious :)Just FYI [1] https://www.postgresql.org/message-id/9726.1542577...@sss.pgh.pa.us Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-29 Thread Andy Fan
On Wed, Apr 29, 2020 at 8:34 AM Andy Fan wrote: > > > On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote: > >> On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote: >> > V6 also includes: >> > 1. Fix the comment misleading you mentioned

Can we remove the other_rels_list parameter for make_rels_by_clause_joins

2020-04-29 Thread Andy Fan
and use root->inital_rels directly. I did the same for make_rels_by_clauseless_joins. The attached is my proposal which should be semantic correctly and more explicitly. Best Regards Andy Fan v1-0001-Remove-other_rels_list-from-make_rels_by_clause_j.patch Description: Binary data

Re: Add "-Wimplicit-fallthrough" to default flags

2020-05-13 Thread Andy Fan
; > > /* Fall through. */ > > default: > > It is generated code by bison. > > $ bison --version > bison (GNU Bison) 3.0.4 > > I just found this just serval minutes ago. Upgrading your bison to the latest version (3.6) is ok. I'd like we have a better way to share this knowledge through. I spend ~30 minutes to troubleshooting this issue. Best Regards Andy Fan

Re: No core file generated after PostgresNode->start

2020-05-12 Thread Andy Fan
On Tue, May 12, 2020 at 3:36 AM Robert Haas wrote: > On Sun, May 10, 2020 at 11:21 PM Andy Fan > wrote: > > Looks this doesn't mean a crash. If the test case(subscription/t/ > 013_partition.pl) > > failed, test framework kill some process, which leads the above &

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
ge the multi_nullvals to true at this case */ >> >> Need a comment explaining this. Generally, I feel, this and other >> functions in >> this file need good comments explaining the logic esp. "why" instead of >> "what". > > > Exactly. > Done in v8. >> Will continue reviewing your new set of patches as time permits. >> > > Thank you! Actually there is no big difference between v6 and v7 > regarding the > UniqueKey part except 2 bug fix. However v7 has some more documents, > comments improvement and code refactor/split, which may be helpful > for review. You may try v7 next time if v8 has not come yet:) > > v8 has come :) Best Regards Andy Fan

Find query characters in respect of optimizer for develop purpose

2020-05-18 Thread Andy Fan
of standard_planner, and the values are increased at make_subplan, set_subquery_pathlist, make_one_rel, create_grouping_paths. later it can be tracked and viewed in pg_stat_statements. What do you think about the requirement and the method I am thinking? Any kind of feedback is welcome. -- Best Regards Andy

Re: No core file generated after PostgresNode->start

2020-05-10 Thread Andy Fan
On Mon, May 11, 2020 at 9:48 AM Andy Fan wrote: > Hi: > > > 2020-05-11 09:37:40.778 CST [69541] sub_viaroot WARNING: terminating > connection because of crash of another server process > > Looks this doesn't mean a crash. If the test case(subscription/t/ 013_partit

No core file generated after PostgresNode->start

2020-05-10 Thread Andy Fan
gt;data_dir, '-l', $self->logfile, '-o', "--cluster-name=$name", 'start'); } Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Wed, May 13, 2020 at 11:48 PM Andy Fan wrote: > >> My impression about the one row stuff, is that there is too much >> special casing around it. We should somehow structure the UniqueKey >> data so that one row unique keys come naturally rather than special >&

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Thu, May 14, 2020 at 6:20 AM David Rowley wrote: > On Thu, 14 May 2020 at 03:48, Andy Fan wrote: > > On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> My impression about the one row stuff, is that there is too much > &

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-05-18 Thread Andy Fan
in nature. However I don't think the above 2 will cause big issues. I added the columns to V1_8 rather than adding a new version. this can be changed at final patch. Any suggestions? Best Regards Andy Fan v1-0001-Add-query-characters-information-to-pg_stat_state.patch Description: Binary data

Re: Add "-Wimplicit-fallthrough" to default flags

2020-05-14 Thread Andy Fan
On Wed, May 13, 2020 at 10:02 PM Tom Lane wrote: > Andy Fan writes: > >> FWIW, I got a warning for jsonpath_gram.c. > > Ugh. Confirmed here on Fedora 30 (bison 3.0.5). > > > I just found this just serval minutes ago. Upgrading your bison to the > > latest vers

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat wrote: > On Fri, May 8, 2020 at 7:27 AM Andy Fan wrote: > > >> +else if (inner_is_onerow) > >> +{ > >> +/* Since rows in innerrel can't be duplicat

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-17 Thread Andy Fan
Hi David: On Wed, Mar 18, 2020 at 12:13 PM David Rowley wrote: > On Wed, 18 Mar 2020 at 15:57, Andy Fan wrote: > > I'm now writing the code for partition index stuff, which > > is a bit of boring, since every partition may have different unique > index. > > Why i

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-17 Thread Andy Fan
Hi David: Thanks for your time. On Wed, Mar 18, 2020 at 9:56 AM David Rowley wrote: > On Mon, 16 Mar 2020 at 06:01, Andy Fan wrote: > > > > Hi All: > > > > I have re-implemented the patch based on David's suggestion/code, Looks > it > > works well.

[PATCH] Keeps tracking the uniqueness with UniqueKey

2020-03-23 Thread Andy Fan
-%2BL4L2%2B0xknsEqpfcs9FF7SeiO9TmpeZOg%40mail.gmail.com#f5d97cc66b9cd330add2fbb004a4d107 [2] https://www.postgresql.org/message-id/CAKU4AWqOORqW900O-%2BL4L2%2B0xknsEqpfcs9FF7SeiO9TmpeZOg%40mail.gmail.com Best regards Andy Fan v1-0001-Maintain-the-uniqueness-of-a-Query-from-bottom-to.patch Description: Binary data

Re: Index Skip Scan

2020-03-23 Thread Andy Fan
> > > On Mon, Mar 23, 2020 at 1:55 AM Floris Van Nee > wrote: > > I'm unsure which version number to give this patch (to continue with > numbers from previous skip scan patches, or to start numbering from scratch > again). It's a rather big change, so one could argue it's mostly a separate >

Re: Index Skip Scan

2020-03-24 Thread Andy Fan
On Wed, Mar 25, 2020 at 12:41 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Wed, Mar 11, 2020 at 06:56:09PM +0800, Andy Fan wrote: > > > > There was a dedicated thread [1] where David explain his idea very > > detailed, and you can also check tha

Re: Implementing Incremental View Maintenance

2020-05-07 Thread Andy Fan
id you test it? AFAIK, we can test it with: 1. For any query like SELECT xxx, we create view like CREATE MATERIAL VIEW mv_name as SELECT xxx; to test if the features in the query are supported. 2. Update the data and then compare the result with SELECT XXX with SELECT * from mv_name to test if the data is correctly sync. Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-07 Thread Andy Fan
h the positions across > subquery and the outer query. > but I think it should be rel->subroot->processed_tlist rather than reltarget? Actually I still a bit of uneasy about rel->subroot->processed_tlist for some DML case, which the processed_tlist is different and I still not figure out its impact. > Will continue reviewing your new set of patches as time permits. > Thank you! Actually there is no big difference between v6 and v7 regarding the UniqueKey part except 2 bug fix. However v7 has some more documents, comments improvement and code refactor/split, which may be helpful for review. You may try v7 next time if v8 has not come yet:) Best Regards Andy Fan

Re: Subplan result caching

2020-05-20 Thread Andy Fan
On Wed, May 20, 2020 at 7:47 PM David Rowley wrote: > On Mon, 27 Apr 2020 at 00:37, Andy Fan wrote: > > I was feeling that we may have to maintain some extra status if we use > hash > > table rather than tuple store, but that might be not a major concern. I > can > &

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-05-21 Thread Andy Fan
e where j1o.i = 1; -- Best Regards Andy Fan

  1   2   3   4   5   6   >