Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-06-04 Thread Andy Fan
On Fri, Jun 5, 2020 at 10:57 AM David Rowley wrote: > On Fri, 5 Jun 2020 at 14:36, Andy Fan wrote: > > On Mon, May 25, 2020 at 2:34 AM David Rowley > wrote: > >> > >> On Sun, 24 May 2020 at 04:14, Dmitry Dolgov <9erthali...@gmail.com> > wrote: > >&g

Re: A wrong index choose issue because of inaccurate statistics

2020-06-08 Thread Andy Fan
On Fri, Jun 5, 2020 at 2:30 PM Pavel Stehule wrote: > > > pá 5. 6. 2020 v 8:19 odesílatel David Rowley > napsal: > >> On Mon, 1 Jun 2020 at 01:24, Andy Fan wrote: >> > The one-line fix describe the exact idea in my mind: >> > >> > +++ b/src/

Re: A wrong index choose issue because of inaccurate statistics

2020-06-08 Thread Andy Fan
pá 5. 6. 2020 v 8:19 odesílatel David Rowley > napsal: > >> > >> On Mon, 1 Jun 2020 at 01:24, Andy Fan wrote: > >> > The one-line fix describe the exact idea in my mind: > >> > > >> > +++ b/src/backend/optimizer/path/costsize.c &g

Re: Index Skip Scan (new UniqueKeys)

2020-06-11 Thread Andy Fan
1 (2 rows) demo=# set enable_indexskipscan to on; SET demo=# select distinct im5, i, im100 from j1 where i < 2; im5 | i | im100 -+---+--- 1 | 1 | 0 (1 row) -- Best Regards Andy Fan

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-06-11 Thread Andy Fan
On Wed, Jun 3, 2020 at 10:36 AM Andy Fan wrote: > >> Thanks for running those tests. I had a quick look at the results and >> I think to say that all 4 are better is not quite right. One is >> actually a tiny bit slower and one is only faster due to a plan >> change

hash as an search key and hash collision

2020-06-19 Thread Andy Fan
ntrib/pg_stat_statements/pg_stat_statements.c#L154 -- Best Regards Andy Fan

Re: hash as an search key and hash collision

2020-06-19 Thread Andy Fan
On Sat, Jun 20, 2020 at 12:34 AM Tomas Vondra wrote: > On Fri, Jun 19, 2020 at 04:24:01PM +0800, Andy Fan wrote: > >I want to maintain an internal table which the primary key is sql_text and > >planstmt::text, it is efficient since it both may be very long. So a > >gene

Re: Commitfest 2020-07

2020-06-28 Thread Andy Fan
en up that I've missed, I'm happy to >> volunteer to >> > run CFM for this one. >> >> No one has volunteered that I recall, so the baton is yours. >> >> > Enjoy! > > //Magnus > > Thanks for the volunteering! -- Best Regards Andy Fan

Re: Implementing Incremental View Maintenance

2020-07-06 Thread Andy Fan
Thanks for the patch! > Query checks for following restrictions are added: Are all known supported cases listed below? > - inheritance parent table > ... > - targetlist containing IVM column > - simple subquery is only supported > How to understand 3 items above? - Best Regards Andy Fan

Re: Implementing Incremental View Maintenance

2020-07-09 Thread Andy Fan
quot;not" word:( > > >> - inheritance parent table > >> ... > >> - targetlist containing IVM column > >> - simple subquery is only supported > >> > > > > How to understand 3 items above? > > The best way to understand them is looking into regression test. > Thanks for sharing, I will look into it. -- Best Regards Andy Fan

Difference for Binary format vs Text format for client-server communication

2020-07-16 Thread Andy Fan
?column? 2. (1 row) If the user uses text format, I can just hack some numeric_out function, but if they use binary format, looks I have to change the driver they used for it. Am I understand it correctly? -- Best Regards Andy Fan

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 3:40 PM Amit Langote wrote: > On Tue, Apr 14, 2020 at 4:13 PM Richard Guo > wrote: > > On Tue, Apr 14, 2020 at 2:44 PM Amit Langote > wrote: > >> Maybe I am missing something obvious, but is it intentional that > >> enable_indexscan is checked by cost_index(), that is, *

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 4:58 PM Amit Langote wrote: > On Tue, Apr 14, 2020 at 5:29 PM Andy Fan wrote: > > On Tue, Apr 14, 2020 at 3:40 PM Amit Langote > wrote: > >> On Tue, Apr 14, 2020 at 4:13 PM Richard Guo > wrote: > >> > On Tue, Apr 14, 2020 at 2:44 PM

Re: index paths and enable_indexscan

2020-04-14 Thread Andy Fan
On Tue, Apr 14, 2020 at 5:12 PM Andy Fan wrote: > > > On Tue, Apr 14, 2020 at 4:58 PM Amit Langote > wrote: > >> On Tue, Apr 14, 2020 at 5:29 PM Andy Fan >> wrote: >> > On Tue, Apr 14, 2020 at 3:40 PM Amit Langote >> wrote: >> >>

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-14 Thread Andy Fan
Hi David: Thanks for your time. > 1. Out of date comment in join.sql > > -- join removal is not possible when the GROUP BY contains a column that is > -- not in the join condition. (Note: as of 9.6, we notice that b.id is a > -- primary key and so drop b.c_id from the GROUP BY of the resulting

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-16 Thread Andy Fan
On Thu, Apr 16, 2020 at 8:36 PM Ashutosh Bapat wrote: > On Thu, Apr 16, 2020 at 7:47 AM Andy Fan wrote: > > > (9 rows) > > > > With this feature: > > explain analyze select a, sum(c) from grp2 group by a; > >

Re: WIP: Aggregation push-down

2020-04-21 Thread Andy Fan
onin: The more tests on your patch, the more powerful I feel it is! At the same time, I think the most difficult part to understand your design is you can accept any number of generic join clauses, so I guess more explanation on this part may be helpful. At the code level, I d

Re: WIP: Aggregation push-down

2020-04-21 Thread Andy Fan
e too precious :)Just FYI [1] https://www.postgresql.org/message-id/9726.1542577...@sss.pgh.pa.us Best Regards Andy Fan

Re: Trying to pull up EXPR SubLinks

2020-04-23 Thread Andy Fan
ath(final_rel, best_path); But when we take this action to production case, how to cost this strategy is challenge since it can neither reduce the total_cost nor result in a new PathKey. I will check other place to see how this kind can be added. Best Regards Andy Fan test.sql Description: Bi

Re: Trying to pull up EXPR SubLinks

2020-04-24 Thread Andy Fan
more than that, like. (1, 2, 1) and (1, 1, 2) is same for your suggestion, but they are not different in this path. and we also may be think about if we can get a lower cost if we add a new sort path. Best Regards Andy Fan

Re: Trying to pull up EXPR SubLinks

2020-04-24 Thread Andy Fan
On Fri, Apr 24, 2020 at 5:24 PM David Rowley wrote: > On Fri, 24 Apr 2020 at 15:26, Andy Fan wrote: > > > > Actually I have a different opinion to handle this issue, to execute the > > a > (select avg(a) from tinner where x = touer.x); The drawback of > current

Re: Subplan result caching

2020-04-26 Thread Andy Fan
w parameter to node->chgParams only if the datum really changed, will it still be semantic correctly. It's great to see someone working on this. > I'd like to have a try. Best Regards Andy Fan

Re: WIP: Aggregation push-down

2020-04-26 Thread Andy Fan
On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska wrote: > Andy Fan wrote: > > > The more tests on your patch, the more powerful I feel it is! > > Thanks for the appreciation. Given the poor progress it's increasingly hard > for me to find motivation to work on it. I'

Re: Subplan result caching

2020-04-26 Thread Andy Fan
On Sun, Apr 26, 2020 at 5:49 PM David Rowley wrote: > On Sun, 26 Apr 2020 at 19:08, Andy Fan wrote: > > If we want to handle this case as well, one of the changes would > > be it needs to cache multi records for one input parameter, or return > > one row each time but retur

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-28 Thread Andy Fan
On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote: > On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote: > > V6 also includes: > > 1. Fix the comment misleading you mentioned above. > > 2. Fixed a concern case for `relation_has_uniquekeys_for` function. > > Over o

Re: Pulling up sublink may break join-removal logic

2020-04-28 Thread Andy Fan
s. > You can see [1] for the discuss for this issue with UniqueKey respect. search "In the past we have some limited ability to detect the unqiueness after join, so that's would be ok. Since we have such ability now, this may be another opportunity to improve the join_is_removable function" I'm checking it today and will have a feedback soon. [1] https://www.postgresql.org/message-id/CAKU4AWrGrs0Vk5OrZmS1gbTA2ijDH18NHKnXZTPZNuupn%2B%2Bing%40mail.gmail.com Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-04-29 Thread Andy Fan
On Wed, Apr 29, 2020 at 8:34 AM Andy Fan wrote: > > > On Wed, Apr 29, 2020 at 8:29 AM David Rowley wrote: > >> On Thu, 16 Apr 2020 at 14:17, Andy Fan wrote: >> > V6 also includes: >> > 1. Fix the comment misleading you mentioned abov

Can we remove the other_rels_list parameter for make_rels_by_clause_joins

2020-04-29 Thread Andy Fan
ameter and use root->inital_rels directly. I did the same for make_rels_by_clauseless_joins. The attached is my proposal which should be semantic correctly and more explicitly. Best Regards Andy Fan v1-0001-Remove-other_rels_list-from-make_rels_by_clause_j.patch Description: Binary data

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-07 Thread Andy Fan
e able to use > reltarget of any of its paths since all of those should match the positions across > subquery and the outer query. > but I think it should be rel->subroot->processed_tlist rather than reltarget? Actually I still a bit of uneasy about rel->subroot->processed_tlist for some DML case, which the processed_tlist is different and I still not figure out its impact. > Will continue reviewing your new set of patches as time permits. > Thank you! Actually there is no big difference between v6 and v7 regarding the UniqueKey part except 2 bug fix. However v7 has some more documents, comments improvement and code refactor/split, which may be helpful for review. You may try v7 next time if v8 has not come yet:) Best Regards Andy Fan

Re: Implementing Incremental View Maintenance

2020-05-07 Thread Andy Fan
id you test it? AFAIK, we can test it with: 1. For any query like SELECT xxx, we create view like CREATE MATERIAL VIEW mv_name as SELECT xxx; to test if the features in the query are supported. 2. Update the data and then compare the result with SELECT XXX with SELECT * from mv_name to test if the data is correctly sync. Best Regards Andy Fan

No core file generated after PostgresNode->start

2020-05-10 Thread Andy Fan
7;pg_ctl', "-c", '-D', $self->data_dir, '-l', $self->logfile, '-o', "--cluster-name=$name", 'start'); } Best Regards Andy Fan

Re: No core file generated after PostgresNode->start

2020-05-10 Thread Andy Fan
On Mon, May 11, 2020 at 9:48 AM Andy Fan wrote: > Hi: > > > 2020-05-11 09:37:40.778 CST [69541] sub_viaroot WARNING: terminating > connection because of crash of another server process > > Looks this doesn't mean a crash. If the test case(subscription/t/ 013_p

Re: No core file generated after PostgresNode->start

2020-05-12 Thread Andy Fan
On Tue, May 12, 2020 at 3:36 AM Robert Haas wrote: > On Sun, May 10, 2020 at 11:21 PM Andy Fan > wrote: > > Looks this doesn't mean a crash. If the test case(subscription/t/ > 013_partition.pl) > > failed, test framework kill some process, which leads the ab

Re: Add "-Wimplicit-fallthrough" to default flags

2020-05-13 Thread Andy Fan
yp != '\\') > > goto do_not_strip_quotes; > > /* Fall through. */ > > default: > > It is generated code by bison. > > $ bison --version > bison (GNU Bison) 3.0.4 > > I just found this just serval minutes ago. Upgrading your bison to the latest version (3.6) is ok. I'd like we have a better way to share this knowledge through. I spend ~30 minutes to troubleshooting this issue. Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat wrote: > On Fri, May 8, 2020 at 7:27 AM Andy Fan wrote: > > >> +else if (inner_is_onerow) > >> +{ > >> +/* Since rows in innerrel can't be dup

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
/* Change the multi_nullvals to true at this case */ >> >> Need a comment explaining this. Generally, I feel, this and other >> functions in >> this file need good comments explaining the logic esp. "why" instead of >> "what". > > > Exactly. > Done in v8. >> Will continue reviewing your new set of patches as time permits. >> > > Thank you! Actually there is no big difference between v6 and v7 > regarding the > UniqueKey part except 2 bug fix. However v7 has some more documents, > comments improvement and code refactor/split, which may be helpful > for review. You may try v7 next time if v8 has not come yet:) > > v8 has come :) Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Thu, May 14, 2020 at 6:20 AM David Rowley wrote: > On Thu, 14 May 2020 at 03:48, Andy Fan wrote: > > On Wed, May 13, 2020 at 8:04 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> My impression about the one row stuff, is that there is too much > &

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-13 Thread Andy Fan
On Wed, May 13, 2020 at 11:48 PM Andy Fan wrote: > >> My impression about the one row stuff, is that there is too much >> special casing around it. We should somehow structure the UniqueKey >> data so that one row unique keys come naturally rather than special >> case

Re: Add "-Wimplicit-fallthrough" to default flags

2020-05-14 Thread Andy Fan
On Wed, May 13, 2020 at 10:02 PM Tom Lane wrote: > Andy Fan writes: > >> FWIW, I got a warning for jsonpath_gram.c. > > Ugh. Confirmed here on Fedora 30 (bison 3.0.5). > > > I just found this just serval minutes ago. Upgrading your bison to the > > latest vers

Find query characters in respect of optimizer for develop purpose

2020-05-18 Thread Andy Fan
standard_planner, and the values are increased at make_subplan, set_subquery_pathlist, make_one_rel, create_grouping_paths. later it can be tracked and viewed in pg_stat_statements. What do you think about the requirement and the method I am thinking? Any kind of feedback is welcome. -- Best Regards Andy

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-05-18 Thread Andy Fan
t as Counter in nature. However I don't think the above 2 will cause big issues. I added the columns to V1_8 rather than adding a new version. this can be changed at final patch. Any suggestions? Best Regards Andy Fan v1-0001-Add-query-characters-information-to-pg_stat_state.patch Description: Binary data

Re: Subplan result caching

2020-05-20 Thread Andy Fan
On Wed, May 20, 2020 at 7:47 PM David Rowley wrote: > On Mon, 27 Apr 2020 at 00:37, Andy Fan wrote: > > I was feeling that we may have to maintain some extra status if we use > hash > > table rather than tuple store, but that might be not a major concern. I > can > &

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-21 Thread Andy Fan
a SEMI/ANTI join, it might be useful to try those > when planning joins. > > Yes, In current implementation, we also add UniqueKey during create_xxx_paths, xxx may be grouping/union. after the index skipscan patch, we can do the similar things in create_indexskip_path. -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-21 Thread Andy Fan
> if I understand correctly those two are introducing the concept, and others are just using it You are understand it correctly. -- Best Regards Andy Fan

Re: [PATCH] Keeps tracking the uniqueness with UniqueKey

2020-05-21 Thread Andy Fan
On Fri, May 22, 2020 at 4:52 AM David Rowley wrote: > On Thu, 14 May 2020 at 14:39, Andy Fan wrote: > > > > On Thu, May 14, 2020 at 6:20 AM David Rowley > wrote: > >> Having the "onerow" flag was not how I intended it to work. > >> > > Th

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-05-21 Thread Andy Fan
n true where j1o.i = 1; -- Best Regards Andy Fan

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-05-21 Thread Andy Fan
On Thu, May 21, 2020 at 3:17 PM Michael Paquier wrote: > On Thu, May 21, 2020 at 08:49:53AM +0200, Julien Rouhaud wrote: > > On Tue, May 19, 2020 at 4:29 AM Andy Fan > wrote: > >> Thanks for the excellent extension. I want to add 5 more fields to > satisfy the >

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-05-21 Thread Andy Fan
On Thu, May 21, 2020 at 3:49 PM Julien Rouhaud wrote: > Le jeu. 21 mai 2020 à 09:17, Michael Paquier a > écrit : > >> On Thu, May 21, 2020 at 08:49:53AM +0200, Julien Rouhaud wrote: >> > On Tue, May 19, 2020 at 4:29 AM Andy Fan >> wrote: >> >> Thanks

Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-26 Thread Andy Fan
ble to fix, however I'm open for suggestion on that as well. Any suggestions? -- Best Regards Andy Fan

Re: Planning counters in pg_stat_statements (using pgss_store)

2020-05-26 Thread Andy Fan
On Fri, May 22, 2020 at 9:51 PM Fujii Masao wrote: > > > On 2020/05/22 15:10, Andy Fan wrote: > > > > > > On Thu, May 21, 2020 at 3:49 PM Julien Rouhaud <mailto:rjuju...@gmail.com>> wrote: > > > > Le jeu. 21 mai 2020 à 09:17, Michae

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-26 Thread Andy Fan
On Tue, May 26, 2020 at 9:59 PM Ashutosh Bapat wrote: > On Tue, May 26, 2020 at 1:52 PM Andy Fan wrote: > > > > > > Consider the below example: > > > > create table j1(i int, im5 int, im100 int, im1000 int); > > insert into j1 select i, i%5, i%100, i%10

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-26 Thread Andy Fan
issue. -- Best Regards Andy Fan index_choose.sql Description: Binary data

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-27 Thread Andy Fan
On Wed, May 27, 2020 at 8:01 PM Ashutosh Bapat < ashutosh.ba...@2ndquadrant.com> wrote: > > > On Wed, 27 May 2020 at 04:43, Andy Fan wrote: > >> You can use the attached sql to reproduce this issue, but I'm not sure >> you can >> get the above resu

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-28 Thread Andy Fan
here expensive_func(col1) = X. we may change it cpu_tuple_cost + qpqual_cost.per_tuple + (0.0001) * list_lenght(qpquals). -- Best Regards Andy Fan

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-28 Thread Andy Fan
Thanks all of you for your feedback. On Fri, May 29, 2020 at 9:04 AM Tom Lane wrote: > Tomas Vondra writes: > > On Wed, May 27, 2020 at 09:58:04PM +0800, Andy Fan wrote: > >> so we need to optimize the cost model for such case, the method is the > >> patch I mentio

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-29 Thread Andy Fan
On Fri, May 29, 2020 at 9:37 PM Ashutosh Bapat wrote: > On Fri, May 29, 2020 at 6:40 AM Andy Fan wrote: > > > > > >> > >> >so we need to optimize the cost model for such case, the method is the > >> >patch I mentioned above. > >> > &

Re: Make the qual cost on index Filter slightly higher than qual cost on index Cond.

2020-05-29 Thread Andy Fan
only show 2 digits in cost, which is not accurate enough to show the difference. However with a nest loop, the overall plan shows the cost difference. [1] https://ankane.org/tpc-h -- Best Regards Andy Fan normal.log Description: Binary data patched.log Description: Binary data

A wrong index choose issue because of inaccurate statistics

2020-05-31 Thread Andy Fan
https://postgrespro.com/list/id/8810.1590714...@sss.pgh.pa.us [2] https://ankane.org/tpc-h -- Best Regards Andy Fan normal.log Description: Binary data patched.log Description: Binary data reproduce.sql Description: Binary data

Re: Index Skip Scan

2020-06-02 Thread Andy Fan
use cases (hopefully it was covered by the v33, but I still need a > confirmation from David, like blinking twice or something). > > * Suspicious performance difference between different type of workload, > mentioned by Tomas (unfortunately I had no chance yet to investigate). > > * Thinking about supporting conditions, that are not covered by the index, > to make skipping more flexible (one of the potential next steps in the > future, as suggested by Floris). > Looks this is the latest patch, which commit it is based on? Thanks -- Best Regards Andy Fan

Re: Index Skip Scan

2020-06-02 Thread Andy Fan
On Tue, Jun 2, 2020 at 9:38 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Jun 02, 2020 at 08:36:31PM +0800, Andy Fan wrote: > > > > > Other than that to summarize current open points for future readers > > > (this thread somehow became quite b

Re: Hybrid Hash/Nested Loop joins and caching results from subplans

2020-06-02 Thread Andy Fan
https://github.com/zhihuiFan/tpch-postgres -- Best Regards Andy Fan

When is the MessageContext released?

2019-02-26 Thread Andy Fan
Hi : I run a query like "select * from t" and set the break like this: break exec_simple_query break MemoryContextDelete commands p context->name c end I can see most of the MemoryContext is relased, but never MessageContext, when will it be released? /* * Create the memory context w

Re: When is the MessageContext released?

2019-02-27 Thread Andy Fan
Thanks you Andres for your time! this context is free with AllocSetReset rather than AllocSetDelete, that makes my breakpoint doesn't catch it. On Wed, Feb 27, 2019 at 2:15 PM Andres Freund wrote: > On 2019-02-27 14:08:47 +0800, Andy Fan wrote: > > Hi : > > I run a query

some hints to understand the plsql cursor.

2019-02-27 Thread Andy Fan
actually I'm hacking pg for a function like : 1. define a select query. 2. client ask for some data. and server reply some data. server will do NOTHING if client doesn't ask any more.. 3. client ask some data more data with a batch and SERVER reply some data then. then do NOTHING. currently the

Re: some hints to understand the plsql cursor.

2019-02-27 Thread Andy Fan
code definitely makes the debug much quicker. Thank you very much! On Wed, Feb 27, 2019 at 11:35 PM Dilip Kumar wrote: > On Wed, Feb 27, 2019 at 4:42 PM Andy Fan wrote: > > > > actually I'm hacking pg for a function like : > > 1. define a select query. > >

[PATCH] Erase the distinctClause if the result is unique by definition

2020-01-31 Thread Andy Fan
Hi: I wrote a patch to erase the distinctClause if the result is unique by definition, I find this because a user switch this code from oracle to PG and find the performance is bad due to this, so I adapt pg for this as well. This patch doesn't work for a well-written SQL, but some drawback of

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-05 Thread Andy Fan
update the patch with considering the semi/anti join. Can anyone help to review this patch? Thanks On Fri, Jan 31, 2020 at 8:39 PM Andy Fan wrote: > Hi: > > I wrote a patch to erase the distinctClause if the result is unique by > definition, I find this because a user switch th

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-07 Thread Andy Fan
PlannerInfo, and when we create a node for Unique/HashAggregate/Group, we can just create a dummy node? > 5. Have you tested this OUTER joins, which can render inner side nullable? > Yes, that part was missed in the test case. I just added them. On Thu, Feb 6, 2020 at 11:31 AM Andy Fan w

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-10 Thread Andy Fan
On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > > >> >> [PATCH] Erase the distinctClause if the result is unique by >> definition >> > > I forgot to mention this in the last round of comments. Your patch was > actually removing distictClause from the Query

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Andy Fan
On Tue, Feb 11, 2020 at 3:56 PM Julien Rouhaud wrote: > On Tue, Feb 11, 2020 at 10:57:26AM +0800, Andy Fan wrote: > > On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < > > ashutosh.bapat@gmail.com> wrote: > > > > > I forgot to mention this in the la

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-11 Thread Andy Fan
On Tue, Feb 11, 2020 at 3:56 PM Julien Rouhaud wrote: > > > > and if we prepare sql outside a transaction, and execute it in the > > transaction, the other session can't drop the constraint until the > > transaction is ended. > > And what if you create a view on top of a query containing a distin

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-13 Thread Andy Fan
On Thu, Feb 13, 2020 at 5:39 PM Julien Rouhaud wrote: > On Tue, Feb 11, 2020 at 10:06:17PM +0530, Ashutosh Bapat wrote: > > On Tue, Feb 11, 2020 at 8:27 AM Andy Fan > wrote: > > > > > On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < > > &

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-24 Thread Andy Fan
On Wed, Feb 12, 2020 at 12:36 AM Ashutosh Bapat < ashutosh.bapat@gmail.com> wrote: > > > On Tue, Feb 11, 2020 at 8:27 AM Andy Fan wrote: > >> >> >> On Tue, Feb 11, 2020 at 12:22 AM Ashutosh Bapat < >> ashutosh.bapat@gmail.com> wrote: >&

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-02-24 Thread Andy Fan
ments. Thanks On Mon, Feb 24, 2020 at 8:38 PM Andy Fan wrote: > > > On Wed, Feb 12, 2020 at 12:36 AM Ashutosh Bapat < > ashutosh.bapat@gmail.com> wrote: > >> >> >> On Tue, Feb 11, 2020 at 8:27 AM Andy Fan >> wrote: >> >

Re: Trying to pull up EXPR SubLinks

2020-02-27 Thread Andy Fan
On Fri, Feb 28, 2020 at 2:35 PM Richard Guo wrote: > Hi All, > > Currently we will not consider EXPR_SUBLINK when pulling up sublinks and > this would cause performance issues for some queries with the form of: > 'a > (SELECT agg(b) from ...)' as described in [1]. > > So here is a patch as an att

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-02 Thread Andy Fan
Thank you Tom for the review! On Mon, Mar 2, 2020 at 4:46 AM Tom Lane wrote: > Andy Fan writes: > > Please see if you have any comments. Thanks > > The cfbot isn't at all happy with this. Its linux build is complaining > about a possibly-uninitialized variable, and th

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-02 Thread Andy Fan
On Tue, Mar 3, 2020 at 1:24 AM Andy Fan wrote: > Thank you Tom for the review! > > On Mon, Mar 2, 2020 at 4:46 AM Tom Lane wrote: > >> Andy Fan writes: >> > Please see if you have any comments. Thanks >> >> The cfbot isn't at all happy with this

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-04 Thread Andy Fan
> > >> * There are some changes in existing regression cases that aren't >> visibly related to the stated purpose of the patch, eg it now >> notices that "select distinct max(unique2) from tenk1" doesn't >> require an explicit DISTINCT step. That's not wrong, but I wonder >> if maybe you should su

Re: [PATCH] Erase the distinctClause if the result is unique by definition

2020-03-06 Thread Andy Fan
But since the current patch not tied with this closely, so I would put this patch for review first. On Wed, Mar 4, 2020 at 9:13 PM Andy Fan wrote: > > >> >>> * There are some changes in existing regression cases that aren't >>> visibly related to th

Questions about the CI process and proposal

2020-03-06 Thread Andy Fan
ql.org/27/2433/ Regards Andy Fan.

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-20 Thread Andy Fan
ge_cost happens to not change very much. /u/y/g/fdirect> sudo ./main fs_cache_lat = 0.569031us, seq_read_lat = 18.901749us, random_page_lat = 148.650589us cache hit ratio: 1.00 random_page_cost 1.00 cache hit ratio: 0.90 random_page_cost 6.401019 cache hit ratio: 0.50 random_page_cost 7.663772 cache hit ratio: 0.10 random_page_cost 7.841498 cache hit ratio: 0.00 random_page_cost 7.864383 This result looks much different from "we should use 1.1 ~ 1.5 for SSD". The attached is the modified detection program. [1] https://www.cdw.com/product/lsi-megaraid-sas-9271-8i-storage-controller-raid-sas-pcie-3.0-x8/4576538#PO -- Best Regards Andy Fan main.c Description: Binary data

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-21 Thread Andy Fan
Thanks Ashutosh for coming:) On Mon, Sep 21, 2020 at 9:03 PM Ashutosh Bapat wrote: > On Mon, Sep 21, 2020 at 9:11 AM Andy Fan wrote: > > > > Here are some changes for my detection program. > > > > | | seq_read_lat (

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-21 Thread Andy Fan
by ~10%. [1] https://www.postgresql.org/message-id/CAKU4AWpRv50k8E3tC3tiLWGe2DbKaoZricRh_YJ8y_zK%2BHdSjQ%40mail.gmail.com -- Best Regards Andy Fan

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-25 Thread Andy Fan
On Fri, Sep 25, 2020 at 5:15 PM Ashutosh Bapat wrote: > On Tue, Sep 22, 2020 at 10:57 AM Andy Fan > wrote: > > > > > > My tools set the random_page_cost to 8.6, but based on the fio data, it > should be > > set to 12.3 on the same hardware. and I do see th

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-26 Thread Andy Fan
On Sat, Sep 26, 2020 at 1:51 PM Julien Rouhaud wrote: > On Sat, Sep 26, 2020 at 8:17 AM Andy Fan wrote: > > > > As for the testing with cache considered, I found how to estimate cache > hit > > ratio is hard or how to control a hit ratio to test is hard. Recently I &g

Partition prune with stable Expr

2020-09-27 Thread Andy Fan
ment_1 Filter: (logdate = to_date('2006-03-02'::text, '-mm-dd'::text)) (4 rows) IMO, we should do it. Why not? The attached is used to show the things in my mind. Btw, why the to_date function is declared as stable rather than immutable since it always delivers th

Re: Partition prune with stable Expr

2020-09-27 Thread Andy Fan
Thank you David for coming:) On Mon, Sep 28, 2020 at 4:46 AM David Rowley wrote: > On Mon, 28 Sep 2020 at 08:59, Andy Fan wrote: > > I find we can't prune partitions in the planner if the qual is a stable > function. > > > IMO, we should do it. Why not? > > T

Re: Partition prune with stable Expr

2020-09-27 Thread Andy Fan
On Mon, Sep 28, 2020 at 7:15 AM Tom Lane wrote: > Andy Fan writes: > > On Mon, Sep 28, 2020 at 4:46 AM David Rowley > wrote: > >> Thanks for showing an interest in partition pruning. Unfortunately, > >> it's not possible to use stable functions to pr

Re: Partition prune with stable Expr

2020-09-27 Thread Andy Fan
On Mon, Sep 28, 2020 at 9:15 AM Tom Lane wrote: > Andy Fan writes: > > Well, that's very interesting. Specific to my user case, > > SELECT * FROM p WHERE pkey = to_date('2018-12-13', '-mm-dd)'; > > p has 1500+ partitions and planning takes

Re: Partition prune with stable Expr

2020-09-28 Thread Andy Fan
On Mon, Sep 28, 2020 at 2:44 PM Jesse Zhang wrote: > On Sun, Sep 27, 2020 at 7:52 PM Andy Fan wrote: > > > > > > On Mon, Sep 28, 2020 at 9:15 AM Tom Lane wrote: > >> > >> Andy Fan writes: > >> > Well, that's very interesting. Specific to m

Re: Partition prune with stable Expr

2020-09-28 Thread Andy Fan
On Mon, Sep 28, 2020 at 7:15 AM Tom Lane wrote: > Andy Fan writes: > > On Mon, Sep 28, 2020 at 4:46 AM David Rowley > wrote: > >> Thanks for showing an interest in partition pruning. Unfortunately, > >> it's not possible to use stable functions to pr

Re: Partition prune with stable Expr

2020-09-28 Thread Andy Fan
On Mon, Sep 28, 2020 at 9:17 PM Thomas Kellerer wrote: > Andy Fan schrieb am 28.09.2020 um 02:54: > > Well, that's very interesting. Specific to my user case, > > SELECT * FROM p WHERE pkey = to_date('2018-12-13', '-mm-dd)'; > > Why use to_date(

Re: BLOB / CLOB support in PostgreSQL

2020-09-28 Thread Andy Fan
methods on the Clob interface must be fully implemented if the JDBC Spec> * JDBC driver supports the data type. What would be the sense behind this? This is not reasonable based on limited experience. To be short, I am supporting Andrew's proposal for now. -- Best Regards Andy Fan

Improve choose_custom_plan for initial partition prune case

2020-10-01 Thread Andy Fan
the partition pruned at plan time from custom plan and then reducing such costs from the generic plan. I just went through the related code but didn't write anything now. I'd like to see if this is a correct direction to go. -- Best Regards Andy Fan

Re: Improve choose_custom_plan for initial partition prune case

2020-10-02 Thread Andy Fan
Hi Amit: Very glad to see your comment! On Fri, Oct 2, 2020 at 4:21 PM Amit Langote wrote: > Hi Andy, > > On Fri, Oct 2, 2020 at 1:04 AM Andy Fan wrote: > > > > Given the plan example: > > > > CREATE TABLE measurement ( > > city_id int not

Re: [HACKERS] Runtime Partition Pruning

2020-10-04 Thread Andy Fan
ased on such information. Any thought about this? I'd be sorry if I missed some already existing discussion on this topic. [1] https://www.postgresql.org/message-id/CA%2BHiwqGsP2L0BW1ad58HRSj1NouNSVHLfL5pm7%3DPBTvL0b%2B-BQ%40mail.gmail.com -- Best Regards Andy Fan

Re: Improve choose_custom_plan for initial partition prune case

2020-10-06 Thread Andy Fan
Hi Ashutosh: Thanks for coming. On Mon, Oct 5, 2020 at 9:27 PM Ashutosh Bapat wrote: > On Thu, Oct 1, 2020 at 9:34 PM Andy Fan wrote: > > > > Given the plan example: > > > > CREATE TABLE measurement ( > > city_id int not null, >

Re: Improve choose_custom_plan for initial partition prune case

2020-10-06 Thread Andy Fan
On Sat, Oct 3, 2020 at 10:05 AM Andy Fan wrote: > Hi Amit: > > Very glad to see your comment! > > On Fri, Oct 2, 2020 at 4:21 PM Amit Langote > wrote: > >> Hi Andy, >> >> On Fri, Oct 2, 2020 at 1:04 AM Andy Fan wrote: >> > >> > G

Re: Improve choose_custom_plan for initial partition prune case

2020-10-06 Thread Andy Fan
tkey2 = $2; > > I just came up with a new idea. Since this situation should be rare, we can just come back to our original method (totally ignore the cost reduction) or use the average number. Fixing the 99% cases would be a big winner as well IMO. So any hint on this will be appreciated..

Re: Improve choose_custom_plan for initial partition prune case

2020-10-07 Thread Andy Fan
On Wed, Oct 7, 2020 at 2:43 PM Andy Fan wrote: > >> 2. Associate them with RelationOid, and we can record such information in >> the >> Append node as well. The bad part is the same relation Oid may appear >> multiple >> times in a query. for example: SELEC

<    1   2   3   4   5   6   7   >