Re: Columns correlation and adaptive query optimization

2021-11-04 Thread Daniel Gustafsson
> On 14 Jul 2021, at 13:13, vignesh C wrote: > "C:\projects\postgresql\pgsql.sln" (default target) (1) -> > "C:\projects\postgresql\auto_explain.vcxproj" (default target) (45) -> > (ClCompile target) -> > contrib/auto_explain/auto_explain.c(658): error C2039: 'mt_plans' : is > not a member of

Re: Columns correlation and adaptive query optimization

2021-07-14 Thread vignesh C
On Fri, Mar 19, 2021 at 10:28 PM Konstantin Knizhnik wrote: > > > > On 19.03.2021 12:17, Yugo NAGATA wrote: > > On Wed, 10 Mar 2021 03:00:25 +0100 > > Tomas Vondra wrote: > > > >> What is being proposed here - an extension suggesting which statistics > >> to create (and possibly creating them

Re: Columns correlation and adaptive query optimization

2021-03-22 Thread Yugo NAGATA
Hello Konstantin, I tested this patch as a statistics advisor using TPC-H queries. The used parameters are: auto_explain.add_statistics_suggest_only = on auto_explain.add_statistics_threshold = 0.1 auto_explain.log_analyze = on auto_explain.log_min_duration = 0 auto_explain suggested to

Re: Columns correlation and adaptive query optimization

2021-03-21 Thread Yugo NAGATA
On Fri, 19 Mar 2021 19:58:27 +0300 Konstantin Knizhnik wrote: > > > On 19.03.2021 12:17, Yugo NAGATA wrote: > > On Wed, 10 Mar 2021 03:00:25 +0100 > > Tomas Vondra wrote: > > > >> What is being proposed here - an extension suggesting which statistics > >> to create (and possibly creating them

Re: Columns correlation and adaptive query optimization

2021-03-20 Thread Konstantin Knizhnik
On 19.03.2021 20:32, Zhihong Yu wrote: Hi, In AddMultiColumnStatisticsForQual(), +   /* Loop until we considered all vars */ +   while (vars != NULL) ... +       /* Contruct list of unique vars */ +       foreach (cell, vars) What if some cell / node, gets into the else block: +            

Re: Columns correlation and adaptive query optimization

2021-03-19 Thread Zhihong Yu
Hi, In AddMultiColumnStatisticsForQual(), + /* Loop until we considered all vars */ + while (vars != NULL) ... + /* Contruct list of unique vars */ + foreach (cell, vars) What if some cell / node, gets into the else block: + else + { +

Re: Columns correlation and adaptive query optimization

2021-03-19 Thread Konstantin Knizhnik
On 19.03.2021 12:17, Yugo NAGATA wrote: On Wed, 10 Mar 2021 03:00:25 +0100 Tomas Vondra wrote: What is being proposed here - an extension suggesting which statistics to create (and possibly creating them automatically) is certainly useful, but I'm not sure I'd call it "adaptive query

Re: Columns correlation and adaptive query optimization

2021-03-19 Thread Yugo NAGATA
On Wed, 10 Mar 2021 03:00:25 +0100 Tomas Vondra wrote: > What is being proposed here - an extension suggesting which statistics > to create (and possibly creating them automatically) is certainly > useful, but I'm not sure I'd call it "adaptive query optimization". I > think "adaptive" means the

Re: Columns correlation and adaptive query optimization

2021-03-10 Thread Tomas Vondra
On 3/10/21 3:00 AM, Tomas Vondra wrote: > Hello Konstantin, > > > Sorry for not responding to this thread earlier. I definitely agree the > features proposed here are very interesting and useful, and I appreciate > you kept rebasing the patch. > > I think the patch improving join estimates can

Re: Columns correlation and adaptive query optimization

2021-03-09 Thread Tomas Vondra
Hello Konstantin, Sorry for not responding to this thread earlier. I definitely agree the features proposed here are very interesting and useful, and I appreciate you kept rebasing the patch. I think the patch improving join estimates can be treated as separate, and I see it already has a

Re: Columns correlation and adaptive query optimization

2021-01-27 Thread Konstantin Knizhnik
On 27.01.2021 8:45, Yugo NAGATA wrote: On Mon, 25 Jan 2021 16:27:25 +0300 Konstantin Knizhnik wrote: Hello, Thank you for review. My answers are inside. Thank you for updating the patch and answering my questions. (2) If I understand correctly, your proposal consists of the following

Re: Columns correlation and adaptive query optimization

2021-01-26 Thread Yugo NAGATA
On Mon, 25 Jan 2021 16:27:25 +0300 Konstantin Knizhnik wrote: > Hello, > > Thank you for review. > My answers are inside. Thank you for updating the patch and answering my questions. > > (2) > > If I understand correctly, your proposal consists of the following two > > features. > > > > 1.

Re: Columns correlation and adaptive query optimization

2021-01-25 Thread Konstantin Knizhnik
Hello, Thank you for review. My answers are inside. On 21.01.2021 15:30, Yugo NAGATA wrote: Hello, On Thu, 26 Mar 2020 18:49:51 +0300 Konstantin Knizhnik wrote: Attached please find new version of the patch with more comments and descriptions added. Adaptive query optimization is very

Re: Columns correlation and adaptive query optimization

2021-01-21 Thread Yugo NAGATA
Hello, On Thu, 26 Mar 2020 18:49:51 +0300 Konstantin Knizhnik wrote: > Attached please find new version of the patch with more comments and > descriptions added. Adaptive query optimization is very interesting feature for me, so I looked into this patch. Here are some comments and questions.

Re: Columns correlation and adaptive query optimization

2020-03-26 Thread Konstantin Knizhnik
On 25.03.2020 20:04, Rafia Sabih wrote: Also, there is no description about any of the functions here, wouldn’t hurt having some more comments there. Attached please find new version of the patch with more comments and descriptions added. -- Konstantin Knizhnik Postgres Professional:

Re: Columns correlation and adaptive query optimization

2020-03-26 Thread Konstantin Knizhnik
Thank you very much for review. On 25.03.2020 20:04, Rafia Sabih wrote: +static void +AddMultiColumnStatisticsForNode(PlanState *planstate, ExplainState *es); + This doesn't look like the right place for it, you might want to declare it with other functions in the starting of the file. Also,

Re: Columns correlation and adaptive query optimization

2020-03-25 Thread Rafia Sabih
Hello, This sounded like an interesting addition to postgresql. I gave some time to it today to review, here are few comments, On Wed, 25 Mar 2020 at 14:28, Konstantin Knizhnik wrote: > > > > On 25.03.2020 16:00, David Steele wrote: > > On 3/25/20 6:57 AM, Konstantin Knizhnik wrote: > >> > >> >

Re: Columns correlation and adaptive query optimization

2020-03-25 Thread Konstantin Knizhnik
On 25.03.2020 16:00, David Steele wrote: On 3/25/20 6:57 AM, Konstantin Knizhnik wrote: On 24.03.2020 20:12, David Steele wrote: On 12/24/19 3:15 AM, Konstantin Knizhnik wrote: New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer

Re: Columns correlation and adaptive query optimization

2020-03-25 Thread David Steele
On 3/25/20 6:57 AM, Konstantin Knizhnik wrote: On 24.03.2020 20:12, David Steele wrote: On 12/24/19 3:15 AM, Konstantin Knizhnik wrote: New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer for more precise estimation of join

Re: Columns correlation and adaptive query optimization

2020-03-25 Thread Konstantin Knizhnik
On 24.03.2020 20:12, David Steele wrote: On 12/24/19 3:15 AM, Konstantin Knizhnik wrote: New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer for more precise estimation of join selectivity. This patch fixes race condition while

Re: Columns correlation and adaptive query optimization

2020-03-24 Thread David Steele
On 12/24/19 3:15 AM, Konstantin Knizhnik wrote: New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer for more precise estimation of join selectivity. This patch fixes race condition while adding statistics and restricts generated

Re: Columns correlation and adaptive query optimization

2019-12-24 Thread Konstantin Knizhnik
New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer for more precise estimation of join selectivity. This patch fixes race condition while adding statistics and restricts generated statistic name to fit in 64 bytes (NameData). --

Re: Columns correlation and adaptive query optimization

2019-10-18 Thread Konstantin Knizhnik
Smarter version of join selectivity patch handling cases like this: explain select * from outer_tab join inner_tab using(x,y) where x=1;    QUERY PLAN  Nested

Re: Columns correlation and adaptive query optimization

2019-10-15 Thread Konstantin Knizhnik
On 15.10.2019 1:20, legrand legrand wrote: Hello Konstantin, What you have proposed regarding join_selectivity and multicolumn statistics is a very good new ! Regarding your auto_explain modification, maybe an "advisor" mode would also be helpfull (with

Re: Columns correlation and adaptive query optimization

2019-10-14 Thread legrand legrand
Hello Konstantin, What you have proposed regarding join_selectivity and multicolumn statistics is a very good new ! Regarding your auto_explain modification, maybe an "advisor" mode would also be helpfull (with auto_explain_add_statistics_threshold=-1 for exemple). This would allow to track