Re: Is it useful to record whether plans are generic or custom?

2021-04-05 Thread torikoshia
On 2021-03-26 17:46, Fujii Masao wrote: On 2021/03/26 0:33, torikoshia wrote: On 2021-03-25 22:14, Fujii Masao wrote: On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported

Re: Is it useful to record whether plans are generic or custom?

2021-03-26 Thread Fujii Masao
On 2021/03/26 0:33, torikoshia wrote: On 2021-03-25 22:14, Fujii Masao wrote: On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported that the patched version failed to be

Re: Is it useful to record whether plans are generic or custom?

2021-03-25 Thread torikoshia
On 2021-03-25 22:14, Fujii Masao wrote: On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported that the patched version failed to be compiled at Windows. Could you fix this

Re: Is it useful to record whether plans are generic or custom?

2021-03-25 Thread Fujii Masao
On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported that the patched version failed to be compiled at Windows. Could you fix this issue?

Re: Is it useful to record whether plans are generic or custom?

2021-03-23 Thread torikoshia
On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! I just tried this feature. When I set plan_cache_mode to force_generic_plan and executed the following queries, I found that pg_stat_statements.generic_calls and pg_prepared_statements.generic_plans were not the same. Is this

Re: Is it useful to record whether plans are generic or custom?

2021-03-05 Thread Fujii Masao
On 2021/02/08 14:02, torikoshia wrote: On 2021-02-04 11:19, Kyotaro Horiguchi wrote: At Thu, 04 Feb 2021 10:16:47 +0900, torikoshia wrote in Chengxi Sun, Yamada-san, Horiguchi-san, Thanks for all your comments. Adding only the number of generic plan execution seems acceptable. On Mon,

Re: Is it useful to record whether plans are generic or custom?

2021-03-05 Thread Fujii Masao
On 2021/01/28 8:11, Tatsuro Yamada wrote: Hi Toricoshi-san, On 2021/01/12 20:36, torikoshia wrote: I suppose it would be normal practice to store past results of pg_stat_statements for future comparisons. If this is the case, I think that if we only add the number of generic plan execution,

Re: Is it useful to record whether plans are generic or custom?

2021-02-07 Thread torikoshia
On 2021-02-04 11:19, Kyotaro Horiguchi wrote: At Thu, 04 Feb 2021 10:16:47 +0900, torikoshia wrote in Chengxi Sun, Yamada-san, Horiguchi-san, Thanks for all your comments. Adding only the number of generic plan execution seems acceptable. On Mon, Jan 25, 2021 at 2:10 PM Kyotaro Horiguchi

Re: Is it useful to record whether plans are generic or custom?

2021-02-03 Thread Kyotaro Horiguchi
At Thu, 04 Feb 2021 10:16:47 +0900, torikoshia wrote in > Chengxi Sun, Yamada-san, Horiguchi-san, > > Thanks for all your comments. > Adding only the number of generic plan execution seems acceptable. > > On Mon, Jan 25, 2021 at 2:10 PM Kyotaro Horiguchi > wrote: > > Note that ActivePortal

Re: Is it useful to record whether plans are generic or custom?

2021-02-03 Thread torikoshia
Chengxi Sun, Yamada-san, Horiguchi-san, Thanks for all your comments. Adding only the number of generic plan execution seems acceptable. On Mon, Jan 25, 2021 at 2:10 PM Kyotaro Horiguchi wrote: Note that ActivePortal is the closest nested portal. So it gives the wrong result for nested

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Hi Toricoshi-san, On 2021/01/12 20:36, torikoshia wrote: I suppose it would be normal practice to store past results of pg_stat_statements for future comparisons. If this is the case, I think that if we only add the number of generic plan execution, it will give us a hint to notice the cause of

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Horiguchi-san, On 2020/12/04 15:37, Kyotaro Horiguchi wrote: And I'm also struggling with the following. | However, I also began to wonder how effective it would be to just | distinguish between generic and custom plans. Custom plans can | include all sorts of plans. and thinking cache

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
Torikoshi-san, On 2020/12/04 15:03, torikoshia wrote: ISTM now that creating pg_stat_statements_xxx views both for generic andcustom plans is better than my PoC patch. And I'm also struggling with the following. | However, I also began to wonder how effective it would be to just |

Re: Is it useful to record whether plans are generic or custom?

2021-01-27 Thread Tatsuro Yamada
On 2020/12/04 14:29, Fujii Masao wrote: On 2020/11/30 15:24, Tatsuro Yamada wrote: Hi Torikoshi-san, In this patch, exposing new columns is mandatory, but I think it's better to make it optional by adding a GUC something like 'pgss.track_general_custom_plans. I also feel it makes the

Re: Is it useful to record whether plans are generic or custom?

2021-01-24 Thread Kyotaro Horiguchi
At Tue, 12 Jan 2021 20:36:58 +0900, torikoshia wrote in > I suppose it would be normal practice to store past results of > pg_stat_statements for future comparisons. > If this is the case, I think that if we only add the number of > generic plan execution, it will give us a hint to notice the

Re: Is it useful to record whether plans are generic or custom?

2021-01-22 Thread Chengxi Sun
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested Hi Atsushi, I just run a few test on your latest patch. It works

Re: Is it useful to record whether plans are generic or custom?

2021-01-12 Thread torikoshia
wrote in ISTM now that creating pg_stat_statements_xxx views both for generic andcustom plans is better than my PoC patch. On my second thought, it also makes pg_stat_statements too complicated compared to what it makes possible.. I'm also worrying that whether taking generic and custom

Re: Is it useful to record whether plans are generic or custom?

2020-12-03 Thread Kyotaro Horiguchi
At Fri, 04 Dec 2020 15:03:25 +0900, torikoshia wrote in > On 2020-12-04 14:29, Fujii Masao wrote: > > On 2020/11/30 15:24, Tatsuro Yamada wrote: > >> Hi Torikoshi-san, > >> > >>> In this patch, exposing new columns is mandatory, but I think > >>> it's better to make it optional by adding a GUC

Re: Is it useful to record whether plans are generic or custom?

2020-12-03 Thread torikoshia
On 2020-12-04 14:29, Fujii Masao wrote: On 2020/11/30 15:24, Tatsuro Yamada wrote: Hi Torikoshi-san, In this patch, exposing new columns is mandatory, but I think it's better to make it optional by adding a GUC something like 'pgss.track_general_custom_plans. I also feel it makes the number

Re: Is it useful to record whether plans are generic or custom?

2020-12-03 Thread Fujii Masao
On 2020/11/30 15:24, Tatsuro Yamada wrote: Hi Torikoshi-san, In this patch, exposing new columns is mandatory, but I think it's better to make it optional by adding a GUC something like 'pgss.track_general_custom_plans. I also feel it makes the number of columns too many. Just adding the

Re: Is it useful to record whether plans are generic or custom?

2020-11-29 Thread Tatsuro Yamada
Hi Torikoshi-san, In this patch, exposing new columns is mandatory, but I think it's better to make it optional by adding a GUC something like 'pgss.track_general_custom_plans. I also feel it makes the number of columns too many. Just adding the total time may be sufficient. I think this

Re: Is it useful to record whether plans are generic or custom?

2020-11-17 Thread Pavel Stehule
út 17. 11. 2020 v 15:21 odesílatel torikoshia napsal: > On 2020-11-12 14:23, Pavel Stehule wrote: > > > yes, the plan self is very interesting information - and information > > if plan was generic or not is interesting too. It is other dimension > > of query - maybe there can be rule - for any

Re: Is it useful to record whether plans are generic or custom?

2020-11-17 Thread torikoshia
On 2020-11-12 14:23, Pavel Stehule wrote: yes, the plan self is very interesting information - and information if plan was generic or not is interesting too. It is other dimension of query - maybe there can be rule - for any query store max 100 most slows plans with all attributes. The next

Re: Is it useful to record whether plans are generic or custom?

2020-11-11 Thread Pavel Stehule
čt 12. 11. 2020 v 2:50 odesílatel torikoshia napsal: > On 2020-09-29 02:39, legrand legrand wrote: > > Hi Atsushi, > > > > +1: Your proposal is a good answer for time based performance analysis > > (even if parsing durationor blks are not differentiated) . > > > > As it makes pgss number of

Re: Is it useful to record whether plans are generic or custom?

2020-11-11 Thread torikoshia
On 2020-09-29 02:39, legrand legrand wrote: Hi Atsushi, +1: Your proposal is a good answer for time based performance analysis (even if parsing durationor blks are not differentiated) . As it makes pgss number of columns wilder, may be an other solution would be to create a

Re: Is it useful to record whether plans are generic or custom?

2020-09-28 Thread legrand legrand
Hi Atsushi, +1: Your proposal is a good answer for time based performance analysis (even if parsing durationor blks are not differentiated) . As it makes pgss number of columns wilder, may be an other solution would be to create a pg_stat_statements_xxx view with the same key as pgss

Re: Is it useful to record whether plans are generic or custom?

2020-09-28 Thread torikoshia
On 2020-09-17 13:46, Michael Paquier wrote: On Fri, Jul 31, 2020 at 06:47:48PM +0900, torikoshia wrote: Oops, sorry about that. I just fixed it there for now. The regression tests of the patch look unstable, and the CF bot is reporting a failure here:

Re: Is it useful to record whether plans are generic or custom?

2020-09-16 Thread Michael Paquier
On Fri, Jul 31, 2020 at 06:47:48PM +0900, torikoshia wrote: > Oops, sorry about that. > I just fixed it there for now. The regression tests of the patch look unstable, and the CF bot is reporting a failure here: https://travis-ci.org/github/postgresql-cfbot/postgresql/builds/727833416 -- Michael

RE: Is it useful to record whether plans are generic or custom?

2020-08-17 Thread legrand legrand
I thought it might be preferable to make a GUC to enable or disable this feature, but changing the hash key makes it harder. >> >>> What happens if the server was running with this option enabled and then >>> restarted with the option disabled? Firstly two entries for the same

Re: Is it useful to record whether plans are generic or custom?

2020-08-17 Thread Fujii Masao
On 2020/07/30 16:34, legrand legrand wrote: >> Main purpose is to decide (1) the user interface and (2) the way to get the plan type from pg_stat_statements. (1) the user interface I added a new boolean column 'generic_plan' to both pg_stat_statements view and the member of the hash key of

Re: Is it useful to record whether plans are generic or custom?

2020-07-31 Thread torikoshia
On 2020-07-30 14:31, Fujii Masao wrote: On 2020/07/22 16:49, torikoshia wrote: On 2020-07-20 13:57, torikoshia wrote: As I proposed earlier in this thread, I'm now trying to add information about generic/cudstom plan to pg_stat_statements. I'll share the idea and the poc patch soon.

RE: Is it useful to record whether plans are generic or custom?

2020-07-30 Thread legrand legrand
>> Main purpose is to decide (1) the user interface and (2) the >> way to get the plan type from pg_stat_statements. >> >> (1) the user interface >> I added a new boolean column 'generic_plan' to both >> pg_stat_statements view and the member of the hash key of >> pg_stat_statements. >> >> This is

Re: Is it useful to record whether plans are generic or custom?

2020-07-29 Thread Fujii Masao
On 2020/07/22 16:49, torikoshia wrote: On 2020-07-20 13:57, torikoshia wrote: As I proposed earlier in this thread, I'm now trying to add information about generic/cudstom plan to pg_stat_statements. I'll share the idea and the poc patch soon. Attached a poc patch. Thanks for the POC

Re: Is it useful to record whether plans are generic or custom?

2020-07-22 Thread torikoshia
On 2020-07-20 13:57, torikoshia wrote: As I proposed earlier in this thread, I'm now trying to add information about generic/cudstom plan to pg_stat_statements. I'll share the idea and the poc patch soon. Attached a poc patch. Main purpose is to decide (1) the user interface and (2) the way

Re: Is it useful to record whether plans are generic or custom?

2020-07-19 Thread torikoshia
On 2020-07-20 11:57, Fujii Masao wrote: On 2020/07/17 16:25, Fujii Masao wrote: On 2020/07/16 11:50, torikoshia wrote: On 2020-07-15 11:44, Fujii Masao wrote: On 2020/07/14 21:24, torikoshia wrote: On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On

Re: Is it useful to record whether plans are generic or custom?

2020-07-19 Thread Fujii Masao
On 2020/07/17 16:25, Fujii Masao wrote: On 2020/07/16 11:50, torikoshia wrote: On 2020-07-15 11:44, Fujii Masao wrote: On 2020/07/14 21:24, torikoshia wrote: On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On

Re: Is it useful to record whether plans are generic or custom?

2020-07-17 Thread Fujii Masao
On 2020/07/16 11:50, torikoshia wrote: On 2020-07-15 11:44, Fujii Masao wrote: On 2020/07/14 21:24, torikoshia wrote: On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On

Re: Is it useful to record whether plans are generic or custom?

2020-07-15 Thread torikoshia
On 2020-07-15 11:44, Fujii Masao wrote: On 2020/07/14 21:24, torikoshia wrote: On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10

Re: Is it useful to record whether plans are generic or custom?

2020-07-14 Thread Fujii Masao
On 2020/07/14 21:24, torikoshia wrote: On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +   

Re: Is it useful to record whether plans are generic or custom?

2020-07-14 Thread torikoshia
On 2020-07-10 10:49, torikoshia wrote: On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +    TupleDescInitEntry(tupdesc, (AttrNumber)

Re: Is it useful to record whether plans are generic or custom?

2020-07-09 Thread torikoshia
On 2020-07-08 16:41, Fujii Masao wrote: On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +    TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_plan", This could be a problem

Re: Is it useful to record whether plans are generic or custom?

2020-07-08 Thread Fujii Masao
On 2020/07/08 10:14, torikoshia wrote: On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +    TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_plan", This could be a problem if we showed the last plan in this

Re: Is it useful to record whether plans are generic or custom?

2020-07-07 Thread torikoshia
On 2020-07-06 22:16, Fujii Masao wrote: On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +    TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_plan", This could be a problem if we showed the last plan in this view.  I think "last_plan_type" would be

Re: Is it useful to record whether plans are generic or custom?

2020-07-06 Thread Fujii Masao
On 2020/06/11 14:59, torikoshia wrote: On 2020-06-10 18:00, Kyotaro Horiguchi wrote: +    TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_plan", This could be a problem if we showed the last plan in this view.  I think "last_plan_type" would be better. +    if

Re: Is it useful to record whether plans are generic or custom?

2020-06-11 Thread torikoshia
On 2020-06-10 18:00, Kyotaro Horiguchi wrote: + TupleDescInitEntry(tupdesc, (AttrNumber) 8, "last_plan", This could be a problem if we showed the last plan in this view. I think "last_plan_type" would be better. + if (prep_stmt->plansource->last_plan_type == PLAN_CACHE_TYPE_CUSTOM)

Re: Is it useful to record whether plans are generic or custom?

2020-06-10 Thread Kyotaro Horiguchi
At Wed, 10 Jun 2020 10:50:58 +0900, torikoshia wrote in > On 2020-06-08 20:45, Masahiro Ikeda wrote: > > > BTW, I found that the dependency between function's comments and > > the modified code is broken at latest patch. Before this is > > committed, please fix it. > > ``` > > diff --git

Re: Is it useful to record whether plans are generic or custom?

2020-06-09 Thread torikoshia
On 2020-06-08 20:45, Masahiro Ikeda wrote: BTW, I found that the dependency between function's comments and the modified code is broken at latest patch. Before this is committed, please fix it. ``` diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index

Re: Is it useful to record whether plans are generic or custom?

2020-06-08 Thread Masahiro Ikeda
On 2020-06-04 17:04, Atsushi Torikoshi wrote: On Mon, May 25, 2020 at 10:54 AM Atsushi Torikoshi wrote: On Thu, May 21, 2020 at 5:10 PM Kyotaro Horiguchi wrote: Cost numbers would look better if it is cooked a bit. Is it worth being in core? I didn't come up with ideas about how to use

Re: Is it useful to record whether plans are generic or custom?

2020-06-04 Thread Atsushi Torikoshi
On Mon, May 25, 2020 at 10:54 AM Atsushi Torikoshi wrote: > On Thu, May 21, 2020 at 5:10 PM Kyotaro Horiguchi > wrote: > >> Cost numbers would look better if it is cooked a bit. Is it worth >> being in core? > > > I didn't come up with ideas about how to use them. > IMHO they might not so

Re: Is it useful to record whether plans are generic or custom?

2020-05-24 Thread Atsushi Torikoshi
Thanks for writing a patch! On Thu, May 21, 2020 at 5:10 PM Kyotaro Horiguchi wrote: > At Thu, 21 May 2020 12:18:16 +0900, Fujii Masao < > masao.fu...@oss.nttdata.com> wrote in > > I like the idea exposing more CachedPlanSource fields in > > pg_prepared_statements. I agree it's useful, e.g.,

Re: Is it useful to record whether plans are generic or custom?

2020-05-21 Thread Tatsuro Yamada
Hi Torikoshi-san! On 2020/05/21 17:10, Kyotaro Horiguchi wrote: At Thu, 21 May 2020 12:18:16 +0900, Fujii Masao wrote in On 2020/05/20 21:56, Atsushi Torikoshi wrote: On Wed, May 20, 2020 at 1:32 PM Kyotaro Horiguchi mailto:horikyota@gmail.com>> wrote: At Tue, 19 May 2020

Re: Is it useful to record whether plans are generic or custom?

2020-05-21 Thread Kyotaro Horiguchi
At Thu, 21 May 2020 12:18:16 +0900, Fujii Masao wrote in > > > On 2020/05/20 21:56, Atsushi Torikoshi wrote: > > On Wed, May 20, 2020 at 1:32 PM Kyotaro Horiguchi > > mailto:horikyota@gmail.com>> wrote: > > At Tue, 19 May 2020 22:56:17 +0900, Atsushi Torikoshi > >

Re: Is it useful to record whether plans are generic or custom?

2020-05-20 Thread Fujii Masao
On 2020/05/20 21:56, Atsushi Torikoshi wrote: On Wed, May 20, 2020 at 1:32 PM Kyotaro Horiguchi mailto:horikyota@gmail.com>> wrote: At Tue, 19 May 2020 22:56:17 +0900, Atsushi Torikoshi mailto:ato...@gmail.com>> wrote in > On Sat, May 16, 2020 at 6:01 PM legrand legrand

Re: Is it useful to record whether plans are generic or custom?

2020-05-20 Thread Atsushi Torikoshi
On Wed, May 20, 2020 at 1:32 PM Kyotaro Horiguchi wrote: > At Tue, 19 May 2020 22:56:17 +0900, Atsushi Torikoshi > wrote in > > On Sat, May 16, 2020 at 6:01 PM legrand legrand < > legrand_legr...@hotmail.com> > > wrote: > > > > BTW, I'd also appreciate other opinions about recording the number

Re: Is it useful to record whether plans are generic or custom?

2020-05-19 Thread Kyotaro Horiguchi
At Tue, 19 May 2020 22:56:17 +0900, Atsushi Torikoshi wrote in > On Sat, May 16, 2020 at 6:01 PM legrand legrand > wrote: > > > > To track executed plan types, I think execution layer hooks > > > are appropriate. > > > These hooks, however, take QueryDesc as a param and it does > > > not

Re: Is it useful to record whether plans are generic or custom?

2020-05-19 Thread Atsushi Torikoshi
On Sat, May 16, 2020 at 6:01 PM legrand legrand wrote: > > To track executed plan types, I think execution layer hooks > > are appropriate. > > These hooks, however, take QueryDesc as a param and it does > > not include cached plan information. > > It seems that the same QueryDesc entry is

Re: Is it useful to record whether plans are generic or custom?

2020-05-16 Thread legrand legrand
> To track executed plan types, I think execution layer hooks > are appropriate. > These hooks, however, take QueryDesc as a param and it does > not include cached plan information. It seems that the same QueryDesc entry is reused when executing a generic plan. For exemple marking

Re: Is it useful to record whether plans are generic or custom?

2020-05-15 Thread Atsushi Torikoshi
On Thu, May 14, 2020 at 2:28 AM legrand legrand wrote: > Hello, > > yes this can be usefull, under the condition of differentiating all the > counters > for a queryid using a generic plan and the one using a custom one. > > For me one way to do that is adding a generic_plan column to >

Re: Is it useful to record whether plans are generic or custom?

2020-05-13 Thread legrand legrand
Hello, yes this can be usefull, under the condition of differentiating all the counters for a queryid using a generic plan and the one using a custom one. For me one way to do that is adding a generic_plan column to pg_stat_statements key, someting like: - userid, - dbid, - queryid, -

Is it useful to record whether plans are generic or custom?

2020-05-11 Thread Atsushi Torikoshi
Hi, When we run prepared statements, as far as I know, running EXPLAIN is the only way to know whether executed plans are generic or custom. There seems no way to know how many times a prepared statement was executed as generic and custom. I think it may be useful to record the number of generic