Re: compute_query_id and pg_stat_statements

2021-05-21 Thread Julien Rouhaud
Le sam. 22 mai 2021 à 02:27, Bruce Momjian a écrit : > On Fri, May 21, 2021 at 02:19:13PM -0400, Andrew Dunstan wrote: > > > > On 5/16/21 11:13 PM, Bruce Momjian wrote: > > > On Sun, May 16, 2021 at 08:39:33PM +0800, Julien Rouhaud wrote: > > >> Le dim. 16 mai 2021 à 11:23, Alvaro Herrera > a éc

Re: compute_query_id and pg_stat_statements

2021-05-21 Thread Bruce Momjian
On Fri, May 21, 2021 at 02:19:13PM -0400, Andrew Dunstan wrote: > > On 5/16/21 11:13 PM, Bruce Momjian wrote: > > On Sun, May 16, 2021 at 08:39:33PM +0800, Julien Rouhaud wrote: > >> Le dim. 16 mai 2021 à 11:23, Alvaro Herrera a > >> écrit : > >> > >> On 2021-May-15, Bruce Momjian wrote: > >

Re: compute_query_id and pg_stat_statements

2021-05-21 Thread Andrew Dunstan
On 5/16/21 11:13 PM, Bruce Momjian wrote: > On Sun, May 16, 2021 at 08:39:33PM +0800, Julien Rouhaud wrote: >> Le dim. 16 mai 2021 à 11:23, Alvaro Herrera a >> écrit : >> >> On 2021-May-15, Bruce Momjian wrote: >> >> > On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: >> >>

Re: compute_query_id and pg_stat_statements

2021-05-16 Thread Bruce Momjian
On Sun, May 16, 2021 at 08:39:33PM +0800, Julien Rouhaud wrote: > Le dim. 16 mai 2021 à 11:23, Alvaro Herrera a écrit  > : > > On 2021-May-15, Bruce Momjian wrote: > > > On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: > > > > I also added Alvaro as an author of the co

Re: compute_query_id and pg_stat_statements

2021-05-16 Thread Bruce Momjian
On Sat, May 15, 2021 at 07:01:25PM -0400, Álvaro Herrera wrote: > On 2021-May-15, Bruce Momjian wrote: > > > On Sat, May 15, 2021 at 02:21:59PM -0400, Álvaro Herrera wrote: > > > > I wonder why the initial line says "query hash" instead of "query > > > identifier". Do we want to say "hash" every

Re: compute_query_id and pg_stat_statements

2021-05-16 Thread Bruce Momjian
On Sat, May 15, 2021 at 11:23:25PM -0400, Álvaro Herrera wrote: > On 2021-May-15, Bruce Momjian wrote: > > > On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: > > > > I also added Alvaro as an author of the compute_query_id item. > >

Re: compute_query_id and pg_stat_statements

2021-05-16 Thread Julien Rouhaud
Le dim. 16 mai 2021 à 11:23, Alvaro Herrera a écrit : > On 2021-May-15, Bruce Momjian wrote: > > > On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: > > > > I also added Alvaro as an author of the compute_query_id item. > > --

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Alvaro Herrera
On 2021-May-15, Bruce Momjian wrote: > On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: > > I also added Alvaro as an author of the compute_query_id item. > -- > > Based on the commit message, adding Alvaro was incorrect

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Bruce Momjian
On Sat, May 15, 2021 at 05:32:58PM -0400, Bruce Momjian wrote: > OK, new text is: > > > > > > Move query hash computation from pg_stat_statements to the core > server (Julien Rouhaud) > > > > The new server variable compute_query_i

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Alvaro Herrera
On 2021-May-15, Bruce Momjian wrote: > On Sat, May 15, 2021 at 02:21:59PM -0400, Álvaro Herrera wrote: > > I wonder why the initial line says "query hash" instead of "query > > identifier". Do we want to say "hash" everywhere? Why didn't we name > > the GUC "compute_query_hash" in that case? >

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Bruce Momjian
On Sat, May 15, 2021 at 02:21:59PM -0400, Álvaro Herrera wrote: > I commented out the release notes para that is now wrong. What remains > is this: > > Move query hash computation from pg_stat_statements to the core server > (Julien Rouhaud) > > We could perhaps add something like > > Exte

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Alvaro Herrera
On 2021-May-16, Julien Rouhaud wrote: > On Sat, May 15, 2021 at 10:00 PM Bruce Momjian wrote: > > > > I am no longer the committer in charge of this feature, but I would like > > to remind the group that beta1 will be wrapped on Monday, and it is hard > > to change non-read-only GUCs after beta s

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Julien Rouhaud
On Sat, May 15, 2021 at 10:00 PM Bruce Momjian wrote: > > I am no longer the committer in charge of this feature, but I would like > to remind the group that beta1 will be wrapped on Monday, and it is hard > to change non-read-only GUCs after beta since the settings are embedded > in postgresql.co

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Bruce Momjian
On Sat, May 15, 2021 at 04:09:32PM +0800, Julien Rouhaud wrote: > > While thinking about this patch it occurred to that an useful gadget > > might be to let pg_stat_statements be loaded always, but with > > compute_query_id=false; so it's never active; except if you > > ALTER {DATABASE, USER} foo

Re: compute_query_id and pg_stat_statements

2021-05-15 Thread Julien Rouhaud
On Sat, May 15, 2021 at 7:50 AM Alvaro Herrera wrote: > > I took out the new WARNING in pg_stat_statements. It's not clear to me > that that's terribly useful (it stops working as soon as you have one > query in the pg_stat_statements stash and later disable everything). If no query_id is calcul

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Justin Pryzby
On Fri, May 14, 2021 at 07:50:13PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7643,7 +7643,12 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > identifier to be computed. Note that an external module can > alternatively be used if the i

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Alvaro Herrera
On 2021-May-14, Julien Rouhaud wrote: > On Fri, May 14, 2021 at 12:20:00PM +0900, Fujii Masao wrote: > > +void > > +EnableQueryId(void) > > +{ > > + if (compute_query_id == COMPUTE_QUERY_ID_AUTO) > > + auto_query_id_enabled = true; > > > > Shouldn't EnableQueryId() enable auto_query_

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Pavel Stehule
pá 14. 5. 2021 v 19:28 odesílatel Bruce Momjian napsal: > On Fri, May 14, 2021 at 12:21:23PM -0400, Bruce Momjian wrote: > > On Fri, May 14, 2021 at 12:04:05PM -0400, Andrew Dunstan wrote: > > > > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > > > > > > > On Thu, May 13, 2021 at 09:41:42

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Bruce Momjian
On Fri, May 14, 2021 at 12:21:23PM -0400, Bruce Momjian wrote: > On Fri, May 14, 2021 at 12:04:05PM -0400, Andrew Dunstan wrote: > > > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > > > > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > > > I think keeping the output as

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Pavel Stehule
pá 14. 5. 2021 v 18:21 odesílatel Bruce Momjian napsal: > On Fri, May 14, 2021 at 12:04:05PM -0400, Andrew Dunstan wrote: > > > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > > > > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > > > I think keeping the output as 'auto'

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Bruce Momjian
On Fri, May 14, 2021 at 12:04:05PM -0400, Andrew Dunstan wrote: > > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > > I think keeping the output as 'auto', and documenting that this query > > must be run to determine if t

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Julien Rouhaud
On Fri, May 14, 2021 at 12:04:05PM -0400, Andrew Dunstan wrote: > > > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > >>> On Fri, May 14, 2021 at 09:40:15AM +0800, Julien Rouhaud wrote: > >>> On Fri, May 14, 2021 at 8:13

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Andrew Dunstan
Sent from my iPhone > On May 14, 2021, at 8:35 AM, Bruce Momjian wrote: > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: >>> On Fri, May 14, 2021 at 09:40:15AM +0800, Julien Rouhaud wrote: >>> On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: On Thu, May 13,

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Julien Rouhaud
On Fri, May 14, 2021 at 08:57:41AM -0400, Bruce Momjian wrote: > On Fri, May 14, 2021 at 08:35:14AM -0400, Bruce Momjian wrote: > > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > > > I think if we keep the output as 'auto', and document that you check > > > pg_stat_activity for a

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Bruce Momjian
On Fri, May 14, 2021 at 08:35:14AM -0400, Bruce Momjian wrote: > On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > > I think if we keep the output as 'auto', and document that you check > > pg_stat_activity for a hash to see if it is enabled, that gets us pretty > > far. > > I think

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Bruce Momjian
On Thu, May 13, 2021 at 09:41:42PM -0400, Bruce Momjian wrote: > On Fri, May 14, 2021 at 09:40:15AM +0800, Julien Rouhaud wrote: > > On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: > > > > > > On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > > > > Here's a first attempt at wh

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Bruce Momjian
On Fri, May 14, 2021 at 12:26:23AM -0400, Tom Lane wrote: > I then tried a really dumb xor'ing implementation, and > that gives me a slowdown of 2.2%. This could undoubtedly > be improved on further, say by unrolling the loop or > processing multiple bytes at once. One problem with it > is that I

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
I wrote: > Maybe we should revert this thing pending somebody doing the work to > make a version of queryid labeling that actually is negligibly cheap. > It certainly seems like that could be done; one more traversal of the > parse tree can't be that expensive in itself. I suspect that the > perfo

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 12:20:00PM +0900, Fujii Masao wrote: > > > On 2021/05/14 9:04, Alvaro Herrera wrote: > > Here's a first attempt at what was suggested. If you say "auto" it > > remains auto in SHOW, but it gets enabled if a module asks for it. > > > > Not final yet, but I thought I'd thr

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Fujii Masao
On 2021/05/14 9:04, Alvaro Herrera wrote: Here's a first attempt at what was suggested. If you say "auto" it remains auto in SHOW, but it gets enabled if a module asks for it. Not final yet, but I thought I'd throw it out for early commentary ... Many thanks! The patch basically looks good

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 09:47:02PM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: > >> I was surprised it was ~2%. > > > Just to be clear, the 2% was a worst case scenario, ie. a very fast > > read-only query on small data returning a sin

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Julien Rouhaud writes: > On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: >> I was surprised it was ~2%. > Just to be clear, the 2% was a worst case scenario, ie. a very fast > read-only query on small data returning a single row. As soon as you > get something more realistic / expensive th

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Fri, May 14, 2021 at 09:40:15AM +0800, Julien Rouhaud wrote: > On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: > > > > On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > > > Here's a first attempt at what was suggested. If you say "auto" it > > > remains auto in SHOW, but i

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: > > On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > > Here's a first attempt at what was suggested. If you say "auto" it > > remains auto in SHOW, but it gets enabled if a module asks for it. > > > > Not final yet, but I thoug

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: > > > Maybe we should revert this thing pending somebody doing the work to > > make a version of queryid labeling that actually is negligibly cheap. > > It certainly seems like that could be done; one more traversal of the > > parse tree can't b

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > Here's a first attempt at what was suggested. If you say "auto" it > remains auto in SHOW, but it gets enabled if a module asks for it. > > Not final yet, but I thought I'd throw it out for early commentary ... I certainly like th

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Alvaro Herrera
Here's a first attempt at what was suggested. If you say "auto" it remains auto in SHOW, but it gets enabled if a module asks for it. Not final yet, but I thought I'd throw it out for early commentary ... -- Álvaro Herrera Valdivia, Chile diff --git a/contrib/pg_stat_statements/expected/p

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 07:19:11PM -0400, Andrew Dunstan wrote: > On 5/13/21 3:04 PM, Alvaro Herrera wrote: >> I'm happy to act as committer for that if he wants to step away from it. >> I'm already used to being lapidated at every corner anyway. > > Many thanks Alvaro, among other things for teac

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 07:19:11PM -0400, Andrew Dunstan wrote: > > On 5/13/21 3:04 PM, Alvaro Herrera wrote: > > On 2021-May-13, Stephen Frost wrote: > > > >> Or just accept that this is a bit hokey with the 'auto' approach. I get > >> Bruce has concerns about it but I'm not convinced that it's

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Andrew Dunstan
On 5/13/21 3:04 PM, Alvaro Herrera wrote: > On 2021-May-13, Stephen Frost wrote: > >> Or just accept that this is a bit hokey with the 'auto' approach. I get >> Bruce has concerns about it but I'm not convinced that it's actually all >> that bad to go with that. > Yeah, I think the alleged confu

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 03:04:30PM -0400, Álvaro Herrera wrote: > On 2021-May-13, Stephen Frost wrote: > > > Or just accept that this is a bit hokey with the 'auto' approach. I get > > Bruce has concerns about it but I'm not convinced that it's actually all > > that bad to go with that. > > Yeah

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 02:29:09PM -0400, Tom Lane wrote: > Stephen Frost writes: > > There's a ridiculously simple option here which is: drop the idea that > > we support an extension redefining the query id and then just make it > > on/off with the default to be 'on'. > > I do not think that de

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Stephen Frost wrote: > Or just accept that this is a bit hokey with the 'auto' approach. I get > Bruce has concerns about it but I'm not convinced that it's actually all > that bad to go with that. Yeah, I think the alleged confusion there is overstated. I'm happy to act as comm

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > There's a ridiculously simple option here which is: drop the idea that > > we support an extension redefining the query id and then just make it > > on/off with the default to be 'on'. > > I do not think that defaultin

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Stephen Frost writes: > There's a ridiculously simple option here which is: drop the idea that > we support an extension redefining the query id and then just make it > on/off with the default to be 'on'. I do not think that defaulting it to 'on' is acceptable unless you can show that the added o

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, May 13, 2021 at 07:39:45PM +0200, Christoph Berg wrote: > > Re: Bruce Momjian > > > Well, now that we have clear warnings when it is misconfigured, > > > especially when querying the pg_stat_statements view, are these > > > complaints

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 07:39:45PM +0200, Christoph Berg wrote: > Re: Bruce Momjian > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? Also, how is modifying > > shared_preload_librari

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:51:07PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > > > I'm coming around to have a similar feeling. While having an > > > alternative query ID might be usefu

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > > I'm coming around to have a similar feeling. While having an > > alternative query ID might be useful, I have a hard time seeing it as > > likely to be a hugely popular featur

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > I'm coming around to have a similar feeling. While having an > alternative query ID might be useful, I have a hard time seeing it as > likely to be a hugely popular feature that is worth a lot of users > complaining (as we've seen al

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Christoph Berg
Re: Bruce Momjian > Well, now that we have clear warnings when it is misconfigured, > especially when querying the pg_stat_statements view, are these > complaints still valid? Also, how is modifying > shared_preload_libraries zero-config, but modifying > shared_preload_libraries and compute_query

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:17:16PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > The only thing that bugs me is that we're pretty damn late in the > > process to be engaging in this amount of design. > > Indeed. I feel that this feature was forced in before it was really > ready. The user

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andrew Dunstan writes: > > The only thing that bugs me is that we're pretty damn late in the > > process to be engaging in this amount of design. > > Indeed. I feel that this feature was forced in before it was really > ready. I'm coming arou

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Andrew Dunstan writes: > The only thing that bugs me is that we're pretty damn late in the > process to be engaging in this amount of design. Indeed. I feel that this feature was forced in before it was really ready. regards, tom lane

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 12:45:07PM -0400, Andrew Dunstan wrote: > > On 5/13/21 12:18 AM, Fujii Masao wrote: > > > > > > > > If we do this, compute_query_id=auto seems to be similar to > > huge_pages=try. > > When huge_pages=try, whether huge pages is actually used is defined > > depending > > outs

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Andrew Dunstan
On 5/13/21 12:18 AM, Fujii Masao wrote: > > > > If we do this, compute_query_id=auto seems to be similar to > huge_pages=try. > When huge_pages=try, whether huge pages is actually used is defined > depending > outside PostgreSQL (i.e, OS setting in this case). Neither > pg_settings.setting nor >

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 09:30:55AM -0700, Maciek Sakrejda wrote: > On Thu, May 13, 2021 at 8:38 AM Julien Rouhaud wrote: > > > > On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > > > > > The warning makes it clear that there's something wrong, but not how > > > to fix it > > > >

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Maciek Sakrejda
On Thu, May 13, 2021 at 8:38 AM Julien Rouhaud wrote: > > On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > > > The warning makes it clear that there's something wrong, but not how > > to fix it > > I'm confused, are we talking about the new warning in v2 as suggested by > Pave

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 11:35:13PM +0800, Julien Rouhaud wrote: > On Thu, May 13, 2021 at 10:41:43AM -0400, Bruce Momjian wrote: > > > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? >

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? Also, how is modifying > > shared_preload_libraries zero-config, but

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > The warning makes it clear that there's something wrong, but not how > to fix it I'm confused, are we talking about the new warning in v2 as suggested by Pavel? As it should make things quite clear: +SELECT count(*) FROM pg_sta

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 10:41:43AM -0400, Bruce Momjian wrote: > > Well, now that we have clear warnings when it is misconfigured, > especially when querying the pg_stat_statements view, are these > complaints still valid? I'm personally fine with it, and I can send a new version with just the wa

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Maciek Sakrejda
On Thu, May 13, 2021 at 7:42 AM Bruce Momjian wrote: > > On Thu, May 13, 2021 at 12:03:42PM +0800, Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > > I don't know what to say. So here is a summary of the complaints that I'm > > aware of: > > > > - > > ht

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 12:03:42PM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > I don't know what to say. So here is a summary of the complaints that I'm > aware of: > > - > https://www.postgresql.org/message-id/1953aec168224b95b0c962a622bef0794

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 04:15:30PM +0900, Kyotaro Horiguchi wrote: > > > > what if you want to have some other extensions like pg_stat_kcache or > > pg_store_plans that need a query_id but don't really care if > > pg_stat_statements is enabled or not? should they all declare their own > > Thanks

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 12:33:47 +0800, Julien Rouhaud wrote in > Le jeu. 13 mai 2021 à 12:26, Kyotaro Horiguchi a > écrit : > > > At Thu, 13 May 2021 12:11:12 +0900 (JST), Kyotaro Horiguchi < > > horikyota@gmail.com> wrote in > > pg_stat_statemnts defines its own query-id provider function in

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 10:31:01PM -0700, Maciek Sakrejda wrote: > On Wed, May 12, 2021 at 9:58 PM Julien Rouhaud wrote: > > Le jeu. 13 mai 2021 à 12:52, Maciek Sakrejda a écrit > > : > >> > >> For what it's worth, I don't think the actual changing of an extra > >> setting is that big a burden:

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Maciek Sakrejda
On Wed, May 12, 2021 at 9:58 PM Julien Rouhaud wrote: > Le jeu. 13 mai 2021 à 12:52, Maciek Sakrejda a écrit : >> >> For what it's worth, I don't think the actual changing of an extra >> setting is that big a burden: it's the figuring out that you need to >> change it, and how you should configur

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
Le jeu. 13 mai 2021 à 12:52, Maciek Sakrejda a écrit : > > For what it's worth, I don't think the actual changing of an extra > setting is that big a burden: it's the figuring out that you need to > change it, and how you should configure it, that is the problem. > Especially since all major sear

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Maciek Sakrejda
On Wed, May 12, 2021 at 9:03 PM Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > > How do they know to set shared_preload_libraries then? We change the > > user API all the time, especially in GUCs, and even rename them, but for > > some reason we don't t

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
Le jeu. 13 mai 2021 à 12:26, Kyotaro Horiguchi a écrit : > At Thu, 13 May 2021 12:11:12 +0900 (JST), Kyotaro Horiguchi < > horikyota@gmail.com> wrote in > > As the result, even if we take the DLL approach, still not need to > > split out the query-id provider part. By the following config: >

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 13:26:37 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 13 May 2021 12:11:12 +0900 (JST), Kyotaro Horiguchi > wrote in > > As the result, even if we take the DLL approach, still not need to > > split out the query-id provider part. By the following config: > > > > > qu

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 12:11:12 +0900 (JST), Kyotaro Horiguchi wrote in > As the result, even if we take the DLL approach, still not need to > split out the query-id provider part. By the following config: > > > query_id_provider = 'pg_stat_statements' > > the core can obtain the entrypoint of, s

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
Le jeu. 13 mai 2021 à 12:18, Fujii Masao a écrit : > > I like leaving compute_query_id=auto instead of overwriting it to "on", > even when queryIsWanted() is called, as I told upthread. Then we can decide > that query id computation is necessary when compute_query_id=auto and > the special flag i

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Fujii Masao
On 2021/05/13 13:03, Julien Rouhaud wrote: On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: On Thu, May 13, 2021 at 11:16:13AM +0800, Julien Rouhaud wrote: On Wed, May 12, 2021 at 11:06:52PM -0400, Bruce Momjian wrote: On Thu, May 13, 2021 at 09:57:00AM +0800, Julien Rouhaud w

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > On Thu, May 13, 2021 at 11:16:13AM +0800, Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:06:52PM -0400, Bruce Momjian wrote: > > > On Thu, May 13, 2021 at 09:57:00AM +0800, Julien Rouhaud wrote: > > > > > > > source? What if y

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Bruce Momjian
On Thu, May 13, 2021 at 11:16:13AM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 11:06:52PM -0400, Bruce Momjian wrote: > > On Thu, May 13, 2021 at 09:57:00AM +0800, Julien Rouhaud wrote: > > > > > source? What if you have for instance pg_stat_statements, pg_stat_kcache, > > > pg_store_p

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 12:11:12PM +0900, Kyotaro Horiguchi wrote: > At Thu, 13 May 2021 10:43:03 +0800, Julien Rouhaud wrote > in > > On Thu, May 13, 2021 at 11:26:29AM +0900, Kyotaro Horiguchi wrote: > > > > > > I believe any "real" > > > alternative query-id provider is supposed to be hooked

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 11:06:52PM -0400, Bruce Momjian wrote: > On Thu, May 13, 2021 at 09:57:00AM +0800, Julien Rouhaud wrote: > > > source? What if you have for instance pg_stat_statements, pg_stat_kcache, > > pg_store_plans and pg_wait_sampling installed? All those extensions need a > > quer

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 10:43:03 +0800, Julien Rouhaud wrote in > On Thu, May 13, 2021 at 11:26:29AM +0900, Kyotaro Horiguchi wrote: > > > > I believe any "real" > > alternative query-id provider is supposed to be hooked "before" > > pg_stat_statements. (It is a kind of magic to control the order o

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Bruce Momjian
On Thu, May 13, 2021 at 09:57:00AM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 09:13:25PM -0400, Bruce Momjian wrote: > > On Thu, May 13, 2021 at 08:52:36AM +0800, Julien Rouhaud wrote: > > > > > > Well, as implemented you can get the value of compute_query_id, and if > > > it's > > >

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 11:49:34AM +0900, Kyotaro Horiguchi wrote: > At Thu, 13 May 2021 10:39:20 +0800, Julien Rouhaud wrote > in > > On Thu, May 13, 2021 at 11:30:56AM +0900, Kyotaro Horiguchi wrote: > > > At Thu, 13 May 2021 11:26:29 +0900 (JST), Kyotaro Horiguchi > > > wrote in > > > > At

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 10:39:20 +0800, Julien Rouhaud wrote in > On Thu, May 13, 2021 at 11:30:56AM +0900, Kyotaro Horiguchi wrote: > > At Thu, 13 May 2021 11:26:29 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > At Thu, 13 May 2021 10:02:45 +0800, Julien Rouhaud > > > wrote in > > > Yes, I

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 11:26:29AM +0900, Kyotaro Horiguchi wrote: > > I believe any "real" > alternative query-id provider is supposed to be hooked "before" > pg_stat_statements. (It is a kind of magic to control the order of > plugins, though.. Indeed, you have to configure shared_preload_libra

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 11:30:56AM +0900, Kyotaro Horiguchi wrote: > At Thu, 13 May 2021 11:26:29 +0900 (JST), Kyotaro Horiguchi > wrote in > > At Thu, 13 May 2021 10:02:45 +0800, Julien Rouhaud > > wrote in > > > On Thu, May 13, 2021 at 10:51:52AM +0900, Kyotaro Horiguchi wrote: > > > > At T

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 11:26:29 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 13 May 2021 10:02:45 +0800, Julien Rouhaud wrote > in > > On Thu, May 13, 2021 at 10:51:52AM +0900, Kyotaro Horiguchi wrote: > > > At Thu, 13 May 2021 09:59:43 +0900 (JST), Kyotaro Horiguchi > > > wrote in > > >

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 10:02:45 +0800, Julien Rouhaud wrote in > On Thu, May 13, 2021 at 10:51:52AM +0900, Kyotaro Horiguchi wrote: > > At Thu, 13 May 2021 09:59:43 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > How about adding a GUC_INTERNAL "current_query_provider" or such? > > > > On the

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 10:51:52AM +0900, Kyotaro Horiguchi wrote: > At Thu, 13 May 2021 09:59:43 +0900 (JST), Kyotaro Horiguchi > wrote in > > How about adding a GUC_INTERNAL "current_query_provider" or such? > > On the second thought, I wonder why we don't just call JumbleQuery in > pgss_post

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 09:13:25PM -0400, Bruce Momjian wrote: > On Thu, May 13, 2021 at 08:52:36AM +0800, Julien Rouhaud wrote: > > > > Well, as implemented you can get the value of compute_query_id, and if it's > > still "auto" then it's not enabled as calling queryIdWanted() would turn it > >

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 09:59:43 +0900 (JST), Kyotaro Horiguchi wrote in > How about adding a GUC_INTERNAL "current_query_provider" or such? On the second thought, I wonder why we don't just call JumbleQuery in pgss_post_parse_analyze when compute_query_id is "off". We can think this behavior as t

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Thu, May 13, 2021 at 09:59:43AM +0900, Kyotaro Horiguchi wrote: > > The query_id of its own is provided because pg_stat_statements did not > expose query_id. And it has been preserved only for the case the > plugin is used without pg_stat_statements activated. Now that the > in-core query_id i

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Bruce Momjian
On Thu, May 13, 2021 at 08:52:36AM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 08:36:18PM -0400, Bruce Momjian wrote: > > The problem with compute_query_id=auto is that there is no way to know > > if the query id is actually enabled, unless you guess from the installed > > extensions, or

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Wed, 12 May 2021 20:36:18 -0400, Bruce Momjian wrote in > On Wed, May 12, 2021 at 05:51:49PM +0800, Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:42:12AM +0200, Pavel Stehule wrote: > > > this check just can check if there is "any" query-id provider. In this > > > context is not importa

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Kyotaro Horiguchi
At Wed, 12 May 2021 18:09:30 +0800, Julien Rouhaud wrote in > On Wed, May 12, 2021 at 06:37:24PM +0900, Kyotaro Horiguchi wrote: > > At Wed, 12 May 2021 14:05:16 +0800, Julien Rouhaud > > wrote in > > > > > > And if I'm not mistaken, pg_store_plans also wants a different queryid > > > implem

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 08:36:18PM -0400, Bruce Momjian wrote: > On Wed, May 12, 2021 at 05:51:49PM +0800, Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:42:12AM +0200, Pavel Stehule wrote: > > > this check just can check if there is "any" query-id provider. In this > > > context is not impor

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Bruce Momjian
On Wed, May 12, 2021 at 05:51:49PM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 11:42:12AM +0200, Pavel Stehule wrote: > > this check just can check if there is "any" query-id provider. In this > > context is not important if it is buildin or external > > Yes, the idea is that if you exe

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 05:30:26PM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 10:57:25AM +0200, Pavel Stehule wrote: > > > > My second proposal can work for your example too. pg_stat_statements have > > to require any active queryid computing. And when it is not available, then > > the

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 06:37:24PM +0900, Kyotaro Horiguchi wrote: > At Wed, 12 May 2021 14:05:16 +0800, Julien Rouhaud wrote > in > > > > And if I'm not mistaken, pg_store_plans also wants a different queryid > > implementation, but has to handle a secondary queryid on top of it > > (https://g

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Julien Rouhaud
On Wed, May 12, 2021 at 11:42:12AM +0200, Pavel Stehule wrote: > st 12. 5. 2021 v 11:39 odesílatel Kyotaro Horiguchi > napsal: > > > At Wed, 12 May 2021 17:30:26 +0800, Julien Rouhaud > > wrote in > > > On Wed, May 12, 2021 at 10:57:25AM +0200, Pavel Stehule wrote: > > > > > > > > My second prop

Re: compute_query_id and pg_stat_statements

2021-05-12 Thread Pavel Stehule
st 12. 5. 2021 v 11:39 odesílatel Kyotaro Horiguchi napsal: > At Wed, 12 May 2021 17:30:26 +0800, Julien Rouhaud > wrote in > > On Wed, May 12, 2021 at 10:57:25AM +0200, Pavel Stehule wrote: > > > > > > My second proposal can work for your example too. pg_stat_statements > have > > > to require

  1   2   >