Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-04-05 Thread m . litsarev
Hi! Thank you for detailed explanations. Respectfully, Mikhail Litsarev, Postgres Professional: https://postgrespro.com

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-04-03 Thread Tom Lane
m.litsa...@postgrespro.ru writes: >> What does this patch give on aglobal scale? Does it save much memory or >> increase performance? How many times? > This patch makes the code semantically more correct and we don't lose > anything. It is obviously not about performance or memory optimisation.

Re: pg_stat_statements and "IN" conditions

2025-03-22 Thread Sami Imseih
> Sure, it's important and I'm planning to tackle this next. If you want, > we can collaborate on a patch for that. I spent some time looking some more at this, and I believe all that needs to be done is check for a PRAM node with a type of PARAM_EXTERN. During planning the planner turns the Para

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 02:54:18PM GMT, Sami Imseih wrote: > I want to mention that the current patch does not deal > with external parameters ( prepared statements ) [0] [1]. This could be a > follow-up, as it may need some further discussion. It is important to > address this case, IMO. Sure,

Re: pg_stat_statements and "IN" conditions

2025-03-19 Thread Dmitry Dolgov
> On Tue, Mar 18, 2025 at 07:33:25PM GMT, Álvaro Herrera wrote: > > By the way, I'm still open to adding the 'powers' mechanism that was > discussed earlier and other simple additions on top of what's there now, > if you have some spare energy to spend on this. (For full disclosure: > the "powers"

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Álvaro Herrera
On 2025-Mar-18, Sami Imseih wrote: > I want to mention that the current patch does not deal > with external parameters ( prepared statements ) [0] [1]. This could be a > follow-up, as it may need some further discussion. It is important to > address this case, IMO. Yes, I realize that. Feel free

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Álvaro Herrera
On 2025-Mar-18, Dmitry Dolgov wrote: > Thanks, much appreciated! I've inspected the diff between patches and > run few tests, at the first glance everything looks fine. Thanks for looking once more. > > I am tempted to say that explicit casts should also be considered > > squashable (that is, in

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Sami Imseih
I want to mention that the current patch does not deal with external parameters ( prepared statements ) [0] [1]. This could be a follow-up, as it may need some further discussion. It is important to address this case, IMO. [0] https://www.postgresql.org/message-id/CAA5RZ0uGfxXyzhp9N5nfsS%2BZqF5ng

Re: pg_stat_statements and "IN" conditions

2025-03-18 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 08:14:16PM GMT, Álvaro Herrera wrote: > On 2025-Mar-17, Álvaro Herrera wrote: > > > You can see my patch on top of yours here: > > https://github.com/alvherre/postgres/commits/query_id_squash_values/ > > and the CI run here: > > https://cirrus-ci.com/build/5660053472018432

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Álvaro Herrera
On 2025-Mar-17, Álvaro Herrera wrote: > You can see my patch on top of yours here: > https://github.com/alvherre/postgres/commits/query_id_squash_values/ > and the CI run here: > https://cirrus-ci.com/build/5660053472018432 Heh, this blew up on bogus SGML markup :-( Fixed and running again: http

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Álvaro Herrera
On 2025-Feb-14, Dmitry Dolgov wrote: > This should do it. The last patch for today, otherwise I'll probably add > more bugs than features :) Thank you. I've spent some time with this patch in the last few days, and I propose a few changes. I renamed everything from "merge" to "squash"; apart fr

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread vignesh C
On Mon, 17 Mar 2025 at 13:42, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Mon, Mar 17, 2025 at 12:07:44PM GMT, vignesh C wrote: > > > > I noticed that the feedback from Sami at [1] has not yet been > > addressed, I have changed the status to Waiting on Author, kindly > > address them and

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Álvaro Herrera
On 2025-Mar-17, Dmitry Dolgov wrote: > I'm afraid there is a disagreement about this part of the feedback. I'm > not yet convinced about the idea suggested over there (treating mutable > functions in the same way as constants) and not planning to change > anything, at least not in the current vers

Re: pg_stat_statements and "IN" conditions

2025-03-17 Thread Dmitry Dolgov
> On Mon, Mar 17, 2025 at 12:07:44PM GMT, vignesh C wrote: > > I noticed that the feedback from Sami at [1] has not yet been > addressed, I have changed the status to Waiting on Author, kindly > address them and update the status to Needs review. > [1] - > https://www.postgresql.org/message-id/CAA

Re: pg_stat_statements and "IN" conditions

2025-03-16 Thread vignesh C
On Mon, 3 Mar 2025 at 17:26, Álvaro Herrera wrote: > > On 2025-Feb-18, Sami Imseih wrote: > > > > It's not a question about whether it's possible to implement this, > > > but about whether it makes sense. In case of plain constants it's > > > straightforward -- they will not change anything meanin

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Sami Imseih
> > > It's not a question about whether it's possible to implement this, > > > but about whether it makes sense. In case of plain constants it's > > > straightforward -- they will not change anything meaningfully and > > > hence could be squashed from the query. Now for a function, that > > > might

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Dmitry Dolgov
> On Mon, Mar 03, 2025 at 12:56:24PM GMT, Álvaro Herrera wrote: > In the meantime, here's v28 which is Dmitry's v27 plus pgindent. No > other changes. Dmitry, were you planning to submit a new version? Nope, there was nothing I wanted to add so far.

Re: pg_stat_statements and "IN" conditions

2025-03-03 Thread Álvaro Herrera
On 2025-Feb-18, Sami Imseih wrote: > > It's not a question about whether it's possible to implement this, > > but about whether it makes sense. In case of plain constants it's > > straightforward -- they will not change anything meaningfully and > > hence could be squashed from the query. Now for

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Michael Paquier
On Tue, Feb 18, 2025 at 08:48:43AM -0600, Sami Imseih wrote: >> Btw, if you would like to share a code delta, please do not post it as a >> patch or diff. This hijacks the CI pipeline, because CFbot thinks that's >> a new version of the original patch. > > You're right. Sorry about that. Exchangi

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Sami Imseih
> > > This test was to catch a crash that was happening in older version of > > > the patch, so it doesn't have to verify the actual pgss entry. > > > > It seems odd to keep this test because of crash behavior experienced > > in a previous version of the patch. if the crash reason was understood >

Re: pg_stat_statements and "IN" conditions

2025-02-18 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 01:50:00PM GMT, Sami Imseih wrote: > > This test was to catch a crash that was happening in older version of > > the patch, so it doesn't have to verify the actual pgss entry. > > It seems odd to keep this test because of crash behavior experienced > in a previous version

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Sami Imseih
> This test was to catch a crash that was happening in older version of > the patch, so it doesn't have to verify the actual pgss entry. It seems odd to keep this test because of crash behavior experienced in a previous version of the patch. if the crash reason was understood and resolved, why kee

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Dmitry Dolgov
> On Mon, Feb 17, 2025 at 09:51:32AM GMT, Sami Imseih wrote: > > This should do it. The last patch for today, > > I looked at v27 today and have a few comments. > > 1/ It looks like the CTE test is missing a check for results. This test was to catch a crash that was happening in older version of t

Re: pg_stat_statements and "IN" conditions

2025-02-17 Thread Sami Imseih
> This should do it. The last patch for today, I looked at v27 today and have a few comments. 1/ It looks like the CTE test is missing a check for results. """ -- Test constants evaluation in a CTE, which was causing issues in the past WITH cte AS ( SELECT 'const' as const FROM test_merge ) SELE

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > Wouldn't doing something like this inside IsMergeableConst > > """ > > if (!IsA(arg, Const) && !IsA(arg, Param)) > > """ > > > > instead of > > """ > > if (!IsA(arg, Const)) > > """ > > > > be sufficient? > > That's exactly what the original rejected implementation was doing. I > guess to answe

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > > I perhap meant "missing chunk" instead of "trimming". To me it just > > > looked like a trimmed text, which was wrong. Looks like v25 > > > deals with that better at least. I am just not sure about all that we are > > > doing > > > here as I believe it may open up big changes for bugs genera

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > I perhap meant "missing chunk" instead of "trimming". To me it just > > looked like a trimmed text, which was wrong. Looks like v25 > > deals with that better at least. I am just not sure about all that we are > > doing > > here as I believe it may open up big changes for bugs generating the

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 09:06:24AM GMT, Sami Imseih wrote: > I think it will be sad to not include this very common case from > the start, because it is going to be one of the most common > cases. > > Wouldn't doing something like this inside IsMergeableConst > """ > if (!IsA(arg, Const) && !IsA(

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 11:12:25PM GMT, Julien Rouhaud wrote: > > > There seems to be an off-by-1 error in parameter numbering when merging > > > them. > > > > There are indeed three constants, but the second is not visible in the > > query text. Maybe makes sense to adjust the number in this ca

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 03:56:32PM +0100, Dmitry Dolgov wrote: > > On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > > There seems to be an off-by-1 error in parameter numbering when merging > > them. > > There are indeed three constants, but the second is not visible in the > query

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > I do see the discussion here [1], sorry for not noticing it. > > > > I am not sure about this though. At minimum this needs to be documented, > > However, I think the prepared statement case is too common of a case to > > skip for the first release of tis feature, and users that will likely >

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 10:39:45PM GMT, Julien Rouhaud wrote: > There seems to be an off-by-1 error in parameter numbering when merging them. There are indeed three constants, but the second is not visible in the query text. Maybe makes sense to adjust the number in this case, let me try. > Not

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 03:20:24PM +0100, Dmitry Dolgov wrote: > > Btw, there was another mistake in the last version introducing > "$1 /*, ... */" format, the constant position has to be of course > calculated as usual. I'm not sure what you mean here, but just in case: > +SELECT * FROM test_me

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:26:19AM GMT, Sami Imseih wrote: > > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > > Constants passed as parameters to a prepared statement will not be > > > handled as expected. I did not not test explicit PREPARE/EXECUTE > > > statement, > > > but I

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Sami Imseih
> > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold or some other fancy modes), maybe you could > > > instead > > > differentiate the merged version by have 2 constants rather than this > > > "..." or > > > something like tha

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Fri, Feb 14, 2025 at 05:57:01PM GMT, Julien Rouhaud wrote: > On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > > On 2025-Feb-14, Julien Rouhaud wrote: > > > > > Since the merging is a yes/no option (I think there used to be some > > > discussions > > > about having a threshold

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
On Fri, Feb 14, 2025 at 10:36:48AM +0100, Álvaro Herrera wrote: > On 2025-Feb-14, Julien Rouhaud wrote: > > > Since the merging is a yes/no option (I think there used to be some > > discussions > > about having a threshold or some other fancy modes), maybe you could instead > > differentiate the

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Álvaro Herrera
On 2025-Feb-14, Julien Rouhaud wrote: > Since the merging is a yes/no option (I think there used to be some > discussions > about having a threshold or some other fancy modes), maybe you could instead > differentiate the merged version by have 2 constants rather than this "..." or > something lik

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Julien Rouhaud
Hi, On Fri, Feb 14, 2025 at 09:36:08AM +0100, Dmitry Dolgov wrote: > > On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > > This case with an array passed to aa function seems to cause a regression > > in pg_stat_statements query text. As you can see the text is incomplete. > > I've alre

Re: pg_stat_statements and "IN" conditions

2025-02-14 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 05:08:45PM GMT, Sami Imseih wrote: > Constants passed as parameters to a prepared statement will not be > handled as expected. I did not not test explicit PREPARE/EXECUTE statement, > but I assume it will have the same issue. This is the same question of supporting variou

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Sami Imseih
Hi, Thanks for the updated patch! I spent some time looking at v24 today, and I have some findings/comments. 1/ Constants passed as parameters to a prepared statement will not be handled as expected. I did not not test explicit PREPARE/EXECUTE statement, but I assume it will have the same issue.

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Thu, Feb 13, 2025 at 01:47:01PM GMT, Álvaro Herrera wrote: > Also, how wed are you to > "query_id_merge_values" as a name? It's not in any way obvious that > this is about values in arrays. How about query_id_squash_arrays? Or > are you thinking in having values in other types of structures

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Álvaro Herrera
On 2025-Feb-13, Dmitry Dolgov wrote: > Here is how it looks like (posting only the first patch, since we > concentrate on it). This version handles just a little more to cover > simpe cases like the implicit convertion above. The GUC is also moved > out from pgss and renamed to query_id_merge_valu

Re: pg_stat_statements and "IN" conditions

2025-02-13 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 08:48:03PM GMT, Dmitry Dolgov wrote: > > On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > > The nastiness level of this seems quite low, compared to what happens to > > this other example if we didn't handle these easy cases: > > > > create table t (a float)

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Wed, Feb 12, 2025 at 07:39:39PM GMT, Álvaro Herrera wrote: > The nastiness level of this seems quite low, compared to what happens to > this other example if we didn't handle these easy cases: > > create table t (a float); > select i from t where i in (1, 2); > select i from t where i in (1, '

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Sami Imseih
> The nastiness level of this seems quite low, compared to what happens to > this other example if we didn't handle these easy cases: > > create table t (a float); > select i from t where i in (1, 2); > select i from t where i in (1, '2'); > select i from t where i in ('1', 2); > select i from t wh

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Álvaro Herrera
On 2025-Feb-12, Dmitry Dolgov wrote: > I've been experimenting with this today, and while it's easy to > implement, Great. > there is one annoying thing for which I don't have a solution > yet. When generating a normalized version for such merged queries in > pgss we rely on expression location,

Re: pg_stat_statements and "IN" conditions

2025-02-12 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 08:00:27PM GMT, Dmitry Dolgov wrote: > > Hmm, what about doing something much simpler, such as testing whether > > there's just a CoerceViaIO/RelabelType around a Const or a one-parameter > > function call of an immutable boostrap-OID function that has a Const as > > argum

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 07:51:43PM GMT, Álvaro Herrera wrote: > On 2025-Feb-11, Dmitry Dolgov wrote: > > > > On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > > > I have only looked at 0001, but I am wondering why > > > query_id_const_merge is a pg_stat_statements GUC > > > rather than

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Sami Imseih wrote: > I have only looked at 0001, but I am wondering why > query_id_const_merge is a pg_stat_statements GUC > rather than a core GUC? I was wondering the same thing and found the explanation here: https://postgr.es/m/ztmuctymis3n3...@paquier.xyz > Other extensions

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Dmitry Dolgov wrote: > > On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > > I have only looked at 0001, but I am wondering why > > query_id_const_merge is a pg_stat_statements GUC > > rather than a core GUC? > > It was moved from being a core GUC into a pg_stat_stateme

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Dmitry Dolgov
> On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > I have only looked at 0001, but I am wondering why > query_id_const_merge is a pg_stat_statements GUC > rather than a core GUC? It was moved from being a core GUC into a pg_stat_statements GUC on the request from the reviewers. Communi

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Sami Imseih wrote: > I do not have an explanation from the patch yet, but I have a test > that appears to show unexpected results. I only tested a few datatypes, > but from what I observe, some merge as expected and others do not; > i.e. int columns merge correctly but bigint do no

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Sami Imseih
I do not have an explanation from the patch yet, but I have a test that appears to show unexpected results. I only tested a few datatypes, but from what I observe, some merge as expected and others do not; i.e. int columns merge correctly but bigint do not. """ show pg_stat_statements.query_id_con

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Sami Imseih
I have only looked at 0001, but I am wondering why query_id_const_merge is a pg_stat_statements GUC rather than a core GUC? The dependency of pg_stat_statements to take advantage of this useful feature does not seem right. For example if the user does not have pg_stat_statements enabled, but are

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
Hello I noticed something that surprised me at first, but on further looking it should have been obvious: setting pg_stat_statements.query_id_const_merge affects the query ID for all readers of it, not just pg_stat_statement. This is good because it preserves the property that pg_stat_activity ent

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-22 Thread m . litsarev
What does this patch give on aglobal scale? Does it save much memory or increase performance? How many times? This patch makes the code semantically more correct and we don't lose anything. It is obviously not about performance or memory optimisation. This will only reduce the size of the $P

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread Sami Imseih
This will only reduce the size of the $PGDATA/pg_stat/pg_stat_statements.txt file. Even with 100k entries, the most I have seen pg_stat_statements.max set to, that will be less than 1 MB of disk saving. The default config of 5k entries will be much less. Regards, Sami

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread Ivan Kush
What does this patch give on aglobal scale? Does it save much memory or increase performance? How many times? On 1/21/25 13:51, m.litsa...@postgrespro.ru wrote: // Mutex should be last field, as this field isn't copied to dump file Updated. 2) You didn't take into account the upgrade. Saved i

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-21 Thread m . litsarev
// Mutex should be last field, as this field isn't copied to dump file Updated. 2) You didn't take into account the upgrade. Saved in Postgres with this byte and try to load in version without this byte. The PGSS_DUMP_FILE format is defined by PGSS_FILE_HEADER magic number (the first four byte

Re: pg_stat_statements: improve loading and saving routines for the dump file

2025-01-20 Thread Ivan Kush
Hello, Mikhail. 1) I'd add to comment a reason, why mutex should be last. // Mutex should be last field, as this field isn't copied to dump file + /* protects the counters only. Should be the very last field, as this field isn't copied to dump file + slock_t mutex; } pgssE

Re: pg_stat_statements and "IN" conditions

2024-12-03 Thread Dmitry Dolgov
> On Thu, Nov 28, 2024 at 08:36:47PM GMT, Kirill Reshke wrote: > > Hi! Can you please send a rebased version of this? Sure, here it is. >From 2de1af6489d46449b2884a9194515cd1090d5e8c Mon Sep 17 00:00:00 2001 From: Dmitrii Dolgov <9erthali...@gmail.com> Date: Tue, 3 Dec 2024 14:55:45 +0100 Subject:

Re: pg_stat_statements and "IN" conditions

2024-11-28 Thread Kirill Reshke
On Wed, 14 Aug 2024 at 01:06, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Sun, Aug 11, 2024 at 09:34:55PM GMT, Dmitry Dolgov wrote: > > > On Sun, Aug 11, 2024 at 07:54:05PM +0300, Sergei Kornilov wrote: > > > > > > This feature will improve my monitoring. Even in patch 0001. I think > >

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-10 Thread Michael Paquier
On Fri, Nov 08, 2024 at 05:49:45PM +0800, wenhui qiu wrote: > Agree , It reduces the lock time , The new comment are short and concise, > It sounds good . Thanks for the double-check. Applied on HEAD. -- Michael signature.asc Description: PGP signature

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-08 Thread wenhui qiu
Hi Karina Liskevich > + /* > + * There is no need to hold entry->mutex when reading stats_since and > + * minmax_stats_since for (unlike counters) they are always written > + * while holding pgss->lock exclusively. We are holding pgss->lock > +

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-07 Thread Michael Paquier
On Thu, Nov 07, 2024 at 04:08:30PM +0300, Karina Litskevich wrote: > Thank you for your feedback and the shorter wording of the comment. > I used it in the new version of the patch. After a second look, sounds good to me. Let's wait a bit and see of others have comments or thoughts to share. -- M

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-07 Thread Karina Litskevich
On Thu, Nov 7, 2024 at 11:17 AM Michael Paquier wrote: > The comment could be simpler, say a "The spinlock is not required when > reading these two as they are always updated when holding pgss->lock > exclusively.". Or something like that. Thank you for your feedback and the shorter wording of t

Re: pg_stat_statements: Avoid holding excessive lock

2024-11-07 Thread Michael Paquier
On Tue, Nov 05, 2024 at 08:37:08PM +0300, Karina Litskevich wrote: > I suggest eliminating holding the excessive lock. See the attached patch. > This would also restore the consistency between the code and the comments > about entry's mutex spinlock usage. You are right. minmax_stats_since and st

Re: pg_stat_statements and "IN" conditions

2024-08-13 Thread Dmitry Dolgov
> On Sun, Aug 11, 2024 at 09:34:55PM GMT, Dmitry Dolgov wrote: > > On Sun, Aug 11, 2024 at 07:54:05PM +0300, Sergei Kornilov wrote: > > > > This feature will improve my monitoring. Even in patch 0001. I think there > > are many other people in the thread who think this is useful. So maybe we > >

Re: pg_stat_statements and "IN" conditions

2024-08-11 Thread Dmitry Dolgov
> On Sun, Aug 11, 2024 at 07:54:05PM +0300, Sergei Kornilov wrote: > > This feature will improve my monitoring. Even in patch 0001. I think there > are many other people in the thread who think this is useful. So maybe we > should move it forward? Any complaints about the overall design? I see in

Re: pg_stat_statements and "IN" conditions

2024-08-11 Thread Sergei Kornilov
Hello This feature will improve my monitoring. Even in patch 0001. I think there are many other people in the thread who think this is useful. So maybe we should move it forward? Any complaints about the overall design? I see in the discussion it was mentioned that it would be good to measure p

Re: pg_stat_statements and "IN" conditions

2024-05-12 Thread Dmitry Dolgov
> On Tue, Apr 23, 2024 at 10:18:15AM +0200, Dmitry Dolgov wrote: > > On Mon, Apr 15, 2024 at 06:09:29PM +0900, Sutou Kouhei wrote: > > > > Thanks. I'm not familiar with this code base but I've > > reviewed these patches because I'm interested in this > > feature too. > > Thanks for the review! The

Re: pg_stat_statements and "IN" conditions

2024-04-23 Thread Dmitry Dolgov
> On Mon, Apr 15, 2024 at 06:09:29PM +0900, Sutou Kouhei wrote: > > Thanks. I'm not familiar with this code base but I've > reviewed these patches because I'm interested in this > feature too. Thanks for the review! The commentaries for the first patch make sense to me, will apply. > 0003: > > >

Re: pg_stat_statements and "IN" conditions

2024-04-15 Thread Sutou Kouhei
Hi, In <20240404143514.a26f7ttxrbdfc73a@erthalion.local> "Re: pg_stat_statements and "IN" conditions" on Thu, 4 Apr 2024 16:35:14 +0200, Dmitry Dolgov <9erthali...@gmail.com> wrote: > Here is the rebased version. Thanks. I'm not familiar with this c

Re: pg_stat_statements and "IN" conditions

2024-04-04 Thread Dmitry Dolgov
> On Wed, Mar 27, 2024 at 08:56:12AM +0900, Yasuo Honda wrote: > Thanks for the useful info. > > Ruby on Rails uses bigint as a default data type for the primary key > and prepared statements have been enabled by default for PostgreSQL. > I'm looking forward to these current patches being merged as

Re: pg_stat_statements and "IN" conditions

2024-03-26 Thread Yasuo Honda
Thanks for the useful info. Ruby on Rails uses bigint as a default data type for the primary key and prepared statements have been enabled by default for PostgreSQL. I'm looking forward to these current patches being merged as a first step and future versions of pg_stat_statements will support nor

Re: pg_stat_statements and "IN" conditions

2024-03-26 Thread Dmitry Dolgov
> On Tue, Mar 26, 2024 at 04:21:46PM +0900, Yasuo Honda wrote: > Yes. The script uses prepared statements because Ruby on Rails enables > prepared statements by default for PostgreSQL databases. > > Then I tested this branch > https://github.com/yahonda/postgres/tree/pg_stat_statements without > us

Re: pg_stat_statements and "IN" conditions

2024-03-26 Thread Yasuo Honda
Yes. The script uses prepared statements because Ruby on Rails enables prepared statements by default for PostgreSQL databases. Then I tested this branch https://github.com/yahonda/postgres/tree/pg_stat_statements without using prepared statements as follows and all of them do not normalize in cla

Re: pg_stat_statements and "IN" conditions

2024-03-25 Thread Dmitry Dolgov
> On Sun, Mar 24, 2024 at 11:36:38PM +0900, Yasuo Honda wrote: > Thanks for the information. I can apply these 4 patches from > 0eb23285a2 . I tested this branch from Ruby on Rails and it gets some > unexpected behavior from my point of view. > Setting pg_stat_statements.query_id_const_merge_thresh

Re: pg_stat_statements and "IN" conditions

2024-03-24 Thread Yasuo Honda
Thanks for the information. I can apply these 4 patches from 0eb23285a2 . I tested this branch from Ruby on Rails and it gets some unexpected behavior from my point of view. Setting pg_stat_statements.query_id_const_merge_threshold = 5 does not normalize sql queries whose number of in clauses excee

Re: pg_stat_statements and "IN" conditions

2024-03-23 Thread Dmitry Dolgov
> On Sat, Mar 23, 2024 at 04:13:44PM +0900, Yasuo Honda wrote: > Hi, I'm interested in this feature. It looks like these patches have > some conflicts. > > http://cfbot.cputube.org/patch_47_2837.log > > Would you rebase these patches? Sure, I can rebase, give me a moment. If you don't want to wait

Re: pg_stat_statements and "IN" conditions

2024-03-23 Thread Yasuo Honda
Hi, I'm interested in this feature. It looks like these patches have some conflicts. http://cfbot.cputube.org/patch_47_2837.log Would you rebase these patches? Thanks, -- Yasuo Honda On Sat, Mar 23, 2024 at 4:11 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Oh, I see, thanks. Give me a m

Re: pg_stat_statements and "IN" conditions

2024-01-22 Thread Dmitry Dolgov
> On Mon, Jan 22, 2024 at 06:07:27PM +0100, Dmitry Dolgov wrote: > > Please notice that at the moment, it's not being tested at all because > > of a patch-apply failure -- that's what the little triangular symbol > > means. The rest of the display concerns the test results from the > > last succes

Re: pg_stat_statements and "IN" conditions

2024-01-22 Thread Dmitry Dolgov
> On Mon, Jan 22, 2024 at 11:35:22AM -0500, Tom Lane wrote: > Dmitry Dolgov <9erthali...@gmail.com> writes: > >> On Mon, Jan 22, 2024 at 05:33:26PM +1100, Peter Smith wrote: > >> Hi, This patch has a CF status of "Needs Review" [1], but it seems > >> there was a CFbot test failure last time it was

Re: pg_stat_statements and "IN" conditions

2024-01-22 Thread Tom Lane
Dmitry Dolgov <9erthali...@gmail.com> writes: >> On Mon, Jan 22, 2024 at 05:33:26PM +1100, Peter Smith wrote: >> Hi, This patch has a CF status of "Needs Review" [1], but it seems >> there was a CFbot test failure last time it was run [2]. Please have a >> look and post an updated version if necess

Re: pg_stat_statements and "IN" conditions

2024-01-22 Thread Dmitry Dolgov
> On Mon, Jan 22, 2024 at 05:33:26PM +1100, Peter Smith wrote: > 2024-01 Commitfest. > > Hi, This patch has a CF status of "Needs Review" [1], but it seems > there was a CFbot test failure last time it was run [2]. Please have a > look and post an updated version if necessary. > > == > [1] http

Re: pg_stat_statements and "IN" conditions

2024-01-21 Thread Peter Smith
2024-01 Commitfest. Hi, This patch has a CF status of "Needs Review" [1], but it seems there was a CFbot test failure last time it was run [2]. Please have a look and post an updated version if necessary. == [1] https://commitfest.postgresql.org/46/2837/ [2] https://cirrus-ci.com/task/6688578

Re: pg_stat_statements and "IN" conditions

2024-01-13 Thread Dmitry Dolgov
> On Mon, Jan 08, 2024 at 05:10:20PM +0100, Dmitry Dolgov wrote: > > On Sat, Jan 06, 2024 at 09:04:54PM +0530, vignesh C wrote: > > > > CFBot shows documentation build has failed at [1] with: > > [07:44:55.531] time make -s -j${BUILD_JOBS} -C doc > > [07:44:57.987] postgres.sgml:572: element xref:

Re: pg_stat_statements and "IN" conditions

2024-01-08 Thread Dmitry Dolgov
> On Sat, Jan 06, 2024 at 09:04:54PM +0530, vignesh C wrote: > > CFBot shows documentation build has failed at [1] with: > [07:44:55.531] time make -s -j${BUILD_JOBS} -C doc > [07:44:57.987] postgres.sgml:572: element xref: validity error : IDREF > attribute linkend references an unknown ID > "guc-

Re: pg_stat_statements and "IN" conditions

2024-01-06 Thread vignesh C
On Tue, 31 Oct 2023 at 14:36, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Fri, Oct 27, 2023 at 05:02:44PM +0200, Dmitry Dolgov wrote: > > > On Thu, Oct 26, 2023 at 09:08:42AM +0900, Michael Paquier wrote: > > > typedef struct ArrayExpr > > > { > > > + pg_node_attr(custom_query_jumble)

Re: pg_stat_statements: more test coverage

2023-12-31 Thread Tom Lane
Peter Eisentraut writes: > It looks like the failing configurations are exactly all the big-endian > ones: s390x, sparc, powerpc. So it's possible that this is actually a > bug? But unless someone can reproduce this locally and debug it, we > should probably revert this for now. The reason f

Re: pg_stat_statements: more test coverage

2023-12-31 Thread Tom Lane
Peter Eisentraut writes: > It looks like the failing configurations are exactly all the big-endian > ones: s390x, sparc, powerpc. So it's possible that this is actually a > bug? But unless someone can reproduce this locally and debug it, we > should probably revert this for now. I see it fai

Re: pg_stat_statements: more test coverage

2023-12-31 Thread Peter Eisentraut
On 31.12.23 10:26, Julien Rouhaud wrote: On Sun, Dec 31, 2023 at 2:28 PM Michael Paquier wrote: On Sat, Dec 30, 2023 at 08:39:47PM +0100, Peter Eisentraut wrote: Ok, I have committed these two patches. Please note that the buildfarm has turned red, as in: https://buildfarm.postgresql.org/cg

Re: pg_stat_statements: more test coverage

2023-12-31 Thread Julien Rouhaud
On Sun, Dec 31, 2023 at 2:28 PM Michael Paquier wrote: > > On Sat, Dec 30, 2023 at 08:39:47PM +0100, Peter Eisentraut wrote: > > Ok, I have committed these two patches. > > Please note that the buildfarm has turned red, as in: > https://buildfarm.postgresql.org/cgi-bin/show_stagxe_log.pl?nm=pipit&

Re: pg_stat_statements: more test coverage

2023-12-30 Thread Michael Paquier
On Sat, Dec 30, 2023 at 08:39:47PM +0100, Peter Eisentraut wrote: > Ok, I have committed these two patches. Please note that the buildfarm has turned red, as in: https://buildfarm.postgresql.org/cgi-bin/show_stagxe_log.pl?nm=pipit&dt=2023-12-31%2001%3A12%3A22&stg=misc-check pg_stat_statements's r

Re: pg_stat_statements: more test coverage

2023-12-30 Thread Michael Paquier
On Sat, Dec 30, 2023 at 08:39:47PM +0100, Peter Eisentraut wrote: > On 29.12.23 06:14, Julien Rouhaud wrote: >> I agree with Michael on this one, the only times I saw this pattern >> was to comply with some company internal policy for minimal coverage >> numbers. > > Ok, skipped that. Just to clo

Re: pg_stat_statements: more test coverage

2023-12-30 Thread Peter Eisentraut
On 29.12.23 06:14, Julien Rouhaud wrote: It looks good to me. One minor complaint, I'm a bit dubious about those queries: SELECT count(*) <= 100 AND count(*) > 0 FROM pg_stat_statements; Is it to actually test that pg_stat_statements won't store more than pg_stat_statements.max records? Note

Re: pg_stat_statements: more test coverage

2023-12-28 Thread Julien Rouhaud
On Wed, Dec 27, 2023 at 8:53 PM Peter Eisentraut wrote: > > On 27.12.23 09:08, Julien Rouhaud wrote: > > > > I was a bit surprised by that so I checked locally. It does work as > > expected provided that you set pg_stat_statements.track to all: > > Ok, here is an updated patch set that does it th

Re: pg_stat_statements: more test coverage

2023-12-27 Thread Peter Eisentraut
On 27.12.23 09:08, Julien Rouhaud wrote: Hi, On Tue, Dec 26, 2023 at 10:03 PM Peter Eisentraut wrote: On 24.12.23 03:03, Michael Paquier wrote: - Use a DO block of a PL function, say with something like that to ensure an amount of N queries? Say with something like that after tweaking pg_st

Re: pg_stat_statements: more test coverage

2023-12-27 Thread Julien Rouhaud
Hi, On Tue, Dec 26, 2023 at 10:03 PM Peter Eisentraut wrote: > > On 24.12.23 03:03, Michael Paquier wrote: > > - Use a DO block of a PL function, say with something like that to > > ensure an amount of N queries? Say with something like that after > > tweaking pg_stat_statements.track: > > CREAT

  1   2   3   >