Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Robert Haas
On Wed, Mar 17, 2021 at 12:48 PM Julien Rouhaud wrote: > I originally suggested to make it clearer by having an enum GUC rather than a > boolean, say compute_queryid = [ none | core | external ], and if set to > external then a hook would be explicitely called. Right now, "none" and > "external"

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Julien Rouhaud
On Wed, Mar 17, 2021 at 12:24:44PM -0400, Bruce Momjian wrote: > On Wed, Mar 17, 2021 at 05:16:50PM +0100, Pavel Stehule wrote: > > > > > > st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > > > > Bruce Momjian writes: > > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote:

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 05:16:50PM +0100, Pavel Stehule wrote: > > > st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Pavel Stehule
st 17. 3. 2021 v 17:03 odesílatel Tom Lane napsal: > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID > calculation > >> method that was designed only for pg_stat_statement's needs as the one

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 12:01:38PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > >> I still say that it's a serious mistake to sanctify a query ID calculation > >> method that was designed only for pg_stat_statement's needs as the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Tom Lane
Bruce Momjian writes: > On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: >> I still say that it's a serious mistake to sanctify a query ID calculation >> method that was designed only for pg_stat_statement's needs as the one >> true way to do it. But that's what exposing it in a core

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Wed, Mar 17, 2021 at 11:28:38AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > We are reaching the two-year mark on this feature, that everyone seems > > to agree is needed. Is any committer going to work on this to get it > > into PG 14? Should I take it? > > I still say that it's a

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Tom Lane
Bruce Momjian writes: > We are reaching the two-year mark on this feature, that everyone seems > to agree is needed. Is any committer going to work on this to get it > into PG 14? Should I take it? I still say that it's a serious mistake to sanctify a query ID calculation method that was

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-17 Thread Bruce Momjian
On Sun, Mar 14, 2021 at 04:06:45PM +0800, Julien Rouhaud wrote: > Recent conflict, thanks to cfbot. v18 attached. We are reaching the two-year mark on this feature, that everyone seems to agree is needed. Is any committer going to work on this to get it into PG 14? Should I take it? I just

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-14 Thread Julien Rouhaud
Recent conflict, thanks to cfbot. v18 attached. >From fa94eba58ee0ca098cfde0d17de72dc230ee471c Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Wed, 14 Oct 2020 02:11:37 +0800 Subject: [PATCH v18 1/3] Move pg_stat_statements query jumbling to core. A new compute_queryid GUC is also added, to

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-01 Thread Julien Rouhaud
On Wed, Jan 20, 2021 at 12:43 AM Julien Rouhaud wrote: > > On Fri, Jan 8, 2021 at 1:07 AM Julien Rouhaud wrote: > > > > v15 that fixes recent conflicts. > > Rebase only, thanks to the cfbot! V16 attached. Recent commit exposed that the explain_filter() doesn't filter negative sign. This can

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Julien Rouhaud
Hello Yamada-san, On Wed, Jan 20, 2021 at 10:06 AM Tatsuro Yamada wrote: > > Hi Julien, > > > >> Rebase only, thanks to the cfbot! V16 attached. > > > > I tested the v16 patch on a0efda88a by using "make installcheck-parallel", > > and > > my result is the following. Attached file is

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Tatsuro Yamada
Hi Julien, Rebase only, thanks to the cfbot!  V16 attached. I tested the v16 patch on a0efda88a by using "make installcheck-parallel", and my result is the following. Attached file is regression.diffs. Sorry, my environment was not suitable for the test when I sent my previous email. I

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Tatsuro Yamada
Hi Julien, Rebase only, thanks to the cfbot! V16 attached. I tested the v16 patch on a0efda88a by using "make installcheck-parallel", and my result is the following. Attached file is regression.diffs. 1 of 202 tests failed. The differences

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-19 Thread Julien Rouhaud
On Fri, Jan 8, 2021 at 1:07 AM Julien Rouhaud wrote: > > v15 that fixes recent conflicts. Rebase only, thanks to the cfbot! V16 attached. From a0388c53d9755cfd706513f7f02a08b31a48aacb Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 18 Mar 2019 18:55:50 +0100 Subject: [PATCH v16 2/3]

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-01-07 Thread Julien Rouhaud
On Sun, Oct 18, 2020 at 4:12 PM Julien Rouhaud wrote: > > On Sun, Oct 18, 2020 at 12:20 PM Tom Lane wrote: > > > > Alvaro Herrera writes: > > > Wait ... what? I've been thinking that this GUC is just to enable or > > > disable the computation of query ID, not to change the algorithm to do > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-18 Thread Julien Rouhaud
On Sun, Oct 18, 2020 at 12:20 PM Tom Lane wrote: > > Alvaro Herrera writes: > > Wait ... what? I've been thinking that this GUC is just to enable or > > disable the computation of query ID, not to change the algorithm to do > > so. Do we really need to allow different algorithms in different >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-17 Thread Tom Lane
Alvaro Herrera writes: > Wait ... what? I've been thinking that this GUC is just to enable or > disable the computation of query ID, not to change the algorithm to do > so. Do we really need to allow different algorithms in different > sessions? We established that some time ago, no?

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-17 Thread Alvaro Herrera
On 2020-Oct-17, Tom Lane wrote: > Fair point, but if we allow several different values to be set in > different sessions, what ends up happening in pg_stat_statements? > > On the other hand, maybe that's just a matter for documentation. > "If the 'same' query is processed with two different

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-17 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Oct-17, Julien Rouhaud wrote: >> On Sat, Oct 17, 2020 at 12:23 AM Tom Lane wrote: >>> then there's a potential security issue if the GUC is USERSET level: >>> a user could hide her queries from pg_stat_statement by turning the >>> GUC off. So this line of

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-17 Thread Alvaro Herrera
On 2020-Oct-17, Julien Rouhaud wrote: > On Sat, Oct 17, 2020 at 12:23 AM Tom Lane wrote: > > then there's a potential security issue if the GUC is USERSET level: > > a user could hide her queries from pg_stat_statement by turning the > > GUC off. So this line of thought suggests the GUC needs

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Julien Rouhaud
On Fri, Oct 16, 2020 at 11:04 PM Bruce Momjian wrote: > > On Thu, Oct 15, 2020 at 11:41:23AM +0800, Julien Rouhaud wrote: > > On Wed, Oct 14, 2020 at 10:40 PM Bruce Momjian wrote: > > > There is that, and log_line_prefix, which I can imaging being useful. > > > My point is that if the queryid is

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Julien Rouhaud
On Sat, Oct 17, 2020 at 12:23 AM Tom Lane wrote: > > Alvaro Herrera writes: > > In this case, I suppose using pg_stat_statement would require to have it > > enabled, and it'd just not collect anything if disabled. Yes, my idea was to be able to have pg_stat_statements enabled even if no queryid

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Bruce Momjian
On Fri, Oct 16, 2020 at 01:03:55PM -0300, Álvaro Herrera wrote: > On 2020-Oct-16, Bruce Momjian wrote: > > > On Thu, Oct 15, 2020 at 11:41:23AM +0800, Julien Rouhaud wrote: > > > > I did some naive benchmarking. Using a custom pgbench script with this > > > query: > > > > I can see around 2%

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Tom Lane
Alvaro Herrera writes: > In this case, I suppose using pg_stat_statement would require to have it > enabled, and it'd just not collect anything if disabled. Alternatively, pg_stat_statement might be able to force it on (applying a non-overridable PGC_INTERNAL-level setting) on load? Not sure if

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Alvaro Herrera
On 2020-Oct-16, Bruce Momjian wrote: > On Thu, Oct 15, 2020 at 11:41:23AM +0800, Julien Rouhaud wrote: > > I did some naive benchmarking. Using a custom pgbench script with this > > query: > > I can see around 2% overhead (this query is reported with ~ 3ms > > latency average). Adding a few

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-16 Thread Bruce Momjian
On Thu, Oct 15, 2020 at 11:41:23AM +0800, Julien Rouhaud wrote: > On Wed, Oct 14, 2020 at 10:40 PM Bruce Momjian wrote: > > There is that, and log_line_prefix, which I can imaging being useful. > > My point is that if the queryid is visible, there should be a reason it > > defaults to show empty.

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Julien Rouhaud
On Wed, Oct 14, 2020 at 10:40 PM Bruce Momjian wrote: > > On Wed, Oct 14, 2020 at 10:34:31PM +0800, Julien Rouhaud wrote: > > On Wed, Oct 14, 2020 at 10:31 PM Tom Lane wrote: > > > > > > Bruce Momjian writes: > > > > Is there a measureable overhead when this is turned on, since it is off > > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Bruce Momjian
On Wed, Oct 14, 2020 at 10:34:31PM +0800, Julien Rouhaud wrote: > On Wed, Oct 14, 2020 at 10:31 PM Tom Lane wrote: > > > > Bruce Momjian writes: > > > Is there a measureable overhead when this is turned on, since it is off > > > by default and maybe should default to on. > > > > I don't believe

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Julien Rouhaud
On Wed, Oct 14, 2020 at 10:31 PM Tom Lane wrote: > > Bruce Momjian writes: > > Is there a measureable overhead when this is turned on, since it is off > > by default and maybe should default to on. > > I don't believe that "default to on" can even be in the discussion. > There is no in-core

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Tom Lane
Bruce Momjian writes: > Is there a measureable overhead when this is turned on, since it is off > by default and maybe should default to on. I don't believe that "default to on" can even be in the discussion. There is no in-core feature that would use this by default.

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Bruce Momjian
On Wed, Oct 14, 2020 at 10:21:24PM +0800, Julien Rouhaud wrote: > On Wed, Oct 14, 2020 at 10:09 PM Bruce Momjian wrote: > > > > OK, I came up with the hash idea only to address one of your concerns > > > > about mismatched hashes for algorithm improvements/changes. Seems we > > > > might as well

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Julien Rouhaud
On Wed, Oct 14, 2020 at 10:09 PM Bruce Momjian wrote: > > On Wed, Oct 14, 2020 at 05:43:33PM +0800, Julien Rouhaud wrote: > > On Tue, Oct 13, 2020 at 4:53 AM Bruce Momjian wrote: > > > > > > On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > > > > Bruce Momjian writes: > > > > > On

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Bruce Momjian
On Wed, Oct 14, 2020 at 05:43:33PM +0800, Julien Rouhaud wrote: > On Tue, Oct 13, 2020 at 4:53 AM Bruce Momjian wrote: > > > > On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > > > Bruce Momjian writes: > > > > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > > > >> Yeah, I

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Julien Rouhaud
On Wed, Oct 14, 2020 at 5:43 PM Julien Rouhaud wrote: > > On Tue, Oct 13, 2020 at 4:53 AM Bruce Momjian wrote: > > > > On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > > > Bruce Momjian writes: > > > > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > > > >> Yeah, I agree

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-14 Thread Julien Rouhaud
On Tue, Oct 13, 2020 at 4:53 AM Bruce Momjian wrote: > > On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > > >> Yeah, I agree --- a version number is the wrong way to think about this. > > > > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > >> Yeah, I agree --- a version number is the wrong way to think about this. > > > The version number was to invalidate _all_ query hashes if the > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Tom Lane
Bruce Momjian writes: > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: >> Yeah, I agree --- a version number is the wrong way to think about this. > The version number was to invalidate _all_ query hashes if the > algorithm is slightly modified, rather than invalidating just some of >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > Robert Haas writes: > > I don't really understand how a version number helps. It's not like > > there is going to be a v2 that is in all ways better than v1. If there > > are different algorithms here, they are going to be customized for

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Tom Lane
Robert Haas writes: > I don't really understand how a version number helps. It's not like > there is going to be a v2 that is in all ways better than v1. If there > are different algorithms here, they are going to be customized for > different needs. Yeah, I agree --- a version number is the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Robert Haas
On Mon, Oct 12, 2020 at 10:14 AM Bruce Momjian wrote: > On Mon, Oct 12, 2020 at 04:20:05PM +0800, Julien Rouhaud wrote: > > But there are many people that aren't happy with the current hashing > > approach. If we're going to move the computation in core, shouldn't > > we listen to their

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 04:20:05PM +0800, Julien Rouhaud wrote: > But there are many people that aren't happy with the current hashing > approach. If we're going to move the computation in core, shouldn't > we listen to their complaints and let them pay some probably quite > high overhead to base

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Julien Rouhaud
On Wed, Oct 7, 2020 at 9:53 AM Bruce Momjian wrote: > > On Wed, Oct 7, 2020 at 10:42:49AM +0900, Michael Paquier wrote: > > On Tue, Oct 06, 2020 at 09:22:29AM -0400, Bruce Momjian wrote: > > > I propose moving the pg_stat_statements queryid hash code into the > > > server (with a version

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-06 Thread Bruce Momjian
On Wed, Oct 7, 2020 at 10:42:49AM +0900, Michael Paquier wrote: > On Tue, Oct 06, 2020 at 09:22:29AM -0400, Bruce Momjian wrote: > > I propose moving the pg_stat_statements queryid hash code into the > > server (with a version number), and also adding a postgresql.conf > > variable that lets you

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-06 Thread Michael Paquier
On Tue, Oct 06, 2020 at 09:22:29AM -0400, Bruce Momjian wrote: > I propose moving the pg_stat_statements queryid hash code into the > server (with a version number), and also adding a postgresql.conf > variable that lets you control how detailed the queryid hash is > computed. This addresses the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-06 Thread Bruce Momjian
On Tue, Oct 6, 2020 at 02:34:58PM +0900, Michael Paquier wrote: > On Mon, Oct 05, 2020 at 11:23:50PM -0400, Bruce Momjian wrote: > > On Tue, Oct 6, 2020 at 11:11:27AM +0800, Julien Rouhaud wrote: > >> Maybe we could add a new hook for only queryid computation, and add a > >> GUC to let people

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Michael Paquier
On Mon, Oct 05, 2020 at 11:23:50PM -0400, Bruce Momjian wrote: > On Tue, Oct 6, 2020 at 11:11:27AM +0800, Julien Rouhaud wrote: >> Maybe we could add a new hook for only queryid computation, and add a >> GUC to let people choose between no queryid computed, core computation >> (current

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Michael Paquier
On Mon, Oct 05, 2020 at 05:24:06PM -0400, Bruce Momjian wrote: > (Also, did we decide _not_ to make the pg_stat_statements queryid > always a positive value?) This specific point has been discussed a couple of years ago, please see cff440d and its related thread:

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Bruce Momjian
On Tue, Oct 6, 2020 at 11:11:27AM +0800, Julien Rouhaud wrote: > > I do think the queryid has to display independent of pg_stat_statements, > > because I can see people using queryid for log file and pg_stat_activity > > comparisons. I also think the ability to have queryid accessible is an > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Julien Rouhaud
On Tue, Oct 6, 2020 at 10:18 AM Bruce Momjian wrote: > > On Mon, Oct 5, 2020 at 07:58:42PM -0300, Álvaro Herrera wrote: > > On 2020-Oct-05, Tom Lane wrote: > > > > > FWIW, I think this proposal is a mess. I was willing to hold my nose > > > and have a queryId field in the internal Query struct

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Bruce Momjian
On Mon, Oct 5, 2020 at 07:58:42PM -0300, Álvaro Herrera wrote: > On 2020-Oct-05, Tom Lane wrote: > > > FWIW, I think this proposal is a mess. I was willing to hold my nose > > and have a queryId field in the internal Query struct without any solid > > consensus about what its semantics are and

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Alvaro Herrera
On 2020-Oct-05, Tom Lane wrote: > FWIW, I think this proposal is a mess. I was willing to hold my nose > and have a queryId field in the internal Query struct without any solid > consensus about what its semantics are and which extensions get to use it. > Exposing it to end users seems like a

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Tom Lane
Bruce Momjian writes: > I would like to apply this patch (I know it has been in the commitfest > since July 2019), but I have some questions about the user API. Does it > make sense to have a column in pg_stat_actvity and an option in > log_line_prefix that will be empty unless

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-05 Thread Bruce Momjian
On Wed, Aug 19, 2020 at 04:19:30PM +0200, Julien Rouhaud wrote: > Similarly to other fields in pg_stat_activity, only the queryid from the top > level statements are exposed, and if the backends status isn't active then the > queryid from the last executed statements is displayed. > > Also add a

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-08-19 Thread Julien Rouhaud
On Tue, Jul 28, 2020 at 10:55:04AM +0200, Julien Rouhaud wrote: > On Tue, Jul 28, 2020 at 10:07 AM torikoshia > wrote: > > > > Thanks for updating! > > I tested the patch setting log_statement = 'all', but %Q in > > log_line_prefix > > was always 0 even when pg_stat_statements.queryid and > >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-07-28 Thread Julien Rouhaud
On Tue, Jul 28, 2020 at 10:07 AM torikoshia wrote: > > Thanks for updating! > I tested the patch setting log_statement = 'all', but %Q in > log_line_prefix > was always 0 even when pg_stat_statements.queryid and > pg_stat_activity.queryid are not 0. > > Is this an intentional behavior? > >[...]

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-07-28 Thread torikoshia
On 2020-07-14 20:24, Julien Rouhaud wrote: On Tue, Jul 14, 2020 at 07:11:02PM +0900, Atsushi Torikoshi wrote: Hi, v9 patch fails to apply to HEAD, could you check and rebase it? Thanks for the notice, v10 attached! And here are minor typos. 79 +* utility statements. Note that we

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-07-14 Thread Julien Rouhaud
On Tue, Jul 14, 2020 at 07:11:02PM +0900, Atsushi Torikoshi wrote: > Hi, > > v9 patch fails to apply to HEAD, could you check and rebase it? Thanks for the notice, v10 attached! > And here are minor typos. > > 79 +* utility statements. Note that we don't compute a queryId > for

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-07-14 Thread Atsushi Torikoshi
Hi, v9 patch fails to apply to HEAD, could you check and rebase it? And here are minor typos. 79 +* utility statements. Note that we don't compute a queryId for prepared 80 +* statemets related utility, as those will inherit from the underlying 81 +* statements's one

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-04-08 Thread Julien Rouhaud
On Tue, Apr 7, 2020 at 8:40 AM Tatsuro Yamada wrote: > > Hi Julien, > > On 2020/04/02 22:25, Julien Rouhaud wrote: > > New conflict, rebased v9 attached. > > I tested the patch on the head (c7654f6a3) and > the result was fine. See below: > > $ make installcheck-world > = >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-04-07 Thread Tatsuro Yamada
Hi Julien, On 2020/04/02 22:25, Julien Rouhaud wrote: New conflict, rebased v9 attached. I tested the patch on the head (c7654f6a3) and the result was fine. See below: $ make installcheck-world = All 1 tests passed. = Regards, Tatsuro Yamada

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-04-02 Thread Julien Rouhaud
New conflict, rebased v9 attached. >From 26b98194d8add282158c65f6ac46c721ba80f498 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 18 Mar 2019 18:55:50 +0100 Subject: [PATCH v9 1/2] Expose queryid in pg_stat_activity and log_line_prefix Similarly to other fields in pg_stat_activity, only

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-03-16 Thread Julien Rouhaud
On Sat, Mar 14, 2020 at 06:53:51PM +0100, Julien Rouhaud wrote: > On Tue, Mar 03, 2020 at 04:24:59PM +0100, Julien Rouhaud wrote: > > > > cfbot reports a failure since 2f9661311b (command completion tag > > change), so here's a rebased v6, no change otherwise. > > > Conflict with 8e8a0becb3 (Unify

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-03-14 Thread Julien Rouhaud
On Tue, Mar 03, 2020 at 04:24:59PM +0100, Julien Rouhaud wrote: > > cfbot reports a failure since 2f9661311b (command completion tag > change), so here's a rebased v6, no change otherwise. Conflict with 8e8a0becb3 (Unify several ways to tracking backend type), thanks again to cfbot, rebased v7

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-03-08 Thread Julien Rouhaud
On Fri, Jun 28, 2019 at 11:49:53AM -0700, Peter Geoghegan wrote: > On Tue, Mar 19, 2019 at 12:38 PM legrand legrand > wrote: > > Would it make sense to add it in auto explain ? > > I don't know for explain itself, but maybe ... > > I think that it should appear in EXPLAIN. pg_stat_statements

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-03-03 Thread Julien Rouhaud
On Fri, Feb 7, 2020 at 11:12 AM Julien Rouhaud wrote: > > On Thu, Feb 06, 2020 at 02:59:09PM -0500, Robert Haas wrote: > > On Wed, Feb 5, 2020 at 9:32 AM Julien Rouhaud > > wrote: > > > There's also the possibility to reserve 1 bit of the hash to know if > > > this is a utility command or not,

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-02-07 Thread Julien Rouhaud
On Thu, Feb 06, 2020 at 02:59:09PM -0500, Robert Haas wrote: > On Wed, Feb 5, 2020 at 9:32 AM Julien Rouhaud wrote: > > There's also the possibility to reserve 1 bit of the hash to know if > > this is a utility command or not, although I don't recall right now > > all the possible issues with

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-02-06 Thread Robert Haas
On Wed, Feb 5, 2020 at 9:32 AM Julien Rouhaud wrote: > There's also the possibility to reserve 1 bit of the hash to know if > this is a utility command or not, although I don't recall right now > all the possible issues with utility commands and some special > handling of them. I'll work on it

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-02-05 Thread Julien Rouhaud
On Sat, Feb 1, 2020 at 12:30 PM Tomas Vondra wrote: > > On Fri, Nov 29, 2019 at 09:39:09AM +0100, Julien Rouhaud wrote: > >On Fri, Nov 29, 2019 at 7:21 AM Michael Paquier wrote: > >> > >> On Fri, Nov 29, 2019 at 03:19:49PM +0900, Michael Paquier wrote: > >> > On Wed, Nov 13, 2019 at 12:53:09PM

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-02-01 Thread Tomas Vondra
On Fri, Nov 29, 2019 at 09:39:09AM +0100, Julien Rouhaud wrote: On Fri, Nov 29, 2019 at 7:21 AM Michael Paquier wrote: On Fri, Nov 29, 2019 at 03:19:49PM +0900, Michael Paquier wrote: > On Wed, Nov 13, 2019 at 12:53:09PM +0100, Julien Rouhaud wrote: >> I'd really like to have the queryid

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-29 Thread Julien Rouhaud
On Fri, Nov 29, 2019 at 7:21 AM Michael Paquier wrote: > > On Fri, Nov 29, 2019 at 03:19:49PM +0900, Michael Paquier wrote: > > On Wed, Nov 13, 2019 at 12:53:09PM +0100, Julien Rouhaud wrote: > >> I'd really like to have the queryid function available through SQL, > >> but I think that this

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-28 Thread Michael Paquier
On Fri, Nov 29, 2019 at 03:19:49PM +0900, Michael Paquier wrote: > On Wed, Nov 13, 2019 at 12:53:09PM +0100, Julien Rouhaud wrote: >> I'd really like to have the queryid function available through SQL, >> but I think that this specific case wouldn't work very well for >> pg_stat_statements'

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-28 Thread Michael Paquier
On Wed, Nov 13, 2019 at 12:53:09PM +0100, Julien Rouhaud wrote: > I'd really like to have the queryid function available through SQL, > but I think that this specific case wouldn't work very well for > pg_stat_statements' approach as it's working with oid. The query > string in pg_stat_activity

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-13 Thread Julien Rouhaud
On Wed, Nov 13, 2019 at 4:15 AM Bruce Momjian wrote: > > On Mon, Nov 11, 2019 at 05:37:30PM +0900, Michael Paquier wrote: > > On Wed, Sep 11, 2019 at 06:30:22PM +0200, Julien Rouhaud wrote: > > > The thing is that pg_stat_statements assigns a 0 queryid in the > > > post_parse_analyze_hook to

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-12 Thread Bruce Momjian
On Mon, Nov 11, 2019 at 05:37:30PM +0900, Michael Paquier wrote: > On Wed, Sep 11, 2019 at 06:30:22PM +0200, Julien Rouhaud wrote: > > The thing is that pg_stat_statements assigns a 0 queryid in the > > post_parse_analyze_hook to recognize utility statements and avoid > > tracking instrumentation

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-11-11 Thread Michael Paquier
On Wed, Sep 11, 2019 at 06:30:22PM +0200, Julien Rouhaud wrote: > The thing is that pg_stat_statements assigns a 0 queryid in the > post_parse_analyze_hook to recognize utility statements and avoid > tracking instrumentation twice in case of utility statements, and then > compute a queryid base on

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-09-11 Thread Julien Rouhaud
Thanks for looking at it! On Wed, Sep 11, 2019 at 6:45 AM Michael Paquier wrote: > > An invalid query ID is assumed to be 0 in the patch, per the way it is > defined in pg_stat_statements. However this also maps with the case > where we have a utility statement. Oh indeed. Which means that if

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-09-10 Thread Michael Paquier
On Wed, Aug 07, 2019 at 09:03:21AM +, Evgeny Efimkin wrote: > The new status of this patch is: Ready for Committer I may be wrong of course, but it looks that this is wanted and the current shape of the patch looks sensible: - Register the query ID using a backend entry. - Only consider the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-07 Thread Evgeny Efimkin
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed HI! patch is look good for me. The new status of this patch

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-05 Thread legrand legrand
Kyotaro Horiguchi-4 wrote > At Sun, 4 Aug 2019 00:04:01 -0700 (MST), legrand legrand > legrand_legrand@ > wrote in < > 1564902241482-0.post@.nabble >> >> > However having the nested queryid in >> > pg_stat_activity would be convenient to track >> > what is a long stored functions currently

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-05 Thread Julien Rouhaud
On Mon, Aug 5, 2019 at 9:28 AM Kyotaro Horiguchi wrote: > > At Sun, 4 Aug 2019 00:04:01 -0700 (MST), legrand legrand > wrote in <1564902241482-0.p...@n3.nabble.com> > > > However having the nested queryid in > > > pg_stat_activity would be convenient to track > > > what is a long stored

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-05 Thread Kyotaro Horiguchi
At Sun, 4 Aug 2019 00:04:01 -0700 (MST), legrand legrand wrote in <1564902241482-0.p...@n3.nabble.com> > > However having the nested queryid in > > pg_stat_activity would be convenient to track > > what is a long stored functions currently doing. > > +1 > > And this could permit to get wait

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-04 Thread legrand legrand
> However having the nested queryid in > pg_stat_activity would be convenient to track > what is a long stored functions currently doing. +1 And this could permit to get wait event sampling per queryid when pg_stat_statements.track = all Regards PAscal -- Sent from:

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-03 Thread Julien Rouhaud
Hi, On Sat, Aug 3, 2019 at 1:21 AM Andres Freund wrote: > > On 2019-08-02 10:54:35 +0200, Julien Rouhaud wrote: > > However having the nested queryid in > > pg_stat_activity would be convenient to track what is a long stored > > functions currently doing. Maybe we could expose something like >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-02 Thread Andres Freund
Hi, On 2019-08-02 10:54:35 +0200, Julien Rouhaud wrote: > On Thu, Aug 1, 2019 at 11:05 PM Andres Freund wrote: > > > > I'm actually quite unconvinced that it's sensible to update the global > > value for nested queries. That'll mean e.g. the log_line_prefix and > > pg_stat_activity values are

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-02 Thread Julien Rouhaud
On Thu, Aug 1, 2019 at 11:05 PM Andres Freund wrote: > > I'm actually quite unconvinced that it's sensible to update the global > value for nested queries. That'll mean e.g. the log_line_prefix and > pg_stat_activity values are most of the time going to be bogus while > nested, because the

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Julien Rouhaud
On Thu, Aug 1, 2019 at 10:52 PM Andres Freund wrote: > > On 2019-08-01 22:42:23 +0200, Julien Rouhaud wrote: > > Sure, but it requires extra wrapper functions, and the st_changecount > > dance when writing the new value. > > So? You need a wrapper function anyway, there's no way we're going to >

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Andres Freund
Hi, On 2019-08-01 22:49:48 +0200, Julien Rouhaud wrote: > On Thu, Aug 1, 2019 at 8:36 PM Andres Freund wrote: > > > > On 2019-08-01 14:20:46 -0400, Robert Haas wrote: > > > However, I think that the fact that this patch adds 15 new calls to > > > pg_atomic_write_u64(>queryId, ...) is probably

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Andres Freund
Hi, On 2019-08-01 22:42:23 +0200, Julien Rouhaud wrote: > On Thu, Aug 1, 2019 at 8:36 PM Andres Freund wrote: > > > > On 2019-08-01 08:45:45 +0200, Julien Rouhaud wrote: > > > On Wed, Jul 31, 2019 at 11:59 PM Andres Freund wrote: > > > > And if it were necessary, why wouldn't any of the other

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Julien Rouhaud
On Thu, Aug 1, 2019 at 8:36 PM Andres Freund wrote: > > On 2019-08-01 14:20:46 -0400, Robert Haas wrote: > > However, I think that the fact that this patch adds 15 new calls to > > pg_atomic_write_u64(>queryId, ...) is probably not a good > > sign. It seems like we ought to be able to centralize

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Julien Rouhaud
On Thu, Aug 1, 2019 at 8:36 PM Andres Freund wrote: > > On 2019-08-01 08:45:45 +0200, Julien Rouhaud wrote: > > On Wed, Jul 31, 2019 at 11:59 PM Andres Freund wrote: > > > And if it were necessary, why wouldn't any of the other fields in > > > PgBackendStatus need it? There's plenty of other

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Andres Freund
Hi, On 2019-08-01 08:45:45 +0200, Julien Rouhaud wrote: > On Wed, Jul 31, 2019 at 11:59 PM Andres Freund wrote: > > And if it were necessary, why wouldn't any of the other fields in > > PgBackendStatus need it? There's plenty of other fields written to > > without a lock, and several of those

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Andres Freund
On 2019-08-01 14:20:46 -0400, Robert Haas wrote: > However, I think that the fact that this patch adds 15 new calls to > pg_atomic_write_u64(>queryId, ...) is probably not a good > sign. It seems like we ought to be able to centralize it better than > that. +1

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Robert Haas
On Thu, Aug 1, 2019 at 2:46 AM Julien Rouhaud wrote: > This patch is actually storing the queryid in PGPROC, not in > PgBackendStatus, thus the need for an atomic. I used PGPROC because > the value needs to be available in log_line_prefix() and spi.c, so > pgstat.c / PgBackendStatus didn't seem

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-08-01 Thread Julien Rouhaud
On Wed, Jul 31, 2019 at 11:59 PM Andres Freund wrote: > > On 2019-07-31 23:51:40 +0200, Julien Rouhaud wrote: > > On Wed, Jul 31, 2019 at 10:55 AM Evgeny Efimkin > > wrote: > > > What reason to use pg_atomic_uint64? > > > > The queryid is read and written without holding any lock on the PGPROC

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-07-31 Thread Andres Freund
Hi, On 2019-07-31 23:51:40 +0200, Julien Rouhaud wrote: > On Wed, Jul 31, 2019 at 10:55 AM Evgeny Efimkin > wrote: > > What reason to use pg_atomic_uint64? > > The queryid is read and written without holding any lock on the PGPROC > entry, so the pg_atomic_uint64 will guarantee that we get a

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-07-31 Thread Julien Rouhaud
Hello, On Wed, Jul 31, 2019 at 10:55 AM Evgeny Efimkin wrote: > > What reason to use pg_atomic_uint64? The queryid is read and written without holding any lock on the PGPROC entry, so the pg_atomic_uint64 will guarantee that we get a consistent value in pg_stat_get_activity(). Other reads

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-07-31 Thread Evgeny Efimkin
What reason to use pg_atomic_uint64? In docs: occured - > occurred

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-06-28 Thread Peter Geoghegan
On Tue, Mar 19, 2019 at 12:38 PM legrand legrand wrote: > Would it make sense to add it in auto explain ? > I don't know for explain itself, but maybe ... I think that it should appear in EXPLAIN. pg_stat_statements already cannot have a query hash of zero, so it might be okay to display it only

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-06-28 Thread Peter Geoghegan
On Tue, Mar 19, 2019 at 12:00 PM Robert Haas wrote: > On the other hand, it also appears that a lot of people would be very, > very happy to just be able to see the query ID field that already > exists, both in pg_stat_statements in pg_stat_activity, and we > shouldn't throw up unnecessary

<    1   2   3   >