Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread legrand legrand
my understanding is * pg_stat_statements.track = 'none' or 'top' (default) or 'all' to make queryId optionally computed * a new GUC: pg_stat_statements.queryid_based = 'oids' (default) or 'names' or 'fullnames' to choose the queryid computation algorithm am I rigth ? -- Sent from:

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread Julien Rouhaud
On Mon, Aug 12, 2019 at 4:01 PM Evgeniy Efimkin wrote: > > > One problem with pg_stat_statement's normalized query is that it's not > > stable, it's storing the normalized version of the first query string > > passed when an entry is created. So you could have different strings > > depending on

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread Evgeniy Efimkin
> One problem with pg_stat_statement's normalized query is that it's not > stable, it's storing the normalized version of the first query string > passed when an entry is created. So you could have different strings > depending on whether the query was fully qualified or relying on > search

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread Julien Rouhaud
On Mon, Aug 12, 2019 at 2:52 PM Evgeniy Efimkin wrote: > > Hi! > What about adding new column in pg_stat_statements e.g. sql_id it's hash from > normalized query. Аnd add function which get that hash (using raw_parser, > raw_expression_tree_walker) for any query > ` > postgres=# select

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread Julien Rouhaud
On Mon, Aug 12, 2019 at 2:40 PM Jim Finnerty wrote: > > If hashing names instead of using OIDs is too expensive for some workload, > then that workload would need to be able to turn statement hashing off. So > it needs to be optional, just like queryId is optionally computed today. > For many

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-12 Thread Evgeniy Efimkin
Hi! What about adding new column in pg_stat_statements e.g. sql_id it's hash from normalized query. Аnd add function which get that hash (using raw_parser, raw_expression_tree_walker) for any query ` postgres=# select get_queryid('select 1'); get_queryid - 680388963 (1 row) `

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-10 Thread legrand legrand
Hi Jim, Its never too later, as nothing has been concluded about that survey ;o) For information, I thought It would be possible to get a more stable QueryId, by hashing relation name or fully qualified names. With the support of Julien Rouhaud, I tested with this kind of code: case

Re: [survey] New "Stable" QueryId based on normalized query text

2019-08-10 Thread Julien Rouhaud
On Sat, Aug 10, 2019 at 3:27 AM Jim Finnerty wrote: > > I missed this thread. I'd be happy to post the code for what we use as the > stable query identifier, but we could definitely come up with a more > efficient algorithm if we're willing to assume that the sql statements are > the same if and

Re: [survey] New "Stable" QueryId based on normalized query text

2019-04-10 Thread Julien Rouhaud
On Tue, Apr 9, 2019 at 11:26 PM Bruce Momjian wrote: > > On Wed, Mar 20, 2019 at 03:19:58PM -0700, legrand legrand wrote: > > > The rest of thread raise quite a lot of concerns about the semantics, > > > the cost and the correctness of this patch. After 5 minutes checking, > > > it wouldn't

Re: [survey] New "Stable" QueryId based on normalized query text

2019-04-09 Thread Bruce Momjian
On Wed, Mar 20, 2019 at 03:19:58PM -0700, legrand legrand wrote: > > The rest of thread raise quite a lot of concerns about the semantics, > > the cost and the correctness of this patch. After 5 minutes checking, > > it wouldn't suits your need if you use custom functions, custom types, > >

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 11:10 PM legrand legrand wrote: > > Thank you Julien for the workaround, > It is not easy to build "cross tables" in excel to join metrics per query > text ... then keep only one queryid over all environments, that's easy enough in SQL: SELECT min(queryid) OVER

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 10:30 PM legrand legrand > > legrand_legrand@ > wrote: >> >> maybe this patch (with a GUC) >> https://www.postgresql.org/message-id/ > 55E51C48.1060102@ >> would be enough for thoses actually using a text normalization function. > > The rest of

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 10:18 PM legrand legrand > > legrand_legrand@ > wrote: >> >> On my personal point of view, I need to get the same Queryid between >> (OLAP) >> environments >> to be able to compare Production, Pre-production, Qualif performances >> (and I don't

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 10:30 PM legrand legrand wrote: > > maybe this patch (with a GUC) > https://www.postgresql.org/message-id/55e51c48.1060...@uptime.jp > would be enough for thoses actually using a text normalization function. The rest of thread raise quite a lot of concerns about the

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 10:18 PM legrand legrand wrote: > > On my personal point of view, I need to get the same Queryid between (OLAP) > environments > to be able to compare Production, Pre-production, Qualif performances > (and I don't need Fully qualified relation names). Today to do that, >

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
maybe this patch (with a GUC) https://www.postgresql.org/message-id/55e51c48.1060...@uptime.jp would be enough for thoses actually using a text normalization function. -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 8:39 PM legrand legrand > > legrand_legrand@ > wrote: >> >> Yes, I would like first to understand what are the main needs, > > I don't really see one implementation that suits every need, as > probably not everyone will agree on using relation

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 8:39 PM legrand legrand wrote: > > Yes, I would like first to understand what are the main needs, I don't really see one implementation that suits every need, as probably not everyone will agree on using relation name vs fully qualified relation name for starter. The

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
> From "Kyotaro HORIGUCHI-2" >>At Wed, 20 Mar 2019 00:23:30 +, "Tsunakawa, Takayuki" >>> From: legrand legrand [mailto:legrand_legrand@] >>> norm.9: comments aware >> Is this to distinguish queries that have different comments for optimizer >> hints? If yes, I agree. > Or, any means to give

RE: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
> From: "Tsunakawa, Takayuki" >> From: legrand legrand [mailto:legrand_legrand@] >> There are many projects that use alternate QueryId >> distinct from the famous pg_stat_statements jumbling algorithm. >I'd like to welcome the standard QueryID that DBAs and extension developers can depend on.

RE: [survey] New "Stable" QueryId based on normalized query text

2019-03-19 Thread Tsunakawa, Takayuki
From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] > > > needs.1: stable accross different databases, > > > > Does this mean different database clusters, not different databases in > a single database cluster? > > Does this mean you want different QueryID for the same-looking >

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-19 Thread Kyotaro HORIGUCHI
At Wed, 20 Mar 2019 00:23:30 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1FBE20A4@G01JPEXMBYT05> > From: legrand legrand [mailto:legrand_legr...@hotmail.com] > > There are many projects that use alternate QueryId > > distinct from the famous pg_stat_statements

RE: [survey] New "Stable" QueryId based on normalized query text

2019-03-19 Thread Tsunakawa, Takayuki
From: legrand legrand [mailto:legrand_legr...@hotmail.com] > There are many projects that use alternate QueryId > distinct from the famous pg_stat_statements jumbling algorithm. I'd like to welcome the standard QueryID that DBAs and extension developers can depend on. Are you surveying the