Re: Multiple Query IDs for a rewritten parse tree

2022-01-31 Thread Andrey Lepikhov
On 29/1/2022 12:51, Julien Rouhaud wrote: 4. We should reserve position of default in-core generator From the discussion above I was under the impression that the core generator should be distinguished by a predefined kind. I don't really like this approach. IIUC, this patch as-is is meant

Re: Multiple Query IDs for a rewritten parse tree

2022-01-31 Thread Dmitry Dolgov
> On Mon, Jan 31, 2022 at 02:59:17PM +0500, Andrey V. Lepikhov wrote: > On 1/28/22 9:51 PM, Dmitry Dolgov wrote: > > > On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > > > Registration of an queryId generator implemented by analogy with > > > extensible > > > methods machinery

Re: Multiple Query IDs for a rewritten parse tree

2022-01-31 Thread Andrey V. Lepikhov
On 1/28/22 9:51 PM, Dmitry Dolgov wrote: On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: Registration of an queryId generator implemented by analogy with extensible methods machinery. Why not more like suggested with stakind and slots in some data structure? All of those gen

Re: Multiple Query IDs for a rewritten parse tree

2022-01-29 Thread Dmitry Dolgov
> On Sun, Jan 30, 2022 at 01:48:20AM +0800, Julien Rouhaud wrote: > Hi, > > On Sat, Jan 29, 2022 at 06:12:05PM +0100, Dmitry Dolgov wrote: > > > On Sat, Jan 29, 2022 at 03:51:33PM +0800, Julien Rouhaud wrote: > > > > > > I'm also unsure of how are extensions supposed to cooperate in general, as > >

Re: Multiple Query IDs for a rewritten parse tree

2022-01-29 Thread Julien Rouhaud
Hi, On Sat, Jan 29, 2022 at 06:12:05PM +0100, Dmitry Dolgov wrote: > > On Sat, Jan 29, 2022 at 03:51:33PM +0800, Julien Rouhaud wrote: > > > > I'm also unsure of how are extensions supposed to cooperate in general, as > > I feel that queryids should be implemented for some "intent" (like > > moni

Re: Multiple Query IDs for a rewritten parse tree

2022-01-29 Thread Dmitry Dolgov
> On Sat, Jan 29, 2022 at 03:51:33PM +0800, Julien Rouhaud wrote: > Hi, > > On Fri, Jan 28, 2022 at 05:51:56PM +0100, Dmitry Dolgov wrote: > > > On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > > > On 1/9/22 5:49 AM, Tom Lane wrote: > > > > The idea I'd been vaguely thinking ab

Re: Multiple Query IDs for a rewritten parse tree

2022-01-28 Thread Julien Rouhaud
Hi, On Fri, Jan 28, 2022 at 05:51:56PM +0100, Dmitry Dolgov wrote: > > On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > > On 1/9/22 5:49 AM, Tom Lane wrote: > > > The idea I'd been vaguely thinking about is to allow attaching a list > > > of query-hash nodes to a Query, where

Re: Multiple Query IDs for a rewritten parse tree

2022-01-28 Thread Dmitry Dolgov
> On Fri, Jan 21, 2022 at 11:33:22AM +0500, Andrey V. Lepikhov wrote: > On 1/9/22 5:49 AM, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > > of query-hash nodes to a Query, where each node would contain a "tag" > > identifying the specific hash calculatio

Re: Multiple Query IDs for a rewritten parse tree

2022-01-20 Thread Andrey V. Lepikhov
On 1/9/22 5:49 AM, Tom Lane wrote: The idea I'd been vaguely thinking about is to allow attaching a list of query-hash nodes to a Query, where each node would contain a "tag" identifying the specific hash calculation method, and also the value of the query's hash calculated according to that meth

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
On 10/1/2022 15:39, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 03:24:46PM +0500, Andrey Lepikhov wrote: On 10/1/2022 13:56, Julien Rouhaud wrote: Yes. the same input query string doesn't prove that frozen query plan can be used, because rewrite rules could be changed. So we use only a query t

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Julien Rouhaud
On Mon, Jan 10, 2022 at 03:24:46PM +0500, Andrey Lepikhov wrote: > On 10/1/2022 13:56, Julien Rouhaud wrote: > > > > I don't know the details of that extension, but I think that it should work > > as > > long as you have the constants information in the jumble state, whatever the > > resulting no

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Andrey Lepikhov
On 10/1/2022 13:56, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 12:37:34PM +0500, Andrey V. Lepikhov wrote: I think, pg_stat_statements can live with an queryId generator of the sr_plan extension. But It replaces all constants with $XXX parameter at the query string. In our extension user defi

Re: Multiple Query IDs for a rewritten parse tree

2022-01-10 Thread Julien Rouhaud
On Mon, Jan 10, 2022 at 12:37:34PM +0500, Andrey V. Lepikhov wrote: > I think, pg_stat_statements can live with an queryId generator of the > sr_plan extension. But It replaces all constants with $XXX parameter at the > query string. In our extension user defines which plan is optimal and which > c

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Andrey V. Lepikhov
On 1/10/22 9:51 AM, Julien Rouhaud wrote: On Mon, Jan 10, 2022 at 09:10:59AM +0500, Andrey V. Lepikhov wrote: I can add one more use case. Our extension for freezing query plan uses query tree comparison technique to prove, that the plan can be applied (and we don't need to execute planning proc

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Mon, Jan 10, 2022 at 09:10:59AM +0500, Andrey V. Lepikhov wrote: > On 1/9/22 5:13 PM, Julien Rouhaud wrote: > > For now the queryid mixes two different things: fingerprinting and query > > text > > normalization. Should each calculation method be allowed to do a different > > normalization too

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Andrey V. Lepikhov
On 1/9/22 5:13 PM, Julien Rouhaud wrote: For now the queryid mixes two different things: fingerprinting and query text normalization. Should each calculation method be allowed to do a different normalization too, and if yes where should be stored the state data needed for that? If not, we would

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Sat, Jan 08, 2022 at 07:49:59PM -0500, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > of query-hash nodes to a Query, where each node would contain a "tag" > identifying the specific hash calculation method, and also the value > of the query's hash ca

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Julien Rouhaud
On Sun, Jan 09, 2022 at 12:43:06PM +0100, Dmitry Dolgov wrote: > > An extension that wants a slightly modified version of hash calculation > implementation from the core would still need to copy everything. The > core probably has to provide more than one (hash, method) pair to cover > some basic n

Re: Multiple Query IDs for a rewritten parse tree

2022-01-09 Thread Dmitry Dolgov
> On Sat, Jan 08, 2022 at 07:49:59PM -0500, Tom Lane wrote: > > The idea I'd been vaguely thinking about is to allow attaching a list > of query-hash nodes to a Query, where each node would contain a "tag" > identifying the specific hash calculation method, and also the value > of the query's hash

Re: Multiple Query IDs for a rewritten parse tree

2022-01-08 Thread Tom Lane
Andrey Lepikhov writes: > On 5/1/2022 10:13, Tom Lane wrote: >>> I feel like we need to get away from the idea that there is just >>> one query hash, and somehow let different extensions attach >>> differently-calculated hashes to a query. I don't have any immediate >>> ideas about how to do that

Multiple Query IDs for a rewritten parse tree

2022-01-08 Thread Andrey Lepikhov
On 5/1/2022 10:13, Tom Lane wrote: > I feel like we need to get away from the idea that there is just > one query hash, and somehow let different extensions attach > differently-calculated hashes to a query. I don't have any immediate > ideas about how to do that in a reasonably inexpensive way.