Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-12-01 Thread Greg Smith
On 11/25/2011 10:52 AM, Jeff Janes wrote: Given Peter's patch on the same subject, should we now mark this one as rejected in the commitfest app? https://commitfest.postgresql.org/action/patch_view?id=681 That may be premature. While the testing I've done so far suggests Peter's idea is the

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-25 Thread Jeff Janes
On Sat, Nov 5, 2011 at 8:42 PM, Tomas Vondra t...@fuzzy.cz wrote: Dne 6.11.2011 03:16, Peter Geoghegan napsal(a): 2011/11/6 Tomas Vondra t...@fuzzy.cz: Hi everyone, The patch implements a simple cleaning that replaces the parameter values with generic strings - e.g. numbers are turned to :n,

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Robert Haas
On Sun, Nov 6, 2011 at 2:56 PM, Greg Smith g...@2ndquadrant.com wrote: Peter's patch adds a planner hook and hashes at that level.  Since this cat is rapidly escaping its bag and impacting other contributors, we might as well share the work in progress early if anyone has a burning desire to

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Greg Smith
On 11/07/2011 09:03 AM, Robert Haas wrote: I think it's an established principle that the design for features like this should, for best results, be discussed on -hackers before writing a lot of code. You can see from the commit history this idea is less than a month old. Do we need to get

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Greg Smith
On 11/06/2011 06:00 PM, Tom Lane wrote: Peter Geogheganpe...@2ndquadrant.com writes: A major consideration was backwards compatibility; This is not a consideration that the community is likely to weigh heavily, or indeed at all. We aren't going to back-port this feature into prior release

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-07 Thread Robert Haas
On Mon, Nov 7, 2011 at 10:27 AM, Greg Smith g...@2ndquadrant.com wrote: On 11/07/2011 09:03 AM, Robert Haas wrote: I think it's an established principle that the design for features like this should, for best results, be discussed on -hackers before writing a lot of code. You can see from

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: I'm a couple of days away from posting a much better principled implementation of pg_stat_statements normalisation. To normalise, we perform a selective serialisation of the query tree, which is hashed. That seems like an interesting approach, and

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Dimitri Fontaine
Tomas Vondra t...@fuzzy.cz writes: If it was possible to compare the actual plan (or at least a hash of the plan), and keeping one record for each plan, that'd be extremely nice. You'd see that the query was executed with 3 different plans, number of calls, average duration etc. I like that

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Tomas Vondra
On 6 Listopad 2011, 16:08, Tom Lane wrote: Peter Geoghegan pe...@2ndquadrant.com writes: I'm a couple of days away from posting a much better principled implementation of pg_stat_statements normalisation. To normalise, we perform a selective serialisation of the query tree, which is hashed.

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Greg Smith
On 11/06/2011 01:08 PM, Tom Lane wrote: Peter Geogheganpe...@2ndquadrant.com writes: ... It also does things like intelligently distinguishing between queries with different limit/offset constant values, as these constants are deemed to be differentiators of queries for our purposes. A

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Peter Geoghegan
On 6 November 2011 15:08, Tom Lane t...@sss.pgh.pa.us wrote: Are you intending to hash the raw grammar output tree, the parse analysis result, the rewriter result, or the actual plan tree? I'm hashing the Query tree from a planner plugin (function assigned to planner_hook), immediately prior

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-06 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 6 November 2011 15:08, Tom Lane t...@sss.pgh.pa.us wrote: Are you intending to hash the raw grammar output tree, the parse analysis result, the rewriter result, or the actual plan tree? I'm hashing the Query tree from a planner plugin

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-05 Thread Peter Geoghegan
2011/11/6 Tomas Vondra t...@fuzzy.cz: Hi everyone, The patch implements a simple cleaning that replaces the parameter values with generic strings - e.g. numbers are turned to :n, so the queries mentioned above are turned to   SELECT abalance FROM pgbench_accounts WHERE aid = :n and thus

Re: [HACKERS] [PATCH] optional cleaning queries stored in pg_stat_statements

2011-11-05 Thread Tomas Vondra
Dne 6.11.2011 03:16, Peter Geoghegan napsal(a): 2011/11/6 Tomas Vondra t...@fuzzy.cz: Hi everyone, The patch implements a simple cleaning that replaces the parameter values with generic strings - e.g. numbers are turned to :n, so the queries mentioned above are turned to SELECT abalance