[HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
pg_stat_statements' fingerprinting logic considers the following two statements as distinct: select 1 in (1, 2, 3); select 1 in (1, 2, 3, 4); This is because the ArrayExpr jumble case jumbles any ArrayExpr's list of elements recursively. In this case it's a list of Const nodes, and the

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Robert Haas
On Tue, Dec 10, 2013 at 4:30 AM, Peter Geoghegan p...@heroku.com wrote: pg_stat_statements' fingerprinting logic considers the following two statements as distinct: select 1 in (1, 2, 3); select 1 in (1, 2, 3, 4); This is because the ArrayExpr jumble case jumbles any ArrayExpr's list of

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 1:09 PM, Robert Haas robertmh...@gmail.com wrote: I am very wary of implementing special-case logic here even though I know it could be useful to some people, simply because I fear that there could be a near-infinite variety of situations where, in a particular

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Andres Freund
On 2013-12-10 16:09:02 -0500, Robert Haas wrote: On Tue, Dec 10, 2013 at 4:30 AM, Peter Geoghegan p...@heroku.com wrote: I'm not sure that I agree, but there is anecdata that suggests that it isn't uncommon for these sorts of queries to be broken out when they're all traceable back to a

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 10, 2013 at 4:30 AM, Peter Geoghegan p...@heroku.com wrote: pg_stat_statements' fingerprinting logic considers the following two statements as distinct: select 1 in (1, 2, 3); select 1 in (1, 2, 3, 4); [ and some people think it

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 1:41 PM, Andres Freund and...@2ndquadrant.com wrote: FWIW, I hit exactly this issue every single time I have looked at pg_stat_statements in some client's database making it nearly useless for analysis. So improving it might be worthwile. I think the thing that makes me

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Andres Freund
On 2013-12-10 14:30:36 -0800, Peter Geoghegan wrote: Did you really find pg_stat_statements to be almost useless in such situations? That seems worse than I thought. It's very hard to see where you should spend efforts when every logical query is split into hundreds of pg_stat_statement

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 2:38 PM, Andres Freund and...@2ndquadrant.com wrote: It's very hard to see where you should spend efforts when every logical query is split into hundreds of pg_stat_statement entries. Suddenly it's important whether a certain counts of parameters are more frequent than

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Robert Haas
On Tue, Dec 10, 2013 at 5:38 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-12-10 14:30:36 -0800, Peter Geoghegan wrote: Did you really find pg_stat_statements to be almost useless in such situations? That seems worse than I thought. It's very hard to see where you should spend

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 2:46 PM, Robert Haas robertmh...@gmail.com wrote: Right, but the flip side is that you could collapse things that people don't want collapsed. If you've got lots of query that differ only in that some of them say user_id IN (const1, const2) and others say user_id IN

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Andres Freund
On 2013-12-10 17:46:56 -0500, Robert Haas wrote: On Tue, Dec 10, 2013 at 5:38 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-12-10 14:30:36 -0800, Peter Geoghegan wrote: Did you really find pg_stat_statements to be almost useless in such situations? That seems worse than I thought.

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 2:55 PM, Peter Geoghegan p...@heroku.com wrote: You might get lucky and have this exact case, and be able to leverage the knowledge that the 2 constants in the ArrayExpr must be the latter and 1 must be the former, but experience suggests very probably not. When things

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Right, but the flip side is that you could collapse things that people don't want collapsed. If you've got lots of query that differ only in that some of them say user_id IN (const1, const2) and others say user_id IN (const1, const2, const3) and the

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Daniel Farina
On Tue, Dec 10, 2013 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: So my objection to what Peter is suggesting is not that it's a bad idea in isolation, but that I don't see where he's going to stop, short of reinventing every query-normalization behavior that exists in the planner. If this

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: A different point of view is that it's more or less an implementation artifact that pg_stat_statements doesn't already see the cases as equivalent; that happens only because it looks at the querytree before the planner gets

Re: [HACKERS] pg_stat_statements fingerprinting logic and ArrayExpr

2013-12-10 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Tue, Dec 10, 2013 at 3:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm wondering whether this doesn't indicate that we need to rethink where the fingerprinter has been plugged in. I'm not sure that somewhere in the planner, post-constant-folding, would