Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Fri, Oct 4, 2013 at 7:22 AM, Fujii Masao wrote: > On Thu, Oct 3, 2013 at 5:11 PM, Sameer Thakur wrote: >> On Wed, Oct 2, 2013 at 6:40 PM, Sameer Thakur wrote: Looks pretty good. Do you want to package up the patch with your change and do the honors and re-submit it? Thanks for

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Thu, Oct 3, 2013 at 5:11 PM, Sameer Thakur wrote: > On Wed, Oct 2, 2013 at 6:40 PM, Sameer Thakur wrote: >>> >>> Looks pretty good. Do you want to package up the patch with your >>> change and do the honors and re-submit it? Thanks for helping out so >>> much! >> Sure, will do. Need to add a b

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Wed, Oct 2, 2013 at 6:40 PM, Sameer Thakur wrote: >> >> Looks pretty good. Do you want to package up the patch with your >> change and do the honors and re-submit it? Thanks for helping out so >> much! > Sure, will do. Need to add a bit of documentation explaining > statistics session as well.

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

> > Looks pretty good. Do you want to package up the patch with your > change and do the honors and re-submit it? Thanks for helping out so > much! Sure, will do. Need to add a bit of documentation explaining statistics session as well. I did some more basic testing around pg_stat_statements.max, n

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sun, Sep 29, 2013 at 10:58 PM, Daniel Farina wrote: > I remember hacking that out for testing sake. > > I can only justify it as a foot-gun to prevent someone from being > stuck restarting the database to get a reasonable number in there. > Let's CC Peter; maybe he can remember some thoughts ab

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Tue, Oct 1, 2013 at 5:32 AM, Sameer Thakur wrote: > On Tue, Oct 1, 2013 at 12:48 AM, Daniel Farina-5 [via PostgreSQL] > <[hidden email]> wrote: > >> >> On Sep 30, 2013 4:39 AM, "Sameer Thakur" <[hidden email]> wrote: >>> >>> > Also, for onlookers, I have changed this patch around to do the >>>

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Tue, Oct 1, 2013 at 12:48 AM, Daniel Farina-5 [via PostgreSQL] wrote: > > On Sep 30, 2013 4:39 AM, "Sameer Thakur" <[hidden email]> wrote: >> >> > Also, for onlookers, I have changed this patch around to do the >> > date-oriented stuff but want to look it over before stapling it up and >> > sen

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Tue, Oct 1, 2013 at 12:48 AM, Daniel Farina-5 [via PostgreSQL] wrote: > > On Sep 30, 2013 4:39 AM, "Sameer Thakur" <[hidden email]> wrote: >> >> > Also, for onlookers, I have changed this patch around to do the >> > date-oriented stuff but want to look it over before stapling it up and >> > sen

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sep 30, 2013 4:39 AM, "Sameer Thakur" wrote: > > > Also, for onlookers, I have changed this patch around to do the > > date-oriented stuff but want to look it over before stapling it up and > > sending it. If one cannot wait, one can look at > > https://github.com/fdr/postgres/tree/queryid. T

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

> Also, for onlookers, I have changed this patch around to do the > date-oriented stuff but want to look it over before stapling it up and > sending it. If one cannot wait, one can look at > https://github.com/fdr/postgres/tree/queryid. The squashed-version of > that history contains a reasonable

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sun, Sep 29, 2013 at 10:25 AM, Sameer Thakur wrote: > Yes i was. Just saw a warning when pg_stat_statements is loaded that > valid values for pg_stat_statements.max is between 100 and 2147483647. > Not sure why though. I remember hacking that out for testing sake. I can only justify it as a f

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Mon, Sep 23, 2013 at 1:26 PM, samthakur74 wrote: >> >> I forgot about removal of the relevant SGML, amended here in v6. > > Thank you for this! > i did a quick test with following steps: > 1. Applied v6 patch > 2. make and make install on pg_stat_statements; > 3. Restarted Postgres with pg_stat

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

Daniel Farina escribió: > On Fri, Sep 20, 2013 at 1:11 AM, Daniel Farina wrote: > > I think the n-call underestimation propagation may not be quite precise for > > various detailed reasons (having to do with 'sticky' queries) and to make it > > precise is probably more work than it's worth. And,

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

> > I forgot about removal of the relevant SGML, amended here in v6. Thank you for this! i did a quick test with following steps: 1. Applied v6 patch 2. make and make install on pg_stat_statements; 3. Restarted Postgres with pg_stat_statements loaded with pg_stat_statements.max = 4 4. Dropped and

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Fri, Sep 20, 2013 at 1:11 AM, Daniel Farina wrote: > I think the n-call underestimation propagation may not be quite precise for > various detailed reasons (having to do with 'sticky' queries) and to make it > precise is probably more work than it's worth. And, on more reflection, I'm > also h

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Thu, Sep 19, 2013 at 11:32 AM, Fujii Masao-2 [via PostgreSQL] < ml-node+s1045698n5771565...@n5.nabble.com> wrote: > On Thu, Sep 19, 2013 at 2:25 PM, samthakur74 <[hidden > email]> > wrote: > > >>I got the segmentation fault when I tested th

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Thu, Sep 19, 2013 at 2:25 PM, samthakur74 wrote: >>I got the segmentation fault when I tested the case where the >> least-executed >>query statistics is discarded, i.e., when I executed different queries more >> than >>pg_stat_statements.max times. I guess that the patch might have a bug. > Tha

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

>I got the segmentation fault when I tested the case where the least-executed >query statistics is discarded, i.e., when I executed different queries more than >pg_stat_statements.max times. I guess that the patch might have a bug. Thanks, will try to fix it. >pg_stat_statements--1.1.sql should be

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Thu, Sep 19, 2013 at 2:41 AM, Fujii Masao wrote: > On Wed, Sep 18, 2013 at 2:41 PM, Sameer Thakur wrote: >You seem to have forgotten to include the pg_stat_statements--1.2.sql >and pg_stat_statements--1.1--1.2.sql in the patch. Sorry again. Please find updated patch attached. >>

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Wed, Sep 18, 2013 at 2:41 PM, Sameer Thakur wrote: >>> >You seem to have forgotten to include the pg_stat_statements--1.2.sql >>> >and pg_stat_statements--1.1--1.2.sql in the patch. >>> Sorry again. Please find updated patch attached. > > I did not add pg_stat_statements--1.2.sql. I have added

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

>> >You seem to have forgotten to include the pg_stat_statements--1.2.sql >> >and pg_stat_statements--1.1--1.2.sql in the patch. >> Sorry again. Please find updated patch attached. I did not add pg_stat_statements--1.2.sql. I have added that now and updated the patch again. The patch attached sho

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Tue, Sep 17, 2013 at 10:48 PM, samthakur74 wrote: > > > > > >You seem to have forgotten to include the pg_stat_statements--1.2.sql > >and pg_stat_statements--1.1--1.2.sql in the patch. >> >> >> Sorry again. Please find updated patch attached. pg_stat_statements--1.2.sql is missing. Could you c

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

>You seem to have forgotten to include the pg_stat_statements--1.2.sql >and pg_stat_statements--1.1--1.2.sql in the patch. > > Sorry again. Please find updated patch attached. > > > > NAML

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sun, Sep 15, 2013 at 3:54 PM, samthakur74 wrote: > > >> >You have added this email to the commit fest, but it contains no patch. >> >> >Please add the email with the actual patch. > > I hope its attached now! You seem to have forgotten to include the pg_stat_statements--1.2.sql and pg_stat_st

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

> >You have added this email to the commit fest, but it contains no patch. > >Please add the email with the actual patch. > I hope its attached now! > Maybe the author should be > >given a chance to update the patches, though, because they are quite > >old. > I did connect with Daniel and he di

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sat, 2013-09-14 at 03:51 -0700, samthakur74 wrote: > We have a need to see this patch committed hence the revived interest > in this thread You have added this email to the commit fest, but it contains no patch. Please add the email with the actual patch. Maybe the author should be given a ch

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

>This patch needs documentation. At a minimum, the new calls_underest >field needs to be listed in the description of the pg_stat_statements. I have attached a version which includes documentation. pg_stat_statements-identification-v4.patch.gz

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On 30.12.2012 08:31, Daniel Farina wrote: A version implementing that is attached, except I generate an additional 64-bit session not exposed to the client to prevent even casual de-leaking of the session state. That may seem absurd, until someone writes a tool that de-xors things and relies on

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sat, Dec 29, 2012 at 7:16 PM, Daniel Farina wrote: > On Sat, Dec 29, 2012 at 7:12 PM, Peter Geoghegan > wrote: >> On 30 December 2012 02:45, Daniel Farina wrote: >>> As I recall, the gist of this objection had to do with a false sense >>> of stability of the hash value, and the desire to enf

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sat, Dec 29, 2012 at 7:12 PM, Peter Geoghegan wrote: > On 30 December 2012 02:45, Daniel Farina wrote: >> As I recall, the gist of this objection had to do with a false sense >> of stability of the hash value, and the desire to enforce the ability >> to alter it. Here's an option: xor the has

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On 30 December 2012 02:45, Daniel Farina wrote: > As I recall, the gist of this objection had to do with a false sense > of stability of the hash value, and the desire to enforce the ability > to alter it. Here's an option: xor the hash value with the > 'statistics session id', so it's *known* to

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sat, Dec 29, 2012 at 6:37 PM, Peter Geoghegan wrote: > On 29 December 2012 12:21, Daniel Farina wrote: >> These were not express goals of the patch, but so long as you are >> inviting features, attached is a bonus patch that exposes the queryid >> and also the notion of a "statistics session"

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On 29 December 2012 12:21, Daniel Farina wrote: > These were not express goals of the patch, but so long as you are > inviting features, attached is a bonus patch that exposes the queryid > and also the notion of a "statistics session" that is re-rolled > whenever the stats file could not be read

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On Sat, Dec 29, 2012 at 4:21 AM, Daniel Farina wrote: > These were not express goals of the patch, but so long as you are > inviting features, attached is a bonus patch that exposes the queryid > and also the notion of a "statistics session" that is re-rolled > whenever the stats file could not be

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

Attached is a cumulative patch attempting to address the below. One can see the deltas to get there at https://github.com/fdr/postgres.git error-prop-pg_stat_statements-v2. On Fri, Dec 28, 2012 at 9:58 AM, Peter Geoghegan wrote: > However, with this approach, calls_underest values might appear t

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

On 28 December 2012 11:43, Daniel Farina wrote: > Without further ado, the cover letter taken from the top of the patch: > > This tries to establish a maximum under-estimate of the number of > calls for a given pg_stat_statements entry. That means the number of > calls to the canonical form of th

[HACKERS] pg_stat_statements: calls under-estimation propagation

Hello, After long delay (sorry) here's a patch implementing what was hand-waved at in http://archives.postgresql.org/pgsql-hackers/2012-10/msg00176.php I am still something at a loss at how to test it besides prodding it by hand; it seems like it's going to involve infrastructure or introducing h

Re: [HACKERS] pg_stat_statements temporary file

On Friday, May 25, 2012 05:19:28 PM Tom Lane wrote: > Andres Freund writes: > > On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: > >> Where do you suggest the file be written to? > > > > One could argue stats_temp_directory would be the correct place. > > No, that would be exactly the

Re: [HACKERS] pg_stat_statements temporary file

On Friday, May 25, 2012, Tom Lane wrote: > Andres Freund > writes: > > On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: > >> Where do you suggest the file be written to? > > > One could argue stats_temp_directory would be the correct place. > > No, that would be exactly the *wrong* place

Re: [HACKERS] pg_stat_statements temporary file

On Friday, May 25, 2012, Peter Geoghegan wrote: > On 25 May 2012 14:13, Magnus Hagander > > wrote: > > Here's a patch that does the two easy fixes: > > 1) writes the file to a temp file and rename()s it over the main file > > as it writes down. This removes the (small) risk of corruption because >

Re: [HACKERS] pg_stat_statements temporary file

> Why would they want that? PSS only writes the tempfile on shutdown and > reads it on startup. Unlike pgstats which reads and writes it all the > time. Ah, ok! Didn't know that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] pg_stat_statements temporary file

On Fri, May 25, 2012 at 6:49 PM, Josh Berkus wrote: > On 5/25/12 8:19 AM, Tom Lane wrote: >> Andres Freund writes: >>> On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: Where do you suggest the file be written to? >> >>> One could argue stats_temp_directory would be the correct plac

Re: [HACKERS] pg_stat_statements temporary file

On 5/25/12 8:19 AM, Tom Lane wrote: > Andres Freund writes: >> On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: >>> Where do you suggest the file be written to? > >> One could argue stats_temp_directory would be the correct place. > > No, that would be exactly the *wrong* place, becaus

Re: [HACKERS] pg_stat_statements temporary file

Andres Freund writes: > On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: >> Where do you suggest the file be written to? > One could argue stats_temp_directory would be the correct place. No, that would be exactly the *wrong* place, because that directory can be on a RAM disk. We need

Re: [HACKERS] pg_stat_statements temporary file

Magnus Hagander writes: > On Fri, May 25, 2012 at 4:09 PM, Tom Lane wrote: >> Given that pgstats keeps its permanent file in global/, I think the >> argument that pg_stat_statements should not do likewise is pretty thin. > Fair enough. As long as the file is unlinked after reading (per my > patc

Re: [HACKERS] pg_stat_statements temporary file

On Fri, May 25, 2012 at 4:09 PM, Tom Lane wrote: > Peter Geoghegan writes: >> On 25 May 2012 14:13, Magnus Hagander wrote: >>> I still think we should consider the placement of this file to not be >>> in the global/ directory, but this is a quick (back-patchable) fix... > >> Where do you suggest

Re: [HACKERS] pg_stat_statements temporary file

Peter Geoghegan writes: > On 25 May 2012 14:13, Magnus Hagander wrote: >> I still think we should consider the placement of this file to not be >> in the global/ directory, but this is a quick (back-patchable) fix... > Where do you suggest the file be written to? Given that pgstats keeps its pe

Re: [HACKERS] pg_stat_statements temporary file

On Friday, May 25, 2012 04:03:49 PM Peter Geoghegan wrote: > > I still think we should consider the placement of this file to not be > > in the global/ directory, but this is a quick (back-patchable) fix... > > Where do you suggest the file be written to? One could argue stats_temp_directory would

Re: [HACKERS] pg_stat_statements temporary file

On 25 May 2012 14:13, Magnus Hagander wrote: > Here's a patch that does the two easy fixes: > 1) writes the file to a temp file and rename()s it over the main file > as it writes down. This removes the (small) risk of corruption because > of a crash during write > > 2) unlinks the file after readi

Re: [HACKERS] pg_stat_statements temporary file

On Thu, May 24, 2012 at 2:19 PM, Magnus Hagander wrote: > On Thu, May 24, 2012 at 2:16 PM, Peter Geoghegan > wrote: >> On 24 May 2012 12:42, Magnus Hagander wrote: >>> What actually happens if it tries to repalloc() something huge? palloc >>> will throw an elog(ERROR), and since this happens du

Re: [HACKERS] pg_stat_statements temporary file

On Thu, May 24, 2012 at 2:16 PM, Peter Geoghegan wrote: > On 24 May 2012 12:42, Magnus Hagander wrote: >> What actually happens if it tries to repalloc() something huge? palloc >> will throw an elog(ERROR), and since this happens during postmaster >> startup, are you sure it won't prevent the ser

Re: [HACKERS] pg_stat_statements temporary file

On 24 May 2012 12:42, Magnus Hagander wrote: > What actually happens if it tries to repalloc() something huge? palloc > will throw an elog(ERROR), and since this happens during postmaster > startup, are you sure it won't prevent the server from starting? Oh, yes, missed that. /*

Re: [HACKERS] pg_stat_statements temporary file

On Thu, May 24, 2012 at 1:36 PM, Peter Geoghegan wrote: > On 24 May 2012 11:43, Magnus Hagander wrote: >> In general, should a contrib module really store data in the global/ >> directory? Seems pretty ugly to me... > > I think the case could be made for moving pg_stat_statements into > core, as

Re: [HACKERS] pg_stat_statements temporary file

On 24 May 2012 11:43, Magnus Hagander wrote: > In general, should a contrib module really store data in the global/ > directory? Seems pretty ugly to me... I think the case could be made for moving pg_stat_statements into core, as an optionally enabled view, like pg_stat_user_functions, since pg_

[HACKERS] pg_stat_statements temporary file

If pg_stat_statements is set to store it's data across restarts, it stores it in global/pg_stat_statements.stat. This causes some interesting things to happen in combination with a base backup - namely, if you take a base backup *after* you have restarted th emaster, the slave will get a snapshot o

Re: [HACKERS] pg_stat_statements and planning time

On 8 March 2012 14:44, Tom Lane wrote: > I thought the proposal was to add it to (1) pg_stat_statement and (2) > EXPLAIN, both of which are not in the normal code execution path. > pg_stat_statement is already a drag on a machine with slow gettimeofday, > but it's not clear why users of it would t

Re: [HACKERS] pg_stat_statements and planning time

Peter Geoghegan writes: > On 8 March 2012 13:09, Robert Haas wrote: >>  Then again, considering that gettimeofday is kinda >> expensive, I suppose that would have to be optional if we were to have >> it at all. > +1. I'm not opposed to having such a mechanism, but it really ought to > impose exa

Re: [HACKERS] pg_stat_statements and planning time

On 8 March 2012 13:09, Robert Haas wrote: > Then again, considering that gettimeofday is kinda > expensive, I suppose that would have to be optional if we were to have > it at all. +1. I'm not opposed to having such a mechanism, but it really ought to impose exactly no overhead on the common case

Re: [HACKERS] pg_stat_statements and planning time

On Wed, Mar 7, 2012 at 9:59 PM, Fujii Masao wrote: >> I'd like to have the planning time in a number of other places as >> well, such as EXPLAIN, and maybe statistics views. > > +1 for EXPLAIN. But which statistics views are in your mind? I don't know. I'm not sure if it's interesting to be able

Re: [HACKERS] pg_stat_statements and planning time

On Thu, Mar 8, 2012 at 1:07 AM, Tom Lane wrote: > Fujii Masao writes: >> In the patch, I didn't change the column name "total_time" meaning >> the time spent in the executor because of the backward compatibility. >> But once new column "plan_time" is added, "total_time" is confusing and >> ISTM i

Re: [HACKERS] pg_stat_statements and planning time

On Thu, Mar 8, 2012 at 12:39 AM, Robert Haas wrote: > On Wed, Mar 7, 2012 at 6:45 AM, Fujii Masao wrote: >> pg_stat_statements is basically very helpful to find out slow queries. >> But since it doesn't report the time spent in the planner, we cannot >> find out slow queries which take most time

Re: [HACKERS] pg_stat_statements and planning time

On Wed, Mar 7, 2012 at 8:07 AM, Tom Lane wrote: > Fujii Masao writes: >> In the patch, I didn't change the column name "total_time" meaning >> the time spent in the executor because of the backward compatibility. >> But once new column "plan_time" is added, "total_time" is confusing and >> ISTM i

Re: [HACKERS] pg_stat_statements and planning time

Fujii Masao writes: > In the patch, I didn't change the column name "total_time" meaning > the time spent in the executor because of the backward compatibility. > But once new column "plan_time" is added, "total_time" is confusing and > ISTM it should be renamed... Well, if we were tracking plann

Re: [HACKERS] pg_stat_statements and planning time

On Wed, Mar 7, 2012 at 6:45 AM, Fujii Masao wrote: > pg_stat_statements is basically very helpful to find out slow queries. > But since it doesn't report the time spent in the planner, we cannot > find out slow queries which take most time to do query planning, from > pg_stat_statements. Is there

Re: [HACKERS] pg_stat_statements and planning time

On Wed, Mar 7, 2012 at 9:00 PM, Simon Riggs wrote: > On Wed, Mar 7, 2012 at 11:45 AM, Fujii Masao wrote: > >> Attached patch extends pg_stat_statements so that it reports the >> planning time. Thought? > > If we successfully aggregate SQL in the current patch then this might > be useful as well.

Re: [HACKERS] pg_stat_statements and planning time

On Wed, Mar 7, 2012 at 11:45 AM, Fujii Masao wrote: > Attached patch extends pg_stat_statements so that it reports the > planning time. Thought? If we successfully aggregate SQL in the current patch then this might be useful as well. Until we do that it's not much use. --  Simon Riggs 

[HACKERS] pg_stat_statements and planning time

Hi, pg_stat_statements is basically very helpful to find out slow queries. But since it doesn't report the time spent in the planner, we cannot find out slow queries which take most time to do query planning, from pg_stat_statements. Is there any reason why pg_stat_statements doesn't collect the p

Re: [HACKERS] pg_stat_statements normalization: re-review

On Fri, Feb 24, 2012 at 3:14 AM, Daniel Farina wrote: > At ExecutorFinish (already hookable) all NodeKeys remembered by an > extension should be invalidated, as that memory is free and ready to > be used again. I think this statement is false; I thought it was true because *not* invalidating give

Re: [HACKERS] pg_stat_statements normalization: re-review

On Thu, Feb 23, 2012 at 3:09 AM, Peter Geoghegan wrote: >> These have all been changed in the usual manner to support one new >> field, the queryId, on the toplevel Plan and Query nodes.  The queryId >> is 64-bit field copied from queries to plans to tie together plans "to >> be used by plugins" (

Re: [HACKERS] pg_stat_statements normalization: re-review

On 23 February 2012 11:09, Peter Geoghegan wrote: > On 23 February 2012 09:58, Daniel Farina wrote: >> * The small changes to hashing are probably not strictly required, >> unless collisions are known to get terrible. > > I imagine that collisions would be rather difficult to demonstrate at > all

Re: [HACKERS] pg_stat_statements normalization: re-review

On 23 February 2012 09:58, Daniel Farina wrote: > What I'm going to do here is focus on the back-end source changes that > are not part of the contrib.  The size of the changes are much > reduced, but their semantics are also somewhat more complex...so, here > goes.  Conclusion first: > > * The sm

[HACKERS] pg_stat_statements normalization: re-review

Hello again, I'm reviewing the revised version of pg_stat_statements again. In particular, this new version has done away with the mechanical but invasive change to the lexing that preserved *both* the position and length of constant values. (See http://archives.postgresql.org/message-id/CAEYLb_W

Re: [HACKERS] pg_stat_statements with query tree based normalization

On 10 December 2011 13:56, Greg Smith wrote: > I heard about some bitrot creeping in here too, but it seems gone now; I had > no problem merging Peter's development branch against master.  I've attached > a newer patch of the main code, which fixes most of the earlier issues there > were disclaime

Re: [HACKERS] pg_stat_statements with query tree based normalization

On Wed, Dec 7, 2011 at 03:19, Peter Geoghegan wrote: > The results are...taking the median value of each set of runs as > representative, my patch appears to run marginally faster than head. > Of course, there is no reason to believe that it should, and I'm > certain that the difference can be exp

Re: [HACKERS] pg_stat_statements with query tree based normalization

On 14 November 2011 04:42, Greg Smith wrote: > The approach Peter used adds a single integer to the Const structure in > order to have enough information to substitute "?" in place of those. >  Adding and maintaining that is the only change outside of the extension > made here, and that overhead i

Re: [HACKERS] pg_stat_statements in core

Tom Lane <[EMAIL PROTECTED]> wrote: > It needs to be a contrib or pgfoundry package for awhile, > to shake out feature issues in a context where users will understand > the API is subject to change. Agreed. That is what I want to do at first. > It seems to me that all you're really missing is a

Re: [HACKERS] pg_stat_statements in core

ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: >> Now I'm working on storing statistics into disks on server >> shutdown. If it is impossible unless the module is in core, >> I would change my policy... I'm really not happy with a proposal to put such a feature in core. Once it's in core we'll have

[HACKERS] pg_stat_statements in core

I wrote: > Now I'm working on storing statistics into disks on server > shutdown. If it is impossible unless the module is in core, > I would change my policy... I reconsidered this part and found that pg_stat_statements needs to be in core to write stats in file on server shutdown because: 1.

Re: [HACKERS] pg_stat_statements

On Tue, 2008-06-24 at 12:14 +0900, ITAGAKI Takahiro wrote: > Simon Riggs <[EMAIL PROTECTED]> wrote: > > > I think you want to see the distribution of execution times for > > particular queries without needing to log *every* execution, including > > parameters. I think I understand now what you ar

Re: [HACKERS] pg_stat_statements

Simon Riggs <[EMAIL PROTECTED]> wrote: > I think you want to see the distribution of execution times for > particular queries without needing to log *every* execution, including > parameters. I think I understand now what you are asking for and why you > are asking for it. Yes. In many case, majo

Re: [HACKERS] pg_stat_statements

On Mon, 2008-06-23 at 15:22 +0900, ITAGAKI Takahiro wrote: > I wrote: > > I will try to measure overheads of logging in some implementation: > > 1. Log statements and dump them into server logs. > > 2. Log statements and filter them before to be written. > > 3. Store statements in shared mem

Re: [HACKERS] pg_stat_statements

I wrote: > I will try to measure overheads of logging in some implementation: > 1. Log statements and dump them into server logs. > 2. Log statements and filter them before to be written. > 3. Store statements in shared memory. > I know 1 is slow, but I don't know what part of it is really sl

Re: [HACKERS] pg_stat_statements

Jonah, > It wouldn't be too hard to write the probes in such a way as they > could be used by DTrace or by a loadable timing/counter implementation > for platforms which don't support DTrace. I was under the impression that's the way our feature, the "Generic Monitoring Interface" was written.

Re: [HACKERS] pg_stat_statements

On Mon, Jun 16, 2008 at 10:34 AM, Robert Treat <[EMAIL PROTECTED]> wrote: > Given that PostgreSQL relies on the operating > system for a majority of it's instermentation (ie. we have nothing like v$ > tables in oracle), we should really be thinking of dtrace as the ultimate > tool for DBA's to figu

Re: [HACKERS] pg_stat_statements

On Sunday 15 June 2008 22:31:59 ITAGAKI Takahiro wrote: > Robert Treat <[EMAIL PROTECTED]> wrote: > > On Friday 13 June 2008 12:58:22 Josh Berkus wrote: > > > I can see how this would be useful, but I can also see that it could be > > > a huge performance burden when activated. So it couldn't be p

Re: [HACKERS] pg_stat_statements

I understand there must be "some" overhead because we're collecting extra info. I'm curious if there're considerable amount of overhead to the users who don't want such additional trance. 2008/6/16 ITAGAKI Takahiro <[EMAIL PROTECTED]>: > > Robert Treat <[EMAIL PROTECTED]> wrote: > >> On Friday 1

Re: [HACKERS] pg_stat_statements

Robert Treat <[EMAIL PROTECTED]> wrote: > On Friday 13 June 2008 12:58:22 Josh Berkus wrote: > > I can see how this would be useful, but I can also see that it could be a > > huge performance burden when activated. So it couldn't be part of the > > standard statistics collection. > > A lower ov

Re: [HACKERS] pg_stat_statements

Tom Lane <[EMAIL PROTECTED]> wrote: > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > Postgres 8.4 has pg_stat_user_functions view to track number of calls > > of stored functions and time spent in them. Then, I'm thinking a > > "sql statement" version of similar view -- pg_stat_statements. >

Re: [HACKERS] pg_stat_statements

On Friday 13 June 2008 12:58:22 Josh Berkus wrote: > ITAGAKI Takahiro wrote: > > Hello, > > > > Postgres 8.4 has pg_stat_user_functions view to track number of calls of > > stored functions and time spent in them. Then, I'm thinking a "sql > > statement" version of similar view -- pg_stat_statement

Re: [HACKERS] pg_stat_statements

Simon Riggs <[EMAIL PROTECTED]> writes: > On Fri, 2008-06-13 at 17:33 +0900, ITAGAKI Takahiro wrote: >> We can write sql statements in server logs and gather them using some >> tools (pgfouine and pqa) even now, but statement logging has >> unignorable overhead. > I would prefer to look at ways to

Re: [HACKERS] pg_stat_statements

On Fri, 2008-06-13 at 17:33 +0900, ITAGAKI Takahiro wrote: > We can write sql statements in server logs and gather them using some > tools (pgfouine and pqa) even now, but statement logging has > unignorable overhead. I would prefer to look at ways to reduce the current overhead rather than chang

Re: [HACKERS] pg_stat_statements

ITAGAKI Takahiro wrote: Hello, Postgres 8.4 has pg_stat_user_functions view to track number of calls of stored functions and time spent in them. Then, I'm thinking a "sql statement" version of similar view -- pg_stat_statements. I can see how this would be useful, but I can also see that it co

Re: [HACKERS] pg_stat_statements

ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > Postgres 8.4 has pg_stat_user_functions view to track number of calls of > stored functions and time spent in them. Then, I'm thinking a "sql statement" > version of similar view -- pg_stat_statements. We don't have any system-wide names for statement

[HACKERS] pg_stat_statements

Hello, Postgres 8.4 has pg_stat_user_functions view to track number of calls of stored functions and time spent in them. Then, I'm thinking a "sql statement" version of similar view -- pg_stat_statements. Prepared statements and statements using extended protocol are grouped by their sql strings

<    1   2