Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-30 Thread Greg Stark
On Sun, Apr 29, 2012 at 12:26 AM, Robert Haas robertmh...@gmail.com wrote: As for track_iotiming - track_io_timing, I'm fine with that as well. I'm still grumpy about the idea of a GUC changing the explain analyze output. How would people feel about adding an explain option that explicitly

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-30 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Sun, Apr 29, 2012 at 12:26 AM, Robert Haas robertmh...@gmail.com wrote: As for track_iotiming - track_io_timing, I'm fine with that as well. I'm still grumpy about the idea of a GUC changing the explain analyze output. How would people feel about adding an

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I like the idea of including the word block in there. I don't think it was probably a terribly good idea to abbreviate that to blk everywhere, but at this point it's probably better to be consistent, sigh. As for track_iotiming - track_io_timing,

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ... You might want to revisit the issue of how the new columns in pg_stat_statements are named, as well. I am not sure I'm happy with that, but neither am I sure that I know what I'd like better. It's not too clear that the timing is specifically for

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Tom Lane
... btw, while I'm criticizing names, how about changing track_iotiming to track_io_timing? The former seems inelegant and unreadable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-28 Thread Robert Haas
On Sat, Apr 28, 2012 at 12:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... You might want to revisit the issue of how the new columns in pg_stat_statements are named, as well.  I am not sure I'm happy with that, but neither am I sure that I know what I'd

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: There's no particular reason to think that Moore's Law is going to result in an increase in the fractional precision of timing data. It hasn't done so in the past, for sure.

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: There's no particular reason to think that Moore's Law is going to result in an increase in the fractional

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: However, the main thing here is that we need to do *something* here... Agreed, this has got to be pushed forward. In the interest of

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 1:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Apr 25, 2012 at 12:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: However, the main thing here is that we need to do *something* here...

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Greg Stark
On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Also, as was pointed out upthread, the underlying data in shared memory is almost certainly never going to be infinite-precision; so using numeric in the API seems to me to be more likely to convey a false impression of

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Also, as was pointed out upthread, the underlying data in shared memory is almost certainly never going to be infinite-precision; so using numeric in the API seems to me to be more likely to

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 1:58 PM, Greg Stark st...@mit.edu wrote: On Wed, Apr 25, 2012 at 5:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Also, as was pointed out upthread, the underlying data in shared memory is almost certainly never going to be infinite-precision; so using numeric in the API

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Peter Eisentraut
On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: Perhaps, but nobody's explained what we gain out of NOT using numeric. So if you want to have possibly different internal and external representations, why not use interval for the external one? -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: Perhaps, but nobody's explained what we gain out of NOT using numeric. So if you want to have possibly different internal and external representations, why not use interval for the external one?

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-25 Thread Robert Haas
On Wed, Apr 25, 2012 at 5:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2012-04-23 at 22:03 -0400, Robert Haas wrote: Perhaps, but nobody's explained what we gain out of NOT using numeric. So if you want to have possibly different internal and

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-23 Thread Robert Haas
On Sat, Apr 14, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The internal representation doesn't have to be (and certainly shouldn't be) numeric.  But if you translate to numeric before returning the data to the user, then you have the freedom,

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Greg Smith
On 04/13/2012 06:22 PM, Tom Lane wrote: But (a) I *don't* want to seriously break things, and don't see a need to; (b) interval is expensive and has got its own problems, notably an internal limitation to usec resolution that we would not be able to get rid of easily. A straight float seems

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Robert Haas
On Sat, Apr 14, 2012 at 3:27 AM, Greg Smith g...@2ndquadrant.com wrote: On 04/13/2012 06:22 PM, Tom Lane wrote: But (a) I *don't* want to seriously break things, and don't see a need to; (b) interval is expensive and has got its own problems, notably an internal limitation to usec resolution

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Peter Geoghegan
On 10 April 2012 19:10, Tom Lane t...@sss.pgh.pa.us wrote: Hmm.  Maybe we should think about numeric ms, which would have all the same advantages but without the round-off error. Color me unimpressed ... numeric calculations are vastly more expensive than float, and where are you going to get

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Peter Geoghegan
On 14 April 2012 02:42, Greg Stark st...@mit.edu wrote: Is this not subject to the birthday paradox? If you have a given hash you're worried about a collision with then you have a one-in-four-billion chance. But if you have a collection of hashes and you're worried about any collisions then it

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Robert Haas
On Sat, Apr 14, 2012 at 9:54 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 10 April 2012 19:10, Tom Lane t...@sss.pgh.pa.us wrote: Hmm.  Maybe we should think about numeric ms, which would have all the same advantages but without the round-off error. Color me unimpressed ... numeric

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The internal representation doesn't have to be (and certainly shouldn't be) numeric. But if you translate to numeric before returning the data to the user, then you have the freedom, in the future, to whack around the internal representation however

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Wed, Apr 11, 2012 at 9:02 AM, k...@rice.edu k...@rice.edu wrote: By using all 64-bits of the hash that we currently calculate, instead of the current use of 32-bits only, the collision probabilities are very low. That's probably true, but I'm not sure it's worth worrying about -

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On another note, I think this is a sufficiently major change that we ought to add Peter's name to the Author section of the pg_stat_statements documentation. +1 ... as long as we have those at all, they probably ought to credit anybody who did

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Fri, Apr 13, 2012 at 4:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On another note, I think this is a sufficiently major change that we ought to add Peter's name to the Author section of the pg_stat_statements documentation. +1 ... as long as we

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Geoghegan
On 13 April 2012 20:15, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 11, 2012 at 9:02 AM, k...@rice.edu k...@rice.edu wrote: By using all 64-bits of the hash that we currently calculate, instead of the current use of 32-bits only, the collision probabilities are very low. That's

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Jim Nasby
On 4/10/12 5:07 PM, Greg Smith wrote: I'd prefer to see at least usec resolution and 8 bytes of dynamic range for query related statistics. Any of these would be fine from a UI perspective to me: -float8 seconds -float8 msec -float8 usec -int8 usec I don't think int8 msec will be enough

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Eisentraut
On tis, 2012-04-10 at 09:33 -0400, Robert Haas wrote: So, should we make the new columns exposed by pg_stat_statements use milliseconds, so that the block read/write timings are everywhere in milliseconds, or should we keep them as a float8, so that all the times exposed by pg_stat_statements

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Jim Nasby j...@nasby.net writes: I agree that ms is on it's way out... and frankly it wouldn't surprise me if at some point we actually had need of ns resolution. Given that, I don't think ms or us definitions make sense... float8 seconds seams much more logical to me. Well, the important

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Greg Stark
On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas robertmh...@gmail.com wrote: That's probably true, but I'm not sure it's worth worrying about - one-in-four-billion is a pretty small probability. Is this not subject to the birthday paradox? If you have a given hash you're worried about a collision

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas robertmh...@gmail.com wrote: That's probably true, but I'm not sure it's worth worrying about - one-in-four-billion is a pretty small probability. Is this not subject to the birthday paradox? If you have a given

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Peter Geoghegan
On 14 April 2012 03:01, Tom Lane t...@sss.pgh.pa.us wrote: Realistically, I'm more worried about collisions due to inadequacies in the jumble calculation logic (Peter already pointed out some risk factors in that regard). It's important to have a sense of proportion about the problem. The

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-13 Thread Robert Haas
On Fri, Apr 13, 2012 at 10:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@mit.edu writes: On Fri, Apr 13, 2012 at 8:15 PM, Robert Haas robertmh...@gmail.com wrote: That's probably true, but I'm not sure it's worth worrying about - one-in-four-billion is a pretty small probability.

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-11 Thread k...@rice.edu
On Wed, Apr 11, 2012 at 01:53:06AM +0100, Peter Geoghegan wrote: On 11 April 2012 01:16, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan pe...@2ndquadrant.com writes: On 11 April 2012 00:35, Robert Haas robertmh...@gmail.com wrote: If people need something like that, couldn't they create

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Thu, Apr 5, 2012 at 11:45 AM, Robert Haas robertmh...@gmail.com wrote: Meanwhile, pg_stat_statements converts the same data to seconds but makes it a double rather than a bigint.  I think that was a bad idea and we should make it consistent use a bigint and milliseconds while we still can.

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Hmm. So, on further review, this is not as simple as it seems. I'd like some input from other people on what we should do here. pg_stat_statements has long exposed a column called total_time as a float8. It now exposes columns time_read and

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 10 April 2012 14:33, Robert Haas robertmh...@gmail.com wrote: So, should we make the new columns exposed by pg_stat_statements use milliseconds, so that the block read/write timings are everywhere in milliseconds, or should we keep them as a float8, so that all the times exposed by

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Hmm.  So, on further review, this is not as simple as it seems.  I'd like some input from other people on what we should do here. pg_stat_statements has long exposed a column called

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: No matter what we end up doing here it will be consistent with something; I am reminded of the phrase the good thing about standards is that there are so many to choose from Well, FWIW I vote for making the new columns be float8 msec. If you don't

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2012 at 17:58, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Hmm.  So, on further review, this is not as simple as it seems.  I'd like some input from other people on what

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Tue, Apr 10, 2012 at 17:58, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Given that we've whacked pg_stat_statements' behavior around rather thoroughly in this release, maybe we

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Magnus Hagander
On Tue, Apr 10, 2012 at 18:27, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Tue, Apr 10, 2012 at 17:58, Robert Haas robertmh...@gmail.com wrote: On Tue, Apr 10, 2012 at 10:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Given that we've whacked

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: No matter what we end up doing here it will be consistent with something; I am reminded of the phrase the good thing about standards is that there are so many to choose from

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, FWIW I vote for making the new columns be float8 msec. Ugh. So the three ways of doing timing that we have already aren't enough, and we need a fourth one? Ack! Huh? I

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, FWIW I vote for making the new columns be float8 msec. Ugh.  So the three ways of doing timing that we have

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Huh? I understood what you said upthread to be that we have two ways in existing releases (anything unreleased has zero standing in this discussion): float8 sec in

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Huh?  I understood what you said upthread to be that we have two ways in existing releases (anything unreleased has

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread k...@rice.edu
On Tue, Apr 10, 2012 at 02:01:02PM -0400, Robert Haas wrote: On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 1:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Huh?  I understood what you said upthread to be that

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Apr 10, 2012 at 1:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: The business about underlying microseconds is maybe not so good, but I don't think we want to touch that right now.  In the long run I think it would make sense to converge on float8

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Greg Smith
On 04/10/2012 12:27 PM, Tom Lane wrote: Changing the column name will definitely break anything more specific than select * from pg_stat_statements. However, it's less clear that changing the units in which the column is expressed will break things. It seems likely to me that nobody out there

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 10 April 2012 23:07, Greg Smith g...@2ndquadrant.com wrote: On 04/10/2012 12:27 PM, Tom Lane wrote: I am doing more sophisticated things with it, so I'll celebrate this as my opportunity to say I did something you didn't see coming for 2012. This is why I requested that we expose the

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Robert Haas
On Tue, Apr 10, 2012 at 6:32 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 10 April 2012 23:07, Greg Smith g...@2ndquadrant.com wrote: On 04/10/2012 12:27 PM, Tom Lane wrote: I am doing more sophisticated things with it, so I'll celebrate this as my opportunity to say I did something you

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 10 April 2012 23:07, Greg Smith g...@2ndquadrant.com wrote: On 04/10/2012 12:27 PM, Tom Lane wrote: I am doing more sophisticated things with it, so I'll celebrate this as my opportunity to say I did something you didn't see coming for 2012.

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 11 April 2012 00:35, Robert Haas robertmh...@gmail.com wrote: If people need something like that, couldn't they create it by hashing the normalized query text with an arbitrary algorithm? That supposes that the normalised query text is perfectly stable. It may well not be, particularly for

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 11 April 2012 00:35, Robert Haas robertmh...@gmail.com wrote: If people need something like that, couldn't they create it by hashing the normalized query text with an arbitrary algorithm? That supposes that the normalised query text is

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-10 Thread Peter Geoghegan
On 11 April 2012 01:16, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan pe...@2ndquadrant.com writes: On 11 April 2012 00:35, Robert Haas robertmh...@gmail.com wrote: If people need something like that, couldn't they create it by hashing the normalized query text with an arbitrary

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-05 Thread Robert Haas
On Tue, Mar 27, 2012 at 8:30 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 27, 2012 at 3:22 PM, Ants Aasma a...@cybertec.at wrote: On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the core of this.  I left out the stats collector stuff,

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Thu, Mar 22, 2012 at 7:38 PM, Ants Aasma a...@cybertec.at wrote: On Wed, Mar 21, 2012 at 5:01 PM, Robert Haas robertmh...@gmail.com wrote: This seems to have bitrotted again.  :-( Can you please rebase again? Rebased. I've committed the core of this. I left out the stats collector

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 2:58 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 22, 2012 at 7:38 PM, Ants Aasma a...@cybertec.at wrote: On Wed, Mar 21, 2012 at 5:01 PM, Robert Haas robertmh...@gmail.com wrote: This seems to have bitrotted again.  :-( Can you please rebase again?

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Ants Aasma
On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the core of this.  I left out the stats collector stuff, because it's still per-table and I think perhaps we should back off to just per-database.  I changed it so that it does not conflate wait time with

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 3:22 PM, Ants Aasma a...@cybertec.at wrote: On Tue, Mar 27, 2012 at 9:58 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the core of this.  I left out the stats collector stuff, because it's still per-table and I think perhaps we should back off to just

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Greg Stark
On Tue, Mar 27, 2012 at 7:58 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the core of this.  I left out the stats collector stuff, because it's still per-table and I think perhaps we should back off to just per-database.  I changed it so that it does not conflate wait time with

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-27 Thread Robert Haas
On Tue, Mar 27, 2012 at 8:41 PM, Greg Stark st...@mit.edu wrote: On Tue, Mar 27, 2012 at 7:58 PM, Robert Haas robertmh...@gmail.com wrote: I've committed the core of this.  I left out the stats collector stuff, because it's still per-table and I think perhaps we should back off to just

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-03-21 Thread Robert Haas
On Fri, Feb 24, 2012 at 2:23 PM, Ants Aasma ants.aa...@eesti.ee wrote: On Wed, Feb 22, 2012 at 6:35 PM, Ants Aasma ants.aa...@eesti.ee wrote: Some implementation notes.  This currently fails regression test create_function_3, haven't looked into why yet. I'll take a look at it. The failure

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-02-24 Thread Jim Nasby
On 2/22/12 10:35 AM, Ants Aasma wrote: The reason why I didn't add write timings to relation stats is that I couldn't figure out what the semantics should be. It could be either time spent waiting for this relations blocks to be written out or time spent waiting for some other relations blocks

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-02-22 Thread Ants Aasma
On Wed, Feb 22, 2012 at 4:43 PM, Greg Smith g...@2ndquadrant.com wrote: Attached are updated versions of this feature without the pg_test_timing tool part, since I broke that out into another discussion thread.  I've split the part that updates pg_stat_statistics out from the main feature too,

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-17 Thread Greg Smith
Attached is the pg_test_timing utility portion of this submission, broken out into its own patch. It's a contrib module modeled on pg_test_fsync. The documentation is still a bit rough, I'm not done with that yet. I have included an example of good timing results, switching to a bad clock

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-01-15 Thread Greg Smith
On 01/15/2012 05:14 PM, Ants Aasma wrote: I hope that having a tool to measure the overhead and check the sanity of clock sources is enough to answer the worries about the potential performance hit. We could also check that the clock source is fast enough on start-up/when the guc is changed, but

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28 Listopad 2011, 8:54, Greg Smith wrote: -Have one of the PostgreSQL background processes keep track of a time estimate on its own, only periodically pausing to sync against the real time. Then most calls to gettimeofday() can use that value instead. I was thinking of that idea for

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Robert Haas
On Mon, Nov 28, 2011 at 2:54 AM, Greg Smith g...@2ndquadrant.com wrote: The real problem with this whole area is that we know there are systems floating around where the amount of time taken to grab timestamps like this is just terrible. Assuming the feature is off by default (and I can't

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Greg Stark
On Nov 28, 2011 8:55 AM, Greg Smith g...@2ndquadrant.com wrote: On 11/27/2011 04:39 PM, Ants Aasma wrote: On the AMD I saw about 3% performance drop with timing enabled. On the Intel machine I couldn't measure any statistically significant change. Oh no, it's party pooper time again.

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28 Listopad 2011, 15:40, Greg Stark wrote: On Nov 28, 2011 8:55 AM, Greg Smith g...@2ndquadrant.com wrote: On 11/27/2011 04:39 PM, Ants Aasma wrote: On the AMD I saw about 3% performance drop with timing enabled. On the Intel machine I couldn't measure any statistically significant

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Nov 28, 2011 at 2:54 AM, Greg Smith g...@2ndquadrant.com wrote: The real problem with this whole area is that we know there are systems floating around where the amount of time taken to grab timestamps like this is just terrible. Assuming the

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Martijn van Oosterhout
On Sun, Nov 27, 2011 at 11:54:38PM -0800, Greg Smith wrote: On 11/27/2011 04:39 PM, Ants Aasma wrote: On the AMD I saw about 3% performance drop with timing enabled. On the Intel machine I couldn't measure any statistically significant change. Oh no, it's party pooper time again. Sorry I

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Dimitri Fontaine
Tomas Vondra t...@fuzzy.cz writes: Another option would be to reimplement the vsyscall, even on platforms that don't provide it. The principle is actually quite simple - allocate a shared memory, store there a current time and update it whenever a clock interrupt happens. This is basically

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Jim Nasby
On Nov 28, 2011, at 9:29 AM, Tomas Vondra wrote: I recall a patch similar to this one was submitted by Greg Stark some time ago. It used the info for different reasons--to try and figure out whether reads were cached or not--but I believe it withered rather than being implemented mainly

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 28.11.2011 22:32, Dimitri Fontaine wrote: Tomas Vondra t...@fuzzy.cz writes: Another option would be to reimplement the vsyscall, even on platforms that don't provide it. The principle is actually quite simple - allocate a shared memory, store there a current time and update it whenever a

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tomas Vondra
On 29.11.2011 02:14, Jim Nasby wrote: On Nov 28, 2011, at 9:29 AM, Tomas Vondra wrote: I recall a patch similar to this one was submitted by Greg Stark some time ago. It used the info for different reasons--to try and figure out whether reads were cached or not--but I believe it withered

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: That sounds good for other interesting things, which entails being able to have timing information attached to the XID sequence. If we go this way, how far are we from having a ticker in PostgreSQL? Those of us who are trying to get rid of

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Ants Aasma
Sorry for taking so long to respong, had a pretty busy day at work. Anyway.. On Mon, Nov 28, 2011 at 9:54 AM, Greg Smith g...@2ndquadrant.com wrote: Oh no, it's party pooper time again.  Sorry I have to be the one to do it this round.  The real problem with this whole area is that we know there

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Greg Smith
On 11/28/2011 05:51 AM, Robert Haas wrote: On Mon, Nov 28, 2011 at 2:54 AM, Greg Smithg...@2ndquadrant.com wrote: The real problem with this whole area is that we know there are systems floating around where the amount of time taken to grab timestamps like this is just terrible. Assuming the

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-28 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: On 11/28/2011 05:51 AM, Robert Haas wrote: Assuming the feature is off by default (and I can't imagine we'd consider anything else), I don't see why that should be cause for concern. If the instrumentation creates too much system load, then don't use

Re: [HACKERS] Patch: add timing of buffer I/O requests

2011-11-27 Thread Greg Smith
On 11/27/2011 04:39 PM, Ants Aasma wrote: On the AMD I saw about 3% performance drop with timing enabled. On the Intel machine I couldn't measure any statistically significant change. Oh no, it's party pooper time again. Sorry I have to be the one to do it this round. The real problem with