Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-03-02 Thread Bernd Helmle
--On 28. Februar 2011 15:02:30 -0500 Tom Lane wrote: Because it's fifty times more mechanism than we need here? We don't want a SQL interface (not even a lightweight one) and it's unclear that we ever want the data to go to disk at all. I wonder wether a library like librrd would be a solu

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-03-01 Thread Dimitri Fontaine
Tom Lane writes: > The ideal solution would likely be for the stats collector to expose its > data structures as shared memory, but I don't think we get to do that > under SysV shmem --- it doesn't like variable-size shmem much. Maybe > that's another argument for looking harder into mmap or POSI

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 2:31 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >>> Ultimately we need to think of a reporting mechanism that's a bit >>> smarter than "rewrite the whole file for any update" ... > >> Well, we have these things called "ta

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Euler Taveira de Oliveira writes: > Em 28-02-2011 15:50, Tom Lane escreveu: >> Ultimately we need to think of a reporting mechanism that's a bit >> smarter than "rewrite the whole file for any update" ... > What about splitting statistic file per database? That would improve matters for some usa

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
"Joshua D. Drake" writes: > On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: > Spitballing here, but could sqlite be an intermediate, compromise solution? >> >> For a core PostgreSQL component ?!?!? > Sure, why not? Because it's fifty times more mechanism than we need here? We don't want

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Euler Taveira de Oliveira
Em 28-02-2011 15:50, Tom Lane escreveu: Ultimately we need to think of a reporting mechanism that's a bit smarter than "rewrite the whole file for any update" ... What about splitting statistic file per database? -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hack

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Joshua D. Drake
On Mon, 2011-02-28 at 11:39 -0800, Josh Berkus wrote: > > Spitballing here, but could sqlite be an intermediate, compromise solution? > > For a core PostgreSQL component ?!?!? Sure, why not? It is ACID compliant, has the right kind of license, has a standard API that we are all used to. It seems

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
> Spitballing here, but could sqlite be an intermediate, compromise solution? For a core PostgreSQL component ?!?!? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com --

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Michael Glaesemann
On Feb 28, 2011, at 14:31, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >>> Ultimately we need to think of a reporting mechanism that's a bit >>> smarter than "rewrite the whole file for any update" ... > >> Well, we have these things called "tables

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: >> Ultimately we need to think of a reporting mechanism that's a bit >> smarter than "rewrite the whole file for any update" ... > Well, we have these things called "tables". Any chance of using those? Having the stats coll

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:50 PM, Tom Lane wrote: > Josh Berkus writes: >> On 2/28/11 10:24 AM, Robert Haas wrote: >>> On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: On the other hand, anything which increases the size of pg_statistic would be a nightmare. > >>> Hmm? > >> Like repl

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Tom Lane
Josh Berkus writes: > On 2/28/11 10:24 AM, Robert Haas wrote: >> On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: >>> On the other hand, anything which increases the size of pg_statistic >>> would be a nightmare. >> Hmm? > Like replacing each statistic with a series of time-based buckets, wh

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
On 2/28/11 10:24 AM, Robert Haas wrote: > On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: >> On the other hand, anything which increases the size of pg_statistic >> would be a nightmare. > > Hmm? Like replacing each statistic with a series of time-based buckets, which would then increase the

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Robert Haas
On Mon, Feb 28, 2011 at 1:04 PM, Josh Berkus wrote: > On the other hand, anything which increases the size of pg_statistic > would be a nightmare. Hmm? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Martijn van Oosterhout
On Mon, Feb 28, 2011 at 10:04:54AM -0800, Josh Berkus wrote: > Take, for example, a problem I was recently grappling with for Nagios. > I'd like to do a check as to whether or not tables are getting > autoanalyzed often enough. After all, autovac can fall behind, and we'd > want to be alerted of t

Re: [HACKERS] Why our counters need to be time-based WAS: WIP: cross column correlation ...

2011-02-28 Thread Josh Berkus
> Well, what we have now is a bunch of counters in pg_stat_all_tables > and pg_statio_all_tables. Right. What I'm saying is those aren't good enough, and have never been good enough. Counters without a time basis are pretty much useless for performance monitoring/management (Baron Schwartz ha