Re: [PERFORM] Advice requested on structuring aggregation queries

2010-02-22 Thread Joe Conway
On 02/22/2010 07:01 PM, Dave Crooke wrote: > The original data is keyed by a 3-ary tuple of strings to keep the > row size down, in the new data model I'm actually storing 32-bit int's > in Postgres. The new schema for each table looks like this: > > (a integer, > b integer, > c integer, >

[PERFORM] Advice requested on structuring aggregation queries

2010-02-22 Thread Dave Crooke
Hi folks I have an application which collects performance stats at time intervals, to which I am retro-fitting a table partitioning scheme in order to improve scalability. The original data is keyed by a 3-ary tuple of strings to keep the row size down, in the new data model I'm actually sto

Re: [PERFORM] Planner question - "bit" data types

2010-02-22 Thread Bruce Momjian
Bruce Momjian wrote: > Alvaro Herrera wrote: > > Karl Denninger escribi?: > > > > > The individual boolean fields don't kill me and in terms of some of the > > > application issues they're actually rather easy to code for. > > > > > > The problem with re-coding for them is extensibility (by those

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Scott Marlowe
On Mon, Feb 22, 2010 at 7:21 PM, Scott Marlowe wrote: > On Mon, Feb 22, 2010 at 6:39 PM, Greg Smith wrote: >> Mark Mielke wrote: >>> >>> I had read the above when posted, and then looked up SRAM. SRAM seems to >>> suggest it will hold the data even after power loss, but only for a period >>> of t

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Scott Marlowe
On Mon, Feb 22, 2010 at 6:39 PM, Greg Smith wrote: > Mark Mielke wrote: >> >> I had read the above when posted, and then looked up SRAM. SRAM seems to >> suggest it will hold the data even after power loss, but only for a period >> of time. As long as power can restore within a few minutes, it see

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Greg Smith
Mark Mielke wrote: I had read the above when posted, and then looked up SRAM. SRAM seems to suggest it will hold the data even after power loss, but only for a period of time. As long as power can restore within a few minutes, it seemed like this would be ok? The normal type of RAM everyone u

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Greg Smith
Ron Mayer wrote: I know less about other file systems. Apparently the NTFS guys are aware of such stuff - but don't know what kinds of fsync equivalent you'd need to make it happen. It's actually pretty straightforward--better than ext3. Windows with NTFS has been perfectly aware how to d

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Mark Mielke
On 02/22/2010 08:04 PM, Greg Smith wrote: Arjen van der Meijden wrote: That's weird. Intel's SSD's didn't have a write cache afaik: "I asked Intel about this and it turns out that the DRAM on the Intel drive isn't used for user data because of the risk of data loss, instead it is used as memor

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Greg Smith
Arjen van der Meijden wrote: That's weird. Intel's SSD's didn't have a write cache afaik: "I asked Intel about this and it turns out that the DRAM on the Intel drive isn't used for user data because of the risk of data loss, instead it is used as memory by the Intel SATA/flash controller for d

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Joel Jacobson
Thank you for explaining! Now I understand, makes perfect sense! :-) 2010/2/22 Nikolas Everett : > The planner knows that that particular date range is quite selective so it > doesn't have to BitmapAnd two indexes together. > The problem is that a prepared statement asks the db to plan the query

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Pierre C
Actually, planner was smart in using a bitmap index scan in the prepared query. Suppose you later EXECUTE that canned plan with a date range which covers say half of the table : the indexscan would be a pretty bad choice since it would have to access half the rows in the table in index orde

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Nikolas Everett
The planner knows that that particular date range is quite selective so it doesn't have to BitmapAnd two indexes together. The problem is that a prepared statement asks the db to plan the query without knowing anything about the parameters. I think functions behave in exactly the same way. Its k

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Tom Lane
Joel Jacobson writes: > Hm, it is strange the query planner is using two different strategies > for the same query? They're not the same query. One plan is generic for any value of the parameters, the other is chosen for specific values of those parameters. In particular, the unparameterized q

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Joel Jacobson
db=# \d FlagValueAccountingTransactions Table "public.flagvalueaccountingtransactions" Column| Type |Modifiers -+-- + -

Re: [PERFORM] plpgsql plan cache

2010-02-22 Thread Pierre C
I cannot understand why the index is not being used when in the plpgsql function? I even tried to make a test function containing nothing more than the single query. Still the index is not being used. When running the same query in the sql prompt, the index is in use though. Please post t

[PERFORM] plpgsql plan cache

2010-02-22 Thread Joel Jacobson
Hi, I am trying to make a select query in my plpgsql function to use an index allowing an index scan instead of a seq scan. When running the query in the sql prompt, it works fine, but apparently the index is not used for the same query in the plpgsql function. The problem is not the d

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Ron Mayer
Bruce Momjian wrote: > Greg Smith wrote: >> If you have a regular SATA drive, it almost certainly >> supports proper cache flushing > > OK, but I have a few questions. Is a write to the drive and a cache > flush command the same? I believe they're different as of ATAPI-6 from 2001. >

Re: [PERFORM] shared_buffers

2010-02-22 Thread Kevin Grittner
"George Sexton" wrote: > Could someone please explain what the role of shared buffers is? This Wiki page might be useful to you: http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server The short answer (from that page) is: "The shared_buffers configuration parameter determines how

[PERFORM] shared_buffers

2010-02-22 Thread George Sexton
I'm reading the docs for 8.4.2, section 18.4.1 Memory. I'm trying to figure out what reasonable values for my usage would be. The doc says: shared_buffers (integer) Sets the amount of memory the database server uses for shared memory buffers. While circular definitions are always right, I'm

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Bruce Momjian
Ron Mayer wrote: > Bruce Momjian wrote: > > Agreed, thought I thought the problem was that SSDs lie about their > > cache flush like SATA drives do, or is there something I am missing? > > There's exactly one case I can find[1] where this century's IDE > drives lied more than any other drive with

Re: [PERFORM] SSD + RAID

2010-02-22 Thread Bruce Momjian
Greg Smith wrote: > Ron Mayer wrote: > > Bruce Momjian wrote: > > > >> Agreed, thought I thought the problem was that SSDs lie about their > >> cache flush like SATA drives do, or is there something I am missing? > >> > > > > There's exactly one case I can find[1] where this century's IDE >

Re: [PERFORM] AutoVacuum_NapTime

2010-02-22 Thread Alvaro Herrera
George Sexton wrote: > If I'm cold starting the system, would it vacuum all 330 databases and then > wait 720 minutes and then do them all again, or would it distribute the > databases more or less evenly over the time period? the latter -- Alvaro Herrerahttp://w