Re: [PERFORM] Configuration Recommendations

On 20/05/12 03:47, Jan Nielsen wrote: In this test, the local mount's buffered reads perform best around RA~10k @ 150MB/sec then starts a steady decline. The SAN mount has a similar but more subtle decline with a maximum around RA~5k @ 80MB/sec but with much greater variance. I was surprised at t

Re: [PERFORM] Configuration Recommendations

Oops - couple of corrections and clarifications below... On Sat, May 19, 2012 at 9:47 AM, Jan Nielsen wrote: > After seeing less much performance during pg_dump and pg_restore > operations from a 10x15k SAN RAID1+1 XFS mount > 10x15k RAID1+0 on a SAN with XFS on /dev/sdc > (allocsize=256m,attr

Re: [PERFORM] Configuration Recommendations

After seeing less much performance during pg_dump and pg_restore operations from a 10x15k SAN RAID1+1 XFS mount ( allocsize=256m,attr2,logbufs=8,logbsize=256k,noatime,nobarrier) than the local-storage 2x15k RAID1 EXT4 mount, I ran the following test of the effect of read-ahead (RA): for t in `seq

Re: [PERFORM] Configuration Recommendations

On Tue, May 15, 2012 at 11:53 AM, Greg Sabino Mullane wrote: > >>> Is it established practice in the Postgres world to separate indexes > >>> from tables? I would assume that the reasoning of Richard Foote - > >>> albeit for Oracle databases - is also true for Postgres: > > > >> Yes, it's an estab

Re: [PERFORM] Configuration Recommendations

On Tue, May 15, 2012 at 7:53 PM, Greg Sabino Mullane wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: RIPEMD160 > > Is it established practice in the Postgres world to separate indexes from tables? I would assume that the reasoning of Richard Foote - albeit for Oracle databases

Re: [PERFORM] Configuration Recommendations

-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 >>> Is it established practice in the Postgres world to separate indexes >>> from tables? I would assume that the reasoning of Richard Foote - >>> albeit for Oracle databases - is also true for Postgres: > >> Yes, it's an established practice. I

Re: [PERFORM] Configuration Recommendations

On Fri, May 4, 2012 at 8:07 AM, Jan Nielsen wrote: > Starting to get some quantitative data now. Here is the results from the > pgbench scaling: > >   pgbench -t 2000 -c 32 -S pgbench A single thread of pgbench is probably not enough to saturate 32 sessions. What if you try -j 16 or -j 32? Also

Re: [PERFORM] Configuration Recommendations

Jan Nielsen wrote: > The range 2048-8192 is an area to study in more detail, obviously. > Feedback welcome. I don't see what's to study there, really. Performance drops off when database size grows from 30GB to 60GB on a system with 48GB RAM. And even more when you double database size again.

Re: [PERFORM] Configuration Recommendations

Starting to get some quantitative data now. Here is the results from the pgbench scaling: pgbench -t 2000 -c 32 -S pgbench for scales of 2^n where n=0..14 for scale, DB size in MB, and transactions per second: Scale DB Size TPS --- 1 21 65618 2 36 66060

Re: [PERFORM] Configuration Recommendations

On 03/05/2012 16:46, Craig James wrote: On Thu, May 3, 2012 at 6:42 AM, Jan Nielsen wrote: Hi John, On Thu, May 3, 2012 at 12:54 AM, John Lister wrote: I was wondering if it would be better to put the xlog on the same disk as the OS? Apart from the occasional log writes I'd have thought most

Re: [PERFORM] Configuration Recommendations

> That sounds interesting. How do you identify a page flush storm? Maybe I used the wrong terminology. What effectively happens if you reach the amount of memory specified in dirty_ratio, is that the system goes from asynchronous disk access, to synchronous disk access, and starts flushing that

Re: [PERFORM] Configuration Recommendations

On 05/03/2012 02:28 AM, Robert Klemme wrote: Maybe this also has some additional input: http://www.fccps.cz/download/adv/frr/hdd/hdd.html Be careful with that link. His recommendations for dirty_ratio and dirty_background_ratio would be *very bad* in a database setting. Note this from the ac

Re: [PERFORM] Configuration Recommendations

On 4/25/12 2:29 PM, "Shaun Thomas" wrote: >On 04/25/2012 02:46 AM, John Lister wrote: > >> Hi, I'd be grateful if you could share any XFS performance tweaks as I'm >> not entirely sure I'm getting the most out of my setup and any >> additional guidance would be very helpful. > >Ok, I'll give thi

Re: [PERFORM] Configuration Recommendations

On 5/3/12 8:46 AM, "Craig James" wrote: >On Thu, May 3, 2012 at 6:42 AM, Jan Nielsen >wrote: >> Hi John, >> >> On Thu, May 3, 2012 at 12:54 AM, John Lister >> >> wrote: >>> >>> On 03/05/2012 03:10, Jan Nielsen wrote: >>> >>> >>> 300GB RAID10 2x15k drive for OS on local storage >>> */dev/sda1 R

Re: [PERFORM] Configuration Recommendations

On Thu, May 3, 2012 at 6:42 AM, Jan Nielsen wrote: > Hi John, > > On Thu, May 3, 2012 at 12:54 AM, John Lister > wrote: >> >> On 03/05/2012 03:10, Jan Nielsen wrote: >> >> >> 300GB RAID10 2x15k drive for OS on local storage >> */dev/sda1 RA*    4096 >> */de

Re: [PERFORM] Configuration Recommendations

Hi John, On Thu, May 3, 2012 at 12:54 AM, John Lister wrote: > On 03/05/2012 03:10, Jan Nielsen wrote: > > > 300GB RAID10 2x15k drive for OS on local storage > */dev/sda1 RA*4096 > */dev/sda1 FS*ext4 > */dev/

Re: [PERFORM] Configuration Recommendations

On Thu, May 3, 2012 at 7:05 AM, Shaun Thomas wrote: > I like to follow this: > > http://www.westnet.com/~**gsmith/content/linux-pdflush.**htm > Thanks for the reference, Shaun. > As a note, there are actually new tunables for some of t

Re: [PERFORM] Configuration Recommendations

Hi Robert, On Thu, May 3, 2012 at 1:28 AM, Robert Klemme wrote: > Hi Jan, > > On Thu, May 3, 2012 at 4:10 AM, Jan Nielsen > wrote: > > Below is the hardware, firmware, OS, and PG configuration pieces that I'm > > settling in on. As was noted, the local storage used for OS is actually > two > > d

Re: [PERFORM] Configuration Recommendations

Hi Jan, On Thu, May 3, 2012 at 4:10 AM, Jan Nielsen wrote: > Below is the hardware, firmware, OS, and PG configuration pieces that I'm > settling in on. As was noted, the local storage used for OS is actually two > disks with RAID 10. If anything appears like a mistake or something is > missing,

Re: [PERFORM] Configuration Recommendations

On 03/05/2012 03:10, Jan Nielsen wrote: 300GB RAID10 2x15k drive for OS on local storage */dev/sda1 RA*4096 */dev/sda1 FS*ext4 */dev/sda1 MO* 600GB RAID 10 8x15k drive for $PGDATA on SAN *IO Scheduler sda* n

Re: [PERFORM] Configuration Recommendations

Below is the hardware, firmware, OS, and PG configuration pieces that I'm settling in on. As was noted, the local storage used for OS is actually two disks with RAID 10. If anything appears like a mistake or something is missing, I'd appreciate the feedback. I'm still working on the benchmarks scr

Re: [PERFORM] Configuration Recommendations

On 24/04/2012 20:32, Shaun Thomas wrote: I'm not sure if you've done metrics or not, but XFS performance is highly dependent on your init and mount options. I can give you some guidelines there, but one of the major changes is that the Linux 3.X kernels have some impressive performance improv

Re: [PERFORM] Configuration Recommendations

On 04/23/2012 10:56 PM, Jan Nielsen wrote: We are planning to rebuild our production 50GB PG 9.0 database serving our application platform on the new hardware below. The web-applications are 80/20 read/write and the data gateways are even mix 50/50 read/write; one of the gateways nightly exports

Re: [PERFORM] Configuration Recommendations

On 04/25/2012 02:46 AM, John Lister wrote: Hi, I'd be grateful if you could share any XFS performance tweaks as I'm not entirely sure I'm getting the most out of my setup and any additional guidance would be very helpful. Ok, I'll give this with a huge caveat: these settings came from lots of

Re: [PERFORM] Configuration Recommendations

On Wed, Apr 25, 2012 at 7:08 PM, Greg Sabino Mullane wrote: >> Is it established practice in the Postgres world to separate indexes >> from tables?  I would assume that the reasoning of Richard Foote - >> albeit for Oracle databases - is also true for Postgres: > > Yes, it's an established practi

Re: [PERFORM] Configuration Recommendations

-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > Is it established practice in the Postgres world to separate indexes > from tables? I would assume that the reasoning of Richard Foote - > albeit for Oracle databases - is also true for Postgres: Yes, it's an established practice. I'd call i

Re: [PERFORM] Configuration Recommendations

Oopps; looping in the list... On Tue, Apr 24, 2012 at 8:57 PM, Jan Nielsen wrote: > On Mon, Apr 23, 2012 at 11:53 PM, Robert Klemme < > shortcut...@googlemail.com> wrote: > >> On Tue, Apr 24, 2012 at 4:56 AM, Jan Nielsen >> wrote: >> > We are considering the following drive allocations: >> > >>

Re: [PERFORM] Configuration Recommendations

On Tue, Apr 24, 2012 at 1:32 PM, Shaun Thomas wrote: > On 04/23/2012 09:56 PM, Jan Nielsen wrote: > > The new hardware for the 50GB PG 9.0 machine is: >> * 24 cores across 2 sockets >> * 64 GB RAM >> * 10 x 15k SAS drives on SAN >> * 1 x 15k SAS drive local >> * CentOS 6.2 (2.6.32 kernel) >> > >

Re: [PERFORM] Configuration Recommendations

On 04/23/2012 09:56 PM, Jan Nielsen wrote: The new hardware for the 50GB PG 9.0 machine is: * 24 cores across 2 sockets * 64 GB RAM * 10 x 15k SAS drives on SAN * 1 x 15k SAS drive local * CentOS 6.2 (2.6.32 kernel) This is a pretty good build. Nice and middle-of-the-road for current hardware

Re: [PERFORM] Configuration Recommendations

On Tue, Apr 24, 2012 at 4:56 AM, Jan Nielsen wrote: > We are considering the following drive allocations: > >  * 4 x 15k SAS drives, XFS, RAID 10 on SAN for PG data >  * 4 x 15k SAS drives, XFS, RAID 10 on SAN  for PG indexes >  * 2 x 15k SAS drives, XFS, RAID 1 on SAN  for PG xlog >  * 1 x 15k SA

[PERFORM] Configuration Recommendations

I'm a n00b [1] to tuning DBs so if anyone has a bit of time to provide feedback, I'd sure appreciate any input the community might have on the plan, configuration, etc. I could very well have unintentionally left-off crucial parts of my descriptions below and for that I apologize for wasting time -