Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Simon Waters
On Wednesday 07 January 2009 04:17:10 M. Edward (Ed) Borasky wrote: 1. The package it lives in is called sysstat. Most Linux distros do *not* install sysstat by default. Somebody should beat up on them about that. :) Hehe, although sysstat and friends did have issues on Linux for a long

[PERFORM] Sl_log_1 and sl_log_2 not getting truncated.

2009-01-07 Thread Nimesh Satam
Hi, We have recently installed slony and tsrted replication on one of our test machines. When we start inserting data in to the replicated database, the replication is taking properly. Over a period of time the lag increases between the two database. Looking further we found that, sl_log_1 and

[PERFORM] Fwd: Casting issue!!

2009-01-07 Thread jose fuenmayor
-- Forwarded message -- From: jose fuenmayor jaf...@gmail.com Date: Wed, Jan 7, 2009 at 2:56 PM Subject: Casting issue!! To: psql-ad...@postgresql.org Hi all I am trying to migrate from postgresql 8.2.x to 8.3.x, i have an issue with casting values when i try to perform the auto

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread Matthew Wakeling
On Wed, 7 Jan 2009, jose fuenmayor wrote: Hi all I am trying to migrate from postgresql 8.2.x to  8.3.x, i have an issue with casting values when i try to perform the auto cast , it does not work and I get an error, how can i perform  auto casting on 8.3 without rewrite my source code, I am

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread tv
Hi Jose, why haven't you post an example of the failing query, data and the exact error message? The casting should work on 8.3 (and it works for me) so I guess there are some invalid data, invalid SQL or something like that. Anyway I doubt this is a performance issue - this falls into generic

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, jose fuenmayor jaf...@gmail.com wrote: Hi all I am trying to migrate from postgresql 8.2.x to 8.3.x, i have an issue with casting values when i try to perform the auto cast , it does not work and I get an error, how can i perform auto casting on 8.3 without

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Simon Waters wrote: On Wednesday 07 January 2009 04:17:10 M. Edward (Ed) Borasky wrote: 1. The package it lives in is called sysstat. Most Linux distros do *not* install sysstat by default. Somebody should beat up on them about that. :) Hehe, although sysstat and friends did have issues on

Re: [PERFORM] Sl_log_1 and sl_log_2 not getting truncated.

2009-01-07 Thread Brad Nicholson
On Wed, 2009-01-07 at 18:18 +0530, Nimesh Satam wrote: Hi, We have recently installed slony and tsrted replication on one of our test machines. When we start inserting data in to the replicated database, the replication is taking properly. Over a period of time the lag increases between the

[PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Dmitry Koterov
Hello. Suppose I perform 1000 RANDOM writes into a file. These writes are saved into Linux writeback buffer and are flushed to the disc asynchronously, that's OK. The question is: will physical writes be performed later in the sequence of physical SECTOR position on the disc (minimizing head

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread david
On Wed, 7 Jan 2009, Dmitry Koterov wrote: Hello. Suppose I perform 1000 RANDOM writes into a file. These writes are saved into Linux writeback buffer and are flushed to the disc asynchronously, that's OK. The question is: will physical writes be performed later in the sequence of physical

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Stefano Nichele
Ok, here some information: OS: Centos 5.x (Linux 2.6.18-53.1.21.el5 #1 SMP Tue May 20 09:34:18 EDT 2008 i686 i686 i386 GNU/Linux) RAID: it's a hardware RAID controller The disks are 9600rpm SATA drives (6 disk 1+0 RAID array and 2 separate disks for the OS). About iostat (on sdb I have

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Stefano Nichele
On Tue, Jan 6, 2009 at 7:45 PM, Scott Marlowe scott.marl...@gmail.comwrote: I concur with Merlin you're I/O bound. Adding to his post, what RAID controller are you running, does it have cache, does the cache have battery backup, is the cache set to write back or write through? At the

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 2:02 PM, Stefano Nichele stefano.nich...@gmail.com wrote: On Tue, Jan 6, 2009 at 7:45 PM, Scott Marlowe scott.marl...@gmail.com wrote: I concur with Merlin you're I/O bound. Adding to his post, what RAID controller are you running, does it have cache, does the cache

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
Just to elaborate on the horror that is a Dell perc5e. We have one in a 1950 with battery backed cache (256 Meg I think). It has an 8 disk 500Gig SATA drive RAID-10 array and 4 1.6GHz cpus and 10 Gigs ram. This server currently serves as a mnogo search server. Here's what vmstat 1 looks like

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Dmitry Koterov
OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to the maximum possible value reported by bonnie++ (or simple dd) utility.

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Greg Smith
On Wed, 7 Jan 2009, Scott Marlowe wrote: I cannot understand how Dell stays in business. There's a continuous stream of people who expect RAID5 to perform well, too, yet this surprises you? -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 3:34 PM, Greg Smith gsm...@gregsmith.com wrote: On Wed, 7 Jan 2009, Scott Marlowe wrote: I cannot understand how Dell stays in business. There's a continuous stream of people who expect RAID5 to perform well, too, yet this surprises you? I guess I've underestimated

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread david
On Thu, 8 Jan 2009, Dmitry Koterov wrote: OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to the maximum possible value

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, Scott Marlowe scott.marl...@gmail.com wrote: The really bad news is that you can't generally plug in a real RAID controller on a Dell. We put an Areca 168-LP PCI-x8 in one of our 1950s and it wouldn't even turn on, got a CPU Error. Hmm, I had to pull the perc5i's

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Greg Smith
On Wed, 7 Jan 2009, Dmitry Koterov wrote: The question is: will physical writes be performed later in the sequence of physical SECTOR position on the disc (minimizing head seeking)? Or Linux background writer knows nothing about physical on-disc placement and flushes data in order it is saved

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, Scott Marlowe scott.marl...@gmail.com wrote: Just to elaborate on the horror that is a Dell perc5e. We have one in a 1950 with battery backed cache (256 Meg I think). It has an 8 disk 500Gig SATA drive RAID-10 array and 4 1.6GHz cpus and 10 Gigs ram. Our perc5i

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 4:36 PM, Glyn Astill glynast...@yahoo.co.uk wrote: --- On Wed, 7/1/09, Scott Marlowe scott.marl...@gmail.com wrote: Just to elaborate on the horror that is a Dell perc5e. We have one in a 1950 with battery backed cache (256 Meg I think). It has an 8 disk 500Gig SATA

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Bborie Park
Since the discussion involves Dell PERC controllers, does anyone know if the performance of LSI cards (those with the same chipsets as Dell) also have similarly poor performance? I have a LSI ELP card, so would like to know what other people's experiences are... -bborie Scott Marlowe

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Carey
If you're stuck with a Dell, the Adaptec 5 series works, I'm using 5085's in a pair and get 1200 MB/sec streaming reads best case with 20 SATA drives in RAID 10 (2 sets of 10, software raid 0 on top). Of course, Dell doesn't like you putting in somebody else's RAID card, but they support the

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Merlin Moncure
On Wed, Jan 7, 2009 at 7:11 PM, Scott Carey sc...@richrelevance.com wrote: If you're stuck with a Dell, the Adaptec 5 series works, I'm using 5085's in a pair and get 1200 MB/sec streaming reads best case with 20 SATA drives in RAID 10 (2 sets of 10, software raid 0 on top). Of course, Dell

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 6:19 PM, Merlin Moncure mmonc...@gmail.com wrote: RE: Perc raid controllers Unfortunately switching the disks to jbod and going software raid doesn't seem to help much. The biggest problem with dell Yeah, I noticed that too when I was trying to get a good config from

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Carey
Sequential read performance means precisely squat for most database loads. Depends on the database workload. Many queries for me may scan 50GB of data for aggregation. Besides, it is a good test for making sure your RAID card doesn't suck. Especially running tests with sequential access

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread M. Edward (Ed) Borasky
da...@lang.hm wrote: On Thu, 8 Jan 2009, Dmitry Koterov wrote: OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Scott Marlowe wrote: On Wed, Jan 7, 2009 at 3:34 PM, Greg Smith gsm...@gregsmith.com wrote: On Wed, 7 Jan 2009, Scott Marlowe wrote: I cannot understand how Dell stays in business. There's a continuous stream of people who expect RAID5 to perform well, too, yet this surprises you? I guess