Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Simon Waters
On Wednesday 07 January 2009 04:17:10 M. Edward (Ed) Borasky wrote: > > 1. The package it lives in is called "sysstat". Most Linux distros do > *not* install "sysstat" by default. Somebody should beat up on them > about that. :) Hehe, although sysstat and friends did have issues on Linux for a lo

[PERFORM] Sl_log_1 and sl_log_2 not getting truncated.

2009-01-07 Thread Nimesh Satam
Hi, We have recently installed slony and tsrted replication on one of our test machines. When we start inserting data in to the replicated database, the replication is taking properly. Over a period of time the lag increases between the two database. Looking further we found that, sl_log_1 and sl

[PERFORM] Fwd: Casting issue!!

2009-01-07 Thread jose fuenmayor
-- Forwarded message -- From: jose fuenmayor Date: Wed, Jan 7, 2009 at 2:56 PM Subject: Casting issue!! To: psql-ad...@postgresql.org Hi all I am trying to migrate from postgresql 8.2.x to 8.3.x, i have an issue with casting values when i try to perform the auto cast , it does n

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread Matthew Wakeling
On Wed, 7 Jan 2009, jose fuenmayor wrote: Hi all I am trying to migrate from postgresql 8.2.x to  8.3.x, i have an issue with casting values when i try to perform the auto cast , it does not work and I get an error, how can i perform  auto casting on 8.3 without rewrite my source code, I am usi

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread tv
Hi Jose, why haven't you post an example of the failing query, data and the exact error message? The casting should work on 8.3 (and it works for me) so I guess there are some invalid data, invalid SQL or something like that. Anyway I doubt this is a performance issue - this falls into generic SQ

Re: [PERFORM] Fwd: Casting issue!!

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, jose fuenmayor wrote: > > Hi all I am trying to migrate from postgresql 8.2.x to > 8.3.x, i have an > issue with casting values when i try to perform the auto > cast , it does not > work and I get an error, how can i perform auto casting on > 8.3 without > rewrite my sour

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Simon Waters wrote: > On Wednesday 07 January 2009 04:17:10 M. Edward (Ed) Borasky wrote: >> 1. The package it lives in is called "sysstat". Most Linux distros do >> *not* install "sysstat" by default. Somebody should beat up on them >> about that. :) > > Hehe, although sysstat and friends did hav

Re: [PERFORM] Sl_log_1 and sl_log_2 not getting truncated.

2009-01-07 Thread Brad Nicholson
On Wed, 2009-01-07 at 18:18 +0530, Nimesh Satam wrote: > Hi, > > We have recently installed slony and tsrted replication on one of our > test machines. When we start inserting data in to the replicated > database, the replication is taking properly. Over a period of time > the lag increases betwee

[PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Dmitry Koterov
Hello. Suppose I perform 1000 RANDOM writes into a file. These writes are saved into Linux writeback buffer and are flushed to the disc asynchronously, that's OK. The question is: will physical writes be performed later in the sequence of physical SECTOR position on the disc (minimizing head seek

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread david
On Wed, 7 Jan 2009, Dmitry Koterov wrote: Hello. Suppose I perform 1000 RANDOM writes into a file. These writes are saved into Linux writeback buffer and are flushed to the disc asynchronously, that's OK. The question is: will physical writes be performed later in the sequence of physical SECT

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Stefano Nichele
Ok, here some information: OS: Centos 5.x (Linux 2.6.18-53.1.21.el5 #1 SMP Tue May 20 09:34:18 EDT 2008 i686 i686 i386 GNU/Linux) RAID: it's a hardware RAID controller The disks are 9600rpm SATA drives (6 disk 1+0 RAID array and 2 separate disks for the OS). About iostat (on sdb I have pg_xl

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Stefano Nichele
On Tue, Jan 6, 2009 at 7:45 PM, Scott Marlowe wrote: > I concur with Merlin you're I/O bound. > > Adding to his post, what RAID controller are you running, does it have > cache, does the cache have battery backup, is the cache set to write > back or write through? At the moment I don't have such

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 2:02 PM, Stefano Nichele wrote: > > On Tue, Jan 6, 2009 at 7:45 PM, Scott Marlowe > wrote: >> >> I concur with Merlin you're I/O bound. >> >> Adding to his post, what RAID controller are you running, does it have >> cache, does the cache have battery backup, is the cache se

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
Just to elaborate on the horror that is a Dell perc5e. We have one in a 1950 with battery backed cache (256 Meg I think). It has an 8 disk 500Gig SATA drive RAID-10 array and 4 1.6GHz cpus and 10 Gigs ram. This server currently serves as a mnogo search server. Here's what vmstat 1 looks like dur

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Dmitry Koterov
OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to the maximum possible value reported by bonnie++ (or simple dd) utility. S

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Greg Smith
On Wed, 7 Jan 2009, Scott Marlowe wrote: I cannot understand how Dell stays in business. There's a continuous stream of people who expect RAID5 to perform well, too, yet this surprises you? -- * Greg Smith gsm...@gregsmith.com http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-perfor

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 3:34 PM, Greg Smith wrote: > On Wed, 7 Jan 2009, Scott Marlowe wrote: > >> I cannot understand how Dell stays in business. > > There's a continuous stream of people who expect RAID5 to perform well, too, > yet this surprises you? I guess I've underestimated the human capaci

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread david
On Thu, 8 Jan 2009, Dmitry Koterov wrote: OK, thank you. Now - PostgreSQL-related question. If the system reorders writes to minimize seeking, I suppose that in heavy write-loaded PostgreSQL instalation dstat (or iostat) realtime write statistics should be close to the maximum possible value re

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, Scott Marlowe wrote: > The really bad news is that > you can't > generally plug in a real RAID controller on a Dell. We put > an Areca > 168-LP PCI-x8 in one of our 1950s and it wouldn't even > turn on, got a > CPU Error. > Hmm, I had to pull the perc5i's out of our dell s

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread Greg Smith
On Wed, 7 Jan 2009, Dmitry Koterov wrote: The question is: will physical writes be performed later in the sequence of physical SECTOR position on the disc (minimizing head seeking)? Or Linux background writer knows nothing about physical on-disc placement and flushes data in order it is saved in

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Glyn Astill
--- On Wed, 7/1/09, Scott Marlowe wrote: > Just to elaborate on the horror that is a Dell perc5e. We > have one in > a 1950 with battery backed cache (256 Meg I think). It has > an 8 disk > 500Gig SATA drive RAID-10 array and 4 1.6GHz cpus and 10 > Gigs ram. Our perc5i controllers performed be

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 4:36 PM, Glyn Astill wrote: > --- On Wed, 7/1/09, Scott Marlowe wrote: > >> Just to elaborate on the horror that is a Dell perc5e. We >> have one in >> a 1950 with battery backed cache (256 Meg I think). It has >> an 8 disk >> 500Gig SATA drive RAID-10 array and 4 1.6GHz

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Bborie Park
Since the discussion involves Dell PERC controllers, does anyone know if the performance of LSI cards (those with the same chipsets as Dell) also have similarly poor performance? I have a LSI ELP card, so would like to know what other people's experiences are... -bborie Scott Marlowe wr

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Carey
If you're stuck with a Dell, the Adaptec 5 series works, I'm using 5085's in a pair and get 1200 MB/sec streaming reads best case with 20 SATA drives in RAID 10 (2 sets of 10, software raid 0 on top). Of course, Dell doesn't like you putting in somebody else's RAID card, but they support the r

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Merlin Moncure
On Wed, Jan 7, 2009 at 7:11 PM, Scott Carey wrote: > If you're stuck with a Dell, the Adaptec 5 series works, I'm using 5085's in > a pair and get 1200 MB/sec streaming reads best case with 20 SATA drives in > RAID 10 (2 sets of 10, software raid 0 on top). Of course, Dell doesn't > like you put

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Marlowe
On Wed, Jan 7, 2009 at 6:19 PM, Merlin Moncure wrote: RE: Perc raid controllers > Unfortunately switching the disks to jbod and going software > raid doesn't seem to help much. The biggest problem with dell Yeah, I noticed that too when I was trying to get a good config from the perc 5e. Also

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread Scott Carey
> Sequential read performance means precisely squat for most database > loads. Depends on the database workload. Many queries for me may scan 50GB of data for aggregation. Besides, it is a good test for making sure your RAID card doesn't suck. Especially running tests with sequential access C

Re: [PERFORM] Are random writes optimized sequentially by Linux kernel?

2009-01-07 Thread M. Edward (Ed) Borasky
da...@lang.hm wrote: > On Thu, 8 Jan 2009, Dmitry Koterov wrote: > >> OK, thank you. >> >> Now - PostgreSQL-related question. If the system reorders writes to >> minimize >> seeking, I suppose that in heavy write-loaded PostgreSQL instalation >> dstat >> (or iostat) realtime write statistics shoul

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-07 Thread M. Edward (Ed) Borasky
Scott Marlowe wrote: > On Wed, Jan 7, 2009 at 3:34 PM, Greg Smith wrote: >> On Wed, 7 Jan 2009, Scott Marlowe wrote: >> >>> I cannot understand how Dell stays in business. >> There's a continuous stream of people who expect RAID5 to perform well, too, >> yet this surprises you? > > I guess I've u