Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Ron
At 03:28 PM 1/8/2009, Merlin Moncure wrote: On Thu, Jan 8, 2009 at 9:42 AM, Stefano Nichele wrote: > Merlin Moncure wrote: >> IIRC that's the 'perc 6ir' card...no write caching. You are getting >> killed with syncs. If you can restart the database, you can test with >> fsync=off comparing load

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Scott Marlowe
On Sat, Jan 10, 2009 at 5:40 AM, Ron wrote: > At 03:28 PM 1/8/2009, Merlin Moncure wrote: >> just be aware of the danger . hard reset (power off) class of failure >> when fsync = off means you are loading from backups. > > That's what redundant power conditioning UPS's are supposed to help preven

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Gregory Stark
"Scott Marlowe" writes: > On Sat, Jan 10, 2009 at 5:40 AM, Ron wrote: >> At 03:28 PM 1/8/2009, Merlin Moncure wrote: >>> just be aware of the danger . hard reset (power off) class of failure >>> when fsync = off means you are loading from backups. >> >> That's what redundant power conditioning

[PERFORM] block device benchmarking

2009-01-10 Thread Markus Wanner
Hi, I'm fiddling with a hand-made block device based benchmarking thingie, which I want to run random reads and writes of relatively small blocks (somewhat similar to databases). I'm much less interested in measuring throughput, but rather in latency. Besides varying block sizes, I'm also testing

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sat, 10 Jan 2009, Gregory Stark wrote: ...and of course, those lucky few with bigger budgets can use SSD's and not care what fsync is set to. Would that prevent any corruption if the writes got out of order because of lack of fsync? Or partial writes? Or wouldn't fsync still need to be tu

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Markus Wanner
Hi, da...@lang.hm wrote: > On Sat, 10 Jan 2009, Gregory Stark wrote: >> I think the idea is that with SSDs or a RAID with a battery backed >> cache you >> can leave fsync on and not have any significant performance hit since >> the seek >> times are very fast for SSD. They have limited bandwidth b

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Scott Marlowe
On Sat, Jan 10, 2009 at 12:00 PM, Markus Wanner wrote: > Hi, > > da...@lang.hm wrote: >> On Sat, 10 Jan 2009, Gregory Stark wrote: >>> I think the idea is that with SSDs or a RAID with a battery backed >>> cache you >>> can leave fsync on and not have any significant performance hit since >>> the

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sat, 10 Jan 2009, Markus Wanner wrote: da...@lang.hm wrote: On Sat, 10 Jan 2009, Gregory Stark wrote: I think the idea is that with SSDs or a RAID with a battery backed cache you can leave fsync on and not have any significant performance hit since the seek times are very fast for SSD. They

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Gregory Stark
da...@lang.hm writes: > On Sat, 10 Jan 2009, Markus Wanner wrote: > >> My understanding of SSDs so far is, that they are not that bad at >> writing *on average*, but to perform wear-leveling, they sometimes have >> to shuffle around multiple blocks at once. So there are pretty awful >> spikes for

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sat, 10 Jan 2009, Gregory Stark wrote: da...@lang.hm writes: On Sat, 10 Jan 2009, Markus Wanner wrote: My understanding of SSDs so far is, that they are not that bad at writing *on average*, but to perform wear-leveling, they sometimes have to shuffle around multiple blocks at once. So th

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Ron
At 10:36 AM 1/10/2009, Gregory Stark wrote: "Scott Marlowe" writes: > On Sat, Jan 10, 2009 at 5:40 AM, Ron wrote: >> At 03:28 PM 1/8/2009, Merlin Moncure wrote: >>> just be aware of the danger . hard reset (power off) class of failure >>> when fsync = off means you are loading from backups. >

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sat, 10 Jan 2009, Ron wrote: At 10:36 AM 1/10/2009, Gregory Stark wrote: "Scott Marlowe" writes: > On Sat, Jan 10, 2009 at 5:40 AM, Ron wrote: >> At 03:28 PM 1/8/2009, Merlin Moncure wrote: >>> just be aware of the danger . hard reset (power off) class of failure >>> when fsync = off mea

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Gregory Stark
Ron writes: > At 10:36 AM 1/10/2009, Gregory Stark wrote: >> >> Or a system crash. If the kernel panics for any reason when it has dirty >> buffers in memory the database will need to be restored. > > A power conditioning UPS should prevent a building wide or circuit level bad > power event Exce

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sat, 10 Jan 2009, Luke Lonergan wrote: The new MLC based SSDs have better wear leveling tech and don't suffer the pauses. Intel X25-M 80 and 160 GB SSDs are both pause-free. See Anandtech's test results for details. they don't suffer the pauses, but they still don't have fantasic write

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Luke Lonergan
The new MLC based SSDs have better wear leveling tech and don't suffer the pauses. Intel X25-M 80 and 160 GB SSDs are both pause-free. See Anandtech's test results for details. Intel's SLC SSDs should also be good enough but they're smaller. - Luke - Original Message - From: pgsql-pe

Re: [PERFORM] block device benchmarking

2009-01-10 Thread M. Edward (Ed) Borasky
Markus Wanner wrote: > Hi, > > I'm fiddling with a hand-made block device based benchmarking thingie, > which I want to run random reads and writes of relatively small blocks > (somewhat similar to databases). I'm much less interested in measuring > throughput, but rather in latency. Besides varyi

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Luke Lonergan
I believe they write at 200MB/s which is outstanding for sequential BW. Not sure about the write latency, though the Anandtech benchmark results showed high detail and IIRC the write latencies were very good. - Luke - Original Message - From: da...@lang.hm To: Luke Lonergan Cc: st...@

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread Mark Kirkwood
da...@lang.hm wrote: On Sat, 10 Jan 2009, Luke Lonergan wrote: The new MLC based SSDs have better wear leveling tech and don't suffer the pauses. Intel X25-M 80 and 160 GB SSDs are both pause-free. See Anandtech's test results for details. they don't suffer the pauses, but they still don't

Re: [PERFORM] understanding postgres issues/bottlenecks

2009-01-10 Thread david
On Sun, 11 Jan 2009, Mark Kirkwood wrote: da...@lang.hm wrote: On Sat, 10 Jan 2009, Luke Lonergan wrote: The new MLC based SSDs have better wear leveling tech and don't suffer the pauses. Intel X25-M 80 and 160 GB SSDs are both pause-free. See Anandtech's test results for details. they d