Re: [PERFORM] Typecast bug?

2008-06-25 Thread Craig James
Tom Lane wrote: Craig James <[EMAIL PROTECTED]> writes: This seems like a bug to me, but it shows up as a performance problem. emol_warehouse_1=> explain analyze select version_id, parent_id from version where version_id = 999; If you actually *need* so many 9's her

Re: [PERFORM] Typecast bug?

2008-06-25 Thread Tom Lane
Craig James <[EMAIL PROTECTED]> writes: > This seems like a bug to me, but it shows up as a performance problem. > emol_warehouse_1=> explain analyze select version_id, parent_id from version > where version_id = 999; If you actually *need* so many 9's here as to force it

Re: [PERFORM] RAID 10 Benchmark with different I/O schedulers

2008-06-25 Thread Craig James
This seems like a bug to me, but it shows up as a performance problem. Since the column being queried is an integer, the second query (see below) can't possibly match, yet Postgres uses a typecast, forcing a full table scan for a value that can't possibly be in the table. The application coul

[PERFORM] Typecast bug?

2008-06-25 Thread Craig James
This seems like a bug to me, but it shows up as a performance problem. Since the column being queried is an integer, the second query (see below) can't possibly match, yet Postgres uses a typecast, forcing a full table scan for a value that can't possibly be in the table. The application coul

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Henrik
25 jun 2008 kl. 17.45 skrev Greg Smith: On Wed, 25 Jun 2008, Henrik wrote: Would you turn off fsync if you had a controller with BBU? =) Turning off fsync has some potential to introduce problems even in that environment, so better not to do that. The issue is that you might have, say,

[PERFORM] Query Planner not choosing hash_aggregate appropriately.

2008-06-25 Thread Scott Carey
The query optimizer fails to use a hash aggregate most of the time. This is an inconsistent behavior -- the queries below were happily using hash_aggregate on a previous pg_restore from the data. On one particular class of tables this is especially painful. The example table has 25 million rows,

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Merlin Moncure wrote: So, based on this and other experiences, i'm starting to be more partial to linux distributions with faster moving kernels, mainly because i trust the kernel drivers more than the vendor provided drivers. Depends on how fast. I find it takes a minim

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: I refrained from saying in my reply that I would set up a firewire-based link to ram in a spare old portable (which comes with a battery) if I wanted to do this cheaply. Maybe, but this is kind of a weird setup. Not many people are going to run a p

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:07:25PM -0500, Kevin Grittner wrote: > > It doesn't have to be free software to cut that way. I've actually > found the free software to waste less of my time. No question. But one of the unfortunate facts of the no-charge-for-licenses world is that many people exp

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Kevin Grittner
>>> Andrew Sullivan <[EMAIL PROTECTED]> wrote: > this is a blade that cuts both ways, and the key thing to do is > to ensure you have good testing infrastructure in place to check that > things will work before you deploy to production. (The other way to > say that, of course, is "Linux is only

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
On Wed, Jun 25, 2008 at 01:35:49PM -0400, Merlin Moncure wrote: > experiences, i'm starting to be more partial to linux distributions > with faster moving kernels, mainly because i trust the kernel drivers > more than the vendor provided drivers. While I have some experience that agrees with this,

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 9:03 AM, Matthew Wakeling <[EMAIL PROTECTED]> wrote: > On Wed, 25 Jun 2008, Merlin Moncure wrote: >>> >>> Has anyone done some benchmarks between hardware RAID vs Linux MD >>> software >>> RAID? >> >> I have here: >> >> http://merlinmoncure.blogspot.com/2007/08/following-are

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 11:55 AM, Joshua D. Drake <[EMAIL PROTECTED]> wrote: > On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: >> On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey >> <[EMAIL PROTECTED]> wrote: > >> I'm currently having a problem with a "well known very large >> servermanufactu

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Peter T. Breuer
"Also sprach Greg Smith:" > On Wed, 25 Jun 2008, Peter T. Breuer wrote: > > > You can put the log/bitmap wherever you want in software raid, including > > on a battery-backed local ram disk if you feel so inclined. So there is > > no intrinsic advantage to be gained there at all. > > You are t

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 09:53 -0600, Scott Marlowe wrote: > On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey > <[EMAIL PROTECTED]> wrote: > > Hi > I'm currently having a problem with a "well known very large > servermanufacturer who shall remain unnamed" and their semi-custom > RAID controller firmw

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Scott Marlowe
On Wed, Jun 25, 2008 at 5:05 AM, Adrian Moisey <[EMAIL PROTECTED]> wrote: > Hi > > Has anyone done some benchmarks between hardware RAID vs Linux MD software > RAID? > > I'm keen to know the result. I've had good performance from sw RAID-10 in later kernels, especially if it was handling a mostly

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Henrik wrote: Would you turn off fsync if you had a controller with BBU? =) Turning off fsync has some potential to introduce problems even in that environment, so better not to do that. The issue is that you might have, say, 1GB of OS-level cache but 256MB of BBU cache

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, at a transfer rate of 100MB/s, and doesn't degrade much in p

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Joshua D. Drake
On Wed, 2008-06-25 at 11:30 -0400, Jonah H. Harris wrote: > On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith <[EMAIL PROTECTED]> wrote: > > SMART doesn't detect 100% of drive failures in advance, but you'd be silly > > to setup a database system where you don't get to take advantage of the > > ~50% i

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Jonah H. Harris
On Wed, Jun 25, 2008 at 11:24 AM, Greg Smith <[EMAIL PROTECTED]> wrote: > SMART doesn't detect 100% of drive failures in advance, but you'd be silly > to setup a database system where you don't get to take advantage of the > ~50% it does catch before you lose any data. Can't argue with that one.

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Peter T. Breuer wrote: You can put the log/bitmap wherever you want in software raid, including on a battery-backed local ram disk if you feel so inclined. So there is no intrinsic advantage to be gained there at all. You are technically correct but this is irrelevant.

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, Henrik wrote: 4 x 146 GB SAS disk in RAID 1+0 for database 6 x 750 GB SATA disks in RAID 1+0 or RAID 5 for OS and transactions logs. The transaction logs are not that big, and there's very little value to striping them across even two disks. You should just get more SAS

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Greg Smith
On Wed, 25 Jun 2008, jay wrote: Why postgresql is so slowly? Is the PG MVCC problem? Update is extremely intensive not just because of MVCC, but because a new version of all the rows are being written out. This forces both lots of database commits and lots of complicated disk I/O to accompl

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Peter T. Breuer
"Also sprach Matthew Wakeling:" > >> Has anyone done some benchmarks between hardware RAID vs Linux MD software > >> RAID? ... > > The upshot is I don't really see a difference in performance. > > The main difference is that you can get hardware RAID with > battery-backed-up cache, which means

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Merlin Moncure wrote: Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I have here: http://merlinmoncure.blogspot.com/2007/08/following-are-results-of-our-testing-of.html The upshot is I don't really see a difference in performance. The ma

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Merlin Moncure
On Wed, Jun 25, 2008 at 7:05 AM, Adrian Moisey <[EMAIL PROTECTED]> wrote: > Has anyone done some benchmarks between hardware RAID vs Linux MD software > RAID? > > I'm keen to know the result. I have here: http://merlinmoncure.blogspot.com/2007/08/following-are-results-of-our-testing-of.html I als

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Henrik wrote: Would you turn off fsync if you had a controller with BBU? =) No, certainly not. Fsync is what makes the data move from the volatile OS cache to the non-volatile disc system. It'll just be a lot quicker on a controller with a BBU cache, because it won't need

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Claus Guttesen
>> If you have a good RAID controller with BBU cache, then there's no point >> splitting the discs into two sets. You're only creating an opportunity to >> under-utilise the system. I'd get ten identical discs and put them in a >> single array, probably RAID 10. > > OK, thats good to know. Really w

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Henrik
25 jun 2008 kl. 13.15 skrev Matthew Wakeling: On Wed, 25 Jun 2008, Henrik wrote: What are your suggestions. What we are currently looking at is. Dual Quad Core Intel 8 - 12 GB RAM More RAM would be helpful. It's not that expensive, compared to the rest of your system. True, as long as I

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Henrik
25 jun 2008 kl. 12.56 skrev Claus Guttesen: We have a database with lots of small simultaneous writes and reads (millions every day) and are looking at buying a good hardware for this. What are your suggestions. What we are currently looking at is. Dual Quad Core Intel 8 - 12 GB RAM 10 di

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Matthew Wakeling
On Wed, 25 Jun 2008, Henrik wrote: What are your suggestions. What we are currently looking at is. Dual Quad Core Intel 8 - 12 GB RAM More RAM would be helpful. It's not that expensive, compared to the rest of your system. 10 disks total. 4 x 146 GB SAS disk in RAID 1+0 for database 6 x 7

[PERFORM] Hardware vs Software RAID

2008-06-25 Thread Adrian Moisey
Hi Has anyone done some benchmarks between hardware RAID vs Linux MD software RAID? I'm keen to know the result. -- Adrian Moisey Systems Administrator | CareerJunction | Your Future Starts Here. Web: www.careerjunction.co.za | Email: [EMAIL PROTECTED] Phone: +27 21 686 6820 | Mobile: +27 82

Re: [PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Claus Guttesen
> We have a database with lots of small simultaneous writes and reads > (millions every day) and are looking at buying a good hardware for this. > > What are your suggestions. What we are currently looking at is. > > Dual Quad Core Intel > 8 - 12 GB RAM > > 10 disks total. > > 4 x 146 GB SAS disk i

[PERFORM] Hardware suggestions for high performance 8.3

2008-06-25 Thread Henrik
Hi list, We have a database with lots of small simultaneous writes and reads (millions every day) and are looking at buying a good hardware for this. What are your suggestions. What we are currently looking at is. Dual Quad Core Intel 8 - 12 GB RAM 10 disks total. 4 x 146 GB SAS disk in R

Re: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread Heikki Linnakangas
jay wrote: I've a table with about 34601755 rows ,when I execute 'update msg_table set type=0;' is very very slow, cost several hours, but still not complete? Why postgresql is so slowly? Is the PG MVCC problem? Possibly. Because of MVCC, a full-table update will actually create a new versio

Re: [PERFORM] PostgreSQL and Ruby on Rails - better accessibility

2008-06-25 Thread Nikhils
Hi, i, Could anybody comment on the postgres-pr driver, from performance point > of view, is it faster than others? > I guess, a more appropriate place to check out for ruby/rails postgres drivers would be rubyforge.org itself. There is a libpq based postgres driver available there (ruby-postgres)

[PERFORM] PostgreSQL and Ruby on Rails - better accessibility

2008-06-25 Thread Amol Pujari
Hi, Could anybody comment on the postgres-pr driver, from performance point of view, is it faster than others? What other options are available to access postgresql in ruby/ruby on rails? which of them is most popular, better? regards Amol DISCLAIMER == This e-mail may contain privile

[PERFORM] 答复: [PERFORM] Postgresql update op is very very slow

2008-06-25 Thread jay
Hi Rusty, The "type" is not in a index. The number of checkpoint segement is 64 and PG version is 8.3.3 After turn on log, I found something about checkpoints. LOG: 0: checkpoint complete: wrote 174943 buffers (26.7%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=20