Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Magnus Hagander
On Tue, Feb 27, 2007 at 06:36:11PM -0500, Charles Sprickman wrote: > On Tue, 27 Feb 2007, Dave Page wrote: > > >Magnus Hagander wrote: > >> > >>Just as a datapoint, we did try to use mnogosearch for the > >>postgresql.org website+archives search, and it fell over completely. > >>Indexing took way

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread Geoff Tolley
Joe Uhl wrote: [1] What is the performance penalty of software raid over hardware raid? Is it truly significant? We will be working with 100s of GB to 1-2 TB of data eventually. One thing you should appreciate about hw vs sw raid is that with the former you can battery-back it and enable co

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Charles Sprickman
On Tue, 27 Feb 2007, Dave Page wrote: Magnus Hagander wrote: Just as a datapoint, we did try to use mnogosearch for the postgresql.org website+archives search, and it fell over completely. Indexing took way too long, and we had search times several thousand times longer than with tsearch2. Th

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Chris
Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the que

Re: [PERFORM] Two hard drives --- what to do with them?

2007-02-27 Thread Scott Marlowe
On Tue, 2007-02-27 at 13:23, Jeff Davis wrote: > Also, put things in context. The chances of failure due to these kinds > of things are fairly low. If it's more likely that someone spills coffee > on your server than the UPS fails, it doesn't make sense to spend huge > amounts of money on NVRAM (or

Re: [PERFORM] Two hard drives --- what to do with them?

2007-02-27 Thread Jeff Davis
On Tue, 2007-02-27 at 09:27 +0100, Peter Kovacs wrote: > I wonder if running a machine on a UPS + 1 hot standby internal PS is > equivalent, in terms of data integrity, to using battery backed write > cache. Instinctively, I'd think that UPS + 1 hot standby internal PS > is better, since this setup

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread Scott Marlowe
On Tue, 2007-02-27 at 12:28, Joe Uhl wrote: > Really appreciate all of the valuable input. The current server has the > Perc4ei controller. > > The impression I am taking from the responses is that we may be okay with > software raid, especially if raid 1 and 10 are what we intend to use. > > I

Re: [PERFORM] [EMAIL PROTECTED]: Anyone interested in improving postgresql scaling?]

2007-02-27 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Ok, looks like the FreeBSD community is interested in PostgreSQL > performance, or at least enough to investigate it. I think this guy is barking up the wrong tree though, because we don't ever have multiple processes sleeping on the same sema. Not tha

Re: [PERFORM] Two hard drives --- what to do with them?

2007-02-27 Thread Shane Ambler
Peter Kovacs wrote: > The reason this becomes an issue is that many consumer-grade disks have > write cache enabled by default and no way to make sure the cached data > actually gets written. So, essentially, these disks "lie" and say they > wrote the data, when in reality, it's in volatile memo

[PERFORM] [EMAIL PROTECTED]: Anyone interested in improving postgresql scaling?]

2007-02-27 Thread Jim C. Nasby
Ok, looks like the FreeBSD community is interested in PostgreSQL performance, or at least enough to investigate it. Anyone here have experience hacking on FreeBSD? - Forwarded message from Kris Kennaway <[EMAIL PROTECTED]> - X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on noel

[PERFORM] [EMAIL PROTECTED]: Progress on scaling of FreeBSD on 8 CPU systems]

2007-02-27 Thread Jim C. Nasby
Thought I'd pass this along, since the Linux vs FreeBSD performance question comes up fairly regularly... BTW, I've already asked about benchmarking with PostgreSQL, so please don't go over there making trouble. :) - Forwarded message from Kris Kennaway <[EMAIL PROTECTED]> - X-Spam-Check

Re: [PERFORM] Postgres performance Linux vs FreeBSD

2007-02-27 Thread Andrew - Supernews
On 2007-02-21, Mark Kirkwood <[EMAIL PROTECTED]> wrote: > With respect to 'select count(*) from ...' being slower on FreeBSD, > there are a number of things to try to make FreeBSD faster for this sort > of query. Two I'm currently using are: > > - setting sysctl vfs.read_max to 16 or 32 > - rebui

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread Scott Marlowe
On Tue, 2007-02-27 at 07:12, Joe Uhl wrote: > We have been running Postgres on a 2U server with 2 disks configured in > raid 1 for the os and logs and 4 disks configured in raid 10 for the > data. I have since been told raid 5 would have been a better option > given our usage of Dell equipment and

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread mark
Hope you don't mind, Ron. This might be splitting hairs. On Tue, Feb 27, 2007 at 11:05:39AM -0500, Ron wrote: > The real CPU overhead when using SW RAID is when using any form of SW > RAID that does XOR operations as part of writes (RAID 5, 6, 50, ..., > etc). At that point, you are essentially

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread Ron
At 08:12 AM 2/27/2007, Joe Uhl wrote: We have been running Postgres on a 2U server with 2 disks configured in raid 1 for the os and logs and 4 disks configured in raid 10 for the data. I have since been told raid 5 would have been a better option given our usage of Dell equipment and the way the

Re: [PERFORM] Two hard drives --- what to do with them?

2007-02-27 Thread Ben
Just remember that batteries (in both RAID cards and UPSes) wear out and will eventually have to be replaced. It depends how critical your data is, but if you only have a UPS, you risk badness in the off chance that your power fails and you haven't replaced your UPS battery. On Feb 27, 2007

Re: [PERFORM] Opinions on Raid

2007-02-27 Thread Stefan Kaltenbrunner
Joe Uhl wrote: We have been running Postgres on a 2U server with 2 disks configured in raid 1 for the os and logs and 4 disks configured in raid 10 for the data. I have since been told raid 5 would have been a better option given our usage of Dell equipment and the way they handle raid 10. I ha

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Dave Page
Magnus Hagander wrote: > > Just as a datapoint, we did try to use mnogosearch for the > postgresql.org website+archives search, and it fell over completely. > Indexing took way too long, and we had search times several thousand > times longer than with tsearch2. > > That said, I'm sure there are

[PERFORM] Opinions on Raid

2007-02-27 Thread Joe Uhl
We have been running Postgres on a 2U server with 2 disks configured in raid 1 for the os and logs and 4 disks configured in raid 10 for the data. I have since been told raid 5 would have been a better option given our usage of Dell equipment and the way they handle raid 10. I have just a few gen

Re: [PERFORM] Writting a "search engine" for a pgsql DB

2007-02-27 Thread Magnus Hagander
On Mon, Feb 26, 2007 at 04:24:12PM -0500, Charles Sprickman wrote: > On Mon, 26 Feb 2007, Madison Kelly wrote: > > >Hi all, > > > >I'd really like to come up with a more intelligent search engine that > >doesn't take two minutes to return results. :) I know, in the end good > >indexes and underl

Re: [PERFORM] Two hard drives --- what to do with them?

2007-02-27 Thread Peter Kovacs
On 2/27/07, Shane Ambler <[EMAIL PROTECTED]> wrote: Jeff Davis wrote: >> Sorry for for not being familar with storage techonologies... Does >> "battery" here mean battery in the common sense of the word - some >> kind of independent power supply? Shouldn't the disk itself be backed >> by a batte