[PERFORM] DRBD and Postgres: how to improve the perfomance?

2007-09-07 Thread Maila Fatticcioni
Hello. We have made some performance tests with DRBD and Postgresql 8.2.3. We have two identical servers in a cluster (Dell 2950) with a partition of 100 GB managed by DRBD: once we checked Postgres keeping his data folder in a local partition, the second time we moved the data folder in the

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Michael Stone
On Fri, Sep 07, 2007 at 12:33:41PM +0200, Tobias Brox wrote: Advantages: 1. Higher I/O (at least the salesman claims so) Benchmark it. It is extremely unlikely that you'll get I/O *as good as* DAS at a similar price point. 2. Easier to upgrade the disk capacity Is this an issue? You may

Re: [PERFORM] DRBD and Postgres: how to improve the perfomance?

2007-09-07 Thread Heikki Linnakangas
Maila Fatticcioni wrote: Hello. We have made some performance tests with DRBD and Postgresql 8.2.3. We have two identical servers in a cluster (Dell 2950) with a partition of 100 GB managed by DRBD: once we checked Postgres keeping his data folder in a local partition, the second time we

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Tobias Brox
We're also considering to install postgres on SAN - that is, my boss is convinced this is the right way to go. Advantages: 1. Higher I/O (at least the salesman claims so) 2. Easier to upgrade the disk capacity 3. Easy to set up warm standby functionality. (Then again, if the postgres server

Re: [PERFORM] DRBD and Postgres: how to improve the perfomance?

2007-09-07 Thread Merlin Moncure
On 9/7/07, Maila Fatticcioni [EMAIL PROTECTED] wrote: Obviously, working with the database in DRBD, we had two writes instead of only one but we are a bit disappointed about the low results. We would like to know if there is any way to improve the performance in order to have a 3/4 rate

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-07 Thread Scott Marlowe
On 9/7/07, Florian Weimer [EMAIL PROTECTED] wrote: * Scott Marlowe: And there's the issue that with windows / NTFS that when one process opens a file for read, it locks it for all other users. This means that things like virus scanners can cause odd, unpredictable failures of your

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Tom Lane
Richard Yen [EMAIL PROTECTED] writes: Here is a snippet of my log output (I can give more if necessary): Sep 5 18:38:57 tii-db2.oaktown.iparadigms.com Out of Memory: Kill process 11696 (postgres) score 1181671 and children. My understanding is that if any one postgres process's memory

Re: [PERFORM] utilising multi-cpu/core machines?

2007-09-07 Thread Sven Geisler
Hi Thomas, PostgreSQL does scale up very well. But you have to keep in mind that this also depends on profile of the application you're on PostgreSQL. Insufficient memory and slow disk systems can interfere PostgreSQL. Another issue is contention if the server has more than 4 cpus. (Please check

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Gregory Stark
Richard Yen [EMAIL PROTECTED] writes: My understanding is that if any one postgres process's memory usage, plus the shared memory, exceeds the kernel limit of 4GB, then the kernel will kill the process off. Is this true? If so, would postgres have some prevention mechanism that would

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Claus Guttesen
I've recently run into problems with my kernel complaining that I ran out of memory, thus killing off postgres and bringing my app to a grinding halt. I'm on a 32-bit architecture with 16GB of RAM, under Gentoo Linux. Naturally, I have to set my shmmax to 2GB because the kernel can't

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Richard Huxton
Richard Yen wrote: Hi All, I've recently run into problems with my kernel complaining that I ran out of memory, thus killing off postgres and bringing my app to a grinding halt. I'm on a 32-bit architecture with 16GB of RAM, under Gentoo Linux. Naturally, I have to set my shmmax to 2GB

[PERFORM] postgres memory management issues?

2007-09-07 Thread Richard Yen
Hi All, I've recently run into problems with my kernel complaining that I ran out of memory, thus killing off postgres and bringing my app to a grinding halt. I'm on a 32-bit architecture with 16GB of RAM, under Gentoo Linux. Naturally, I have to set my shmmax to 2GB because the kernel

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Matthew Schumacher
I'm getting a san together to consolidate my disk space usage for my servers. It's iscsi based and I'll be pxe booting my servers from it. The idea is to keep spares on hand for one system (the san) and not have to worry about spares for each specific storage system on each server. This also

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-07 Thread Harald Armin Massa
Scott, Well, there've been a lot of issues with anti-virus and postgresql not getting along. I wonder if pgsql takes out a stronger lock, and when it can't get it then the failure happens. Not familiar enough with windows to do more than speculate. without touching the file-concurrency

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Michael Stone
On Thu, Sep 06, 2007 at 09:06:53AM -0700, Richard Yen wrote: My understanding is that if any one postgres process's memory usage, plus the shared memory, exceeds the kernel limit of 4GB, On a 32 bit system the per-process memory limit is a lot lower than 4G. If you want to use 16G

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Michael Stone
On Fri, Sep 07, 2007 at 12:26:23AM -0400, Greg Smith wrote: consider is this: your SAN starts having funky problems, and your database is down because of it. You call the vendor. They find out you're running CentOS instead of RHEL and say that's the cause of your problem (even though it

Re: [PERFORM] Performance on 8CPU's and 32GB of RAM

2007-09-07 Thread Florian Weimer
* Scott Marlowe: And there's the issue that with windows / NTFS that when one process opens a file for read, it locks it for all other users. This means that things like virus scanners can cause odd, unpredictable failures of your database. I think most of them open the file in

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Florian Weimer
* Arjen van der Meijden: The disadvantage of using Areca or 3Ware is obviously the lack of support in A-brand servers and the lack of support for SAS-disks. Only recently Areca has stepped in the SAS-market, but I have no idea how easily those controllers are integrated in standard servers

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Bryan Murphy
We are currently running our database against on SAN share. It looks like this: 2 x RAID 10 (4 disk SATA 7200 each) Raid Group 0 contains the tables + indexes Raid Group 1 contains the log files + backups (pg_dump) Our database server connects to the san via iSCSI over Gig/E using jumbo

Re: [PERFORM] postgres memory management issues?

2007-09-07 Thread Florian Weimer
* Gregory Stark: You might also tweak /proc/sys/vm/overcommit_memory but I don't remember what the values are, you can search to find them. 2 is the interesting value, it turns off overcommit. However, if you're tight on memory, this will only increase your problems because the system fails

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Alan Hodgson
On Friday 07 September 2007 10:56, Bryan Murphy [EMAIL PROTECTED] wrote: Our database server connects to the san via iSCSI over Gig/E using jumbo frames. File system is XFS (noatime). Throughput, however, kinda sucks. I just can't get the kind of throughput to it I was hoping to get. A

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Brian Hurt
Bryan Murphy wrote: Our database server connects to the san via iSCSI over Gig/E using jumbo frames. File system is XFS (noatime). ... Throughput, however, kinda sucks. I just can't get the kind of throughput to it I was hoping to get. When our memory cache is blown, the database can

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alan Hodgson wrote: On Friday 07 September 2007 10:56, Bryan Murphy [EMAIL PROTECTED] wrote: Our database server connects to the san via iSCSI over Gig/E using jumbo frames. File system is XFS (noatime). Throughput, however, kinda sucks. I

Re: [PERFORM] DRBD and Postgres: how to improve the perfomance?

2007-09-07 Thread Simon Riggs
On Fri, 2007-09-07 at 11:37 +0200, Maila Fatticcioni wrote: protocol C; Try protocol B instead. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Greg Smith
On Fri, 7 Sep 2007, Tobias Brox wrote: We're also considering to install postgres on SAN - that is, my boss is convinced this is the right way to go. Advantages: 1. Higher I/O (at least the salesman claims so) Shockingly, the salesman is probably lying to you. The very concept of SAN says

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Vivek Khera
On Sep 6, 2007, at 2:42 PM, Scott Marlowe wrote: I'd recommend against Dell unless you're at a company that orders computers by the hundred lot. My experience with Dell has been that unless you are a big customer you're just another number (a small one at that) on a spreadsheet. I order

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 [EMAIL PROTECTED] wrote: On Fri, 7 Sep 2007, Tobias Brox wrote: We're also considering to install postgres on SAN - that is, my boss is convinced this is the right way to go. Advantages: 1. Higher I/O (at least the salesman claims so) In

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread david
On Fri, 7 Sep 2007, Tobias Brox wrote: We're also considering to install postgres on SAN - that is, my boss is convinced this is the right way to go. Advantages: 1. Higher I/O (at least the salesman claims so) only if you buy better disks for the SAN then for the local system (note that

Re: [PERFORM] DRBD and Postgres: how to improve the perfomance?

2007-09-07 Thread Bernd Helmle
--On Freitag, September 07, 2007 20:00:16 +0100 Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2007-09-07 at 11:37 +0200, Maila Fatticcioni wrote: protocol C; Try protocol B instead. But that would have an impact on transaction safety, wouldn't it? It will return immediately after