Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-04-16 Thread Franck Routier
Le 29/03/2013 15:20, Franck Routier a écrit : Hi, I have a postgresql database (8.4) running in production whose performance is degrading. There is no single query that underperforms, all queries do. Another interesting point is that a generic performance test (https://launchpad.net/tpc-b)

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-30 Thread Cédric Villemain
I don't know that tcp-b does tpcb.jar is a java implementation of the http://www.tpc.org/tpcb/ benchmark. It is not particularly representative of my workload, but gives a synthetic, db-agnostic, view of the system performance. We use it to have quick view to compare differents servers

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-30 Thread Jeff Janes
On Fri, Mar 29, 2013 at 8:31 AM, Franck Routier franck.rout...@axege.comwrote: Hi, I don't know that tcp-b does tpcb.jar is a java implementation of the http://www.tpc.org/tpcb/benchmark. It is not particularly representative of my workload, but gives a synthetic, db-agnostic, view of

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-29 Thread Julien Cigar
On 03/29/2013 15:20, Franck Routier wrote: Hi, Hello, I have a postgresql database (8.4) running in production whose performance is degrading. There is no single query that underperforms, all queries do. Another interesting point is that a generic performance test

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-29 Thread Guillaume Cottenceau
Franck Routier franck.routier 'at' axege.com writes: Hi, I have a postgresql database (8.4) running in production whose performance is degrading. There is no single query that underperforms, all queries do. Another interesting point is that a generic performance test

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-29 Thread Franck Routier
Hi, I don't know that tcp-b does tpcb.jar is a java implementation of the http://www.tpc.org/tpcb/ benchmark. It is not particularly representative of my workload, but gives a synthetic, db-agnostic, view of the system performance. We use it to have quick view to compare differents servers

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-29 Thread Jeff Janes
On Fri, Mar 29, 2013 at 7:20 AM, Franck Routier franck.rout...@axege.comwrote: Hi, I have a postgresql database (8.4) running in production whose performance is degrading. There have been substantial improvements in performance monitoring in newer versions, so using 8.4 limits your options.

Re: [PERFORM] Postgresql performance degrading... how to diagnose the root cause

2013-03-29 Thread Guillaume Cottenceau
Franck Routier franck.routier 'at' axege.com writes: http://wiki.postgresql.org/wiki/Show_database_bloat How do I interpret the output of this query ? Is 1.1 bloat level on a table alarming, or quite ok ? I am not very used to this, but I'd start by comparing the top result in your

Re: [PERFORM] PostgreSQL performance on 64 bit as compared to 32 bit

2012-09-21 Thread Claudio Freire
On Sun, Sep 16, 2012 at 12:48 AM, Umesh Kirdat umesh.kir...@yahoo.com wrote: The issue we have noticed is the 9.0.4 (64 bit) version of PostgreSQL has slower performance as compared to 8.2.2 (32 bit) version on an identical hardware. First of all, that's comparing apples and oranges. Compare

Re: [PERFORM] Postgresql - performance of using array in big database

2012-08-08 Thread Craig Ringer
On 08/03/2012 05:14 PM, robertha...@o2.pl wrote: It is read-only table so every integer column have an index. First tip: Define the table without the indexes. INSERT your data, and only after it is inserted create your indexes. Similarly, if you're making huge changes to the table you

Re: [PERFORM] Postgresql - performance of using array in big database

2012-08-08 Thread Ondrej Ivanič
Hi, On 3 August 2012 19:14, robertha...@o2.pl wrote: I want to add to table Item a column a_elements (array type of big integers) Every record would have not more than 50-60 elements in this column. After that i would create index GIN on this column and typical select should look like this:

Re: [PERFORM] PostgreSQL performance tweaking on new hardware

2011-09-11 Thread Scott Marlowe
On Sun, Sep 11, 2011 at 1:50 PM, Ogden li...@darkstatic.com wrote: I want to thank members on this list which helped me benchmark and conclude that RAID 10 on a XFS filesystem was the way to go over what we had prior. PostgreSQL we have been using with Perl for the last 8 years and it has

Re: [PERFORM] postgreSQL performance 8.2.6 vs 8.3.3

2009-02-23 Thread david
On Fri, 20 Feb 2009, David Rees wrote: On Fri, Feb 20, 2009 at 1:34 PM, Battle Mage battlem...@gmail.com wrote: The amount of tps almost doubled, which is good, but i'm worried about the load. For my application, a load increase is bad and I'd like to keep it just like in 8.2.6 (a load

Re: [PERFORM] postgreSQL performance 8.2.6 vs 8.3.3

2009-02-23 Thread Scott Marlowe
On Mon, Feb 23, 2009 at 2:02 PM, da...@lang.hm wrote: On Fri, 20 Feb 2009, David Rees wrote: On Fri, Feb 20, 2009 at 1:34 PM, Battle Mage battlem...@gmail.com wrote: The amount of tps almost doubled, which is good, but i'm worried about the load. For my application, a load increase is bad

Re: [PERFORM] postgreSQL performance 8.2.6 vs 8.3.3

2009-02-20 Thread Kenneth Marshall
On Fri, Feb 20, 2009 at 04:34:23PM -0500, Battle Mage wrote: I have a server box that has 4GB of RAM, Quad core CPU AMD Opteron 200.152 Mhz (1024 KB cache size each) with plenty of hard drive space. I installed both postgresql 8.2.6 and 8.3.3 on it. I've created a basic test db and used

Re: [PERFORM] postgreSQL performance 8.2.6 vs 8.3.3

2009-02-20 Thread David Rees
On Fri, Feb 20, 2009 at 1:34 PM, Battle Mage battlem...@gmail.com wrote: The amount of tps almost doubled, which is good, but i'm worried about the load. For my application, a load increase is bad and I'd like to keep it just like in 8.2.6 (a load average between 3.4 and 4.3). What parameters

Re: [PERFORM] postgreSQL performance 8.2.6 vs 8.3.3

2009-02-20 Thread Scott Marlowe
On Fri, Feb 20, 2009 at 2:34 PM, Battle Mage battlem...@gmail.com wrote: I have a server box that has 4GB of RAM, Quad core CPU AMD Opteron 200.152 Mhz (1024 KB cache size each) with plenty of hard drive space. I installed both postgresql 8.2.6 and 8.3.3 on it. I've created a basic test db

Re: [PERFORM] PostgreSQL performance on a virtual host

2008-03-05 Thread Moritz Onken
We have very good experiences with openVZ as virtualizer. Since it's not a para virtualization like xen it's very fast. Almost as fast as the host. www.openvz.org Am 04.03.2008 um 16:43 schrieb Theo Kramer: Hi We are thinking of running a PostgreSQL instance on a virtual host under

Re: [PERFORM] postgresql performance

2008-03-05 Thread Franck Routier
Hi, Le mercredi 05 mars 2008 à 11:39 +0100, Steinar H. Gunderson a écrit : Without knowing what a lakhs record is, I had the same question... and Wikipedia gave me the answer : it is an Indian word meaning 10^5, often used in indian english. Franck -- Sent via pgsql-performance mailing

Re: [PERFORM] postgresql performance

2008-03-05 Thread Steinar H. Gunderson
On Wed, Mar 05, 2008 at 02:27:08AM -0800, SPMLINGAM wrote: I have a table with 50 lakhs records, the table has more then 10 fields, i have primary key, i have select query with count(*) without any condition, it takes 17 seconds. Without knowing what a lakhs record is, it's pretty obvious

Re: [PERFORM] postgresql performance

2008-03-05 Thread Claus Guttesen
Without knowing what a lakhs record is, I had the same question... and Wikipedia gave me the answer : it is an Indian word meaning 10^5, often used in indian english. Thank you (both OP and this post) for enlightening us with this word. -- regards Claus When lenity and cruelty play for

Re: [PERFORM] PostgreSQL performance on a virtual host

2008-03-05 Thread Dave Cramer
Hi, I've run it on xen. works OK. Course this is all predicated upon your expectations. If you expect it to be as fast as a dedicated machine, you will be dissapointed. Dave On 5-Mar-08, at 3:54 AM, Moritz Onken wrote: We have very good experiences with openVZ as virtualizer. Since it's

Re: [PERFORM] PostgreSQL performance on a virtual host

2008-03-05 Thread Ivan Zolotukhin
Hello, We had a bad experience with PostgreSQL running in OpenVZ (year and a half year ago): OpenVZ kernel killed postmaster with strange signals from time to time, failcounters of OpenVZ did not worked as expected in this moments, PostgreSQL fighted for the disk with applications in other

Re: [PERFORM] PostgreSQL performance on a virtual host

2008-03-05 Thread Bill Moran
In response to Ivan Zolotukhin [EMAIL PROTECTED]: We had a bad experience with PostgreSQL running in OpenVZ (year and a half year ago): OpenVZ kernel killed postmaster with strange signals from time to time, failcounters of OpenVZ did not worked as expected in this moments, PostgreSQL

Re: [PERFORM] PostgreSQL performance on a virtual host

2008-03-05 Thread Ivan Zolotukhin
Hello, On Wed, Mar 5, 2008 at 5:40 PM, Bill Moran [EMAIL PROTECTED] wrote: In response to Ivan Zolotukhin [EMAIL PROTECTED]: We had a bad experience with PostgreSQL running in OpenVZ (year and a half year ago): OpenVZ kernel killed postmaster with strange signals from time to time,

Re: [PERFORM] postgresql performance

2008-03-05 Thread Dave Dutcher
-Original Message- From: SPMLINGAM Subject: [PERFORM] postgresql performance Dear Friends, I have a table with 50 lakhs records, the table has more then 10 fields, i have primary key, i have select query with count(*) without any condition, it takes 17 seconds. 17 seconds

Re: [PERFORM] postgresql performance

2008-03-05 Thread Kevin Grittner
On Wed, Mar 5, 2008 at 4:39 AM, in message [EMAIL PROTECTED], Steinar H. Gunderson [EMAIL PROTECTED] wrote: it's pretty obvious that you haven't vacuumed in a very long time. Run VACUUM FULL on your tables If you use VACUUM FULL, you should probably throw in ANALYZE with it, and REINDEX,

Re: [PERFORM] postgresql performance

2008-03-05 Thread Bill Moran
In response to Dave Dutcher [EMAIL PROTECTED]: -Original Message- From: SPMLINGAM Subject: [PERFORM] postgresql performance Dear Friends, I have a table with 50 lakhs records, the table has more then 10 fields, i have primary key, i have select query with count(*)

Re: [PERFORM] PostgreSQL performance on various distribution stock kernels

2007-11-26 Thread Alexander Staubo
On 11/26/07, Damon Hart [EMAIL PROTECTED] wrote: So, what's different between these tests? I'm seeing performance differences of between +65% to +90% transactions per second of the OpenVZ kernel running on the HN over the stock Fedora 8 kernel. Is this reflective of different emphasis between

Re: [PERFORM] PostgreSQL performance on various distribution stock kernels

2007-11-26 Thread Scott Marlowe
On Nov 26, 2007 4:50 PM, Damon Hart [EMAIL PROTECTED] wrote: So, what's different between these tests? I'm seeing performance differences of between +65% to +90% transactions per second of the OpenVZ kernel running on the HN over the stock Fedora 8 kernel. Is this reflective of different

Re: [PERFORM] PostgreSQL performance on various distribution stock kernels

2007-11-26 Thread Damon Hart
On Mon, 2007-11-26 at 17:00 -0600, Scott Marlowe wrote: On Nov 26, 2007 4:50 PM, Damon Hart [EMAIL PROTECTED] wrote: So, what's different between these tests? I'm seeing performance differences of between +65% to +90% transactions per second of the OpenVZ kernel running on the HN over the

Re: [PERFORM] PostgreSQL performance on various distribution stock kernels

2007-11-26 Thread Damon Hart
On Mon, 2007-11-26 at 18:06 -0500, Tom Lane wrote: Damon Hart [EMAIL PROTECTED] writes: So, what's different between these tests? I'm seeing performance differences of between +65% to +90% transactions per second of the OpenVZ kernel running on the HN over the stock Fedora 8 kernel. Is

Re: [PERFORM] PostgreSQL performance on various distribution stock kernels

2007-11-26 Thread Greg Smith
On Mon, 26 Nov 2007, Damon Hart wrote: Fedora 8: Linux 2.6.23.1-49.fc8 #1 SMP Thu Nov 8 21:41:26 EST 2007 i686 i686 i386 GNU/Linux OpenVZ: Linux 2.6.18-8.1.15.el5.028stab049.1 #1 SMP Thu Nov 8 16:23:12 MSK 2007 i686 i686 i386 GNU/Linux 2.6.23 introduced a whole new scheduler:

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options: All good ideas and I'll be sure to implement them later. I am curious why you need to query the

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Luke Lonergan
Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu3 : 96.2% us,

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Rusty Conover wrote: On Aug 29, 2006, at 7:52 AM, Willo van der Merwe wrote: Hi, We're running PostgreSQL 8.1.4 on CentOS 4 (Linux version 2.6.9-34.0.1.ELsmp). Hardware specs: 2x AMD Dual-Core Opteron 270 Italy 1Ghz HT 2 x 1MB L2 Cache Socket 940 4 GB Registered ECC PC3200 DDR RAM

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu3

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Luke Lonergan
AM To: Luke Lonergan Cc: Merlin Moncure; pgsql-performance@postgresql.org Subject: Re: [PERFORM] PostgreSQL performance issues Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Alex Hayward
On Wed, 30 Aug 2006, Willo van der Merwe wrote: Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options: All good ideas and I'll be sure to

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
? - Luke -Original Message- From: Willo van der Merwe [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 30, 2006 4:35 AM To: Luke Lonergan Cc: Merlin Moncure; pgsql-performance@postgresql.org Subject: Re: [PERFORM] PostgreSQL performance issues Luke Lonergan wrote: Currently the load

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Alex Hayward wrote: On Wed, 30 Aug 2006, Willo van der Merwe wrote: Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options:

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Dave Cramer
On 30-Aug-06, at 7:35 AM, Willo van der Merwe wrote: Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6%

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Dave Cramer wrote: On 30-Aug-06, at 7:35 AM, Willo van der Merwe wrote: Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa,

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Dave Dutcher
Title: Message That's an interesting situation. Your CPU's are pegged, and you're hardly doing any IO. I wonder if there is some ineficient query, or if its just very high query volume. Maybe you could try setting log_min_duration_statement to try to track down the slowest of the queries.

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Dave Dutcher wrote: That's an interesting situation. Your CPU's are pegged, and you're hardly doing any IO. I wonder if there is some ineficient query, or if its just very high query volume. Maybe you could try setting log_min_duration_statement to try to track down the slowest of the

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Merlin Moncure
On 8/30/06, Willo van der Merwe [EMAIL PROTECTED] wrote: This was just an example. All queries have slowed down. Could it be that I've reached some cut-off and now my disk is thrashing? Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0%

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Alan Hodgson
On Wednesday 30 August 2006 03:48, Willo van der Merwe [EMAIL PROTECTED] wrote: Hi Rusty, Good ideas and I've implemented some of them, and gained about 10%. I'm still sitting on a load avg of about 60. Any ideas on optimizations on my postgresql.conf, that might have an effect? If all of

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread A. Kretschmer
am Tue, dem 29.08.2006, um 15:52:50 +0200 mailte Willo van der Merwe folgendes: and it has 743321 rows and a explain analyze select count(*) from property_values; QUERY PLAN

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread Joshua D. Drake
4 1/2 seconds for a count(*) ? This seems a bit rough - is there anything else I can try to optimize my Database? You can imagine that slightly more complex queries goes out the roof. Well a couple of things. 1. You put all your money in the wrong place.. 1 hard drive!!??!! 2. What is your

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread Willo van der Merwe
Joshua D. Drake wrote: 4 1/2 seconds for a count(*) ? This seems a bit rough - is there anything else I can try to optimize my Database? You can imagine that slightly more complex queries goes out the roof. Well a couple of things. 1. You put all your money in the wrong place.. 1 hard

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread PFC
4 1/2 seconds for a count(*) ? Is this a real website query ? Do you need this query ? ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread A. Kretschmer
am Tue, dem 29.08.2006, um 16:55:11 +0200 mailte Willo van der Merwe folgendes: 4 1/2 seconds for a count(*) ? This seems a bit rough - is there anything else Because of MVCC. http://www.thescripts.com/forum/thread173678.html http://www.varlena.com/GeneralBits/120.php

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread Rusty Conover
On Aug 29, 2006, at 7:52 AM, Willo van der Merwe wrote: Hi, We're running PostgreSQL 8.1.4 on CentOS 4 (Linux version 2.6.9-34.0.1.ELsmp). Hardware specs: 2x AMD Dual-Core Opteron 270 Italy 1Ghz HT 2 x 1MB L2 Cache Socket 940 4 GB Registered ECC PC3200 DDR RAM SuperMicro Server-Class 1U AS1020S

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread Codelogic
On Tue, 2006-08-29 at 15:52 +0200, Willo van der Merwe wrote: (cost=0.00..51848.56 rows=1309356 width=0) It is going through way more number of rows than what is returned by the count(*). It appears that you need to VACUUM the table (not VACUUM ANALYZE). ---(end of

Re: [PERFORM] PostgreSQL performance issues

2006-08-29 Thread Merlin Moncure
On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options: 1. keep a sequence on the property values and query it. if you want exact count you must do some clever locking however.

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Magnus Hagander
There is 64MB on the 6i and 192MB on the 642 controller. I wish the controllers had a wrieback enable option like the LSI MegaRAID adapters have. I have tried splitting the cache accelerator 25/75 75/25 0/100 100/0 but the results really did not improve. They have a writeback option, but you

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Luke Lonergan
Steve, If this is an internal RAID1 on two disks, it looks great. Based on the random seeks though (578 seeks/sec), it looks like maybe it's 6 disks in a RAID10? - Luke On 8/16/06 7:10 PM, Steve Poe [EMAIL PROTECTED] wrote: Everyone, I wanted to follow-up on bonnie results for the

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Bucky Jordan
To: [EMAIL PROTECTED]; Scott Marlowe Cc: Michael Stone; pgsql-performance@postgresql.org Subject: Re: [PERFORM] Postgresql Performance on an HP DL385 and Steve, If this is an internal RAID1 on two disks, it looks great. Based on the random seeks though (578 seeks/sec), it looks like maybe it's 6 disks

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Steve Poe
Luke,Nope. it is only a RAID1 for the 2 internal discs connected to the SmartArray 6i. This is where I *had* the pg_xlog located when the performance was very poor. Also, I just found out the default stripe size is 128k. Would this be a problem for pg_xlog? The 6-disc RAID10 you speak of is on the

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Luke Lonergan
Steve, On 8/18/06 10:39 AM, Steve Poe [EMAIL PROTECTED] wrote: Nope. it is only a RAID1 for the 2 internal discs connected to the SmartArray 6i. This is where I *had* the pg_xlog located when the performance was very poor. Also, I just found out the default stripe size is 128k. Would this be

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Steve Poe
Luke, ISTM that the main performance issue for xlog is going to be the rate at which fdatasync operations complete, and the stripe size shouldn't hurtthat.I thought so. However, I've also tried running the PGDATA off of the RAID1 as a test and it is poor. What are your postgresql.conf settings for

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Luke Lonergan
Title: Re: [PERFORM] Postgresql Performance on an HP DL385 and Steve, One thing here is that wal_sync_method should be set to fdatasync and not fsync. In fact, the default is fdatasync, but because you have uncommented the standard line in the file, it is changed to fsync, which is a lot

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-18 Thread Steve Poe
Luke,I'll try it, but you're right, it should not matter. The two systems are:HP DL385 (dual Opteron 265 I believe) 8GB of RAM, two internal RAID1 U320 10KSun W2100z (dual Opteron 245 I believe) 4GB of RAM, 1 U320 10K drive with LSI MegaRAID 2X 128M driving two external 4-disc arrays U320 10K

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-16 Thread Markus Schaber
Hi, Jim, Jim C. Nasby wrote: Well, if the controller is caching with a BBU, I'm not sure that order matters anymore, because the controller should be able to re-order at will. Theoretically. :) But this is why having some actual data posted somewhere would be great. Well, actually, the

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-16 Thread Guillaume Cottenceau
Hi, Can you run bonnie++ version 1.03a on the machine and report the results here? Do you know if the figures from bonnie++ are able to measure the performance related to the overhead of the 'fsync' option? I had very strange performance differences between two Dell 1850 machines months ago,

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-16 Thread Steve Poe
Everyone, I wanted to follow-up on bonnie results for the internal RAID1 which is connected to the SmartArray 6i. I believe this is the problem, but I am not good at interepting the results. Here's an sample of three runs: scsi disc array

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Jim C. Nasby
On Mon, Aug 14, 2006 at 01:03:41PM -0400, Michael Stone wrote: On Mon, Aug 14, 2006 at 10:38:41AM -0500, Jim C. Nasby wrote: Got any data to back that up? yes. that I'm willing to dig out? no. :) Well, I'm not digging hard numbers out either, so that's fair. :) But it would be very handy if

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Jim C. Nasby
On Mon, Aug 14, 2006 at 01:09:04PM -0400, Michael Stone wrote: On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote: Wow, interesting. IIRC, XFS is lower performing than ext3, For xlog, maybe. For data, no. Both are definately slower than ext2 for xlog, which is another reason to

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Michael Stone
On Tue, Aug 15, 2006 at 11:25:24AM -0500, Jim C. Nasby wrote: Well, if the controller is caching with a BBU, I'm not sure that order matters anymore, because the controller should be able to re-order at will. Theoretically. :) But this is why having some actual data posted somewhere would be

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Michael Stone
On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote: Are 'we' sure that such a setup can't lose any data? Yes. If you check the archives, you can even find the last time this was discussed... The bottom line is that the only reason you need a metadata journalling filesystem is to

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread mark
On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote: On Mon, Aug 14, 2006 at 01:09:04PM -0400, Michael Stone wrote: On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote: Wow, interesting. IIRC, XFS is lower performing than ext3, For xlog, maybe. For data, no. Both are

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread mark
On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote: Are 'we' sure that such a setup can't lose any data? Yes. If you check the archives, you can even find the last time this was discussed... I looked last night

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Michael Stone
On Tue, Aug 15, 2006 at 02:33:27PM -0400, [EMAIL PROTECTED] wrote: On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote: Are 'we' sure that such a setup can't lose any data? Yes. If you check the archives, you can even find

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Jim C. Nasby
On Tue, Aug 15, 2006 at 03:02:56PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 02:33:27PM -0400, [EMAIL PROTECTED] wrote: On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote: Are 'we' sure that such a setup can't

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread mark
On Tue, Aug 15, 2006 at 03:02:56PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 02:33:27PM -0400, [EMAIL PROTECTED] wrote: Are 'we' sure that such a setup can't lose any data? Yes. If you check the archives, you can even find the last time this was discussed... I looked last night

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Tom Lane
[EMAIL PROTECTED] writes: I've been worrying about this myself, and my current conclusion is that ext2 is bad because: a) fsck, and b) data can be lost or corrupted, which could lead to the need to trash the xlog. Even ext3 in writeback mode allows for the indirect blocks to be updated

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread mark
On Tue, Aug 15, 2006 at 04:05:17PM -0400, Tom Lane wrote: [EMAIL PROTECTED] writes: I've been worrying about this myself, and my current conclusion is that ext2 is bad because: a) fsck, and b) data can be lost or corrupted, which could lead to the need to trash the xlog. Even ext3 in

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Tom Lane
[EMAIL PROTECTED] writes: WAL file is never appended - only re-written? If so, then I'm wrong, and ext2 is fine. The requirement is that no file system structures change as a result of any writes that PostgreSQL does. If no file system structures change, then I take everything back as

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Michael Stone
On Tue, Aug 15, 2006 at 02:15:05PM -0500, Jim C. Nasby wrote: Now, if fsync'ing a file also ensures that all the metadata is written, then we're probably fine... ...and it does. Unclean shutdowns cause problems in general because filesystems operate asynchronously. postgres (and other

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Michael Stone
On Tue, Aug 15, 2006 at 03:39:51PM -0400, [EMAIL PROTECTED] wrote: No. This is not true. Updating the file system structure (inodes, indirect blocks) touches a separate part of the disk than the actual data. If the file system structure is modified, say, to extend a file to allow it to contain

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread mark
On Tue, Aug 15, 2006 at 04:58:59PM -0400, Michael Stone wrote: On Tue, Aug 15, 2006 at 03:39:51PM -0400, [EMAIL PROTECTED] wrote: No. This is not true. Updating the file system structure (inodes, indirect blocks) touches a separate part of the disk than the actual data. If the file system

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Jim C. Nasby
On Tue, Aug 15, 2006 at 05:38:43PM -0400, [EMAIL PROTECTED] wrote: I didn't know that the xlog segment only uses pre-allocated space. I ignore mtime/atime as they don't count as file system structure changes to me. It's updating a field in place. No change to the structure. With the

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Steinar H. Gunderson
On Tue, Aug 15, 2006 at 05:20:25PM -0500, Jim C. Nasby wrote: This is only valid if the pre-allocation is also fsync'd *and* fsync ensures that both the metadata and file data are on disk. Anyone actually checked that? :) fsync() does that, yes. fdatasync() (if it exists), OTOH, doesn't sync

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread David Lang
On Tue, 15 Aug 2006 [EMAIL PROTECTED] wrote: This is also wrong. fsck is needed because the file system is broken. nope, the file system *may* be broken. the dirty flag simply indicates that the filesystem needs to be checked to find out whether or not it is broken. Ah, but if we knew it

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-15 Thread Tom Lane
Steinar H. Gunderson [EMAIL PROTECTED] writes: On Tue, Aug 15, 2006 at 05:20:25PM -0500, Jim C. Nasby wrote: This is only valid if the pre-allocation is also fsync'd *and* fsync ensures that both the metadata and file data are on disk. Anyone actually checked that? :) fsync() does that, yes.

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-14 Thread Steve Poe
Jim,I have to say Michael is onto something here to my surprise. I partitioned the RAID10 on the SmartArray 642 adapter into two parts, PGDATA formatted with XFS and pg_xlog as ext2. Performance jumped up to median of 98 TPS. I could reproduce the similar result with the LSI MegaRAID 2X adapter as

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-14 Thread Michael Stone
On Mon, Aug 14, 2006 at 10:38:41AM -0500, Jim C. Nasby wrote: Got any data to back that up? yes. that I'm willing to dig out? no. :) The problem with seperate partitions is that it means more head movement for the drives. If it's all one partition the pg_xlog data will tend to be

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-14 Thread Jim C. Nasby
On Mon, Aug 14, 2006 at 08:51:09AM -0700, Steve Poe wrote: Jim, I have to say Michael is onto something here to my surprise. I partitioned the RAID10 on the SmartArray 642 adapter into two parts, PGDATA formatted with XFS and pg_xlog as ext2. Performance jumped up to median of 98 TPS. I

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-14 Thread Michael Stone
On Mon, Aug 14, 2006 at 12:05:46PM -0500, Jim C. Nasby wrote: Wow, interesting. IIRC, XFS is lower performing than ext3, For xlog, maybe. For data, no. Both are definately slower than ext2 for xlog, which is another reason to have xlog on a small filesystem which doesn't need metadata

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-10 Thread Michael Stone
On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote: I tried as you suggested and my performance dropped by 50%. I went from a 32 TPS to 16. Oh well. If you put data xlog on the same array, put them on seperate partitions, probably formatted differently (ext2 on xlog). Mike Stone

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-10 Thread Luke Lonergan
Mike, On 8/10/06 4:09 AM, Michael Stone [EMAIL PROTECTED] wrote: On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote: I tried as you suggested and my performance dropped by 50%. I went from a 32 TPS to 16. Oh well. If you put data xlog on the same array, put them on seperate

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-10 Thread Scott Marlowe
On Thu, 2006-08-10 at 10:15, Luke Lonergan wrote: Mike, On 8/10/06 4:09 AM, Michael Stone [EMAIL PROTECTED] wrote: On Wed, Aug 09, 2006 at 08:29:13PM -0700, Steve Poe wrote: I tried as you suggested and my performance dropped by 50%. I went from a 32 TPS to 16. Oh well. If you put

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-10 Thread Steve Poe
Scott,I *could* rip out the LSI MegaRAID 2X from my Sun box. This belongs to me for testing. but I don't know if it will fit in the DL385. Do they have full-heigth/length slots? I've not worked on this type of box before. I was thinking this is the next step. In the meantime, I've discovered their

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Steve Poe
Luke,I check dmesg one more time and I found this regarding the cciss driver:Filesystem cciss/c1d0p1: Disabling barriers, not supported by the underlying device.Don't know if it means anything, but thought I'd mention it. SteveOn 8/8/06, Steve Poe [EMAIL PROTECTED] wrote: Luke,I thought so. In my

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Luke Lonergan
you contact them through HP tech support and report back to this list what you find out? - Luke -Original Message- From: Steve Poe [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 08, 2006 11:33 PM To: Luke Lonergan Cc: Alex Turner; pgsql-performance@postgresql.org Subject: Re: [PERFORM

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Steve Poe
support and report backto this list what you find out?- Luke -Original Message- From: Steve Poe [mailto: [EMAIL PROTECTED]] Sent: Tuesday, August 08, 2006 11:33 PM To: Luke Lonergan Cc: Alex Turner; pgsql-performance@postgresql.org Subject: Re: [PERFORM] Postgresql Performance on an HP DL385

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Luke Lonergan
Steve, I will do that. If it is the general impression that this server should perform well with Postgresql, Are the RAID cards, the 6i and 642 sufficient to your knowledge? I am wondering if it is the disc array itself. I think that is the question to be answered by HP support. Ask

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Steve Poe
Luke,I hope so. I'll keep you and the list up-to-date as I learn more.SteveOn 8/8/06, Luke Lonergan [EMAIL PROTECTED] wrote:Steve, I will do that. If it is the general impression that this server should perform well with Postgresql, Are the RAID cards, the 6i and 642 sufficient to your

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Jim C. Nasby
On Tue, Aug 08, 2006 at 10:45:07PM -0700, Steve Poe wrote: Luke, I thought so. In my test, I tried to be fair/equal since my Sun box has two 4-disc arrays each on their own channel. So, I just used one of them which should be a little slower than the 6-disc with 192MB cache. Incidently,

Re: [PERFORM] Postgresql Performance on an HP DL385 and

2006-08-09 Thread Steve Poe
Jim,I'll give it a try. However, I did not see anywhere in the BIOS configuration of the 642 RAID adapter to enable writeback. It may have been mislabled cache accelerator where you can give a percentage to read/write. That aspect did not change the performance like the LSI MegaRAID adapter does.

  1   2   >