[PERFORM] select count(*) on large tables

2004-04-08 Thread Cosimo Streppone
Hello, I've followed the last discussion about the particular case of select count(*)s on large tables being somewhat slow. I've seen also this issue already on the todo list, so I know it is not a simple question. This problem arises for me on very large tables, which I mean starting from 1

[PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-01-31 Thread Cosimo Streppone
Hi all, I've been following this list for nearly a year now. I've always managed to get PostgreSQL 7.1.x right for the job, which in my case is a large and complex oltp system, run under Pg for 6 years now. We were already planning the switch from 7.1 to 7.4 (or even 8.0). The last project we're

Re: [PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-01-31 Thread Cosimo Streppone
Tom Lane wrote: Cosimo writes: 1) What kind of performance gain can I expect switching from 7.1 to 7.4 (or 8.0)? Obviously I'm doing my own testing, but I'm not very impressed by 8.0 speed, may be I'm doing testing on a low end server... Most people report a noticeable speedup in each new

Re: [PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-02-01 Thread Cosimo Streppone
Merlin Moncure wrote: Corollary: use pl/pgsql. It can be 10 times or more faster than query by query editing. Merlin, thanks for your good suggestions. By now, our system has never used stored procedures approach, due to the fact that we're staying on the minimum common SQL features that are

Re: [PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-02-01 Thread Cosimo Streppone
Alex Turner wrote: To be honest I've used compaq, dell and LSI SCSI RAID controllers and got pretty pathetic benchmarks from all of them. I also have seen average-low results for LSI (at least the 1020 card). 2xOpteron 242, Tyan S2885 MoBo, 4GB Ram, 14xSATA WD Raptor drives: 2xRaid 1, 1x4 disk

Re: [PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-02-02 Thread Cosimo Streppone
Merlin Moncure wrote: [...] (...DBI + DBD::Pg), so that switching to 8.0 should automatically enable the single-prepare, multiple-execute behavior, saving a lot of query planner processing, if I understand correctly. [...] I know that the perl people were pushing for certain features into the

Re: [PERFORM] High end server and storage for a PostgreSQL OLTP system

2005-02-28 Thread Cosimo Streppone
Cosimo Streppone wrote: Merlin Moncure wrote: If everything is working the way it's supposed to, 8.0 should be faster than 7.1 (like, twice faster) for what you are probably trying to do. In the next days I will be testing the entire application with the same database only changing the backend

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Cosimo Streppone
Mindaugas Riauba wrote: The vacuum cost parameters can be adjusted to make vacuums fired by pg_autovacuum less of a burden. I haven't got any specific numbers to suggest, but perhaps someone else does. It looks like that not only vacuum causes our problems. vacuum_cost seems to lower vacuum

Re: [PERFORM] postgresql-8.0.1 performance tuning

2005-05-31 Thread Cosimo Streppone
Martin Fandel wrote: i'm trying to tune my postgresql-db but i don't know if the values are I use the following environment for the postgres-db: I assumed you're running Linux here, you don't mention it. # Hardware cpu: 2x P4 3Ghz ram: 1024MB DDR 266Mhz I think 1Gb

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-03 Thread Cosimo Streppone
Stacy White wrote: We're in the process of buying another Opteron server to run Postgres, and based on the suggestions in this list I've asked our IT director to get an LSI MegaRaid controller rather than one of the Adaptecs. But when we tried to place our order, our vendor (Penguin Computing)

Re: [PERFORM] faster search

2005-06-11 Thread Cosimo Streppone
Steve Atkins wrote: On Fri, Jun 10, 2005 at 01:45:05PM -0400, Clark Slater wrote: Hi- Would someone please enlighten me as to why I'm not seeing a faster execution time on the simple scenario below? [...] Create an index on (productlistid, typeid, partnumber) then select * from test

Re: [PERFORM] tricky query

2005-06-28 Thread Cosimo Streppone
Merlin Moncure wrote: I need a fast way (sql only preferred) to solve the following problem: I need the smallest integer that is greater than zero that is not in the column of a table. I've already worked out a query using generate_series (not scalable) and pl/pgsql. An SQL only solution

Re: [PERFORM] tricky query

2005-06-28 Thread Cosimo Streppone
John A Meinel wrote: John A Meinel wrote: Merlin Moncure wrote: I need the smallest integer that is greater than zero that is not in the column of a table. In other words, if an 'id' column has values 1,2,3,4,6 and 7, I need a query that returns the value of 5. [...] Well, I was able

Re: [PERFORM] Monitoring Postgresql performance

2005-09-29 Thread Cosimo Streppone
Arnau wrote: Hi all, I have been googling a bit searching info about a way to monitor postgresql (CPU Memory, num processes, ... ) and I haven't found anything relevant. I'm using munin to monitor others parameters of my servers and I'd like to include postgresql or have a similar tool.

[PERFORM] High concurrency OLTP database performance tuning

2006-08-31 Thread Cosimo Streppone
Good morning, I'd like to ask you some advice on pg tuning in a high concurrency OLTP-like environment. The application I'm talking about is running on Pg 8.0.1. Under average users load, iostat and vmstat show that iowait stays well under 1%. Tables and indexes scan and seek times are also

[PERFORM] Update on high concurrency OLTP application and Postgres 8 tuning

2006-09-20 Thread Cosimo Streppone
Hi all, I was searching tips to speed up/reduce load on a Pg8 app. Thank you for all your suggestions on the matter. Thread is archived here: http://www.mail-archive.com/pgsql-performance@postgresql.org/msg18342.html After intensive application profiling and database workload analysis, I

Re: [PERFORM] Update on high concurrency OLTP application and Postgres

2006-09-20 Thread Cosimo Streppone
Andrew wrote: On Wed, Sep 20, 2006 at 11:09:23AM +0200, Cosimo Streppone wrote: I scheduled a cron job every hour or so that runs an analyze on the 4/5 most intensive relations and sleeps 30 seconds between every analyze. This suggests to me that your statistics need a lot of updating

Re: [PERFORM] Update on high concurrency OLTP application and Postgres

2006-09-23 Thread Cosimo Streppone
Christian Storm wrote: At the moment, my rule of thumb is to check out the ANALYZE VERBOSE messages to see if all table pages are being scanned. INFO: mytable: scanned xxx of yyy pages, containing ... If xxx = yyy, then I keep statistics at the current level. When xxx is way less than yyy,

Re: [PERFORM] Context switch storm

2006-11-03 Thread Cosimo Streppone
Richard Huxton wrote: [EMAIL PROTECTED] wrote: Hi, We've migrated one of our servers from pg 7.4 to 8.1 and from times to times (4 hours) the server start doing a lot of context switching and all transactions become very slow. The average context switching for this server as vmstat shows is

Re: [PERFORM] Context switch storm

2006-11-06 Thread Cosimo Streppone
Andreas Kostyrka wrote: The solution for us has been twofold: upgrade to the newest PG version available at the time while we waited for our new Opteron-based DB hardware to arrive. Do you remember the exact Pg version? -- Cosimo ---(end of

Re: [PERFORM] Easy read-heavy benchmark kicking around?

2006-11-08 Thread Cosimo Streppone
Merlin Moncure wrote: On 11/8/06, Markus Schaber [EMAIL PROTECTED] wrote: Hi, Brian, Brian Hurt wrote: So the question is: is there an easy to install and run, read-heavy benchmark out there that I can wave at them to get them to fix the problem? For sequential read performance, use dd.

Re: [PERFORM] Context switch storm

2006-11-14 Thread Cosimo Streppone
Richard Huxton wrote: Cosimo Streppone wrote: Richard Huxton wrote: The average context switching for this server as vmstat shows is 1 but when the problem occurs it goes to 25. I seem to have the same exact behaviour for an OLTP-loaded 8.0.1 server upgrade from 8.0.1 - the most

Re: [PERFORM] Context switch storm

2006-11-14 Thread Cosimo Streppone
Merlin wrote: On 11/14/06, Jim C. Nasby [EMAIL PROTECTED] wrote: On Tue, Nov 14, 2006 at 09:17:08AM -0500, Merlin Moncure wrote: On 11/14/06, Cosimo Streppone [EMAIL PROTECTED] wrote: I must say I lowered shared_buffers to 8192, as it was before. I tried raising it to 16384, but I can't

[PERFORM] Looking for hw suggestions for high concurrency OLTP app

2006-12-11 Thread Cosimo Streppone
Hi all, I'd like to get suggestions from all you out there for a new Postgresql server that will replace an existing one. My performance analysis shows very *low* iowaits, and very high loads at times of peak system activity. The average concurrent processes number is 3/4, with peaks of 10/15.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Cosimo Streppone
Luke Lonergan wrote: Can you try this with just -O3 versus -O2? Thanks to Daniel for doing these tests. I happen to have done the same tests about 3/4 years ago, and concluded that gcc flags did *not* influence performance. Moved by curiosity, I revamped those tests now on a test machine

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Cosimo Streppone
Michael Stone wrote: On Tue, Dec 12, 2006 at 01:42:06PM +0100, Cosimo Streppone wrote: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps I'm

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Cosimo Streppone
Alexander Staubo wrote: On Dec 15, 2006, at 17:53 , Ron wrote: At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency).

Re: [PERFORM] int4 vs varchar to store ip addr

2007-01-29 Thread Cosimo Streppone
Nicolas wrote: I have an authorization table that associates 1 customer IP to a service IP to determine a TTL (used by a radius server). table auth client varchar(15); service varchar(15); ttl int4; client and service are both ip addr. The number of distinct clients can be rather large

Re: [PERFORM] PostgreSQL in virtual machine

2007-03-13 Thread Cosimo Streppone
Andreas Tille wrote: Are there any experiences about reasonable performance increasing strategies? Are there any special things to regard in a VM? Not directly about Postgresql, but I'm seeing evidence that upgrading from vmware 2.5.3 to 3.0.1 seems to have solved disk access performance

Re: [PERFORM] What`s wrong with JFS configuration?

2007-04-26 Thread Cosimo Streppone
Jim Nasby wrote: On Apr 25, 2007, at 8:51 AM, Paweł Gruszczyński wrote: where u6 stores Fedora Core 6 operating system, and u0 stores 3 partitions with ext2, ext3 and jfs filesystem. Keep in mind that drives have a faster data transfer rate at the outer-edge than they do at the inner edge