Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Mark Wong
On Sun, Dec 21, 2008 at 10:56 PM, Gregory Stark st...@enterprisedb.com wrote: Mark Wong mark...@gmail.com writes: On Dec 20, 2008, at 5:33 PM, Gregory Stark wrote: Mark Wong mark...@gmail.com writes: To recap, dbt2 is a fair-use derivative of the TPC-C benchmark. We are using a 1000

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Greg Smith
On Sat, 20 Dec 2008, Mark Wong wrote: Here are links to how the throughput changes when increasing shared_buffers: http://pugs.postgresql.org/node/505 My first glance takes tells me that the system performance is quite erratic when increasing the shared_buffers. If you smooth that curve out

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Gregory Stark
Mark Wong mark...@gmail.com writes: I'm not sure how bad that is for the benchmarks. The only effect that comes to mind is that it might exaggerate the effects of some i/o intensive operations that under normal conditions might not cause any noticeable impact like wal log file switches or

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Gregory Stark
Mark Wong mark...@gmail.com writes: Thanks for the input. In a more constructive vein: 1) autovacuum doesn't seem to be properly tracked. It looks like you're just tracking the autovacuum process and not the actual vacuum subprocesses which it spawns. 2) The response time graphs would

[PERFORM] Slow table update

2008-12-22 Thread Laszlo Nagy
SQL: update product set sz_category_id=null where am_style_kw1 is not null and sz_category_id is not null query plan: Seq Scan on product (cost=0.00..647053.30 rows=580224 width=1609) Filter: ((am_style_kw1 IS NOT NULL) AND (sz_category_id IS NOT NULL)) Information on the table: row

Re: [PERFORM] Slow table update

2008-12-22 Thread Laszlo Nagy
Laszlo Nagy wrote: SQL: update product set sz_category_id=null where am_style_kw1 is not null and sz_category_id is not null Hmm, this query: select count(*) from product where am_style_kw1 is not null and sz_category_id is not null and sz_category_id4809 opens in 10 seconds. The update

Re: [PERFORM] Slow table update

2008-12-22 Thread Gregory Williamson
Laszlo Nagy wrote: Laszlo Nagy wrote: SQL: update product set sz_category_id=null where am_style_kw1 is not null and sz_category_id is not null Hmm, this query: ?select count(*) from product where am_style_kw1 is not null and sz_category_id is not null and sz_category_id4809

Re: [PERFORM] Slow table update

2008-12-22 Thread Laszlo Nagy
If the table has some sort of FK relations it might be being slowed by the need to check a row meant to be deleted has any children. If you look at my SQL, there is only one column to be updated. That column has no foreign key constraint. (It should have, but we did not want to add that

Re: [PERFORM] Slow table update

2008-12-22 Thread Laszlo Nagy
I just tested the same on a test machine. It only has one processor 1GB memory, and one SATA disk. The same select count(*) was 58 seconds. I started the same UPDATE with EXPLAIN ANALYZE. It is running since 1000 seconds. I'm now 100% sure that the problem is with the database, because this

Re: [PERFORM] Slow table update

2008-12-22 Thread Tom Lane
Laszlo Nagy gand...@shopzeus.com writes: If the table has some sort of FK relations it might be being slowed by the need to check a row meant to be deleted has any children. If you look at my SQL, there is only one column to be updated. That column has no foreign key constraint. That was

Re: [PERFORM] [ADMIN] rebellious pg stats collector (reopened case)

2008-12-22 Thread Laszlo Nagy
and see if its output changes when you start to trace it. %cat test.c #include stdio.h int main() { while(1) { sleep(5); printf(ppid = %d\n, getppid()); } } %gcc -o test test.c %./test ppid = 47653 ppid = 47653 ppid = 47653 # Started truss -p 48864 here! ppid = 49073

Re: [PERFORM] [ADMIN] rebellious pg stats collector (reopened case)

2008-12-22 Thread Laszlo Nagy
Posted to the wrong list by mistake. Sorry. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] [ADMIN] rebellious pg stats collector (reopened case)

2008-12-22 Thread Alvaro Herrera
Laszlo Nagy wrote: %gcc -o test test.c %./test ppid = 47653 ppid = 47653 ppid = 47653 # Started truss -p 48864 here! ppid = 49073 ppid = 49073 ppid = 49073 I think you should report that as a bug to Sun. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The

[PERFORM] temp_tablespaces and RAID

2008-12-22 Thread Marc Mamin
Hello, To improve performances, I would like to try moving the temp_tablespaces locations outside of our RAID system. Is it a good practice ? Thanks, Marc Mamin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription:

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Kevin Grittner
Mark Wong mark...@gmail.com wrote: The DL380 G5 is an 8 core Xeon E5405 with 32GB of memory. The MSA70 is a 25-disk 15,000 RPM SAS array, currently configured as a 25-disk RAID-0 array. number of connections (250): Moving forward, what other parameters (or combinations of) do people

Re: [PERFORM] temp_tablespaces and RAID

2008-12-22 Thread Scott Marlowe
On Mon, Dec 22, 2008 at 7:40 AM, Marc Mamin m.ma...@intershop.de wrote: Hello, To improve performances, I would like to try moving the temp_tablespaces locations outside of our RAID system. Is it a good practice ? Maybe yes, maybe no. If you move it to a single slow drive, then it could

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Mark Wong
On Mon, Dec 22, 2008 at 12:59 AM, Greg Smith gsm...@gregsmith.com wrote: On Sat, 20 Dec 2008, Mark Wong wrote: Here are links to how the throughput changes when increasing shared_buffers: http://pugs.postgresql.org/node/505 My first glance takes tells me that the system performance is quite

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Mark Wong
On Mon, Dec 22, 2008 at 2:56 AM, Gregory Stark st...@enterprisedb.com wrote: Mark Wong mark...@gmail.com writes: Thanks for the input. In a more constructive vein: 1) autovacuum doesn't seem to be properly tracked. It looks like you're just tracking the autovacuum process and not the

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Mark Wong
On Mon, Dec 22, 2008 at 7:27 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Mark Wong mark...@gmail.com wrote: The DL380 G5 is an 8 core Xeon E5405 with 32GB of memory. The MSA70 is a 25-disk 15,000 RPM SAS array, currently configured as a 25-disk RAID-0 array. number of connections

Re: [PERFORM] dbt-2 tuning results with postgresql-8.3.5

2008-12-22 Thread Greg Smith
On Mon, 22 Dec 2008, Mark Wong wrote: The shared_buffers are the default, 24MB. The database parameters are saved, probably unclearly, here's an example link: http://207.173.203.223/~markwkm/community6/dbt2/baseline.1000.1/db/param.out That's a bit painful to slog through to find what was