Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Thu, 14 Dec 2006, Alexander Staubo wrote: Interestingly enough I see that PostgreSQL seems to be writing around 1MB/s during the pgbench run, even though I'm running pgbench in the -S mode. I haven't had the chance to look at the source yet; is it really only doing selects? I've noticed t

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options in addition to -O3 actually =hurts= performance. ...that seems unexpected... Ron Peacet

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 15, 2006, at 01:16 , Ron wrote: At 05:39 PM 12/14/2006, Alexander Staubo wrote: On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS="-O3" and just CFLAGS="-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe" as well ? All right. From my perspective, th

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 05:39 PM 12/14/2006, Alexander Staubo wrote: On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS="-O3" and just CFLAGS="-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe" as well ? All right. From my perspective, the effect of -O3 is significant, whereas a

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Tom Lane
"Kelly Burkhart" <[EMAIL PROTECTED]> writes: > I hope this isn't a "crummy mainboard" but I can't seem to affect > things by changing clock source (kernel 2.6.16 SLES10). I tried > kernel command option clock=XXX where XXX in > (cyclone,hpet,pmtmr,tsc,pit), no option was significantly better than

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Mark Kirkwood
Kelly Burkhart wrote: I hope this isn't a "crummy mainboard" but I can't seem to affect things by changing clock source (kernel 2.6.16 SLES10). I tried kernel command option clock=XXX where XXX in (cyclone,hpet,pmtmr,tsc,pit), no option was significantly better than the default. Anyone know ho

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Kelly Burkhart
On 12/14/06, Tom Lane <[EMAIL PROTECTED]> wrote: Evgeny Gridasov <[EMAIL PROTECTED]> writes: > This is a Linux Debian 3.1 ontop of 2x XEON 3.4 Ghz. > PostgreSQL is 8.2 checked out from CVS REL8_2_STABLE yesterday. > I'm running the same Postgres on another machine, > with Debian Etch and have the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS="-O3" and just CFLAGS="-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe" as well ? All right. From my perspective, the effect of -O3 is significant, whereas architecture-related optimizations have no statis

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-14 Thread Tom Lane
"Steven Flatt" <[EMAIL PROTECTED]> writes: > Regarding your other email -- interesting -- but we are vacuuming pg_class > every hour. So I don't think the answer lies there... That's good, but is the vacuum actually accomplishing anything? I'm wondering if there's also a long-running transaction

Re: [PERFORM] File Systems Compared

2006-12-14 Thread Ron Mayer
Bruno Wolff III wrote: > On Thu, Dec 14, 2006 at 01:39:00 -0500, > Jim Nasby <[EMAIL PROTECTED]> wrote: >> On Dec 11, 2006, at 12:54 PM, Bruno Wolff III wrote: >>> This appears to be changing under Linux. Recent kernels have write >>> barriers implemented using cache flush commands (which >>>

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-14 Thread Steven Flatt
Thanks for your replies. Starting a fresh session (not restarting the postmaster) seems to be sufficient to reset performance (and is an easy enough workaround). Still, it would be nice to know the root cause of the problem. The backend process does not seem to be bloating memory-wise (I'm usin

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
Alexander, Good stuff. Can you do runs with just CFLAGS="-O3" and just CFLAGS="-msse2 -mfpmath=sse -funroll-loops -m64 - march=opteron -pipe" as well ? As it is, you've given a good lower and upper bound on your performance obtainable using compiler options, but you've given no data to sho

Re: [PERFORM] strange query behavior

2006-12-14 Thread Tim Jones
ok thanks Tom I will alter the statistics and re-analyze the table. Tim Jones Healthcare Project Manager Optio Software, Inc. (770) 576-3555 -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Thursday, December 14, 2006 12:49 PM To: Tim Jones Cc: pgsql-performance@postgres

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 14, 2006, at 16:00 , Greg Smith wrote: On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is n

Re: [PERFORM] strange query behavior

2006-12-14 Thread Tom Lane
I wrote: > It's still a bit odd that the case with two batteryidentifiers was > estimated fairly accurately when the other wasn't; I'll go look into > that. For the sake of the archives: I looked into this, and it seems there's not anything going wrong other than the bogusly small n_distinct for o

Re: [PERFORM] strange query behavior

2006-12-14 Thread Tom Lane
"Tim Jones" <[EMAIL PROTECTED]> writes: > 18,273,008 rows in observationresults > [ and n_distinct = 12942 ] OK, so the estimated rowcounts are coming from those two numbers. It's notoriously hard to get a decent n_distinct estimate from a small sample :-(, and I would imagine the number of batter

Re: [PERFORM] File Systems Compared

2006-12-14 Thread Bruno Wolff III
On Thu, Dec 14, 2006 at 01:39:00 -0500, Jim Nasby <[EMAIL PROTECTED]> wrote: > On Dec 11, 2006, at 12:54 PM, Bruno Wolff III wrote: > > > >This appears to be changing under Linux. Recent kernels have write > >barriers > >implemented using cache flush commands (which some drives ignore, > >so

[PERFORM] unsubscribe

2006-12-14 Thread Rohit Prakash Khare
unsubscribe Have you checked out the new-look www.indiatimes.com yet? ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Tom Lane
Mark Lewis <[EMAIL PROTECTED]> writes: > Out of curiosity, how hard would it be to modify the output of EXPLAIN > ANALYZE when doing an insert/update to include the index update times > and/or non-FK constraint checking times and/or the table row update > times? I don't think it'd help much --- in

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Tom Lane
Evgeny Gridasov <[EMAIL PROTECTED]> writes: > This is a Linux Debian 3.1 ontop of 2x XEON 3.4 Ghz. > PostgreSQL is 8.2 checked out from CVS REL8_2_STABLE yesterday. > I'm running the same Postgres on another machine, > with Debian Etch and have the same results. Hmph. With 8.2 on Fedora 5 on a 2.

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Mark Lewis
Out of curiosity, how hard would it be to modify the output of EXPLAIN ANALYZE when doing an insert/update to include the index update times and/or non-FK constraint checking times and/or the table row update times? Or any other numbers that might be useful in circumstances like this. I'm wonderi

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Evgeny Gridasov
Tom, Hello. This is a Linux Debian 3.1 ontop of 2x XEON 3.4 Ghz. PostgreSQL is 8.2 checked out from CVS REL8_2_STABLE yesterday. I'm running the same Postgres on another machine, with Debian Etch and have the same results. On Thu, 14 Dec 2006 11:11:42 -0500 Tom Lane <[EMAIL PROTECTED]> wrote: >

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Arnaud Lesauvage
Tom Lane a écrit : I think the conclusion must be that there was just too much I/O to be done to update all the rows. Have you done any tuning of shared_buffers and so forth? I recall having seen cases where update performance went bad as soon as the upper levels of a large index no longer fit

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Tom Lane
Arnaud Lesauvage <[EMAIL PROTECTED]> writes: > Tom Lane a écrit : >> It seems the time must be going into this trigger function. What >> does it do? > A lot of things ! Indeed, if it runs it will very badly hurt performances > (table > lookups, string manipulation, etc...) ! > But it should onl

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Matthew O'Connor
Tom Lane wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: On Wed, 2006-12-13 at 18:36 -0800, Josh Berkus wrote: Mostly, though, pgbench just gives the I/O system a workout. It's not a really good general workload. It also will not utilize all cpus on a many cpu machine. We recently foun

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Tom Lane
Evgeny Gridasov <[EMAIL PROTECTED]> writes: > Running the same query on pg 8.2 through EXPLAIN ANALYZE takes 4x-10x time as > running it without it. If your machine has slow gettimeofday() this is not surprising. 8.2 is no worse (or better) than any prior version. Some quick arithmetic from you

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 10:00 AM 12/14/2006, Greg Smith wrote: On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is not what

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Michael Stone
On Wed, Dec 13, 2006 at 01:03:04PM -0500, Ron wrote: What I find interesting is that so far Guido's C2D Mac laptop has gotten the highest values by far in this set of experiments, and no one else is even close. The slowest results, Michael's, are on the system with what appears to be the slowes

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Thu, 14 Dec 2006, Tom Lane wrote: The pgbench app itself becomes the bottleneck at high transaction rates. Awhile back I rewrote it to improve its ability to issue commands concurrently, but then desisted from submitting the changes --- if we change the app like that, future numbers would b

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Arnaud Lesauvage
Tom Lane a écrit : Arnaud Lesauvage <[EMAIL PROTECTED]> writes: I must be missing something, so here is the full table description. It looks pretty harmless, except for CREATE TRIGGER parse_log_trigger BEFORE INSERT ON statistiques.log FOR EACH ROW EXECUTE PROCEDURE statistiq

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Tom Lane
Arnaud Lesauvage <[EMAIL PROTECTED]> writes: > I must be missing something, so here is the full table description. It looks pretty harmless, except for > CREATE TRIGGER parse_log_trigger > BEFORE INSERT > ON statistiques.log > FOR EACH ROW > EXECUTE PROCEDURE statistiques.parse_lo

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is not what one would naively expect when benching a IO

[PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-14 Thread Evgeny Gridasov
Hi, everybody! Running the same query on pg 8.2 through EXPLAIN ANALYZE takes 4x-10x time as running it without it. Is it ok? Example: testing=> select count(*) from auth_user; count - 2575675 (1 row) Time: 1450,829 ms testing=> explain analyze select count(*) from auth_user;

Re: [PERFORM] strange query behavior

2006-12-14 Thread Tim Jones
18,273,008 rows in observationresults pg_stats: select * from pg_stats where tablename='observationresults' and attname='batteryidentifier'; schemaname | tablename | attname | null_frac | avg_width | n_distinct | most_common_vals |

Re: [PERFORM] Slow update with simple query

2006-12-14 Thread Arnaud Lesauvage
Tom Lane a écrit : Arnaud Lesauvage <[EMAIL PROTECTED]> writes: Indeed, the new query does not perform that well : "Hash Join (cost=112.75..307504.97 rows=2024869 width=355) (actual time=53.995..246443.811 rows=2020061 loops=1)" ... "Total runtime: 2777844.892 ms" I removed all unnecessa