Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-18 Thread Ron
Sorry for the delay in responding. I had familial obligations. As a matter of fact, I am spending a decent amount of time on this. I don't usually pore through documentation for compilers and OS's to the degree I've been since this thread started. Nor do I usually try and get access to the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Cosimo Streppone
Alexander Staubo wrote: On Dec 15, 2006, at 17:53 , Ron wrote: At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency).

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Ron
At 07:06 PM 12/15/2006, Michael Stone wrote: On Fri, Dec 15, 2006 at 12:24:46PM -0500, Ron wrote: ATM, the most we can say is that in a number of systems with modest physical IO subsystems So I reran it on a 3.2GHz xeon with 6G RAM off a ramdisk; I/O ain't the bottleneck on that one.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Steinar H. Gunderson
On Sat, Dec 16, 2006 at 10:53:21AM -0500, Ron wrote: AFAICT, no one has stated there would be a blow-your-socks-off dramatic performance improvement for pg due to compilation options. Just that there might be some, and there might be some that are arch specific. FWIW, the original claim

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Michael Stone
On Sat, Dec 16, 2006 at 10:53:21AM -0500, Ron wrote: The most important gain IMO is Knowledge, and I'd say there is still more to learn and/or verify IMHO. YMMV. Well, I think there are other areas where I can spend my time where potential gains are more likely. YMMV (although, I note, you

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Alexander Staubo
On Dec 15, 2006, at 04:09 , Ron wrote: At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options in addition to -O3 actually =hurts=

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Steinar H. Gunderson
On Fri, Dec 15, 2006 at 10:53:25AM +0100, Alexander Staubo wrote: The difference is very slight. I'm going to run without -funroll- loops and -pipe (which are not arch-related) to get better data. -pipe does not matter for the generated code; it only affects compiler speed. (It simply means

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 04:54 AM 12/15/2006, Alexander Staubo wrote: On Dec 15, 2006, at 04:09 , Ron wrote: At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Merlin Moncure
On 12/15/06, Ron [EMAIL PROTECTED] wrote: I'm looking more closely into exactly what the various gcc -O optimizations do on Kx's as well. 64b vs 32b gets x86 compatible code access to ~ 2x as many registers; and MMX or SSE instructions get you access to not only more registers, but wider ones as

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Greg Smith
On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so important with current processors that I'd suggest throwing -Os (optimize for size) into

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 09:23 AM 12/15/2006, Merlin Moncure wrote: On 12/15/06, Ron [EMAIL PROTECTED] wrote: It seems unusual that code generation options which give access to more registers would ever result in slower code... The slower is probably due to the unroll loops switch which can actually hurt code

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Merlin Moncure
On 12/15/06, Ron [EMAIL PROTECTED] wrote: At 09:23 AM 12/15/2006, Merlin Moncure wrote: On 12/15/06, Ron [EMAIL PROTECTED] wrote: It seems unusual that code generation options which give access to more registers would ever result in slower code... The slower is probably due to the unroll loops

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so important with current processors that I'd

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 10:55 AM 12/15/2006, Merlin Moncure wrote: On 12/15/06, Ron [EMAIL PROTECTED] wrote: There are many instances of x86 compatible code that get 30-40% speedups just because they get access to 16 rather than 8 GPRs when recompiled for x84-64. ...We benchmarked PostgreSQL internally here and

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Alexander Staubo
On Dec 15, 2006, at 17:53 , Ron wrote: At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Michael Stone
On Fri, Dec 15, 2006 at 12:24:46PM -0500, Ron wrote: ATM, the most we can say is that in a number of systems with modest physical IO subsystems So I reran it on a 3.2GHz xeon with 6G RAM off a ramdisk; I/O ain't the bottleneck on that one. Results didn't show didn't show any signficant

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is not what one would naively expect when benching a

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Thu, 14 Dec 2006, Tom Lane wrote: The pgbench app itself becomes the bottleneck at high transaction rates. Awhile back I rewrote it to improve its ability to issue commands concurrently, but then desisted from submitting the changes --- if we change the app like that, future numbers would

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Michael Stone
On Wed, Dec 13, 2006 at 01:03:04PM -0500, Ron wrote: What I find interesting is that so far Guido's C2D Mac laptop has gotten the highest values by far in this set of experiments, and no one else is even close. The slowest results, Michael's, are on the system with what appears to be the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 10:00 AM 12/14/2006, Greg Smith wrote: On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is not what

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Matthew O'Connor
Tom Lane wrote: Joshua D. Drake [EMAIL PROTECTED] writes: On Wed, 2006-12-13 at 18:36 -0800, Josh Berkus wrote: Mostly, though, pgbench just gives the I/O system a workout. It's not a really good general workload. It also will not utilize all cpus on a many cpu machine. We recently found

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 14, 2006, at 16:00 , Greg Smith wrote: On Wed, 13 Dec 2006, Ron wrote: The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of the results seem to similarly depend on relative CPU performance. This is

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
Alexander, Good stuff. Can you do runs with just CFLAGS=-O3 and just CFLAGS=-msse2 -mfpmath=sse -funroll-loops -m64 - march=opteron -pipe as well ? As it is, you've given a good lower and upper bound on your performance obtainable using compiler options, but you've given no data to show

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS=-O3 and just CFLAGS=-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe as well ? All right. From my perspective, the effect of -O3 is significant, whereas architecture-related optimizations have no

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 05:39 PM 12/14/2006, Alexander Staubo wrote: On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS=-O3 and just CFLAGS=-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe as well ? All right. From my perspective, the effect of -O3 is significant, whereas

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Alexander Staubo
On Dec 15, 2006, at 01:16 , Ron wrote: At 05:39 PM 12/14/2006, Alexander Staubo wrote: On Dec 14, 2006, at 20:28 , Ron wrote: Can you do runs with just CFLAGS=-O3 and just CFLAGS=-msse2 - mfpmath=sse -funroll-loops -m64 - march=opteron -pipe as well ? All right. From my perspective, the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Ron
At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options in addition to -O3 actually =hurts= performance. ...that seems unexpected... Ron

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-14 Thread Greg Smith
On Thu, 14 Dec 2006, Alexander Staubo wrote: Interestingly enough I see that PostgreSQL seems to be writing around 1MB/s during the pgbench run, even though I'm running pgbench in the -S mode. I haven't had the chance to look at the source yet; is it really only doing selects? I've noticed

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Sven Geisler
Hi, Did someone try '-mfpmath=sse -msse3'? Would be interesting to know if -mfpmath=sse boost the performance. I guess, the difference in the generated code isn't that much between i686 and prescott. The bigger step is i386 to i686. '-mfpmath=sse -msse3' will also use the SSE unit, which the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Cosimo Streppone
Michael Stone wrote: On Tue, Dec 12, 2006 at 01:42:06PM +0100, Cosimo Streppone wrote: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps I'm

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Ron
At 11:11 AM 12/13/2006, Cosimo Streppone wrote: Interesting, eh? Cosimo What I find interesting is that so far Guido's C2D Mac laptop has gotten the highest values by far in this set of experiments, and no one else is even close. The slowest results, Michael's, are on the system with what

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Bucky Jordan
What I find interesting is that so far Guido's C2D Mac laptop has gotten the highest values by far in this set of experiments, and no one else is even close. The slowest results, Michael's, are on the system with what appears to be the slowest CPU of the bunch; and the ranking of the rest of

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Ron
At 01:49 PM 12/13/2006, Bucky Jordan wrote: I've only seen pg_bench numbers 2,000 tps on either really large hardware (none of the above mentioned comes close) or the results are in memory due to a small database size (aka measuring update contention). Which makes a laptop achieving such

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Guido Neitzer
On 13.12.2006, at 19:03, Ron wrote: What I find interesting is that so far Guido's C2D Mac laptop has gotten the highest values by far in this set of experiments, and no one else is even close. This might be the case because I have tested with fsync=off as my internal harddrive would be

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Bruce Momjian
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: While skimming over the pgbench source it has looked to me like it's necessary to pass the -s switch (scale factor) to both the initialization (-i) and the subsequent (non -i) runs. No, it's not supposed to be, and I've never found

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Rajesh Kumar Mallah
[offtopic]; hmm quite a long thread below is stats of posting Total Messages:87Total Participants: 27 - 19 Daniel van Ham Colchete 12 Michael Stone 9 Ron 5 Steinar H. Gunderson 5 Alexander Staubo 4 Tom Lane 4

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Josh Berkus
Bruce, pgbench is designed to be a general benchmark, meanining it exercises all parts of the system. I am thinking just reexecuting a single SELECT over and over again would be a better test of the CPU optimizations. Mostly, though, pgbench just gives the I/O system a workout. It's not a

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Joshua D. Drake
On Wed, 2006-12-13 at 18:36 -0800, Josh Berkus wrote: Bruce, pgbench is designed to be a general benchmark, meanining it exercises all parts of the system. I am thinking just reexecuting a single SELECT over and over again would be a better test of the CPU optimizations. Mostly,

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes: On Wed, 2006-12-13 at 18:36 -0800, Josh Berkus wrote: Mostly, though, pgbench just gives the I/O system a workout. It's not a really good general workload. It also will not utilize all cpus on a many cpu machine. We recently found that the only way

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Michael Glaesemann
On Dec 14, 2006, at 14:44 , Tom Lane wrote: The pgbench app itself becomes the bottleneck at high transaction rates. Awhile back I rewrote it to improve its ability to issue commands concurrently, but then desisted from submitting the changes --- if we change the app like that, future numbers

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Ron
Benchmarks, like any other SW, need modernizing and updating from time to time. Given the multi-core CPU approach to higher performance as the current fad in CPU architecture, we need a benchmark that is appropriate. If SPEC feels it is appropriate to rev their benchmark suite regularly, we

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-13 Thread Gregory S. Williamson
; pgsql-performance@postgresql.org; Bruce Momjian; Alvaro Herrera; Alexander Staubo; Michael Stone Subject:Re: [PERFORM] New to PostgreSQL, performance considerations On Dec 14, 2006, at 14:44 , Tom Lane wrote: The pgbench app itself becomes the bottleneck at high transaction rates

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Daniel van Ham Colchete
Mike, are you using -mtune/-mcpu or -march with GCC? Witch GCC version? Are you working with a 32bits OS or 64bits? Daniel On 12/11/06, Michael Stone [EMAIL PROTECTED] wrote: Can anyone else reproduce these results? I'm on similar hardware (2.5GHz P4, 1.5G RAM) and my test results are more

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Guido Neitzer
On 12.12.2006, at 02:37, Michael Stone wrote: Can anyone else reproduce these results? I'm on similar hardware (2.5GHz P4, 1.5G RAM) and my test results are more like this: I'm on totally different hardware / software (MacBook Pro 2.33GHz C2D) and I can't reproduce the tests. I have

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Alexander Staubo
On Dec 11, 2006, at 23:22 , Daniel van Ham Colchete wrote: I ran this test at a Gentoo test machine I have here. It's a Pentium 4 3.0GHz (I don't know witch P4) Try cat /proc/cpuinfo. TESTS RESULTS == On a dual-core Opteron 280 with 4G RAM with an LSI PCI-X Fusion-MPT SAS

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Michael Stone
On Tue, Dec 12, 2006 at 01:35:04AM -0500, Greg Smith wrote: These changes could easily explain the magnitude of difference in results you're seeing, expecially when combined with a 20% greater raw CPU clock. I'm not interested in comparing the numbers between the systems (which is obviously

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Michael Stone
On Tue, Dec 12, 2006 at 07:10:34AM -0200, Daniel van Ham Colchete wrote: are you using -mtune/-mcpu or -march with GCC? I used exactly the options you said you used. Witch GCC version? Are you working with a 32bits OS or 64bits? 3.3.5; 32 Mike Stone ---(end of

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Michael Stone
On Tue, Dec 12, 2006 at 12:29:29PM +0100, Alexander Staubo wrote: I suspect the hardware's real maximum performance of the system is ~150 tps, but that the LSI's write cache is buffering the writes. I would love to validate this hypothesis, but I'm not sure how. With fsync off? The write

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Cosimo Streppone
Luke Lonergan wrote: Can you try this with just -O3 versus -O2? Thanks to Daniel for doing these tests. I happen to have done the same tests about 3/4 years ago, and concluded that gcc flags did *not* influence performance. Moved by curiosity, I revamped those tests now on a test machine

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Steinar H. Gunderson
On Tue, Dec 12, 2006 at 01:42:06PM +0100, Cosimo Streppone wrote: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps For the record, -O3 = -O6

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Michael Stone
On Tue, Dec 12, 2006 at 01:42:06PM +0100, Cosimo Streppone wrote: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps I'm curious now to get the same

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Michael Stone
On Tue, Dec 12, 2006 at 07:48:06AM -0500, Michael Stone wrote: I'd be curious to see -O2 with and without the arch-specific flags, since that's mostly what the discussion is about. That came across more harshly than I intended; I apologize for that. It's certainly a useful data point to

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Brad Nicholson
On Mon, 2006-12-11 at 20:22 -0200, Daniel van Ham Colchete wrote: I'm thinking about writing a script to make all the tests (more than 3 times each), get the data and plot some graphs. I don't have the time right now to do it, maybe next week I'll have. Check out the OSDL test suite stuff.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Alexander Staubo
On Dec 12, 2006, at 13:32 , Michael Stone wrote: On Tue, Dec 12, 2006 at 12:29:29PM +0100, Alexander Staubo wrote: I suspect the hardware's real maximum performance of the system is ~150 tps, but that the LSI's write cache is buffering the writes. I would love to validate this hypothesis,

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Ron
1= In all these results I'm seeing, no one has yet reported what their physical IO subsystem is... ...when we are benching a DB. 2= So far we've got ~ a factor of 4 performance difference between Michael Stone's 1S 1C Netburst era 2.5GHz P4 PC and Guido Neitzer's 1S 2C MacBook Pro 2.33GHz

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Florian Weimer
* Cosimo Streppone: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps -mcpu and -mtune are synonymous. You really should -march here (but the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Bill Moran
In response to Ron [EMAIL PROTECTED]: 3= Daniel van Ham Colchete is running Gentoo. That means every SW component on his box has been compiled to be optimized for the HW it is running on. There may be a combination of effects going on for him that others not running a system optimized

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Tom Lane
Alexander Staubo [EMAIL PROTECTED] writes: No, fsync=on. The tps values are similarly unstable with fsync=off, though -- I'm seeing bursts of high tps values followed by low-tps valleys, a kind of staccato flow indicative of a write caching being filled up and flushed. It's notoriously

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Daniel van Ham Colchete
On 12/12/06, Florian Weimer [EMAIL PROTECTED] wrote: * Cosimo Streppone: -O0 ~ 957 tps -O1 -mcpu=pentium4 -mtune=pentium4 ~ 1186 tps -O2 -mcpu=pentium4 -mtune=pentium4 ~ 1229 tps -O3 -mcpu=pentium4 -mtune=pentium4 ~ 1257 tps -O6 -mcpu=pentium4 -mtune=pentium4 ~ 1254 tps -mcpu and -mtune

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread David Boreham
Alexander Staubo wrote: No, fsync=on. The tps values are similarly unstable with fsync=off, though -- I'm seeing bursts of high tps values followed by low-tps valleys, a kind of staccato flow indicative of a write caching being filled up and flushed. Databases with checkpointing

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Greg Smith
On Tue, 12 Dec 2006, Tom Lane wrote: Um, you entirely missed the point: the hardware speedups you mention are quite independent of any compiler options. The numbers we are looking at are the relative speeds of two different compiles on the same hardware, not whether hardware A is faster than

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Alvaro Herrera
Tom Lane wrote: Alexander Staubo [EMAIL PROTECTED] writes: No, fsync=on. The tps values are similarly unstable with fsync=off, though -- I'm seeing bursts of high tps values followed by low-tps valleys, a kind of staccato flow indicative of a write caching being filled up and

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Greg Smith
On Tue, 12 Dec 2006, Alvaro Herrera wrote: While skimming over the pgbench source it has looked to me like it's necessary to pass the -s switch (scale factor) to both the initialization (-i) and the subsequent (non -i) runs. For non-custom runs, it's computed based on the number of branches.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Daniel van Ham Colchete
Mike, I'm making some other tests here at another hardware (also Gentoo). I found out that PostgreSQL stops for a while if I change the -t parameter on bgbench from 600 to 1000 and I have ~150 tps instead of ~950tps. I don't know why PostgreSQL stoped, but it was longer than 5 seconds and my

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: While skimming over the pgbench source it has looked to me like it's necessary to pass the -s switch (scale factor) to both the initialization (-i) and the subsequent (non -i) runs. No, it's not supposed to be, and I've never found it needed in

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Tom Lane
Daniel van Ham Colchete [EMAIL PROTECTED] writes: I'm making some other tests here at another hardware (also Gentoo). I found out that PostgreSQL stops for a while if I change the -t parameter on bgbench from 600 to 1000 and I have ~150 tps instead of ~950tps. I don't know why PostgreSQL

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Ron
At 10:47 AM 12/12/2006, Tom Lane wrote: It's notoriously hard to get repeatable numbers out of pgbench :-( That's not a good characteristic in bench marking SW... Does the ODSL stuff have an easier time getting reproducible results? A couple of tips: * don't put any faith in short

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Daniel van Ham Colchete
I just made another test with a second Gentoo machine: Pentium 4 3.0Ghz Prescott GCC 4.1.1 Glibc 2.4 PostgreSQL 8.1.5 Kernel 2.6.17 Same postgresql.conf as yesterday's. First test == GLIBC: -O2 -march=i686 PostgreSQL: -O2 -march=i686 Results: 974.638731 975.602142 975.882051

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Ron
At 01:35 PM 12/12/2006, Daniel van Ham Colchete wrote: I just made another test with a second Gentoo machine: snip The results showed no significant change. The conclusion of today's test would be that there are no improvement at PostgreSQL when using -march=prescott. I only see 3

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Bucky Jordan
I just made another test with a second Gentoo machine: Pentium 4 3.0Ghz Prescott GCC 4.1.1 Glibc 2.4 PostgreSQL 8.1.5 Kernel 2.6.17 Same postgresql.conf as yesterday's. First test == GLIBC: -O2 -march=i686 PostgreSQL: -O2 -march=i686 Results: 974.638731 975.602142

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Greg Smith
On Tue, 12 Dec 2006, Daniel van Ham Colchete wrote: I'm making some other tests here at another hardware (also Gentoo). I found out that PostgreSQL stops for a while if I change the -t parameter on bgbench from 600 to 1000 and I have ~150 tps instead of ~950tps. Sure sounds like a checkpoint

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Christopher Browne
[EMAIL PROTECTED] (Alexander Staubo) wrote: On Dec 12, 2006, at 13:32 , Michael Stone wrote: On Tue, Dec 12, 2006 at 12:29:29PM +0100, Alexander Staubo wrote: I suspect the hardware's real maximum performance of the system is ~150 tps, but that the LSI's write cache is buffering the writes.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Alexander Staubo
On Dec 11, 2006, at 04:35 , Tatsuo Ishii wrote: That's not the whole story. UTF-8 and other variable-width encodings don't provide a 1:1 mapping of logical characters to single bytes; in particular, combination characters opens the possibility of multiple different byte sequences mapping to the

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
On 12/11/06, Alexander Staubo [EMAIL PROTECTED] wrote: On Dec 11, 2006, at 02:47 , Daniel van Ham Colchete wrote: I never understood what's the matter between the ASCII/ISO-8859-1/UTF8 charsets to a database. They're all simple C strings that doesn't have the zero-byte in the midlle (like

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
Hi Dave, On 12/11/06, Dave Cramer [EMAIL PROTECTED] wrote: Hi Daniel On 10-Dec-06, at 8:02 PM, Daniel van Ham Colchete wrote: Hi Gene, at my postgresql.conf, the only non-comented lines are: fsync = off This can, and will result in lost data. I know... If there is a power failure things

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Dave Cramer
On 11-Dec-06, at 5:36 AM, Daniel van Ham Colchete wrote: Hi Dave, On 12/11/06, Dave Cramer [EMAIL PROTECTED] wrote: Hi Daniel On 10-Dec-06, at 8:02 PM, Daniel van Ham Colchete wrote: Hi Gene, at my postgresql.conf, the only non-comented lines are: fsync = off This can, and will result

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
Mike, unfortunally I don't have any benchmarks right now. Doing something like this with the same machine would be a 2-day work (at least). Installing a Gentoo and putting it to run well is not a quick task (although is easy). But, trust me on this one. It's worth it. Think of this: PostgreSQL

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
How much memory does this machine have and what version of postgresql are you using? It's only a test server with 512MB RAM, I only used it to see how well would the PostgreSQL do in a ugly case. Given that optimal performance for postgresql can require up to 50% of available memory, you

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 09:05:56AM -0200, Daniel van Ham Colchete wrote: But, trust me on this one. It's worth it. You know what? I don't. Think of this: PostgreSQL and GNU LibC use a lot of complex algorithms: btree, hashes, checksums, strings functions, etc... And you have a lot of ways to

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Michael Stone
On Mon, Dec 11, 2006 at 09:05:56AM -0200, Daniel van Ham Colchete wrote: unfortunally I don't have any benchmarks right now. That's fairly normal for gentoo users pushing their compile options. Mike Stone ---(end of broadcast)--- TIP 4: Have

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
On 12/11/06, Steinar H. Gunderson [EMAIL PROTECTED] wrote: On Mon, Dec 11, 2006 at 09:05:56AM -0200, Daniel van Ham Colchete wrote: But, trust me on this one. It's worth it. You know what? I don't. So test it yourself. Think of this: PostgreSQL and GNU LibC use a lot of complex algorithms:

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
This is very very very true :-)! I just remebered one case with MySQL. When I changed the distro from Conectiva 10 (rpm-based ended brazilian distro) to Gentoo, a MySQL operation that usually took 2 minutes to run, ended in 47 seconds. This is absolutely vage. I don't have how to prove it to

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Christopher Browne
Oops! [EMAIL PROTECTED] (Daniel van Ham Colchete) was seen spray-painting on a wall: But, trust me on this one. It's worth it. No, the point of performance analysis is that you *can't* trust the people that say trust me on this one. If you haven't got a benchmark where you can demonstrate a

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
You are right Christopher. Okay. Let's solve this matter. What PostgreSQL benchmark software should I use??? I'll test PostgreSQL 8.1 on a Fedora Core 6 and on a Gentoo. I'll get the same version FC6 uses and install it at my Gentoo. I'll use the same hardware (diferent partitions to each).

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:09:13AM -0200, Daniel van Ham Colchete wrote: You know what? I don't. So test it yourself. You're making the claims, you're supposed to be proving them... As I said, it is an example. Take floatpoint divisions. You have plenty of ways of doing it: 387, MMX, SSE,

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
On 12/11/06, Steinar H. Gunderson [EMAIL PROTECTED] wrote: On Mon, Dec 11, 2006 at 11:09:13AM -0200, Daniel van Ham Colchete wrote: You know what? I don't. So test it yourself. You're making the claims, you're supposed to be proving them... As I said, it is an example. Take floatpoint

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:17:06AM -0200, Daniel van Ham Colchete wrote: I just remebered one case with MySQL. When I changed the distro from Conectiva 10 (rpm-based ended brazilian distro) to Gentoo, a MySQL operation that usually took 2 minutes to run, ended in 47 seconds. How do you know

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:31:48AM -0200, Daniel van Ham Colchete wrote: What PostgreSQL benchmark software should I use??? Look up the list archives; search for TPC. I'll test PostgreSQL 8.1 on a Fedora Core 6 and on a Gentoo. I'll get the same version FC6 uses and install it at my Gentoo.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Merlin Moncure
On 12/11/06, Daniel van Ham Colchete [EMAIL PROTECTED] wrote: But, trust me on this one. It's worth it. Think of this: PostgreSQL and GNU LibC use a lot of complex algorithms: btree, hashes, checksums, strings functions, etc... And you have a lot of ways to compile it into binary code. Now you

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Chris Browne
[EMAIL PROTECTED] (Daniel van Ham Colchete) writes: You are right Christopher. Okay. Let's solve this matter. What PostgreSQL benchmark software should I use??? pgbench is one option. There's a TPC-W at pgFoundry (http://pgfoundry.org/projects/tpc-w-php/). There's the Open Source

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Ron
This definitely is the correct approach. Actually, Daniel van Ham Colchete may not be as all wet as some around here think. We've had previous data that shows that pg can become CPU bound (see previous posts by Josh Berkus and others regarding CPU overhead in what should be IO bound tasks).

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Michael Stone
On Mon, Dec 11, 2006 at 12:15:51PM -0500, Ron wrote: I'd say the fairest attitude is to do everything we can to support having the proper experiments done w/o presuming the results. Who's presuming results?[1] It is fair to say that making extraordinary claims without any evidence should be

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Ron
Statements like these can not be reasonably interpreted in any manner _except_ that of presuming the results: I expect that you'll discover, if you actually do these tests, that this belief (that using arch specific compiler options lead to better performing SW) is fairly much nonsense.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Luke Lonergan
Michael, On 12/11/06 9:31 AM, Michael Stone [EMAIL PROTECTED] wrote: [1] I will say that I have never seen a realistic benchmark of general code where the compiler flags made a statistically significant difference in the runtime. Here's one - I wrote a general purpose Computational Fluid

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Michael Stone
On Mon, Dec 11, 2006 at 01:20:50PM -0500, Ron wrote: (The validity of the claim has nothing to do with the skills or experience of the claimant or anyone else in the discussion. Only on the evidence.) Please go back and reread the original post. I don't think the response was unwarranted.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Michael Stone
On Mon, Dec 11, 2006 at 10:30:55AM -0800, Luke Lonergan wrote: Here's one - I wrote a general purpose Computational Fluid Dynamics analysis method used by hundreds of people to perform aircraft and propulsion systems analysis. That's kinda the opposite of what I meant by general code. I was

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Merlin Moncure
On 12/11/06, Ron [EMAIL PROTECTED] wrote: Statements like these can not be reasonably interpreted in any manner _except_ that of presuming the results: I expect that you'll discover, if you actually do these tests, that this belief (that using arch specific compiler options lead to better

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Ron
At 01:47 PM 12/11/2006, Michael Stone wrote: On Mon, Dec 11, 2006 at 01:20:50PM -0500, Ron wrote: (The validity of the claim has nothing to do with the skills or experience of the claimant or anyone else in the discussion. Only on the evidence.) Please go back and reread the original post.

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Daniel van Ham Colchete
On 12/11/06, Steinar H. Gunderson [EMAIL PROTECTED] wrote: On Mon, Dec 11, 2006 at 11:17:06AM -0200, Daniel van Ham Colchete wrote: I just remebered one case with MySQL. When I changed the distro from Conectiva 10 (rpm-based ended brazilian distro) to Gentoo, a MySQL operation that usually

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Ron
At 02:28 PM 12/11/2006, Merlin Moncure wrote: also, some people posting here, not necessarily me, are authority figures. :-) merlin Noam Chomsky was one of the most influential thinkers in Linguistics to yet have lived. He was proven wrong a number of times. Even within Linguistics.

  1   2   >