Re: [PERFORM] Sun performance - Major discovery!

2003-10-14 Thread Kenneth Marshall
Jeff,

My first concern with the -fast option is that it makes an executable
that is specific for the platform on which the compilation is run
unless other flags are given. My second concern is the effect it has
on IEEE floating point behavior w.r.t. rounding, error handling, 
And my third concern is that if you use -fast, all other code must
be compiled and linked with the -fast option for correct operation,
this includes any functional languages such as perl, python, R,...
That is a pretty big requirement for a default compilation flag.

Ken Marshall

On Thu, Oct 09, 2003 at 12:07:20PM -0400, Jeff wrote:
 On Thu, 9 Oct 2003, Bruce Momjian wrote:
 
   52 seconds to 19-20 seconds
 
  Wow, that's dramatic.  Do you want to propose some flags for non-gcc
  Solaris?  Is -fast the only one?  Is there one that suppresses those
  warnings or are they OK?
 
 
 Well. As I said, I didn't see an obvious way to hide those warnings.
 I'd love to make those warnings go away.  That is why I suggested perhaps
 printing a message to ensure the user knows that warnings may be printed
 when using sunsoft.
 
 -fast should be all you need - it picks the best settings to use for the
 platform that is doing the compile.
 
 
 --
 Jeff Trout [EMAIL PROTECTED]
 http://www.jefftrout.com/
 http://www.stuarthamm.net/
 
 
 
 ---(end of broadcast)---
 TIP 7: don't forget to increase your free space map settings

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Jeff
On Wed, 8 Oct 2003, Neil Conway wrote:

 Hey Jeff,

 On Wed, 2003-10-08 at 11:46, Jeff wrote:
  Yeah - like I expected it was able to generate much better code for
  _bt_checkkeys which was the #1 function in gcc on both sun  linux.

 If you get a minute, would it be possible to compare the performance of
 your benchmark under linux/gcc and solaris/gcc when PostgreSQL is
 compiled with -O3?

Sun:
gcc:
none: 60 seconds
-O: 21 seconds
-O2: 20 seconds
-O3: 19 seconds

suncc:
none: 52 seconds
-fast: 20 secondsish.

-fast is actually a macro that expands to the best settings for the
platform that is doing the compilation.


Linux:
-O2: 35
-O3: 40
Odd.. I wonder why it took longer. Perhaps gcc built some bad code?
I thought the results were odd there so I ran the test many times.. same
results! Swapped the binaries back (so -O2 was running) and boom. back to
35.

Sun gcc -O2 and suncc -fast both pass make check.


--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian

So you want -fast added as default for non-gcc Solaris?  You mentioned
there is a warning generated that we have to deal with?

---

Jeff wrote:
 On Wed, 8 Oct 2003, Neil Conway wrote:
 
  Hey Jeff,
 
  On Wed, 2003-10-08 at 11:46, Jeff wrote:
   Yeah - like I expected it was able to generate much better code for
   _bt_checkkeys which was the #1 function in gcc on both sun  linux.
 
  If you get a minute, would it be possible to compare the performance of
  your benchmark under linux/gcc and solaris/gcc when PostgreSQL is
  compiled with -O3?
 
 Sun:
 gcc:
 none: 60 seconds
 -O: 21 seconds
 -O2: 20 seconds
 -O3: 19 seconds
 
 suncc:
 none: 52 seconds
 -fast: 20 secondsish.
 
 -fast is actually a macro that expands to the best settings for the
 platform that is doing the compilation.
 
 
 Linux:
 -O2: 35
 -O3: 40
 Odd.. I wonder why it took longer. Perhaps gcc built some bad code?
 I thought the results were odd there so I ran the test many times.. same
 results! Swapped the binaries back (so -O2 was running) and boom. back to
 35.
 
 Sun gcc -O2 and suncc -fast both pass make check.
 
 
 --
 Jeff Trout [EMAIL PROTECTED]
 http://www.jefftrout.com/
 http://www.stuarthamm.net/
 
 
 
 ---(end of broadcast)---
 TIP 3: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly
 

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Jeff
On Thu, 9 Oct 2003, Bruce Momjian wrote:


 So you want -fast added as default for non-gcc Solaris?  You mentioned
 there is a warning generated that we have to deal with?


 Yeah, suncc generates a warning for _every_ file that says:
Warning: -xarch=native has been explicitly specified, or implicitly
specified by a macro option, -xarch=native on this architecture implies
-xarch=v8plusa which generates code that does not run on pre-UltraSPARC
processors

And then I get various warnings here and there...

lots of statement not reached as in ecpg's type.c module
The offending code is a big switch statement like:
case ECPGt_bool:
return (ECPGt_bool);
break;

And then any functiont aht uses PG_RETURN_NULL generates  warning:
end-of-loop code not reached

and a bunch of constant promoted to unsigned long long


And some places such as in fe-exec.c have code like this:
buflen = strlen(strtext);   /* will shrink, also we discover
if

where strtext is an unsigned char * which generates warning: argument #1
is incompatible with prototype:

and then various other type mismatches here and there.

I skimmed through the manpage.. it doesn't look like we can supress
these..


Not sure we want it to look like we have bad code if someone uses cc.
perhaps issue a ./configure notice or something?

gcc compiles things fine.


--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian

What is the performance win for the -fast flag again?

---

Jeff wrote:
 On Thu, 9 Oct 2003, Bruce Momjian wrote:
 
 
  So you want -fast added as default for non-gcc Solaris?  You mentioned
  there is a warning generated that we have to deal with?
 
 
  Yeah, suncc generates a warning for _every_ file that says:
 Warning: -xarch=native has been explicitly specified, or implicitly
 specified by a macro option, -xarch=native on this architecture implies
 -xarch=v8plusa which generates code that does not run on pre-UltraSPARC
 processors
 
 And then I get various warnings here and there...
 
 lots of statement not reached as in ecpg's type.c module
 The offending code is a big switch statement like:
 case ECPGt_bool:
 return (ECPGt_bool);
 break;
 
 And then any functiont aht uses PG_RETURN_NULL generates  warning:
 end-of-loop code not reached
 
 and a bunch of constant promoted to unsigned long long
 
 
 And some places such as in fe-exec.c have code like this:
 buflen = strlen(strtext);   /* will shrink, also we discover
 if
 
 where strtext is an unsigned char * which generates warning: argument #1
 is incompatible with prototype:
 
 and then various other type mismatches here and there.
 
 I skimmed through the manpage.. it doesn't look like we can supress
 these..
 
 
 Not sure we want it to look like we have bad code if someone uses cc.
 perhaps issue a ./configure notice or something?
 
 gcc compiles things fine.
 
 
 --
 Jeff Trout [EMAIL PROTECTED]
 http://www.jefftrout.com/
 http://www.stuarthamm.net/
 
 
 
 ---(end of broadcast)---
 TIP 5: Have you checked our extensive FAQ?
 
http://www.postgresql.org/docs/faqs/FAQ.html
 

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Jeff
On Thu, 9 Oct 2003, Bruce Momjian wrote:

  52 seconds to 19-20 seconds

 Wow, that's dramatic.  Do you want to propose some flags for non-gcc
 Solaris?  Is -fast the only one?  Is there one that suppresses those
 warnings or are they OK?


Well. As I said, I didn't see an obvious way to hide those warnings.
I'd love to make those warnings go away.  That is why I suggested perhaps
printing a message to ensure the user knows that warnings may be printed
when using sunsoft.

-fast should be all you need - it picks the best settings to use for the
platform that is doing the compile.


--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Jeff
On Thu, 9 Oct 2003, Kenneth Marshall wrote:

 Jeff,

 My first concern with the -fast option is that it makes an executable
 that is specific for the platform on which the compilation is run
 unless other flags are given. My second concern is the effect it has
 on IEEE floating point behavior w.r.t. rounding, error handling, 
 And my third concern is that if you use -fast, all other code must
 be compiled and linked with the -fast option for correct operation,
 this includes any functional languages such as perl, python, R,...
 That is a pretty big requirement for a default compilation flag.

 Ken Marshall


So you think we should leave PG alone and let it run horrifically slowly?
Do you have a better idea of how to do this?

And do you have evidence apps compiled with -fast linked to non -fast
(or gcc compiled) have problems?


--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Andrew Sullivan
On Thu, Oct 09, 2003 at 01:04:23PM -0400, Jeff wrote:
 
 So you think we should leave PG alone and let it run horrifically slowly?
 Do you have a better idea of how to do this?

Given the point in the release cycle, mightn't the FAQ_Solaris or
some other place be better for this for now?  I agree with the
concern.  I'd rather have slow'n'stable than fast-but-broken.

A

-- 

Andrew Sullivan 204-4141 Yonge Street
Afilias CanadaToronto, Ontario Canada
[EMAIL PROTECTED]  M2P 2A8
 +1 416 646 3304 x110


---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian
Andrew Sullivan wrote:
 On Thu, Oct 09, 2003 at 01:04:23PM -0400, Jeff wrote:
  
  So you think we should leave PG alone and let it run horrifically slowly?
  Do you have a better idea of how to do this?
 
 Given the point in the release cycle, mightn't the FAQ_Solaris or
 some other place be better for this for now?  I agree with the
 concern.  I'd rather have slow'n'stable than fast-but-broken.

FAQ added.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Jeff
We're keeping the -O2 for gcc in the template and moving the mention of
-fast to the FAQ, correct?

--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian
Jeff wrote:
 We're keeping the -O2 for gcc in the template and moving the mention of
 -fast to the FAQ, correct?

gcc gets -O2, non-gcc gets -O, and -fast is in the FAQ, yea.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Christopher Browne
[EMAIL PROTECTED] (Bruce Momjian) writes:
 5) How can I compile for optimum performance?

 Try using the -fast compile flag.  The binaries might not be portable to
 other Solaris systems, and you might need to compile everything that links
 to PostgreSQL with -fast, but PostgreSQL will run significantly faster,
 50% faster on some tests.

You might also mention something like the following:

  If you are compiling using GCC, you will quite likely want to add in
  the -O2 compile flag.
-- 
let name=cbbrowne and tld=libertyrms.info in String.concat @ [name;tld];;
http://dev6.int.libertyrms.com/
Christopher Browne
(416) 646 3304 x124 (land)

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian
Christopher Browne wrote:
 [EMAIL PROTECTED] (Bruce Momjian) writes:
  5) How can I compile for optimum performance?
 
  Try using the -fast compile flag.  The binaries might not be portable to
  other Solaris systems, and you might need to compile everything that links
  to PostgreSQL with -fast, but PostgreSQL will run significantly faster,
  50% faster on some tests.
 
 You might also mention something like the following:
 
   If you are compiling using GCC, you will quite likely want to add in
   the -O2 compile flag.

We already do that by default in current CVS for gcc, and -O for
non-gcc.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Neil Conway
On Wed, 2003-10-08 at 21:44, Bruce Momjian wrote:
 Agreed.  Do we set them all to -O2, then remove it from the ones we
 don't get successful reports on?

I took the time to compile CVS tip with a few different machines from
HP's TestDrive program, to see if there were any regressions using the
new optimization flags:

(1) (my usual dev machine)

$ uname -a
Linux tokyo 2.4.19-xfs #1 Mon Jan 20 19:12:29 EST 2003 i686 GNU/Linux
$ gcc --version
gcc (GCC) 3.3.2 20031005 (Debian prerelease)

'make check' passes

(2)

$ uname -a
Linux spe161 2.4.18-smp #1 SMP Sat Apr 6 21:42:22 EST 2002 alpha unknown
$ gcc --version
gcc (GCC) 3.3.1

'make check' passes

(3)

$ uname -a
Linux spe170 2.4.17-64 #1 Sat Mar 16 17:31:44 MST 2002 parisc64 unknown
$ gcc --version
3.0.4

'make check' passes

BTW, this platform doesn't have any code written for native spinlocks.

(4)

$ uname -a
Linux spe156 2.4.18-mckinley-smp #1 SMP Thu Jul 11 12:51:02 MDT 2002
ia64 unknown
$ gcc --version

When you compile PostgreSQL without changing the CFLAGS configure picks,
the initdb required for 'make check' fails with:

[...]
initializing pg_depend... ok
creating system views... ok
loading pg_description... ok
creating conversions... ERROR:  could not identify operator 679

I tried to compile PostgreSQL with CFLAGS='-O0' to see if the above
resulted from an optimization-induced compiler error, but I got the
following error:

$ gcc -O0 -Wall -Wmissing-prototypes -Wmissing-declarations
-I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
../../../../src/include/storage/s_lock.h: In function `tas':
../../../../src/include/storage/s_lock.h:125: error: inconsistent
operand constraints in an `asm'

Whereas this works fine:

$ gcc -O2 -Wall -Wmissing-prototypes -Wmissing-declarations
-I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
$

BTW, line 138 of s_lock.h is:

#if defined(__arm__) || defined(__arm__)

That seems a little redundant.

Anyway, I tried running initdb after compiling all of pgsql with -O0',
except for the files that included s_lock.h, but make check still
failed:

creating information schema... ok
vacuuming database template1...
/house/neilc/pgsql/src/test/regress/./tmp_check/install//usr/local/pgsql/bin/initdb: 
line 882: 22035 Segmentation fault  (core dumped) $PGPATH/postgres $PGSQL_OPT 
template1 /dev/null  EOF
ANALYZE;
VACUUM FULL FREEZE;
EOF

The core file seems to indicate a stack overflow due to an infinitely
recursive function:

(gdb) bt 25
#0  0x40645dc0 in hash_search ()
#1  0x40616930 in RelationSysNameCacheGetRelation ()
#2  0x40616db0 in RelationSysNameGetRelation ()
#3  0x40082e40 in relation_openr ()
#4  0x40083910 in heap_openr ()
#5  0x4060e6b0 in ScanPgRelation ()
#6  0x40611d60 in RelationBuildDesc ()
#7  0x40616e70 in RelationSysNameGetRelation ()
#8  0x40082e40 in relation_openr ()
#9  0x40083910 in heap_openr ()
#10 0x4060e6b0 in ScanPgRelation ()
#11 0x40611d60 in RelationBuildDesc ()
#12 0x40616e70 in RelationSysNameGetRelation ()
#13 0x40082e40 in relation_openr ()
#14 0x40083910 in heap_openr ()
#15 0x4060e6b0 in ScanPgRelation ()
#16 0x40611d60 in RelationBuildDesc ()
#17 0x40616e70 in RelationSysNameGetRelation ()
#18 0x40082e40 in relation_openr ()
#19 0x40083910 in heap_openr ()
#20 0x4060e6b0 in ScanPgRelation ()
#21 0x40611d60 in RelationBuildDesc ()
#22 0x40616e70 in RelationSysNameGetRelation ()
#23 0x40082e40 in relation_openr ()
#24 0x40083910 in heap_openr ()
(More stack frames follow...)

(It also dumps core in the same place during initdb if CFLAGS='-O' is
specified.)

So it looks like the Itanium port is a little broken. Does anyone have
an idea what needs to be done to fix it?

-Neil



---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [PERFORM] Sun performance - Major discovery!

2003-10-09 Thread Bruce Momjian

Isn't it great how you have the same directory on every host so you can
download once and run the same tests easily.


Neil Conway wrote:
 $ uname -a
 Linux spe170 2.4.17-64 #1 Sat Mar 16 17:31:44 MST 2002 parisc64 unknown
 $ gcc --version
 3.0.4
 
 'make check' passes

I didn't know there was a pa-risc-64 chip.

 BTW, this platform doesn't have any code written for native spinlocks.
 
 (4)
 
 $ uname -a
 Linux spe156 2.4.18-mckinley-smp #1 SMP Thu Jul 11 12:51:02 MDT 2002
 ia64 unknown
 $ gcc --version
 
 When you compile PostgreSQL without changing the CFLAGS configure picks,
 the initdb required for 'make check' fails with:
 
 [...]
 initializing pg_depend... ok
 creating system views... ok
 loading pg_description... ok
 creating conversions... ERROR:  could not identify operator 679
 
 I tried to compile PostgreSQL with CFLAGS='-O0' to see if the above
 resulted from an optimization-induced compiler error, but I got the
 following error:
 
 $ gcc -O0 -Wall -Wmissing-prototypes -Wmissing-declarations
 -I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
 ../../../../src/include/storage/s_lock.h: In function `tas':
 ../../../../src/include/storage/s_lock.h:125: error: inconsistent
 operand constraints in an `asm'
 
 Whereas this works fine:
 
 $ gcc -O2 -Wall -Wmissing-prototypes -Wmissing-declarations
 -I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
 $
 
 BTW, line 138 of s_lock.h is:
 
 #if defined(__arm__) || defined(__arm__)

Fix just committed.  Thanks.

 That seems a little redundant.
 
 Anyway, I tried running initdb after compiling all of pgsql with -O0',
 except for the files that included s_lock.h, but make check still
 failed:
 
 creating information schema... ok
 vacuuming database template1...
 /house/neilc/pgsql/src/test/regress/./tmp_check/install//usr/local/pgsql/bin/initdb: 
 line 882: 22035 Segmentation fault  (core dumped) $PGPATH/postgres $PGSQL_OPT 
 template1 /dev/null  EOF
 ANALYZE;
 VACUUM FULL FREEZE;
 EOF
 
 The core file seems to indicate a stack overflow due to an infinitely
 recursive function:
 
 (gdb) bt 25
 #0  0x40645dc0 in hash_search ()
 #1  0x40616930 in RelationSysNameCacheGetRelation ()
 #2  0x40616db0 in RelationSysNameGetRelation ()
 #3  0x40082e40 in relation_openr ()
 #4  0x40083910 in heap_openr ()
 #5  0x4060e6b0 in ScanPgRelation ()
 #6  0x40611d60 in RelationBuildDesc ()
 #7  0x40616e70 in RelationSysNameGetRelation ()
 #8  0x40082e40 in relation_openr ()
 #9  0x40083910 in heap_openr ()
 #10 0x4060e6b0 in ScanPgRelation ()
 #11 0x40611d60 in RelationBuildDesc ()
 #12 0x40616e70 in RelationSysNameGetRelation ()
 #13 0x40082e40 in relation_openr ()
 #14 0x40083910 in heap_openr ()
 #15 0x4060e6b0 in ScanPgRelation ()
 #16 0x40611d60 in RelationBuildDesc ()
 #17 0x40616e70 in RelationSysNameGetRelation ()
 #18 0x40082e40 in relation_openr ()
 #19 0x40083910 in heap_openr ()
 #20 0x4060e6b0 in ScanPgRelation ()
 #21 0x40611d60 in RelationBuildDesc ()
 #22 0x40616e70 in RelationSysNameGetRelation ()
 #23 0x40082e40 in relation_openr ()
 #24 0x40083910 in heap_openr ()
 (More stack frames follow...)
 
 (It also dumps core in the same place during initdb if CFLAGS='-O' is
 specified.)
 
 So it looks like the Itanium port is a little broken. Does anyone have
 an idea what needs to be done to fix it?

My guess is that the compiler itself is broken --- what else could it
be?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Andrew Sullivan
On Wed, Oct 08, 2003 at 08:36:56AM -0400, Jeff wrote:
 
 So here's the results using my load tester (single connection per beater,
 repeats the query 1000 times with different input each time (we'll get
 ~20k rows back), the query is a common query around here.

My worry about this test is that it gives us precious little
knowledge about concurrent connection slowness, which is where I find
the most significant problems.  When we tried a Sunsoft cc vs gcc 2.95
on Sol 7 about 1 1/2 years ago, we found more or less no difference
once we added more than 5 connections (and we always have more than 5
connections).  It might be worth trying again, though, since we moved
to Sol 8.

Thanks for the result. 

-- 

Andrew Sullivan 204-4141 Yonge Street
Afilias CanadaToronto, Ontario Canada
[EMAIL PROTECTED]  M2P 2A8
 +1 416 646 3304 x110


---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Neil Conway
On Wed, 2003-10-08 at 08:36, Jeff wrote:
 So here's the results using my load tester (single connection per beater,
 repeats the query 1000 times with different input each time (we'll get
 ~20k rows back), the query is a common query around here.

What is the query?

 Linux - 1x - 35 seconds, 20x - 180 seconds

20x means 20 concurrent testing processes, right?

 Sun - gcc - 1x 60 seconds  20x 245 seconds
 Sun - sunsoft defaults - 1x 52 seonds 20x [similar to gcc most likely]
 Sun - sunsoft -fast  - 1x 28 seconds  20x 164 seconds

Interesting (and surprising that the performance differential is that
large, to me at least). Can you tell if the performance gain comes from
an improvement in a particular subsystem? (i.e. could you get a profile
of Sun/gcc and compare it with Sun/sunsoft).

-Neil



---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Jeff
On Wed, 8 Oct 2003, Andrew Sullivan wrote:

 My worry about this test is that it gives us precious little
 knowledge about concurrent connection slowness, which is where I find
 the most significant problems.  When we tried a Sunsoft cc vs gcc 2.95
 on Sol 7 about 1 1/2 years ago, we found more or less no difference
 once we added more than 5 connections (and we always have more than 5
 connections).  It might be worth trying again, though, since we moved
 to Sol 8.


The 20x column are the results when I fired up 20 beater concurrently.


--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Jeff
On Wed, 8 Oct 2003, Neil Conway wrote:

 What is the query?


It retrieves an index listing for our boards. The boards are flat (not
threaded) and messages are numbered starting at 1 for each board.

If you pass in 0 for the start_from it assumes the latest 60.

And it should be noted - in some cases some boards have nearly 2M posts.
Index on board_name, number.

I cannot give out too too much stuff ;)

create or replace function get_index2(integer, varchar, varchar)
returns setof snippet
as '
DECLARE
p_start alias for $1;
p_board alias for $2;
v_start integer;
v_num integer;
v_body text;
v_sender varchar(35);
v_time timestamptz;
v_finish integer;
v_row record;
v_ret snippet;
BEGIN

v_start := p_start;

if v_start = 0 then
select * into v_start from get_high_msg(p_board);
v_start := v_start - 59;
end if;

v_finish := v_start + 60;

for v_row in
select number, substr(body, 0, 50) as snip, member_handle,
timestamp
from posts
where board_name = p_board and
number = v_start and
number  v_finish
order by number desc
LOOP
return next v_row;
END LOOP;

return;
END;
' language 'plpgsql';


 Interesting (and surprising that the performance differential is that
 large, to me at least). Can you tell if the performance gain comes from
 an improvement in a particular subsystem? (i.e. could you get a profile
 of Sun/gcc and compare it with Sun/sunsoft).


I'll get these later today.

--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Jeff
On Wed, 8 Oct 2003, Neil Conway wrote:

 Interesting (and surprising that the performance differential is that
 large, to me at least). Can you tell if the performance gain comes from
 an improvement in a particular subsystem? (i.e. could you get a profile
 of Sun/gcc and compare it with Sun/sunsoft).


Yeah - like I expected it was able to generate much better code for
_bt_checkkeys which was the #1 function in gcc on both sun  linux.

and as you can see, suncc was just able to generate much nicer code. I'd
look at the assembler output but that won't be useful since I am very
unfamiliar with the [ultra]sparc instruction set..


Here's the prof and gprof output for the latest run:
GCC:
  %   cumulative   self  self total
 time   seconds   secondscalls  ms/call  ms/call  name
 31.52 19.4419.44 internal_mcount
 20.28 31.9512.51  8199466 0.00 0.00  _bt_checkkeys
  5.61 35.41 3.46  8197422 0.00 0.00  _bt_step
  5.01 38.50 3.09 24738620 0.00 0.00  FunctionCall2
  3.00 40.35 1.85  8194186 0.00 0.00  varchareq
  2.61 41.96 1.6124309 0.07 0.28  _bt_next
  2.42 43.45 1.49 1003 1.49 1.51  AtEOXact_Buffers
  2.37 44.91 1.4612642 0.12 0.12  _read
  2.33 46.35 1.44 16517771 0.00 0.00  pg_detoast_datum
  2.08 47.63 1.28  8193186 0.00 0.00  int4lt
  1.35 48.46 0.83  8237204 0.00 0.00  BufferGetBlockNumber
  1.35 49.29 0.83  8193888 0.00 0.00  int4ge
  1.35 50.12 0.83 _mcount


SunCC -pg -fast.
 %Time Seconds Cumsecs  #Calls   msec/call  Name

  23.24.274.27108922056  0.  _mcount
  20.73.828.09 8304052  0.0005  _bt_checkkeys
  13.72.53   10.6225054788  0.0001  FunctionCall2
   5.10.94   11.56   24002  0.0392  _bt_next
   4.40.81   12.37 8301867  0.0001  _bt_step
   3.40.63   13.00 8298219  0.0001  varchareq
   2.70.50   13.5016726855  0.  pg_detoast_datum
   2.40.45   13.95 8342464  0.0001  BufferGetBlockNumber
   2.40.44   14.39 8297941  0.0001  int4ge
   2.20.41   14.801003  0.409   AtEOXact_Buffers
   2.00.37   15.17 4220349  0.0001  lc_collate_is_c
   2.00.37   15.54 8297219  0.  int4lt
   1.60.29   15.83   26537  0.0109  AllocSetContextCreate
   0.90.16   15.991887  0.085   pglz_decompress
   0.70.13   16.12  159966  0.0008  nocachegetattr
   0.70.13   16.25 4220349  0.  varstr_cmp
   0.60.11   16.36  937576  0.0001  MemoryContextAlloc
   0.50.09   16.45  150453  0.0006  hash_search





 -Neil



 ---(end of broadcast)---
 TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]



--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Neil Conway
On Wed, 2003-10-08 at 10:48, Andrew Sullivan wrote:
 My worry about this test is that it gives us precious little
 knowledge about concurrent connection slowness, which is where I find
 the most significant problems.

As Jeff points out, the second set of results is for 20 concurrent
connections. Note that the advantage sunsoft cc has over gcc decreases
as the number of connections increases (which makes sense, as the 20x
workload is likely to be more I/O bound).

-Neil



---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Neil Conway
On Wed, 2003-10-08 at 11:46, Jeff wrote:
 Yeah - like I expected it was able to generate much better code for
 _bt_checkkeys which was the #1 function in gcc on both sun  linux.
 
 and as you can see, suncc was just able to generate much nicer code.

What CFLAGS does configure pick for gcc? From
src/backend/template/solaris, I'd guess it's not enabling any
optimization. Is that the case? If so, some gcc numbers with -O and -O2
would be useful.

-Neil



---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Jeff
On Wed, 8 Oct 2003, Neil Conway wrote:


 What CFLAGS does configure pick for gcc? From
 src/backend/template/solaris, I'd guess it's not enabling any
 optimization. Is that the case? If so, some gcc numbers with -O and -O2
 would be useful.


I can't believe I didn't think of this before! heh.
Turns out gcc was getting nothing for flags.

I added -O2 to CFLAGS and my 60 seconds went down to 21.  A rather mild
improvment huh?

I did a few more tests and suncc still beats it out - but not by too much
now (Not enought to justify buying a license just for compiling pg)

I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
See if they pass the test.

If they do we should consider adding -O2 and -fast to the CFLAGS.

--
Jeff Trout [EMAIL PROTECTED]
http://www.jefftrout.com/
http://www.stuarthamm.net/



---(end of broadcast)---
TIP 8: explain analyze is your friend


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread pgsql-performance

In message [EMAIL PROTECTED], Jeff writes:

I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
See if they pass the test.

My default set of gcc optimization flags is:

-O3 -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -falign-functions 
-mcpu=i686 -march=i686

Obviously the last two flags product CPU specific code, so would have
to differ...autoconf is always possible, but so is just lopping them off.

I have found these flags to produce faster code that a simple -O2, but
I understand the exact combination which is best for you is
code-dependent.  Of course, if you are getting really excited, you can
use -fbranch-probabilities, but as you will see if you investigate
that requires some profiling information, so is not very easy to
actually practically use.

-Seth Robertson

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Bruce Momjian
Jeff wrote:
 On Wed, 8 Oct 2003, Neil Conway wrote:
 
 
  What CFLAGS does configure pick for gcc? From
  src/backend/template/solaris, I'd guess it's not enabling any
  optimization. Is that the case? If so, some gcc numbers with -O and -O2
  would be useful.
 
 
 I can't believe I didn't think of this before! heh.
 Turns out gcc was getting nothing for flags.
 
 I added -O2 to CFLAGS and my 60 seconds went down to 21.  A rather mild
 improvment huh?
 
 I did a few more tests and suncc still beats it out - but not by too much
 now (Not enought to justify buying a license just for compiling pg)
 
 I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
 See if they pass the test.
 
 If they do we should consider adding -O2 and -fast to the CFLAGS.

[ CC added for hackers.]

Well, this is really embarassing.  I can't imagine why we would not set
at least -O on all platforms.  Looking at the template files, I see
these have no optimization set:

darwin
dgux
freebsd (non-alpha)
irix5
nextstep
osf (gcc)
qnx4
solaris
sunos4
svr4
ultrix4

I thought we used to have code that did -O for any platforms that set no
cflags, but I don't see that around anywhere.  I recommend adding -O2,
or at leaset -O to all these platforms --- we can then use platform
testing to make sure they are working.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Neil Conway
On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
 Well, this is really embarassing.  I can't imagine why we would not set
 at least -O on all platforms.

ISTM the most legitimate reason for not enabling compilater
optimizations on a given compiler/OS/architecture combination is might
cause compiler errors / bad code generation.

Can we get these optimizations enabled in time for the next 7.4 beta? It
might also be good to add an item in the release notes about it.

-Neil



---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings


Re: [PERFORM] Sun performance - Major discovery!

2003-10-08 Thread Bruce Momjian
Tom Lane wrote:
 Neil Conway [EMAIL PROTECTED] writes:
  On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
  Well, this is really embarassing.  I can't imagine why we would not set
  at least -O on all platforms.
 
 I believe that autoconf will automatically select -O2 (when CFLAGS isn't
 already set) *if* it's chosen gcc.  It won't select anything for vendor
 ccs.

I think the problem is that template/solaris overrides that with:

  CFLAGS=

  Can we get these optimizations enabled in time for the next 7.4 beta?
 
 I think it's too late in the beta cycle to add optimization flags except
 for platforms we can get specific success results for.  (Solaris is
 probably okay for instance.)  The risk of breaking things seems too
 high.

Agreed.  Do we set them all to -O2, then remove it from the ones we
don't get successful reports on?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings