Re: [HACKERS] Trivial HugeTLB Benchmark

[Jim Nasby]

On Mar 4, 2007, at 3:33 PM, Ryan Cumming wrote:

I did another 18 runs, 9 each for huge pages and normal shared memory.
The database was reinitialized before every third run with pgbench -i
-s 10. The runs themselves were done with pgbench -s 10 -c 5 -t  
1


Rather than doing that, I think you'd be much better off just running  
a very long benchmark and turning on autovaccum. That would at least  
be closer to real-world usage.

--
Jim Nasby[EMAIL PROTECTED]
EnterpriseDB  http://enterprisedb.com  512.569.9461 (cell)



---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Trivial HugeTLB Benchmark

[Ryan Cumming]

On Sun, 2007-03-04 at 10:14 -0800, Tom Lane wrote:
 If you did this only once, the results are not really trustworthy;
 you need to average several similar runs before you can have much
 confidence.  pgbench's inter-run variation is usually upwards of 10%,
 so trying to draw conclusions about half-percentage-point differences
 without averaging is a waste of time.
Good point, thanks

 Also, if scaling factor  number of clients then what you're mostly
 measuring is update-contention behavior.  Try it with -s 10 and -c 5;
 and don't forget to reinitialize the database for each run of tests
 to be sure it's fair.
 
I did another 18 runs, 9 each for huge pages and normal shared memory.
The database was reinitialized before every third run with pgbench -i
-s 10. The runs themselves were done with pgbench -s 10 -c 5 -t 1

Normal shared memory:
tps = 1835.929043 (including connections establishing)
tps = 1697.455165 (including connections establishing)
tps = 1378.393001 (including connections establishing)

tps = 1834.802729 (including connections establishing)
tps = 1630.100895 (including connections establishing)
tps = 1415.504943 (including connections establishing)

tps = 1864.908838 (including connections establishing)
tps = 1726.295622 (including connections establishing)
tps = 1323.679649 (including connections establishing)

Average: 1634.19 tps

Huge pages:
tps = 1867.400381 (including connections establishing)
tps = 1715.269338 (including connections establishing)
tps = 1259.314139 (including connections establishing)

tps = 1880.803336 (including connections establishing)
tps = 1885.351404 (including connections establishing)
tps = 1603.302855 (including connections establishing)

tps = 1884.888431 (including connections establishing)
tps = 1563.452093 (including connections establishing)
tps = 1361.896887 (including connections establishing)

Average: 1669.08

That works out to approximately a 2.1% performance boost for huge pages.
It still doesn't seem very compelling but I could try to put together a
patch for inclusion if people were interested in such a thing.

-Ryan
 
This electronic mail transmission and any accompanying attachments contain 
confidential information intended only for the use of the individual or entity 
named above. Any dissemination, distribution, copying or action taken in 
reliance on the contents of this communication by anyone other than the 
intended recipient is strictly prohibited. If you have received this 
communication in error please immediately delete the e-mail and either notify 
the sender at the above e-mail address or by telephone at 250.386.5323.

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org

Re: [HACKERS] Trivial HugeTLB Benchmark

[Tom Lane]

Ryan Cumming [EMAIL PROTECTED] writes:
 I ran each pgbench after a fresh reboot. I used 85 huge pages reserved at 
 boot for the huge page test, and none for the normal shared memory test.

 Normal shared memory:
 -bash-3.00$ pgbench -c 5 -t 1
 starting vacuum...end.
 transaction type: TPC-B (sort of)
 scaling factor: 1
 number of clients: 5
 number of transactions per client: 1
 number of transactions actually processed: 5/5
 tps = 1669.009344 (including connections establishing)
 tps = 1669.941756 (excluding connections establishing)

If you did this only once, the results are not really trustworthy;
you need to average several similar runs before you can have much
confidence.  pgbench's inter-run variation is usually upwards of 10%,
so trying to draw conclusions about half-percentage-point differences
without averaging is a waste of time.

Also, if scaling factor  number of clients then what you're mostly
measuring is update-contention behavior.  Try it with -s 10 and -c 5;
and don't forget to reinitialize the database for each run of tests
to be sure it's fair.

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly