This is what you get if you merge the previous four reports (and filter out the
hand-tweaked assembly benchmarks).
I generated the report with the following command:
EXCLUDE='^hand/' \
tools/merge.pl \
ghc-armada-thorough-6.9.tgz \
ghc-fischer-thorough-6.9.tgz \
ghc-albatross-thorough-6.9.tgz \
ghc-thorough-6.9.tgz \
> xx
This graph shows the memory usage to be (almost) exactly the same. The
difference can easily be explained with slightly different versions of the
C library (there was also a security update or two from Ubuntu in the last
few days - the kernel was definitely updated and I think the C library was
too). We are also talking about two versions of Ubuntu and one (old) version
of SuSE. And we are talking about three not quite identical versions of ghc.
The fischer and albatross runs were made before the inclusion of the three
benchmarks from Don Stewart (hs/space-bs-c8-count, hs/space-bslc8-count, and
hs/space-bslc8-chunk-4) so they are a couple of holes in the graphs.
The speed pattern is more fun. It makes no sense to compare absolute times
here so the graphs were not rescaled. One would naïvely expect the bars to
be about the same when taken in groups of four but that really turns out to be
far from the case!
The explanation is most likely that we are looking at four very different
microarchitectures.
**********
This should hammer home the point that benchmarking on any single machine
isn't enough!
**********
Two probably aren't enough either...
-Peter
ls-search
ghc 6.9.20071119
Pentium III (Coppermine)
596.932 MHz
TESTKIND=THOROUGH
SUFFIX=
Fischer's machine
ghc 6.9.20071124
AMD Duron(tm) processor
1200.089 MHz
TESTKIND=THOROUGH
SUFFIX=
albatross
ghc 6.9.20071208 (or thereabouts)
Genuine Intel(R) CPU T2300 @ 1.66GHz (Core Duo)
1667.000 MHz
TESTKIND=THOROUGH
SUFFIX=
charybdis
ghc 6.9.20071119
AMD Athlon(tm) 64 Processor 3000+
2009.160 MHz
TESTKIND=THOROUGH
SUFFIX=
Time (byte counting) std
-------------------- avg dev slack
hs/byte-bs----acc: 3.274 1â° 0.1 ââââ
|
-- 1.892 21â° 0.4 âââ
|
-- 0.918 4â° 0.1 ââââââ
|
-- 0.705 7â° 0.1 ââââ
|
hs/byte-bs----foldlx: 4.027 0â° 0.0 ââââ
|
-- 2.258 3â° 0.1 ââââ
|
-- 1.014 2â° 0.2 ââââââ
|
-- 1.002 1â° 0.5 ââââââ
|
hs/byte-bs----foldrx: 4.184 1â° 0.0 ââââ
|
-- 2.933 0â° 0.1 âââââ
|
-- 0.999 2â° 0.1 ââââââ
|
-- 1.112 2â° 0.1 ââââââ
|
hs/byte-bsl---acc: 28.005 10â° 0.0
âââââââââââââââââââââââââââ
|
-- 14.319 45â° 0.1
âââââââââââââââââââââ |
-- 1.957 1â° 0.1 ââââââââââââ
|
-- 2.595 14â° 0.1
ââââââââââââââ |
hs/byte-xxxxx-acc-1: 25.852 4â° 0.0
âââââââââââââââââââââââââ
|
-- 20.915 17â° 4.0
ââââââââââââââââââââââââââââââ
|
-- 3.591 2â° 0.1
ââââââââââââââââââââââ
|
-- 4.436 5â° 0.0
ââââââââââââââââââââââââ
|
hs/byte-xxxxx-acc-2: 25.622 5â° 0.0
âââââââââââââââââââââââââ
|
-- 20.691 8â° 0.1
ââââââââââââââââââââââââââââââ
|
-- 3.598 4â° 0.1
ââââââââââââââââââââââ
|
-- 4.473 8â° 0.0
ââââââââââââââââââââââââ
|
hs/byte-xxxxx-foldl: 25.803 5â° 0.0
âââââââââââââââââââââââââ
|
-- 20.610 5â° 1.4
ââââââââââââââââââââââââââââââ
|
-- 3.594 1â° 0.1
ââââââââââââââââââââââ
|
-- 4.412 10â° 0.0
âââââââââââââââââââââââ
|
c/byte-getchar: 13.016 0â° 0.0
âââââââââââââ |
-- 9.042 0â° 0.1
âââââââââââââ |
-- 3.761 2â° 0.1
âââââââââââââââââââââââ
|
-- 2.422 27â° 0.1
âââââââââââââ |
c/byte-getchar-u: 1.662 1â° 0.1 ââ
|
-- 1.314 3â° 0.2 ââ
|
-- 0.719 2â° 0.3 âââââ
|
-- 0.632 3â° 0.3 ââââ
|
c/byte-4k: 0.543 2â° 0.2 â
|
-- 0.419 5â° 0.5 â
|
-- 0.101 10â° 3.1 â
|
-- 0.094 26â° 2.2 â
|
Memory: Peak
------- KB
hs/byte-bs----acc: 147752
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147492
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147784
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147752
ââââââââââââââââââââââââââââââââââââââââ
|
hs/byte-bs----foldlx: 147756
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147492
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147812
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147748
ââââââââââââââââââââââââââââââââââââââââ
|
hs/byte-bs----foldrx: 147760
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147488
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147788
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147744
ââââââââââââââââââââââââââââââââââââââââ
|
hs/byte-bsl---acc: 3180 â
|
-- 2896 â
|
-- 3228 â
|
-- 3172 â
|
hs/byte-xxxxx-acc-1: 1916 â
|
-- 1612 â
|
-- 1904 â
|
-- 1904 â
|
hs/byte-xxxxx-acc-2: 1912 â
|
-- 1612 â
|
-- 1908 â
|
-- 1904 â
|
hs/byte-xxxxx-foldl: 1912 â
|
-- 1612 â
|
-- 1908 â
|
-- 1900 â
|
c/byte-getchar: 436 â
|
-- 384 â
|
-- 432 â
|
-- 440 â
|
c/byte-getchar-u: 432 â
|
-- 384 â
|
-- 440 â
|
-- 440 â
|
c/byte-4k: 436 â
|
-- 380 â
|
-- 436 â
|
-- 440 â
|
Time (space counting) std
--------------------- avg dev slack
hs/space-bs-c8-acc-1: 4.318 1â° 0.0 ââââ
|
-- 2.467 1â° 0.3 ââââ
|
-- 1.313 0â° 0.2 ââââââââ
|
-- 1.145 1â° 0.2 ââââââ
|
hs/space-bs-c8-count: 3.118 1â° 0.1 âââ
|
-- n/a
-- n/a
-- 0.521 1â° 0.2 âââ
|
hs/space-bs-c8-foldlx-1: 4.631 1â° 0.0 âââââ
|
-- 2.585 2â° 0.1 ââââ
|
-- 1.326 1â° 0.1 ââââââââ
|
-- 1.221 1â° 0.1 âââââââ
|
hs/space-bs-c8-foldlx-2: 4.632 1â° 0.0 âââââ
|
-- 2.576 2â° 0.3 ââââ
|
-- 1.328 1â° 0.1 ââââââââ
|
-- 1.219 2â° 0.2 âââââââ
|
hs/space-bs-c8-foldrx: 4.678 0â° 0.0 âââââ
|
-- 2.982 8â° 2.3 âââââ
|
-- 1.128 1â° 0.2 âââââââ
|
-- 1.172 2â° 0.3 âââââââ
|
hs/space-bs-c8-lenfil: 4.634 1â° 0.1 âââââ
|
-- 2.599 1â° 0.2 ââââ
|
-- 1.327 2â° 0.2 ââââââââ
|
-- 1.223 1â° 0.2 âââââââ
|
hs/space-bslc8-acc-1: 32.733 7â° 0.0
âââââââââââââââââââââââââââââââ
|
-- 15.228 8â° 0.1
ââââââââââââââââââââââ
|
-- 3.002 1â° 0.1
ââââââââââââââââââ |
-- 3.388 10â° 0.1
ââââââââââââââââââ |
hs/space-bslc8-acc-2: 33.112 11â° 0.0
ââââââââââââââââââââââââââââââââ
|
-- 15.855 38â° 0.0
âââââââââââââââââââââââ
|
-- 3.010 2â° 0.0
ââââââââââââââââââ |
-- 3.386 5â° 0.1
ââââââââââââââââââ |
hs/space-bslc8-acc-3: 30.294 12â° 0.0
âââââââââââââââââââââââââââââ
|
-- 14.980 14â° 0.0
ââââââââââââââââââââââ
|
-- 2.586 1â° 0.1
ââââââââââââââââ |
-- 2.832 4â° 0.1
âââââââââââââââ |
hs/space-bslc8-chunk-1: 3.888 0â° 0.1 ââââ
|
-- 2.443 2â° 0.2 ââââ
|
-- 1.192 0â° 0.0 âââââââ
|
-- 1.094 0â° 0.3 ââââââ
|
hs/space-bslc8-chunk-2: 3.887 1â° 0.0 ââââ
|
-- 2.449 1â° 0.3 ââââ
|
-- 1.189 0â° 0.1 âââââââ
|
-- 1.094 1â° 0.3 ââââââ
|
hs/space-bslc8-chunk-3: 3.881 0â° 0.1 ââââ
|
-- 2.534 3â° 0.3 ââââ
|
-- 1.272 1â° 0.2 ââââââââ
|
-- 1.087 1â° 0.3 ââââââ
|
hs/space-bslc8-chunk-4: 3.910 1â° 0.0 ââââ
|
-- n/a
-- n/a
-- 1.099 0â° 0.3 ââââââ
|
hs/space-bslc8-count: 2.355 1â° 0.1 âââ
|
-- n/a
-- n/a
-- 0.477 2â° 0.4 âââ
|
hs/space-bslc8-foldl: 4.674 0â° 0.0 âââââ
|
-- 2.938 1â° 0.2 âââââ
|
-- 1.517 0â° 0.1 âââââââââ
|
-- 1.458 1â° 0.3 ââââââââ
|
hs/space-bslc8-foldlx-1: 4.676 0â° 0.0 âââââ
|
-- 2.928 1â° 0.0 âââââ
|
-- 1.516 0â° 0.0 âââââââââ
|
-- 1.458 0â° 0.1 ââââââââ
|
hs/space-bslc8-foldlx-2: 4.676 0â° 0.0 âââââ
|
-- 2.937 2â° 0.2 âââââ
|
-- 1.511 0â° 0.0 âââââââââ
|
-- 1.457 1â° 0.1 ââââââââ
|
hs/space-bslc8-foldr-1: 5.222 0â° 0.0 âââââ
|
-- 4.043 6â° 0.1 ââââââ
|
-- 1.205 1â° 0.2 ââââââââ
|
-- 1.312 1â° 0.2 âââââââ
|
hs/space-bslc8-foldr-2: 5.224 1â° 0.1 âââââ
|
-- 4.007 4â° 0.1 ââââââ
|
-- 1.202 3â° 0.1 ââââââââ
|
-- 1.315 1â° 0.2 âââââââ
|
hs/space-bslc8-lenfil-1: 4.712 1â° 0.0 âââââ
|
-- 3.240 1â° 0.2 âââââ
|
-- 1.400 0â° 0.0 âââââââââ
|
-- 1.323 1â° 0.1 âââââââ
|
hs/space-bslc8-lenfil-2: 4.714 1â° 0.0 âââââ
|
-- 3.236 1â° 0.2 âââââ
|
-- 1.396 0â° 0.0 âââââââââ
|
-- 1.325 1â° 0.1 âââââââ
|
hs/space-bsl---foldlx: 4.522 0â° 0.1 âââââ
|
-- 2.821 1â° 0.1 ââââ
|
-- 1.476 1â° 0.1 âââââââââ
|
-- 1.391 1â° 0.1 ââââââââ
|
hs/space-xxxxx-acc-1: 27.529 6â° 0.0
ââââââââââââââââââââââââââ
|
-- 21.002 4â° 0.1
ââââââââââââââââââââââââââââââ
|
-- 4.245 1â° 0.0
âââââââââââââââââââââââââ
|
-- 4.991 1â° 0.1
ââââââââââââââââââââââââââ
|
hs/space-xxxxx-acc-2: 27.200 3â° 0.0
ââââââââââââââââââââââââââ
|
-- 21.270 22â° 4.7
âââââââââââââââââââââââââââââââ
|
-- 4.241 2â° 0.1
âââââââââââââââââââââââââ
|
-- 4.985 1â° 0.0
ââââââââââââââââââââââââââ
|
hs/space-xxxxx-foldl: 27.436 8â° 0.0
ââââââââââââââââââââââââââ
|
-- 20.934 1â° 0.1
ââââââââââââââââââââââââââââââ
|
-- 4.239 2â° 0.1
âââââââââââââââââââââââââ
|
-- 5.035 2â° 0.1
âââââââââââââââââââââââââââ
|
hs/space-xxxxx-lenfil: 39.343 3â° 0.0
âââââââââââââââââââââââââââââââââââââ|
-- 25.915 3â° 0.0
âââââââââââââââââââââââââââââââââââââ|
-- 6.287 2â° 0.0
âââââââââââââââââââââââââââââââââââââ|
-- 7.106 8â° 0.1
âââââââââââââââââââââââââââââââââââââ|
c/space-getchar: 13.268 0â° 0.0
âââââââââââââ |
-- 9.354 0â° 0.0
ââââââââââââââ |
-- 4.031 18â° 0.0
ââââââââââââââââââââââââ
|
-- 2.465 15â° 0.1
âââââââââââââ |
c/space-getchar-u: 2.170 0â° 0.1 ââ
|
-- 1.676 2â° 0.2 âââ
|
-- 0.609 2â° 0.7 ââââ
|
-- 0.631 1â° 0.5 ââââ
|
c/space-4k: 1.815 1â° 0.1 ââ
|
-- 1.293 2â° 0.5 ââ
|
-- 0.465 2â° 0.6 âââ
|
-- 0.319 3â° 0.6 ââ
|
c/space-megabuf: 2.763 2â° 0.1 âââ
|
-- 1.830 3â° 0.5 âââ
|
-- 0.613 1â° 0.3 ââââ
|
-- 0.515 3â° 0.4 âââ
|
c/space-getwchar: 15.105 29â° 0.0
âââââââââââââââ |
-- 14.721 1â° 0.1
âââââââââââââââââââââ |
-- 4.584 1â° 0.1
âââââââââââââââââââââââââââ
|
-- 3.956 6â° 0.1
âââââââââââââââââââââ |
c/space-getwchar-u: 8.979 1â° 0.0 âââââââââ
|
-- 4.814 0â° 0.0 âââââââ
|
-- 2.437 0â° 0.1
âââââââââââââââ |
-- 2.217 1â° 0.1 ââââââââââââ
|
c/space-32k: 1.786 9â° 0.1 ââ
|
-- 1.276 2â° 0.6 ââ
|
-- 0.445 1â° 0.5 âââ
|
-- 0.303 3â° 1.0 ââ
|
c/space-32k-8: 1.777 1â° 0.1 ââ
|
-- 1.275 2â° 0.2 ââ
|
-- 0.448 0â° 0.0 âââ
|
-- 0.304 2â° 0.3 ââ
|
Memory: Peak
------- KB
hs/space-bs-c8-acc-1: 147748
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147488
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147772
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147736
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bs-c8-count: 147760
ââââââââââââââââââââââââââââââââââââââââ
|
-- n/a
-- n/a
-- 147744
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bs-c8-foldlx-1: 147760
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147492
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147800
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147756
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bs-c8-foldlx-2: 147764
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147492
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147804
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147752
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bs-c8-foldrx: 147748
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147488
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147776
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147732
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bs-c8-lenfil: 147764
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147492
ââââââââââââââââââââââââââââââââââââââââ
|
-- 147796
âââââââââââââââââââââââââââââââââââââââââ
|
-- 147756
ââââââââââââââââââââââââââââââââââââââââ
|
hs/space-bslc8-acc-1: 3196 â
|
-- 2896 â
|
-- 3192 â
|
-- 3192 â
|
hs/space-bslc8-acc-2: 3200 â
|
-- 2896 â
|
-- 3192 â
|
-- 3192 â
|
hs/space-bslc8-acc-3: 3204 â
|
-- 2896 â
|
-- 3188 â
|
-- 3196 â
|
hs/space-bslc8-chunk-1: 65204
ââââââââââââââââââ
|
-- 65892
ââââââââââââââââââ
|
-- 65208
ââââââââââââââââââ
|
-- 65192
ââââââââââââââââââ
|
hs/space-bslc8-chunk-2: 65200
ââââââââââââââââââ
|
-- 65892
ââââââââââââââââââ
|
-- 65212
ââââââââââââââââââ
|
-- 65192
ââââââââââââââââââ
|
hs/space-bslc8-chunk-3: 75248
âââââââââââââââââââââ
|
-- 76472
âââââââââââââââââââââ
|
-- 75232
âââââââââââââââââââââ
|
-- 75240
âââââââââââââââââââââ
|
hs/space-bslc8-chunk-4: 76204
âââââââââââââââââââââ
|
-- n/a
-- n/a
-- 76188
âââââââââââââââââââââ
|
hs/space-bslc8-count: 79224
ââââââââââââââââââââââ
|
-- n/a
-- n/a
-- 79204
ââââââââââââââââââââââ
|
hs/space-bslc8-foldl: 85880
ââââââââââââââââââââââââ
|
-- 86772
ââââââââââââââââââââââââ
|
-- 85856
ââââââââââââââââââââââââ
|
-- 85868
ââââââââââââââââââââââââ
|
hs/space-bslc8-foldlx-1: 85880
ââââââââââââââââââââââââ
|
-- 86772
ââââââââââââââââââââââââ
|
-- 85852
ââââââââââââââââââââââââ
|
-- 85864
ââââââââââââââââââââââââ
|
hs/space-bslc8-foldlx-2: 85880
ââââââââââââââââââââââââ
|
-- 86772
ââââââââââââââââââââââââ
|
-- 85856
ââââââââââââââââââââââââ
|
-- 85868
ââââââââââââââââââââââââ
|
hs/space-bslc8-foldr-1: 169704
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169360
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169684
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169700
ââââââââââââââââââââââââââââââââââââââââââââââ|
hs/space-bslc8-foldr-2: 169700
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169360
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169684
ââââââââââââââââââââââââââââââââââââââââââââââ|
-- 169700
ââââââââââââââââââââââââââââââââââââââââââââââ|
hs/space-bslc8-lenfil-1: 109408
ââââââââââââââââââââââââââââââ
|
-- 110704
âââââââââââââââââââââââââââââââ
|
-- 109404
ââââââââââââââââââââââââââââââ
|
-- 109404
ââââââââââââââââââââââââââââââ
|
hs/space-bslc8-lenfil-2: 109404
ââââââââââââââââââââââââââââââ
|
-- 110704
âââââââââââââââââââââââââââââââ
|
-- 109404
ââââââââââââââââââââââââââââââ
|
-- 109404
ââââââââââââââââââââââââââââââ
|
hs/space-bsl---foldlx: 85844
ââââââââââââââââââââââââ
|
-- 86776
ââââââââââââââââââââââââ
|
-- 85880
ââââââââââââââââââââââââ
|
-- 85836
ââââââââââââââââââââââââ
|
hs/space-xxxxx-acc-1: 1912 â
|
-- 1612 â
|
-- 1916 â
|
-- 1904 â
|
hs/space-xxxxx-acc-2: 1912 â
|
-- 1612 â
|
-- 1916 â
|
-- 1904 â
|
hs/space-xxxxx-foldl: 1912 â
|
-- 1612 â
|
-- 1916 â
|
-- 1900 â
|
hs/space-xxxxx-lenfil: 1904 â
|
-- 1588 â
|
-- 1892 â
|
-- 1892 â
|
c/space-getchar: 436 â
|
-- 384 â
|
-- 436 â
|
-- 444 â
|
c/space-getchar-u: 436 â
|
-- 384 â
|
-- 436 â
|
-- 444 â
|
c/space-4k: 472 â
|
-- 412 â
|
-- 468 â
|
-- 468 â
|
c/space-megabuf: 146968
ââââââââââââââââââââââââââââââââââââââââ
|
-- 146904
ââââââââââââââââââââââââââââââââââââââââ
|
-- 146968
ââââââââââââââââââââââââââââââââââââââââ
|
-- 146968
ââââââââââââââââââââââââââââââââââââââââ
|
c/space-getwchar: 484 â
|
-- 440 â
|
-- 484 â
|
-- 496 â
|
c/space-getwchar-u: 484 â
|
-- 440 â
|
-- 480 â
|
-- 500 â
|
c/space-32k: 500 â
|
-- 436 â
|
-- 496 â
|
-- 496 â
|
c/space-32k-8: 496 â
|
-- 436 â
|
-- 500 â
|
-- 500 â
|
_______________________________________________
Haskell-Cafe mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/haskell-cafe
