More numbers. On my dual 2GHz G5 I got 31,446. I then ran it again in
python without quitting and got 28,736. Again: 29069.
On my 1.6GHz PowerBook: 27,778. Then I realized I was using /opt/
local/bin/python (v2.4.2, built with GCC 4), so I tried again with /
usr/bin/python (v2.3.5, built with GCC 3.3) and got 24,510. Huh. The
G5 was using 2.4.2/GCC4.
So these numbers have a fuzz factor of about 3% or so, and are highly
impacted by build environment and/or Python version. I am assuming
John's numbers are from using GCC 4 and Python 2.4.2.
I threw these numbers into Excel and added a "stones/GHz" field:
PyStone Benchmarks
System GHz CPU results stones/GHz
1.2Ghz iBook 1.2 G4 18939 15782.5
3GHz P4 XP 3.0 P4 39463 13154.3
3GHz P4 FC4 3.0 P4 27932 9310.7
1.67GHz PowerBook 1.67 G4 27778 16633.5
2x2GHz G5 2.0 G5 31446 15723.0
When you factor in the actual speed of the CPU, it looks like the G4
CPUs in the PowerBook and iBook fare the best, with the Fedora Core
3GHz P4 the worst. I would expect an AMD CPU to outshine the G4's.
These numbers don't really affect the final conclusion all that much
- that Python is CPU limited. And with Apple's switch to Intel, the
differences amongst OS X, XP and Linux should fade.
Reid
On Mar 8, 2006, at 18:50, John Townsend wrote:
Hi John,
I actually had access to an Intel iMac today so I ran your python
test on the machine.
Intel-based iMac (running 1.83 GHz Intel Dual Core) with 1GB of RAM.
The results: 31250 pystones/sec.
By comparison, G4 based PowerBook (1.67GHz PowerPC G4) with 1GB of
RAM:
The results: 22831 pystones/sec.
Just thought you might be interested.
--> towns
On Mar 8, 2006, at 3:09 PM, John Anderson wrote:
During the sprints at PyCon I worked on Performance, focusing on
trying to understand why Chandler is slower on Mac than other
platforms. Preliminary results seem to indicate that OSX Macs are
just overall slower than Windows and Linux PCs. Read on for the
nitty-gritty details.
Launching Chandler with a preexisting repository -- the normal
user experience -- on the 1.2Ghz OSX iBook G4 I had at PyCon took
14 seconds compared to about 4 seconds on my 3Ghz P3 Shuttle
running XP or Fedora Core3. Alec's XP Pentium M laptop was not far
behind the 4 second mark.
The 3.5 times slowdown in launch translated into a significant
loss of productivity for me. When developing Chandler I usually
run in WingIDE. On my 3Ghz P4 shuttle, launching Chandler in Wing
with an existing repository takes 12 seconds and launching while
creating a new repository takes 36 seconds. Multiplying those
times by 3.5 made the iBook too slow for development. This
probably explains why Macintosh developers avoid using Wing or
even running Chandler whenever possible.
Next I decided to run some benchmarks to see how C and Python
performance compared on the different machines. Anyone who has run
benchmarks on modern multitasking operating systems knows that
running the same benchmark can sometimes vary as much as a factor
of two on the same machine for a variety of reasons. Sometimes
it's because some other task is using the CPU. Sometimes the
machine saves power by slowing down the processor. I ran all the
tests several times to try to get representative numbers.
The first test I ran was Python's pystone, which I ran as follows:
from test import pystone
pystone.pystones (loops=pystone.LOOPS)
The iBook did 18,939 pytsones/sec. 3Ghz P4 XP did 39,463. 3Ghz P4
Fedora Core 4 did 27,932. XP Pentium M was comparable to the 3Ghz
P4 XP result. So on first glance, Python performance seems twice
as slow on the iBook compared to the XP machines.
Running the Python "make test" on the iBook vs. XP Pentium M
produced a similar XP speed advantage of roughly a factor of 2.
Since Fedora and OSX use the same gcc compiler, yet XP is 1.4
times faster than Fedora on the same hardware, it looks like a big
part of the difference in performance is explained by the
Microsoft compiler. Not only does the Microsoft compiler produce
much better code, it runs significantly faster -- Bear mentioned
that builds on the iBook are about 5 times slower than Windows.
I have a 10,000 line C program I wrote years ago that I use for
performance measures on different processors. It's an interpreted
language, not unlike Python, which I suspect compares to the kind
of C code that makes up Chandler. The iBook did 141K instructions/
sec. 3Ghz P4 XP did 1226K/sec. 3Ghz P4 Fedora did 800K/sec.
Pentium M XP did about 1000K/sec. A 1Ghz PowerBook did about 300K/
sec. The first NeXT machine did about 30K/sec. So in this test the
iBook was almost 9 times slower than Intel/XP. The Microsoft
compiler advantage, 1.5 times speed up, was comparable to the
pystone tests -- even using -O3, the highest level of optimization
for the gcc builds.
Alec pointed out a C benchmark, GeekBench (http://
www.geekpatrol.ca/geekbench/) which has lots of tests that measure
the speed of different tasks on a variety of processors. Although
their tests didn't compare the same processors I tested, the most
varied CPU test gave the Microsoft/Intel vs OSX/G4 a 2 to 4 speed
advantage. The exciting news, however, is that the iMac Core Duo
2.0 was roughly equal to the best Microsoft/Intel machines.
Here are my conclusions: Although these results don't explain the
whole story, the XP/Intel Chandler launch advantage of 3.5 seems
like it could be entirely be explained by the difference in C
performance. Furthermore, a significant part of this advantage is
due to the Microsoft compiler. There are more tests that would be
interesting to do: profile the C code in Chandler to see where
time is spent during launch and other time consuming tasks. Run
tests on the Intel Macs. Measure how much IO is involved running
Chandler. Construct some Berkeley DB benchmarks and measure them
on the different platforms. Finally, Given Moore's law, the
current state of Chandler, and its probable lifetime (assuming
it's successful), performance probably won't be a significant
obstacle to adoption.
John
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
