Reid Ellis wrote:
More numbers. On my dual 2GHz G5 I got 31,446. I then ran it again in
python without quitting and got 28,736. Again: 29069.
On my 1.6GHz PowerBook: 27,778. Then I realized I was using
/opt/local/bin/python (v2.4.2, built with GCC 4), so I tried again
with /usr/bin/python (v2.3.5, built with GCC 3.3) and got 24,510. Huh.
The G5 was using 2.4.2/GCC4.
So these numbers have a fuzz factor of about 3% or so, and are highly
impacted by build environment and/or Python version. I am assuming
John's numbers are from using GCC 4 and Python 2.4.2.
I used Chandler's latest release build of Python for all my tests.
I threw these numbers into Excel and added a "stones/GHz" field:
PyStone Benchmarks
System GHz CPU results stones/GHz
1.2Ghz iBook 1.2 G4 18939 15782.5
3GHz P4 XP 3.0 P4 39463 13154.3
3GHz P4 FC4 3.0 P4 27932 9310.7
1.67GHz PowerBook 1.67 G4 27778 16633.5
2x2GHz G5 2.0 G5 31446 15723.0
When you factor in the actual speed of the CPU, it looks like the G4
CPUs in the PowerBook and iBook fare the best, with the Fedora Core
3GHz P4 the worst. I would expect an AMD CPU to outshine the G4's.
These numbers don't really affect the final conclusion all that much -
that Python is CPU limited. And with Apple's switch to Intel, the
differences amongst OS X, XP and Linux should fade.
Reid
On Mar 8, 2006, at 18:50, John Townsend wrote:
Hi John,
I actually had access to an Intel iMac today so I ran your python
test on the machine.
Intel-based iMac (running 1.83 GHz Intel Dual Core) with 1GB of RAM.
The results: 31250 pystones/sec.
By comparison, G4 based PowerBook (1.67GHz PowerPC G4) with 1GB of RAM:
The results: 22831 pystones/sec.
Just thought you might be interested.
--> towns
On Mar 8, 2006, at 3:09 PM, John Anderson wrote:
During the sprints at PyCon I worked on Performance, focusing on
trying to understand why Chandler is slower on Mac than other
platforms. Preliminary results seem to indicate that OSX Macs are
just overall slower than Windows and Linux PCs. Read on for the
nitty-gritty details.
Launching Chandler with a preexisting repository -- the normal user
experience -- on the 1.2Ghz OSX iBook G4 I had at PyCon took 14
seconds compared to about 4 seconds on my 3Ghz P3 Shuttle running XP
or Fedora Core3. Alec's XP Pentium M laptop was not far behind the 4
second mark.
The 3.5 times slowdown in launch translated into a significant loss
of productivity for me. When developing Chandler I usually run in
WingIDE. On my 3Ghz P4 shuttle, launching Chandler in Wing with an
existing repository takes 12 seconds and launching while creating a
new repository takes 36 seconds. Multiplying those times by 3.5 made
the iBook too slow for development. This probably explains why
Macintosh developers avoid using Wing or even running Chandler
whenever possible.
Next I decided to run some benchmarks to see how C and Python
performance compared on the different machines. Anyone who has run
benchmarks on modern multitasking operating systems knows that
running the same benchmark can sometimes vary as much as a factor of
two on the same machine for a variety of reasons. Sometimes it's
because some other task is using the CPU. Sometimes the machine
saves power by slowing down the processor. I ran all the tests
several times to try to get representative numbers.
The first test I ran was Python's pystone, which I ran as follows:
from test import pystone
pystone.pystones (loops=pystone.LOOPS)
The iBook did 18,939 pytsones/sec. 3Ghz P4 XP did 39,463. 3Ghz P4
Fedora Core 4 did 27,932. XP Pentium M was comparable to the 3Ghz P4
XP result. So on first glance, Python performance seems twice as
slow on the iBook compared to the XP machines.
Running the Python "make test" on the iBook vs. XP Pentium M
produced a similar XP speed advantage of roughly a factor of 2.
Since Fedora and OSX use the same gcc compiler, yet XP is 1.4 times
faster than Fedora on the same hardware, it looks like a big part of
the difference in performance is explained by the Microsoft
compiler. Not only does the Microsoft compiler produce much better
code, it runs significantly faster -- Bear mentioned that builds on
the iBook are about 5 times slower than Windows.
I have a 10,000 line C program I wrote years ago that I use for
performance measures on different processors. It's an interpreted
language, not unlike Python, which I suspect compares to the kind of
C code that makes up Chandler. The iBook did 141K instructions/sec.
3Ghz P4 XP did 1226K/sec. 3Ghz P4 Fedora did 800K/sec. Pentium M XP
did about 1000K/sec. A 1Ghz PowerBook did about 300K/sec. The first
NeXT machine did about 30K/sec. So in this test the iBook was almost
9 times slower than Intel/XP. The Microsoft compiler advantage, 1.5
times speed up, was comparable to the pystone tests -- even using
-O3, the highest level of optimization for the gcc builds.
Alec pointed out a C benchmark, GeekBench
(http://www.geekpatrol.ca/geekbench/) which has lots of tests that
measure the speed of different tasks on a variety of processors.
Although their tests didn't compare the same processors I tested,
the most varied CPU test gave the Microsoft/Intel vs OSX/G4 a 2 to 4
speed advantage. The exciting news, however, is that the iMac Core
Duo 2.0 was roughly equal to the best Microsoft/Intel machines.
Here are my conclusions: Although these results don't explain the
whole story, the XP/Intel Chandler launch advantage of 3.5 seems
like it could be entirely be explained by the difference in C
performance. Furthermore, a significant part of this advantage is
due to the Microsoft compiler. There are more tests that would be
interesting to do: profile the C code in Chandler to see where time
is spent during launch and other time consuming tasks. Run tests on
the Intel Macs. Measure how much IO is involved running Chandler.
Construct some Berkeley DB benchmarks and measure them on the
different platforms. Finally, Given Moore's law, the current state
of Chandler, and its probable lifetime (assuming it's successful),
performance probably won't be a significant obstacle to adoption.
John
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Open Source Applications Foundation "Dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/dev
