On the recursion limits: Until beta 6 IronPython didn't have proper support for limiting recursion depth. There was some minor support there, but it wasn't right.
In beta 6 we have full support for limiting recursion depth, but by default we allow infinite recursion. If the user explicitly sets the recursion limit then we'll go ahead and enforce it. But all the reasons you outline below are great explanations of the differences. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paolo Molaro Sent: Monday, April 24, 2006 2:53 AM To: Brent Fulgham Cc: [EMAIL PROTECTED]; python-dev@python.org; users@lists.ironpython.com; mono-devel-list@lists.ximian.com Subject: Re: [IronPython] [Mono-dev] IronPython Performance [I continued your large cross-post, even if I'm not subscribed to python-dev: hopefully the moderators will approve the post if needed.] On 04/21/06 Brent Fulgham wrote: > A while ago (nearly two years) an interesting paper was published by Jim > Hugunin > (http://www.python.org/pycon/dc2004/papers/9/) crowing about the significant > speed advantage observed in this interpreter running on top of Microsoft's > .NET > VM stack. I remember being surprised by these results, as Python has always > seemed fairly fast for an interpreted language. > > I've been working on the Programming Language Shootout for a few years now, > and > after growing tired of the repeated requests to include scores for IronPython > I > finally added the implementation this week. > > Comparison of IronPython (1.0 Beta 5) to Python 2.4.3 [1] and IronPython (1.0 > Beta 1) > to Python 2.4.2 [2] do not match the results reported in the 2004 paper. In > fact, > IronPython is consistenly 3 to 4 times slower (in some cases worse than > that), and > seemed to suffer from recursion/stack limitations. You're comparing different benchmarks, so it's not a surprise that you get different numbers. I only tried two benchmarks from the paper, pystone and the globals one (attached) and the results are largely equivalent to the ones from the paper: time mono IronPython-1.0-Beta6/IronPythonConsole.exe -O python-globals-bench2.py real 0m0.861s user 0m0.815s sys 0m0.043s (Note the above includes the startup time that I guess was not considered in the paper and is 0.620 seconds running test(1): just increasing the number of iterations will make the difference converge at about 10x faster mono vs python2.4). time python2.4 -O python-globals-bench2.py real 0m2.239s user 0m2.202s sys 0m0.025s python2.4 -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 50000 passes = 1.4 This machine benchmarks at 35714.3 pystones/second mono IronPython-1.0-Beta6/IronPythonConsole.exe -O /usr/lib/python2.4/test/pystone.py Pystone(1.1) time for 50000 passes = 0.989288 This machine benchmarks at 50541.4 pystones/second So IronPython is 40+% faster than CPython, like in the paper. The mono version I used is roughtly 1.1.14. As for the recursion limitations: what are they? I took the recursive benchmark from the shootout page and both CPython and IronPython need the call to setrecursionlimit (actually cpython needed it earlier for me, anyway). This might be a difference between IronPython beta 5 and the beta6 I'm using, though. It is also interesting because on my system running the attached python-ack.py with 8 as argument, mono/IronPython is 10% faster than CPython, while the shootout page reports CPython 2.4 50% faster. Feel free to investigate if this is because of improvements in mono or IronPython:) > I am aware of the following issues that might affect the results: > 1) Mono vs. Microsoft's CLR. This can certainly be a factor: some things in the MS runtime run faster and some slower than in the Mono runtime, so it depends which features are exercised the most in a particular benchmark. It is very likely that in most benchmarks the MS runtime will score better. > 2) Python 2.1 vs. Python 2.4 The python folks will have more details, but python2.4 seems to be faster than 2.1 and 2.3 in several areas. > 3) Changes in IronPython over the last two years. For some time the IronPython folks have focused on features and correctness issues, so I guess they'll start caring about performance in the future. It is also to note that IronPython now uses generics which have yet to see optimizations work in the mono runtime and libraries (they are a 2.0 feature). > I thought these findings might be interesting to some of the Python, > IronPython, > and Mono developers. Let the flames begin! ;-) Why flames? Hard numbers are just that: useful information. It's just your conclusions that might be incorrect;-) I think the summary to take away is: *) Sometimes IronPython on mono is faster than CPython *) Often CPython is faster than IronPython on mono *) IronPython and mono are improving quickly. Thanks for the numbers: it would be nice if someone with time in his hands could rerun the benchmarks with ironpython beta6, mono from svn (or at least mono 1.1.14/1.1.15) and also with the MS runtime on windows. Better if this includes running pystone to relate to improvements in the published paper and the piethon benchmarks (on the MS runtime, in mono we need to implement some things that newer ironpythons use). In the meantime here are the piethon results running with IronPython 0.6 with a current mono. Mono results: b0 = 2.49623870849609 -- [2.59130096435547, 2.40117645263672] b1 = 0.473392486572266 -- [0.496170043945312, 0.450614929199219] b2 = 0.453083038330078 -- [0.543968200683594, 0.362197875976562] b3 = 0.967914581298828 -- [0.995307922363281, 0.940521240234375] b4 = 1.11585998535156 -- [1.23159027099609, 1.00012969970703] b5 = 3.56014633178711 -- [3.70928192138672, 3.4110107421875] b6 = 1.43364334106445 -- [1.42024230957031, 1.44704437255859] all done in 21.01 sec Python 2.4 results: b0 = 2.185000 -- [2.1800000000000002, 2.1900000000000004] b1 = 0.915000 -- [0.93999999999999995, 0.88999999999999879] b2 = 0.395000 -- [0.39999999999999991, 0.39000000000000057] b3 = 1.015000 -- [1.02, 1.0099999999999998] b4 = 0.585000 -- [0.58999999999999986, 0.58000000000000007] b5 = 0.995000 -- [0.99000000000000021, 1.0] b6 = 1.410000 -- [1.4199999999999999, 1.4000000000000004] all done in 15.00 sec Python 2.3 results: b0 = 2.270000 -- [2.2800000000000002, 2.2600000000000016] b1 = 0.955000 -- [0.98999999999999977, 0.91999999999999993] b2 = 0.395000 -- [0.39000000000000012, 0.39999999999999858] b3 = 1.230000 -- [1.2300000000000004, 1.2300000000000004] b4 = 0.805000 -- [0.80999999999999961, 0.80000000000000071] b5 = 1.145000 -- [1.1500000000000004, 1.1399999999999988] b6 = 1.500000 -- [1.5099999999999989, 1.4900000000000002] all done in 16.60 sec Since Guido designed it to excercise the tricky corners of the python implementation this could be a better general evaluation of the speed of a python runtime. Note that in the oscon paper, mono was reported to be 23 time slower than CPython (but 2 times slower excluding b5 where we had a really unoptimized codepath). Current mono is 27% slower than CPython 2.3 and 40% slower than CPython 2.4. So while the CPython performance increased, mono's rate of improvement is significantly faster. While having mono/IronPython be faster than CPython in more benchmarks would be nice (and I guess we could run a few profile runs if there is interest to see if some quick specific improvement could be done), it is not really necessary. Having mono run python code 1-2 times slower than CPython is pretty good. People interested in raw performance could just write the code in C#, gluelessly use it from python code and get the 1-2 orders of magnitude improvements visible also in the shootout comparison of C# vs python. Thanks. lupus -- ----------------------------------------------------------------- [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better _______________________________________________ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
