BTW time.clock() is what I usually use to measure which works on both CPython and IronPython. On Ipy we use the .NET Stopwatch class which uses a high resolution counter if it's available.
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Foord Sent: Thursday, August 14, 2008 9:42 AM To: Discussion of IronPython Subject: Re: [IronPython] Performance of IronPython 2 Beta 4 and IronPython 1 Just for fun I also compared with CPython. The results are interesting, I'll turn it into a blog post of course... Results in milliseconds with a granularity of about 15ms and so an accuracy of +/- ~60ms. All testing with 10 000 000 operations unless otherwise stated. The version of Python I compared against was Python 2.4. Empty loop (overhead): IP1: 422 IP2: 438 Py: 3578 Create instance newstyle: IP1: 20360 IP2: 1109 Py: 4063 Create instance oldstyle: IP1: 3766 IP2: 3359 Py: 4797 Function call: IP1: 937 IP2: 906 Py: 3313 Create function: 25% slower IP1: 2828 IP2: 3640 Py: 2766 Define newstyle (1 000 000): IP1: 42047 IP2: 20484 Py: 23921 Define oldstyle (1 000 000): 33% slower IP1: 1781 IP2: 2671 Py: 2108 Comparing (== and !=): IP1: 278597 IP2: 117662 Py: 62423 Sets: IP1: 37095 IP2: 30860 Py: 8047 Lists (10 000): 50% slower IP1: 10422 IP2: 16109 Py: 6094 Recursion (10 000): IP1: 1125 IP2: 1000 Py: 3609 Sets2 (100 000): 500% slower IP1: 4984 IP2: 30547 Py: 1203 func_with_args: IP1: 6312 IP2: 5906 Py: 11250 method_with_args: IP1: 20594 IP2: 11813 Py: 14875 method_with_kwargs: IP1: 27953 IP2: 11187 Py: 20032 import: 15% slower IP1: 28469 IP2: 32000 Py: 25782 global: 20% slower IP1: 1047 IP2: 1203 Py: 4141 Exceptions (100000): 40% slower IP1: 4703 IP2: 6125 Py: 266 Engine execution: 8000% slower!! IP1: 1600 IP2: 115002 Michael Foord wrote: > Hello all, > > I've ported Resolver One to run on IronPython 2 Beta 4 to check for > any potential problems (we will only do a *proper* port once IP 2 is > out of beta). > > The basic porting was straightforward and several bugs have been fixed > since IP 2 B3 - many thanks to the IronPython team. > > The good news is that Resolver One is only 30-50% slower than Resolver > One on IronPython 1! (It was 300 - 400% slower on top of IP 2 B3.) > Resolver One is fairly heavily optimised around the performance > hotspots of IronPython 1, so we expect to have to do a fair bit of > profiling and refactoring to readjust to the performance profile of IP 2. > > Having said that, there are a few oddities (and the areas that slow > down vary tremendously depending on which spreadsheet we use to > benchmark it - making it fairly difficult to track down the hotspots). > > We have one particular phase of spreadsheet calculation that takes > 0.4seconds on IP1 and around 6 seconds on IP2, so I have been doing > some micro-benchmarking to try and identify the hotspot. I've > certainly found part of the problem. > > For those that are interested I've attached the very basic > microbenchmarks I've been using. The nice thing is that in *general* > IP2 does outperform IP1. > > The results that stand out in the other direction are: > > Using sets with custom classes (that define '__eq__', '__ne__' and > '__hash__') seems to be 6 times slower in IronPython 2. > > Adding lists together is about 50% slower. > > Defining functions seems to be 25% slower and defining old style > classes about 33% slower. (Creating instances of new style classes is > massively faster though - thanks!) > > The code I used to test sets (sets2.py) is as follows: > > from System import DateTime > > class Thing(object): > def __init__(self, val): > self.val = val > def __eq__(self, other): > return self.val == other.val > > def __neq__(self): > return not self.__eq__(other) > def __hash__(self): > return hash(self.val) > def test(s): > a = set() > for i in xrange(100000): > a.add(Thing(i)) > a.add(Thing(i+1)) > Thing(i) in a > Thing(i+2) in a > return (DateTime.Now -s).TotalMilliseconds > s = DateTime.Now > print test(s) > > > Interestingly the time taken is exactly the same if I remove the > definition of '__hash__'. > > The full set of results below: > > Results in milliseconds with a granularity of about 15ms and so an > accuracy of +/- ~60ms. > All testing with 10 000 000 operations unless otherwise stated. > > Empty loop (overhead): > IP1: 421.9 > IP2: 438 > Create instance newstyle: > IP1: 20360 > IP2: 1109 > Create instance oldstyle: > IP1: 3766 > IP2: 3359 > Function call: > IP1: 937 > IP2: 906 > Create function: 25% slower > IP1: 2828 > IP2: 3640 > Define newstyle (1 000 000): > IP1: 42047 > IP2: 20484 > Define oldstyle (1 000 000): 33% slower > IP1: 1781 > IP2: 2671 > > Comparing (== and !=): > IP1: 278597 > IP2: 117662 > Sets (with numbers): > IP1: 37095 > IP2: 30860 > > Lists (10 000): 50% slower > IP1: 10422 > IP2: 16109 > > Recursion (10 000): > IP1: 1125 > IP2: 1000 > > Sets2 (100 000): 600% slower > IP1: 4984 > IP2: 30547 > > > I'll be doing more as the 600% slow down for sets and the 50% slow > down for lists accounts for some of the dependency analysis problem > but not all of it. > > Many Thanks > > Michael Foord > -- > http://www.resolversystems.com > http://www.ironpythoninaction.com > > > > ---------------------------------------------------------------------- > -- > > _______________________________________________ > Users mailing list > Users@lists.ironpython.com > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.resolverhacks.net/ http://www.theotherdelia.co.uk/ Sample disclaimer text _______________________________________________ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
