Can you tell us about the environment you're running these tests in? Most interesting are the specific OS and the version of mscorwks.dll.
On Thu, Aug 14, 2008 at 9:41 AM, Michael Foord <[EMAIL PROTECTED]>wrote: > Just for fun I also compared with CPython. The results are interesting, > I'll turn it into a blog post of course... > > Results in milliseconds with a granularity of about 15ms and so an accuracy > of +/- ~60ms. > All testing with 10 000 000 operations unless otherwise stated. > > The version of Python I compared against was Python 2.4. > > Empty loop (overhead): > IP1: 422 > IP2: 438 > Py: 3578 > Create instance newstyle: > IP1: 20360 > IP2: 1109 > Py: 4063 > Create instance oldstyle: > IP1: 3766 > IP2: 3359 > Py: 4797 > Function call: > IP1: 937 > IP2: 906 > Py: 3313 > Create function: 25% slower > IP1: 2828 > IP2: 3640 > Py: 2766 > Define newstyle (1 000 000): > IP1: 42047 > IP2: 20484 > Py: 23921 > Define oldstyle (1 000 000): 33% slower > IP1: 1781 > IP2: 2671 > Py: 2108 > > Comparing (== and !=): > IP1: 278597 > IP2: 117662 > Py: 62423 > Sets: > IP1: 37095 > IP2: 30860 > Py: 8047 > > Lists (10 000): 50% slower > IP1: 10422 > IP2: 16109 > Py: 6094 > > Recursion (10 000): > IP1: 1125 > IP2: 1000 > Py: 3609 > Sets2 (100 000): 500% slower > IP1: 4984 > IP2: 30547 > Py: 1203 > > func_with_args: > IP1: 6312 > IP2: 5906 > Py: 11250 > > method_with_args: > IP1: 20594 > IP2: 11813 > Py: 14875 > method_with_kwargs: > IP1: 27953 > IP2: 11187 > Py: 20032 > import: 15% slower > IP1: 28469 > IP2: 32000 > Py: 25782 > > global: 20% slower > IP1: 1047 > IP2: 1203 > Py: 4141 > > Exceptions (100000): 40% slower > IP1: 4703 > IP2: 6125 > Py: 266 > Engine execution: 8000% slower!! > IP1: 1600 > IP2: 115002 > > > Michael Foord wrote: > >> Hello all, >> >> I've ported Resolver One to run on IronPython 2 Beta 4 to check for any >> potential problems (we will only do a *proper* port once IP 2 is out of >> beta). >> >> The basic porting was straightforward and several bugs have been fixed >> since IP 2 B3 - many thanks to the IronPython team. >> >> The good news is that Resolver One is only 30-50% slower than Resolver One >> on IronPython 1! (It was 300 - 400% slower on top of IP 2 B3.) Resolver One >> is fairly heavily optimised around the performance hotspots of IronPython 1, >> so we expect to have to do a fair bit of profiling and refactoring to >> readjust to the performance profile of IP 2. >> >> Having said that, there are a few oddities (and the areas that slow down >> vary tremendously depending on which spreadsheet we use to benchmark it - >> making it fairly difficult to track down the hotspots). >> >> We have one particular phase of spreadsheet calculation that takes >> 0.4seconds on IP1 and around 6 seconds on IP2, so I have been doing some >> micro-benchmarking to try and identify the hotspot. I've certainly found >> part of the problem. >> >> For those that are interested I've attached the very basic microbenchmarks >> I've been using. The nice thing is that in *general* IP2 does outperform >> IP1. >> >> The results that stand out in the other direction are: >> >> Using sets with custom classes (that define '__eq__', '__ne__' and >> '__hash__') seems to be 6 times slower in IronPython 2. >> >> Adding lists together is about 50% slower. >> >> Defining functions seems to be 25% slower and defining old style classes >> about 33% slower. (Creating instances of new style classes is massively >> faster though - thanks!) >> >> The code I used to test sets (sets2.py) is as follows: >> >> from System import DateTime >> >> class Thing(object): >> def __init__(self, val): >> self.val = val >> def __eq__(self, other): >> return self.val == other.val >> >> def __neq__(self): >> return not self.__eq__(other) >> def __hash__(self): >> return hash(self.val) >> def test(s): >> a = set() >> for i in xrange(100000): >> a.add(Thing(i)) >> a.add(Thing(i+1)) >> Thing(i) in a >> Thing(i+2) in a >> return (DateTime.Now -s).TotalMilliseconds >> s = DateTime.Now >> print test(s) >> >> >> Interestingly the time taken is exactly the same if I remove the >> definition of '__hash__'. >> >> The full set of results below: >> >> Results in milliseconds with a granularity of about 15ms and so an >> accuracy of +/- ~60ms. >> All testing with 10 000 000 operations unless otherwise stated. >> >> Empty loop (overhead): >> IP1: 421.9 >> IP2: 438 >> Create instance newstyle: >> IP1: 20360 >> IP2: 1109 >> Create instance oldstyle: >> IP1: 3766 >> IP2: 3359 >> Function call: >> IP1: 937 >> IP2: 906 >> Create function: 25% slower >> IP1: 2828 >> IP2: 3640 >> Define newstyle (1 000 000): >> IP1: 42047 >> IP2: 20484 >> Define oldstyle (1 000 000): 33% slower >> IP1: 1781 >> IP2: 2671 >> >> Comparing (== and !=): >> IP1: 278597 >> IP2: 117662 >> Sets (with numbers): >> IP1: 37095 >> IP2: 30860 >> >> Lists (10 000): 50% slower >> IP1: 10422 >> IP2: 16109 >> >> Recursion (10 000): >> IP1: 1125 >> IP2: 1000 >> Sets2 (100 000): 600% slower >> IP1: 4984 >> IP2: 30547 >> >> >> I'll be doing more as the 600% slow down for sets and the 50% slow down >> for lists accounts for some of the dependency analysis problem but not all >> of it. >> >> Many Thanks >> >> Michael Foord >> -- >> http://www.resolversystems.com >> http://www.ironpythoninaction.com >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Users mailing list >> Users@lists.ironpython.com >> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com >> > > > -- > http://www.ironpythoninaction.com/ > http://www.voidspace.org.uk/ > http://www.trypython.org/ > http://www.ironpython.info/ > http://www.resolverhacks.net/ > http://www.theotherdelia.co.uk/ > > > _______________________________________________ > Users mailing list > Users@lists.ironpython.com > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > >
_______________________________________________ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
