Michael Foord wrote: > Jim Hugunin wrote: > >> I'd like to point out one fact about these benchmarks that seems to have >> been missed. Seo's numbers are for running IronPython 1.1 on Mono - not on >> the Microsoft .NET implementation. The Mono team has done excellent work >> with that project, but today the performance of the .NET implementation is >> still significantly better and is a much better platform for benchmarking >> IronPython's performance. >> >> > And on close examination the results are impressive. Nice one. > > The anomaly with raising exceptions is very weird though (IronPython is > around 30 times slower!) : > > TryExcept: 7ms 100ms -93.4% 11ms 102ms -89.4% > TryRaiseExcept: 3413ms 114ms +2896.2% 3453ms 115ms +2910.6% >
Sorry for all the emails - I see that you addressed this in your comments Jim. Michael > All the best, > > Michael Foord > > > >> I reran pybench on my laptop tonight on the RTM version of .NET that ships >> with Windows Vista - and has been available for XP for quite some time now. >> I compared with CPython-2.5.1 (the latest version). >> >> Running on .NET, I find that IronPython is faster on some tests and CPython >> is faster on others. We know that we still have performance work to do. >> After all, IronPython just reached its 1.1 release and almost all of our >> recent work has been about compatibility and completeness. However, the >> story is already quite interesting. Out of the 51 tests in pybench, CPython >> is more than 2x faster on 10 of the tests and IronPython is more than 2x >> faster on 9 of the tests. Depending on what your code does, either >> implementation could run faster. >> >> The most interesting cases to me are the 5 tests where CPython is more than >> 3x faster than IronPython and the other 5 tests where IronPython is more >> than 3x faster than CPython. CPython's strongest performance is in >> dictionaries with integer and string keys, list slicing, small tuples and >> code that actually throws and catches exceptions. IronPython's strongest >> performance is in calling builtin functions, if/then/else blocks, calling >> python functions, deep recursion, and try/except blocks that don't actually >> catch an exception. >> >> The places where IronPython is performing strongest are where it can use the >> .NET code generation and JIT optimization most effectively. The places >> where it is slowest are mainly in runtime library implementation of core >> datatypes. CPython's list and dictionary datatypes are written in C and >> have been hand-tuned for Python-style workloads over the past 10 years. >> IronPython's datatypes are much newer and clearly need more tuning as the >> implementation matures. >> >> The only two tests that really stand out as showing a deep performance issue >> are the two exception handling ones. These are the only tests where either >> implementation is more than 4x faster than the other. IronPython is 10x >> faster on the try/catch without an exception and CPython is 30x faster when >> an exception is actually raised. This is a deliberate design decision >> within .NET to make code that doesn't throw exceptions run faster - even if >> that means slowing down code that does throw exceptions. I'm fairly >> confident this was the right decision - and even remember the day long ago >> when Guido was discussing Python's exception system and explained that he'd >> accept almost any slow-down to the exceptional case in return for removing a >> single instruction from the non-exceptional path. >> >> Thanks - Jim >> >> My results >> ------------------------------------------------------------------------------- >> PYBENCH 2.0 >> ------------------------------------------------------------------------------- >> * using Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit >> (Int >> el)] >> * disabled garbage collection >> * system check interval set to maximum: 2147483647 >> * using timer: time.clock >> >> ------------------------------------------------------------------------------- >> Benchmark: ipy11.pybench >> ------------------------------------------------------------------------------- >> >> Rounds: 10 >> Warp: 10 >> Timer: time.time >> >> Machine Details: >> Platform ID: cli-32bit >> Processor: >> >> Python: >> Implementation: Python >> Executable: c:\ironpython-1.1\ipy.exe >> Version: 1.1.0 >> Compiler: X >> Bits: 32bit >> Build: 0 0 (#0) >> Unicode: UCS2 >> >> >> ------------------------------------------------------------------------------- >> Comparing with: py25.pybench >> ------------------------------------------------------------------------------- >> >> Rounds: 10 >> Warp: 10 >> Timer: time.clock >> >> Machine Details: >> Platform ID: Microsoft-Windows-32bit-WindowsPE >> Processor: >> >> Python: >> Implementation: Python >> Executable: c:\python25\python.exe >> Version: 2.5.1 >> Compiler: MSC v.1310 32 bit (Intel) >> Bits: 32bit >> Build: Apr 18 2007 08:51:08 (#r251:54863) >> Unicode: UCS2 >> >> >> Test minimum run-time average run-time >> this other diff this other diff >> ------------------------------------------------------------------------------- >> BuiltinFunctionCalls: 44ms 181ms -75.7% 48ms 182ms >> -73.7% >> BuiltinMethodLookup: 335ms 160ms +109.1% 344ms 162ms >> +112.0% >> CompareFloats: 99ms 111ms -11.3% 105ms 112ms >> -6.0% >> CompareFloatsIntegers: 63ms 125ms -49.7% 65ms 126ms >> -48.7% >> CompareIntegers: 102ms 114ms -10.4% 105ms 115ms >> -8.9% >> CompareInternedStrings: 175ms 126ms +39.0% 180ms 127ms >> +41.6% >> CompareLongs: 111ms 105ms +5.3% 114ms 106ms >> +7.2% >> CompareStrings: 167ms 126ms +32.7% 172ms 127ms >> +35.1% >> CompareUnicode: 121ms 125ms -3.4% 123ms 126ms >> -2.0% >> ConcatStrings: 482ms 272ms +77.2% 533ms 275ms >> +93.4% >> ConcatUnicode: 305ms 216ms +41.0% 350ms 217ms >> +61.5% >> CreateInstances: 102ms 148ms -31.1% 106ms 149ms >> -28.9% >> CreateNewInstances: 326ms 131ms +148.0% 335ms 133ms >> +151.9% >> CreateStringsWithConcat: 228ms 147ms +55.9% 238ms 148ms >> +61.0% >> CreateUnicodeWithConcat: 91ms 154ms -41.2% 94ms 157ms >> -40.4% >> DictCreation: 137ms 105ms +30.0% 143ms 106ms >> +33.9% >> DictWithFloatKeys: 301ms 230ms +30.8% 306ms 232ms >> +32.0% >> DictWithIntegerKeys: 343ms 106ms +223.0% 351ms 108ms >> +224.7% >> DictWithStringKeys: 388ms 102ms +282.1% 395ms 102ms >> +285.1% >> ForLoops: 39ms 95ms -59.0% 40ms 97ms >> -58.5% >> IfThenElse: 35ms 111ms -68.9% 37ms 112ms >> -66.8% >> ListSlicing: 468ms 155ms +201.3% 477ms 157ms >> +204.4% >> NestedForLoops: 55ms 120ms -54.1% 57ms 122ms >> -53.0% >> NormalClassAttribute: 276ms 127ms +117.7% 282ms 129ms >> +119.7% >> NormalInstanceAttribute: 106ms 119ms -11.0% 109ms 120ms >> -9.7% >> PythonFunctionCalls: 37ms 130ms -71.7% 40ms 131ms >> -69.5% >> PythonMethodCalls: 127ms 158ms -19.6% 132ms 159ms >> -16.8% >> Recursion: 49ms 177ms -72.5% 50ms 179ms >> -71.8% >> SecondImport: 189ms 126ms +50.4% 195ms 127ms >> +53.6% >> SecondPackageImport: 196ms 134ms +45.8% 202ms 136ms >> +48.5% >> SecondSubmoduleImport: 266ms 177ms +50.6% 272ms 180ms >> +51.3% >> SimpleComplexArithmetic: 97ms 148ms -34.4% 101ms 149ms >> -32.0% >> SimpleDictManipulation: 311ms 118ms +164.4% 318ms 118ms >> +168.0% >> SimpleFloatArithmetic: 64ms 119ms -46.6% 69ms 121ms >> -43.2% >> SimpleIntFloatArithmetic: 41ms 100ms -59.4% 43ms 102ms >> -58.2% >> SimpleIntegerArithmetic: 43ms 100ms -57.2% 44ms 101ms >> -56.4% >> SimpleListManipulation: 121ms 104ms +16.6% 124ms 105ms >> +18.3% >> SimpleLongArithmetic: 126ms 113ms +11.2% 131ms 115ms >> +14.1% >> SmallLists: 223ms 154ms +45.0% 229ms 156ms >> +47.0% >> SmallTuples: 508ms 144ms +252.2% 523ms 145ms >> +259.7% >> SpecialClassAttribute: 275ms 124ms +121.2% 283ms 126ms >> +124.5% >> SpecialInstanceAttribute: 105ms 210ms -50.1% 109ms 212ms >> -48.5% >> StringMappings: 342ms 633ms -46.0% 351ms 637ms >> -44.9% >> StringPredicates: 222ms 217ms +2.3% 228ms 219ms >> +4.4% >> StringSlicing: 245ms 162ms +51.4% 260ms 163ms >> +59.4% >> TryExcept: 7ms 100ms -93.4% 11ms 102ms >> -89.4% >> TryRaiseExcept: 3413ms 114ms +2896.2% 3453ms 115ms >> +2910.6% >> TupleSlicing: 227ms 150ms +51.6% 235ms 151ms >> +54.9% >> UnicodeMappings: 227ms 126ms +79.6% 233ms 129ms >> +81.4% >> UnicodePredicates: 218ms 129ms +68.8% 224ms 132ms >> +70.4% >> UnicodeSlicing: 211ms 182ms +15.8% 223ms 184ms >> +21.3% >> ------------------------------------------------------------------------------- >> Totals: 12781ms 7659ms +66.9% 13191ms 7741ms >> +70.4% >> >> (this=ipy11.pybench, other=py25.pybench) >> _______________________________________________ >> users mailing list >> [email protected] >> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com >> >> >> > > _______________________________________________ > users mailing list > [email protected] > http://lists.ironpython.com/listinfo.cgi/users-ironpython.com > > _______________________________________________ users mailing list [email protected] http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
