Re: [Mono-dev] how to improve mono performance
Hi I'm investigating mono performance regression and improvement. You can see some measurements on http://nenya.ms.mff.cuni.cz/projects/mono/index.phtml And I try to find which lines in source code cause differencies in performance. Peter ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
On Thu, 2005-11-17 at 23:32 -0800, zhu shi song wrote: I insist that performance is the key factor for key applications. For example, we now have one enterprise web query application. Every day between 8:30AM and 12:00AM, it serve about 2 requests. Now we use one P4 2.6G PC Server with 512M DDR running Windows 2003 Server, it works well. If we port it to Linux + Mono, we must upgrade the hardware to meet the same workload. Have you actually ported and measured this? From your previous emails it sounds like you haven't. Just because something can add doubles N times faster does not mean that it will be N times faster for your application. If you'd like us to help you with your application, you are going to have to do something more than show us issues with our GC and Register Allocator that we already know about (See Miguel's blog: http://tirania.org/blog/texts/mono-status.html) What we need is: * Output of --profile from *your application* running on mono under realistic load * (If possible) The source code to your application (I am sure that Miguel could help you if you need an NDA, etc) so that we can better understand why some functions show up in the profile To reiterate: your benchmarks do not help us fix *your* problems, at least not in the short term. I'd also point out: 20,000 requests / (3.5 hours) = 1.5 requests / second Is your server using *all* it's cpu power for 1.5 requests/sec? Maybe you can make small optimizations in your application which will help it on either platform (I recently encountered a web application where submissions were taking 20 seconds each, it turned out that we did not use indexes correctly in sql server. A little bit of DBA action made requests take 1 second each). It's often easier to optimize the application than the platform. This seems like a small price to pay for the benefit of not having to deal with the latest worm, patch Tuesday and the general pain of Windows system administration (not to mention the money one saves from the Windows license!) -- Ben ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
On Fri, 2005-11-18 at 03:19 -0500, Ben Maurer wrote: Is your server using *all* it's cpu power for 1.5 requests/sec? Maybe you can make small optimizations in your application which will help it on either platform (I recently encountered a web application where submissions were taking 20 seconds each, it turned out that we did not use indexes correctly in sql server. A little bit of DBA action made requests take 1 second each). It's often easier to optimize the application than the platform. This seems like a small price to pay for the benefit of not having to deal with the latest worm, patch Tuesday and the general pain of Windows system administration (not to mention the money one saves from the Windows license!) Please don't take this to mean that we expect performance to be our user's problem. Many of the project's hackers (including myself) are dedicated to providing a high performance runtime. However, there's only so much we can do with limited resources. -- Ben ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
Hello, From the measurements I've taken, MS .NET is generally 2 to 3 times faster than Mono, both using micro-benchmarks and applications that do real work. It is easy to spot performance issues with micro-benchmarks, and we typically use these to do some of our development work. For large applications it is harder to find out what pieces are performance issues in the runtime, and which others are operating system pieces. For instance, even if our runtime produces slower code, ASP.NET applications benefit from Linux's TCP/IP and file system async semantics so overall, we come on top in many benchmarks. Quoting `applications that do real work' without pointing to them is useless. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
RE: [Mono-dev] how to improve mono performance
I'd like to point out that the arrival of his requests probably follows a poisson distribution anyway. In which case, his probability of getting more than 9 requests per second is null. So, it stays really reasonable... And WAY under the limit of 732 req/sec for ASP that Miguel just mentioned. You can verify this alegation by looking at the analysis below if anyone feels like it ;). I may be wrong. Max == requests: 2 service: 4:30:00 hh:mm:ss 16200seconds Avg request/second: 1.23 Concurrent | Poisson| Requests Req/sec| probability | concerned ---+-+--- 0 | 29.10%| 5819 1 | 35.92%| 7184 2 | 22.17%| 4435 3 |9.12%| 1825 4 |2.82%|563 5 |0.70%|139 6 |0.14%| 29 7 |0.03%| 5 8 |0.00%| 1 ---+-+--- 100.00% 2 Basically, 5919 requests have 29.1% chance to be the only request in the system for a second, and so on... (based on the excel poisson distrib, with 1.23 as the mean and a cumul of 0) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Miguel de Icaza Sent: Friday, November 18, 2005 12:25 PM To: zhu shi song Cc: Atsushi Eno; Ben Maurer; mono-devel-list@lists.ximian.com Subject: Re: [Mono-dev] how to improve mono performance Hello, I insist that performance is the key factor for key applications. For example, we now have one enterprise web query application. Every day between 8:30AM and 12:00AM, it serve about 2 requests. Now we use one P4 2.6G PC Server with 512M DDR running Windows 2003 Server, it works well. If we port it to Linux + Mono, we must upgrade the hardware to meet the same workload. Ben did a fantastic job in pointing out that 1.5 request per second is hardly a lot. For ASP.NET applications, my laptop (a 2.0 Ghz with a much slower configuration than your server) can dispatch 732 ASP.NET requests per second. The sample TestService web service page which puts a bunch of controls into a page comes out at 371 requests per second. Which puts Mono at handling 4.41 million request per second in the time frame that you described. So, 220 times the load that you have today on your site. So for all intents and purposes Mono 1.1.10 should be able to cope with your load just fine. We are still working to improve Mono's speed (see my status report on my blog for details about what we are doing), but you hardly need these improvements today. Other people might. Miguel. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
(1) I feel a little disappointed at the comparsion results. I think we should not only prove we can do this but we can do even better. The performance fact is very essential for production applications. I hope the mono community can improve performance of mono largely. Then mono will be depoyed in many real apps. We have spent so much effort on it, we shouldn't let it be reference implementation just for lab. (2) when I use msft's csc to compile compare.cs, then I run under linux+mono. I just got 10% faster than that compiled by mono mcs. So maybe mono's jit is not so good as msft's. tks zhu --- Ben Maurer [EMAIL PROTECTED] wrote: On Wed, 2005-11-16 at 20:30 -0800, zhu shi song wrote: I've got one test example and tested it using linux mono and windows .NET on the same machine. The results showed that the performance of linux mono is much bad than windows .NET. Linux Mono: ArrayList strings test.3311 ms StringBuilder test.4008 ms Integer Floating ADD.15145 ms Exception test.621 ms Reflection and recursion...9421 ms Windows .Net: ArrayList strings test.1109 ms StringBuilder test.1437 ms Integer Floating ADD.2734 ms Exception test.4046 ms Reflection and recursion...5843 ms Machine configuration: RAM: 512M DDR CPU: Intel C4 2.0G OS: Windows 2003 Server .NET 1.1 Framework Fedora Core 4 with mono compiled from svn Now I try to port my aspx web application to linux mono from windows 2003 .net framework. But regarding of performance, I wonder whether it is deserved. Maybe I lack some tips to improve linux mono performance. If someone knows pls help me. Chances are, your web application does not add items to an array, create stringbuilders, add integers, throw exceptions and do reflection anything like the tests you have configured. I would suggest benchmarking a ported application rather than these micro measurements. If you want to do such measurements, I would recommend something more relevant to your application (database accesses? the asp.net pipeline?) That being said, some possible reasons for slowness: ArrayList/StringBuilder: Likely the GC. Floating Point: I think msft may be taking advantage of some special hardware stuff. I highly doubt this is relevant to your application. Exceptions: We super optimized this path :-) Reflection: Not sure, might be worth looking at -- Ben __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
zhu shi song wrote: (1) I feel a little disappointed at the comparsion results. I think we should not only prove we can do this but we can do even better. The performance fact is very essential for production applications. I hope the mono community can improve performance of mono largely. Then mono will be depoyed in many real apps. We have spent so much effort on it, we shouldn't let it be reference implementation just for lab. I agree that performance improvements are impressive, but against your expectation mono is already used in practice, including softwares and individual solutions: http://www.mono-project.com/Software http://www.osnews.com/story.php?news_id=10714 (2) when I use msft's csc to compile compare.cs, then I run under linux+mono. I just got 10% faster than that compiled by mono mcs. So maybe mono's jit is not so good as msft's. Wrong assumption. There are three factors in mono to affect on performance: - class libraries - compiler outputs - JIT The most effective optimizations could be done at class libraries (which could be usually done with a few lines of changes on a few critical paths), where many hot users among users turned into hackers and posting several patches here (and there). Actual analysis and thoughts on which affects on performance are welcome (like what Ben wrote). It would be awesome if you try mono --profile (or heap-buddy in our svn repository) to count individual performance, find out where the code is specially slow, and optimize the sources. Even a line of change could result in more than 200% performance boost in some code areas (I often got such hits in DOM/XSLT areas). Atsushi Eno tks zhu --- Ben Maurer [EMAIL PROTECTED] wrote: On Wed, 2005-11-16 at 20:30 -0800, zhu shi song wrote: I've got one test example and tested it using linux mono and windows .NET on the same machine. The results showed that the performance of linux mono is much bad than windows .NET. Linux Mono: ArrayList strings test.3311 ms StringBuilder test.4008 ms Integer Floating ADD.15145 ms Exception test.621 ms Reflection and recursion...9421 ms Windows .Net: ArrayList strings test.1109 ms StringBuilder test.1437 ms Integer Floating ADD.2734 ms Exception test.4046 ms Reflection and recursion...5843 ms Machine configuration: RAM: 512M DDR CPU: Intel C4 2.0G OS: Windows 2003 Server .NET 1.1 Framework Fedora Core 4 with mono compiled from svn Now I try to port my aspx web application to linux mono from windows 2003 .net framework. But regarding of performance, I wonder whether it is deserved. Maybe I lack some tips to improve linux mono performance. If someone knows pls help me. Chances are, your web application does not add items to an array, create stringbuilders, add integers, throw exceptions and do reflection anything like the tests you have configured. I would suggest benchmarking a ported application rather than these micro measurements. If you want to do such measurements, I would recommend something more relevant to your application (database accesses? the asp.net pipeline?) That being said, some possible reasons for slowness: ArrayList/StringBuilder: Likely the GC. Floating Point: I think msft may be taking advantage of some special hardware stuff. I highly doubt this is relevant to your application. Exceptions: We super optimized this path :-) Reflection: Not sure, might be worth looking at -- Ben __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
BTW we have more detailed description on this topic here: http://www.mono-project.com/Performance_Tips Atsushi Eno ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
I very agree with you. The right direction to improve performance should be from : - class libraries - compiler outputs - JIT I don't think we put emphasis on apps. Because we have the same source, but the running results are so different using linux+mono and msft .net. I insist that performance is the key factor for key applications. For example, we now have one enterprise web query application. Every day between 8:30AM and 12:00AM, it serve about 2 requests. Now we use one P4 2.6G PC Server with 512M DDR running Windows 2003 Server, it works well. If we port it to Linux + Mono, we must upgrade the hardware to meet the same workload. tks zhu --- Atsushi Eno [EMAIL PROTECTED] wrote: zhu shi song wrote: (1) I feel a little disappointed at the comparsion results. I think we should not only prove we can do this but we can do even better. The performance fact is very essential for production applications. I hope the mono community can improve performance of mono largely. Then mono will be depoyed in many real apps. We have spent so much effort on it, we shouldn't let it be reference implementation just for lab. I agree that performance improvements are impressive, but against your expectation mono is already used in practice, including softwares and individual solutions: http://www.mono-project.com/Software http://www.osnews.com/story.php?news_id=10714 (2) when I use msft's csc to compile compare.cs, then I run under linux+mono. I just got 10% faster than that compiled by mono mcs. So maybe mono's jit is not so good as msft's. Wrong assumption. There are three factors in mono to affect on performance: - class libraries - compiler outputs - JIT The most effective optimizations could be done at class libraries (which could be usually done with a few lines of changes on a few critical paths), where many hot users among users turned into hackers and posting several patches here (and there). Actual analysis and thoughts on which affects on performance are welcome (like what Ben wrote). It would be awesome if you try mono --profile (or heap-buddy in our svn repository) to count individual performance, find out where the code is specially slow, and optimize the sources. Even a line of change could result in more than 200% performance boost in some code areas (I often got such hits in DOM/XSLT areas). Atsushi Eno tks zhu --- Ben Maurer [EMAIL PROTECTED] wrote: On Wed, 2005-11-16 at 20:30 -0800, zhu shi song wrote: I've got one test example and tested it using linux mono and windows .NET on the same machine. The results showed that the performance of linux mono is much bad than windows .NET. Linux Mono: ArrayList strings test.3311 ms StringBuilder test.4008 ms Integer Floating ADD.15145 ms Exception test.621 ms Reflection and recursion...9421 ms Windows .Net: ArrayList strings test.1109 ms StringBuilder test.1437 ms Integer Floating ADD.2734 ms Exception test.4046 ms Reflection and recursion...5843 ms Machine configuration: RAM: 512M DDR CPU: Intel C4 2.0G OS: Windows 2003 Server .NET 1.1 Framework Fedora Core 4 with mono compiled from svn Now I try to port my aspx web application to linux mono from windows 2003 .net framework. But regarding of performance, I wonder whether it is deserved. Maybe I lack some tips to improve linux mono performance. If someone knows pls help me. Chances are, your web application does not add items to an array, create stringbuilders, add integers, throw exceptions and do reflection anything like the tests you have configured. I would suggest benchmarking a ported application rather than these micro measurements. If you want to do such measurements, I would recommend something more relevant to your application (database accesses? the asp.net pipeline?) That being said, some possible reasons for slowness: ArrayList/StringBuilder: Likely the GC. Floating Point: I think msft may be taking advantage of some special hardware stuff. I highly doubt this is relevant to your application. Exceptions: We super optimized this path :-) Reflection: Not sure, might be worth looking at -- Ben __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list __ Start your day with Yahoo! -
Re: [Mono-dev] how to improve mono performance
From the measurements I've taken, MS .NET is generally 2 to 3 times faster than Mono, both using micro-benchmarks and applications that do real work. I would also point out that your .NET test was on MS, whereas your Mono test was on Linux. Exception-handling on MS is thought to be very slow. So that might explain part of the reason why Mono was so much faster on that component. As for the strings and StringBuilder, in MS, string and StringBuilder share some internal data structures and use a helper class. So their string operations and conversions between string and StringBuilder are very fast. That being said, some possible reasons for slowness: ArrayList/StringBuilder: Likely the GC. Floating Point: I think msft may be taking advantage of some special hardware stuff. I highly doubt this is relevant to your application. Exceptions: We super optimized this path :-) ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
see mono options: --aot Compiles the assembly to native code --optimize=OPT Turns on or off a specific optimization Use --list-opt to get a list of optimizations MS might has similar so it might not mean much for comparisons, but can mean a lot just for your absolute runtimes on linux. some of your times are very short, i.e. 1000ms you might want to make them go longer (iterate) so as to eliminate what might be an unfair startup requirement but i am guessing here. Maybe your timing is done in the code and not just with a time mono .. off the shell. I have done a couple of tests comparing optimized C# to optimized C (on linux) and generally found C#/mono to be 33% slower then C which actually I was very happy to see it so close. Your tests are showing 300%!! average differences to MS C#, so I am very surprized to see your results having such a difference :( The integer/floating add test results just blows me away, as i did a integer add comparison of C# vs. C on linux and again that was only 33% difference. Its almost as if your floating point part was done with out fpu (on linux) and done in the CPU fpu on Windows. -tl On Wed, 16 Nov 2005 20:30:23 -0800 (PST) zhu shi song [EMAIL PROTECTED] wrote: Dear lists, I've got one test example and tested it using linux mono and windows .NET on the same machine. The results showed that the performance of linux mono is much bad than windows .NET. Linux Mono: ArrayList strings test.3311 ms StringBuilder test.4008 ms Integer Floating ADD.15145 ms Exception test.621 ms Reflection and recursion...9421 ms Windows .Net: ArrayList strings test.1109 ms StringBuilder test.1437 ms Integer Floating ADD.2734 ms Exception test.4046 ms Reflection and recursion...5843 ms Machine configuration: RAM: 512M DDR CPU: Intel C4 2.0G OS: Windows 2003 Server .NET 1.1 Framework Fedora Core 4 with mono compiled from svn Now I try to port my aspx web application to linux mono from windows 2003 .net framework. But regarding of performance, I wonder whether it is deserved. Maybe I lack some tips to improve linux mono performance. If someone knows pls help me. tks zhu __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] how to improve mono performance
On Wed, 16 Nov 2005 20:30:23 -0800 (PST) zhu shi song [EMAIL PROTECTED] wrote: Dear lists, I've got one test example and tested it using linux mono and windows .NET on the same machine. The results showed that the performance of linux mono is much bad than windows .NET. I'd like to try to reproduce this on one of my servers. Can you make the source available for download and or send to me offlist. I'd like to make sure this is reproducible before we make any final conclusions. I've traced and optimized my code and found that despite a few places Mono is in _real_ time faster. To this credit I add we do heavy caching and so even IF, not sure how big that is, there is some upfront difference between the two it can be worked around. Cheers, C. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
