Our project is now in a good shape and we started encouraging people to try real applications on top of Harmony. One of the important parts of the feedback will be general impression of stability, reliability and performance of the HDK. We are doing pretty good in fixing bugs and developing new functionality in HDK, and it might be a good time for us to start thinking of performance as well. It might be a little bit preliminary, but do we consider having any performance targets for us? If yes, we may need to focus at least some of our efforts on benchmarking and tuning overall performance of HDK.
One of the main questions here is what should be the targets for us and how should we measure our performance. There are several ways for measuring performance, such us commercial benchmarks, free benchmarks, application startup, small micro suites, etc. Some of free benchmarks have been mentioned in JIRA issues and dev list, nevertheless at the moment we don't have any goals for performance. In spite of application enabling initiative it might be good to consider publicly available benchmarks as the additional list of the software applications which we would enable on Harmony. So I suggest to start discussing performance techniques and methods that can be used for comparing performance between RI and HDK. I think in case we do not consider performance issues, we can get negative feedback from users even if application starts without any errors, exceptions, etc. One of the benchmarks that was mentioned is DaCapo[1]. It's a free open-source benchmark suite and I believe it can be used for regular performance measurement of HDK. I've tried to find other free suites and got the following list: Telco - this one mostly stresses BigInteger/BigDecimal functionality GcOld - the purpose of this one is clear from the name :) SciMark - java benchmark for scientific and numerical computing Linpack java - well-known benchmark solving linear equations The Plasma Benchmark - creates an animated display by continuously summing four sine waves in an applet JavaWorld Benchmark - benchmark for low-level operations: loops, accessing variables, method invocation, arithmetic operators, casting, instantiation, exception handling, thread creation and switching. CaffeineMark 3.0 - low-level benchmark suite, including sieve of Eratosthenes, sorting, logic ops, method invocation, floating point, simple graphics and GUI ops JavaGrande benchmark suite - a set of benchmarks stressing different areas of java. Having in mind that the list of publicly available benchmarks is not too big, sometimes it will be necessary to create micro benches for some of patches (for instance, Harmony-935). IMO micro should be started in case we change some code that the bench covers. Other interesting and possibly more productive way for comparing performance between different implementations are to use non-free benchmarks. For instance, we can use benchmarks from Spec[n], like SpecJVM, SpecJBB, SpecJAppserver. Unfortunately first we should get license for it, but I believe this issue can be solved within the help of companies participating in Harmony :) Thoughts? Comments? [1] http://dacapobench.org [2] http://www.spec.org -- Vladimir Strigun, Intel Middleware Products Division --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
