Hi Rana 2006/9/14, Rana Dasgupta <[EMAIL PROTECTED]>: <SNIP>
One way to write the test would be to loop N times on a scenario that kicks in the optimization say, array bounds check elimination and then loop N times a very similar scenario but such that the bounds check does not get eliminated. Then the test should pass only if the difference in timing is at least X on any platform.
I tried to create a similar test when was testing that resolved IP addresses are cached. Finally I've figured out that this test is not the best pre-commit test as it may accidentally fail if I run other apps on the same machine where I run the tests. And as you know unstable failure is not the most pleasant thing to deal with :) Thanks, Mikhail
I have been forced to do this several times :-) So I couldn't resist spreading the pain. Thanks, Rana > On 14 Sep 2006 12:10:19 +0700, Egor Pasko < [EMAIL PROTECTED]> wrote: > > > > > > Weldon, I am afraid, this is a performance issue and the test would > > show nothing more than a serious performance boost after the fix. I'll > > find someone with a test like this :) and ask to attach it to JIRA. > > But .. do we need performance tests in the regression suite? > > > > Apart of this issue I see that JIT infrastructure is not so > > test-oriented as one would expect. JIT tests should sometimes be more > > sophisticated than those in vm/tests and, I guess, we need a separate > > place for them in the JIT tree. > > > > Many JIT tests are sensitive to various JIT options and cannot be > > reproduced in default mode. For example, to catch a bug in OPT with a > > small test you will have to provide "-Xem opt" options. Thus, in a > > regression test we will need: > > (a) extra options to VM, > > (b) sources (often in jasmin or C++ (for hand-crafted IRs)) > > (c) and even *.emconfig files to set custom sequences of optimizations > > > > (anything else?) > > I am afraid, we will have to hack a lot above JUnit to get all these. > > > > Let's decide whether we need a framework like this at the time. We can > > make a first version quite quickly and improve it further on as-needed > > basis. Design is not quite clear now, though it is expected to be a > > fast-converging discussion. > > > > > > -- > > Egor Pasko, Intel Managed Runtime Division > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
