at least, or better yet move to a new thread. This conversation has moved way beyond discussing the results of a contribution vote.
For the poor people who have hundreds of mails to read ;-) Tim Pavel Ozhdikhin wrote: > Rana, > > To cover most of the performance regressions ideally we might have three > types of tests: > > 1. High-level benchmarks (like SPECs or DaCapo benchmarks) - cover > significant performance issues or issues that are not related to a > particular optimization > 2. Performance regression tests or micro-benchmarks - a > bytecode-based tests covering one optimization or a small group of them > 3. Low-level tests (like VM or JIT unit tests) - tests for > particular code transformations. > > To my mind the second type of tests is the most sensitive to the > environment. In many cases it's difficult to create such test in a > reasonable amount of time. I admit that this type of tests might be a > short-term solution to check if a fix has been integrated properly but in > the long-term we need also 1 and 3 from the list. > > Thanks, > Pavel > > On 9/14/06, Rana Dasgupta <[EMAIL PROTECTED]> wrote: > >> Hi Pavel, >> Platform specific optimizations can be accomodated in the scheme >> described by doing a cpuid check in the test and automatically passing it >> or >> disabling on all other platforms. That shouldn't be too hard. >> I understand that some jit optimizations are deeper and more abstract, >> but ultimately the value of the optimization cannot just be the morphing >> of >> an IR, and the gain cannot be invisible to the user, or the regression >> undetectable. If it needs to be part of a sequence to be effective, the >> scenario in the test needs to be set up accordingly. It is a little >> uncomfortable if a framework does some magic and then comes back and says >> "everything is OK". >> Sorry to sound difficult. >> >> Thanks, >> Rana >> >> >> > On 9/14/06, Pavel Ozhdikhin <[EMAIL PROTECTED]> wrote: >> > > >> > > Hello Rana, >> > > >> > > When I think of an optimization which gives 1% improvement on some >> > > simple >> > > workload or 3% improvement on EM64T platforms only I doubt this >> can be >> > > easily detected with a general-purpose test suite. IMO the >> performance >> > > regression testing should have a specialized framework and a stable >> > > environment which guarantees no user application can spoil the >> results. >> > > >> > > The right solution might also be a JIT testing framework which would >> > > understand the JIT IRs and check if some code patterns have been >> > > optimized >> > > as expected. Such way we can guarantee necessary optimizations are >> done >> > > independently of the user environment. >> > > >> > > Thanks, >> > > Pavel >> > > >> > > >> > > >> > > >> > > >> >> > -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
