On Thursday 21 March 2002 10:24, Jukka Santala wrote: > On Tue, 19 Mar 2002, Dalibor Topic wrote: > > Speaking of benchmarking suites, there is Ashes from the SableVM people. > > http://www.sable.mcgill.ca/ashes/ > > I think it's open source, so I'd prefer that. :) > > Okay, thanks, I'll take a look at that. There are also some open source > conformance test-suites, I believe.
There is mauve. There is jacks (compiler conformance). Then there are the regression test suites of various implementations, like libgcj's. I have run kaffe with mauve. I haven't looked into other conformance test-suites. > Also, it does give you a rough idea on conformance if the benchmarks break > too badly :) I was thinking more in the terms of "breaking some > optimization". The problem with optimizations is that they're quite > platform specific and sometimes longer to run than simple conformance/ > regression tests, so they're difficult for a single developer to run. In > addition Kaffe has pretty good conformance, but it would appear > performance could use some work. Optimizations in core & library code should be beneficial to all platforms. But I guess you are not talking about those, since they are not platform specific. I don't think I understood the term "breaking some optimization" properly. Do you mean "breaking some benchmark" ? If so, yes, meaningful benchmarks will take longer to run than conformance tests in general. You have to run a benchmark for some time to avoid measuring side effects like disk I/O, VM startup etc. Unless of course that's exactly what you want to measure :) Conformance tests usually run for a couple of seconds each. In general, if people run kaffe on benchmarks & put the results on the web, I assume that Jim will link to them. If people volunteer to do so regularly, like the nightly builds done by the flex people, someone could write a couple of scripts to collect & present the information in a nice way. All it takes is to pick the benchmarks, find volunteers and agree on the procedure :) I could submit results on x86-linux, interpreter/jit/jit3. But that's a rather common platform, and not that interesting, I guess. Unless there is a massive amount of interest in seeing how kaffe performs on old, slow computers :) dalibor topic _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com