On Sun, Dec 06, 2009 at 08:06:50AM -0800, Geoffrey Broadwell wrote: > On Sun, 2009-12-06 at 01:33 -0800, Jonathan Leto wrote: > > This one is all over the place and trunk is not looking very pretty. > > http://gist.github.com/250139 > > 1. Make sure HEAD is in the same grouping as the fastest time, for all > benchmarks we can find. > 2. Make sure HEAD has the fastest time on all of our benchmarks. > > Right now I think we should be focusing on #1, both because #2 is too > far ahead of where we are now, and because violations of #1 will tend to > tell us when we've found a particular use case that needs more immediate > attention.
This thread and discussion bothers me a fair bit, because it seems to imply that we should be focusing our limited resources on optimizing Parrot for code that is *completely unlike* anything our target HLLs are likely to be generating anytime in the near future. Given Parrot's other limitations, there is almost no way that we could get from PHP, Perl 5, Perl 6, or Python source to the PIR code that is being measured in this benchmark (except perhaps by HLL compilers that are specifically tuned to this purpose). As such, these results are completely unrepresentative of HLL performance on Parrot. Indeed, one reason why older Parrots are able to perform better is because they were able to ignore the very features (and overhead) that have been added to support HLL development. Beyond that, I do not agree with any premise that HLLs will benefit from the optimizations this sort of analysis can achieve. I will even go so far as to say that many of the HLL performance and code difficulties we see today are *precisely* because of past efforts to optimize Parrot around unrealistic PIR snippets like these. What Parrot needs to do instead is to analyze, benchmark, and optimize around the type of code that HLLs will actually generate. So, if there are Parrot developers out there who are only interested in optimizing benchmarks like these, then I'm fine with those individuals pursing that interest. But generalizing to say that Parrot development as a whole should focus on optimizing _away_ from today's HLL developer needs bothers me a lot. Pm _______________________________________________ http://lists.parrot.org/mailman/listinfo/parrot-dev
