Getting back more to the original topic, while basic comparisons can be useful, differences between individual machines can overwhelm other factors. Also, we should bear in mind what Roger has said many times previously that anything less than about a factor of two (?) difference is not worth pursuing. So, it may not be as important to pursue fine differences as to measure how something scales: I'd like to have some idea when I'll start paging as my data increases in size.
So, I would like to see comparisons like the following when deciding between different versions of "foo": NB. Arbitrary thing to test: foo=: %. NB. Get timings for different-sized arguments. tms1e2=. (10$1) 6!:2 'foo rm' [ rm=. <:+:1e2 1e2?@$0 tms1e3=. (10$1) 6!:2 'foo rm' [ rm=. <:+:1e3 1e3?@$0 NB.* usus: Usual stats: min, max, mean, SD usus=: (<./,>./,mean,stddev)"1 NB. Basic stats are only so helpful: usus&>tms1e2;tms1e3 0.0041076009 0.0054712414 0.0042859661 0.00041699928 2.0378695 2.0685407 2.0539971 0.0094757391 NB. Timings per unit are better for comparison: usus&>(tms1e2;tms1e3)%&.>*:1e2 1e3 4.1076009e_7 5.4712414e_7 4.2859661e_7 4.1699928e_8 2.0378695e_6 2.0685407e_6 2.0539971e_6 9.4757391e_9 NB. Now we can get some idea of how this scales: %/usus&>(tms1e2;tms1e3)%&.>*:1e2 1e3 0.20156349 0.26449764 0.20866466 4.4007046 On Thu, Oct 20, 2011 at 4:04 PM, Roger Hui <[email protected]>wrote: > This message is addressed to Forum members who are knowledgeable in > statistics. > > The objective is to test whether the same expression is faster, > slower, or takes the same amount of time, on the two different > versions of the interpreter. We know that due to vagaries of the > ... -- Devon McCormick, CFA ^me^ at acm. org is my preferred e-mail ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
