Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-02 Thread Nadav Har'El
On Tue, Feb 01, 2011, Tzafrir Cohen wrote about "Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > On Tue, Feb 01, 2011 at 12:49:00PM +0200, Elazar Leibovich wrote: > > > The program they tested[1] is strictly CPU bound. BTW, standard deviation > > w

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Nadav Har'El
On Tue, Feb 01, 2011, Oleg Goldshmidt wrote about "Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > awareness. It's a far cry from saying that computers are "not deterministic" > though. If you want to measure A and in fact you measure something

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Tzafrir Cohen
On Tue, Feb 01, 2011 at 12:49:00PM +0200, Elazar Leibovich wrote: > The program they tested[1] is strictly CPU bound. BTW, standard deviation > wouldn't work as well, but it shows (I think) that there's no such think as > "ideal minimal runtime". > > [1] > static int i = 0, j = 0, k = 0; > int m

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Oleg Goldshmidt
2011/2/1 Elazar Leibovich > > See this paper[2] which is referred in the slides. > > [2] http://www-plan.cs.colorado.edu/diwan/asplos09.pdf This morning I took part in a meeting here at work where something related was discussed, so I am hot for the topic. Apologies all around. The paper you r

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Shlomi Fish
On Tuesday 01 Feb 2011 12:09:17 Nadav Har'El wrote: > On Tue, Feb 01, 2011, Elazar Leibovich wrote about "Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > > Long story short, he claims there that modern computers are now highly > > non-deterministic, he

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Elazar Leibovich
On Tue, Feb 1, 2011 at 12:09 PM, Nadav Har'El wrote: > But if he runs his deterministic application 5 times and gets 5 different > durations, each duration is composed of the deterministic run-time of > the application plus a random delay caused by other things on the system. > The *minimum* of th

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-02-01 Thread Nadav Har'El
On Tue, Feb 01, 2011, Elazar Leibovich wrote about "Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > Long story short, he claims there that modern computers are now highly > non-deterministic, he demonstrated 20% running time variation by the same > JVM runnin

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2011-01-31 Thread Elazar Leibovich
On Mon, Jul 19, 2010 at 12:18 PM, Nadav Har'El wrote: > Imagine, for example, that you run a certain program 5 times and get the > times: 20.0, 18.0, 18.1, 27.0, 18.1 > Evidently, the first run was slower because things were not in the cache, > and the run that took 27.0 was delayed by some other

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-19 Thread Elazar Leibovich
On Mon, Jul 19, 2010 at 11:18 AM, Nadav Har'El wrote: > On Sun, Jul 18, 2010, Elazar Leibovich wrote about "Re: New Freecell Solver > gcc-4.5.0 vs. LLVM+clang Benchmark": > > The standard deviation can give you an estimation of the minimal running > > time. (99

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-19 Thread Nadav Har'El
On Sun, Jul 18, 2010, Elazar Leibovich wrote about "Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > The standard deviation can give you an estimation of the minimal running > time. (99. of the samples are within X standard deviations below the > average. Pick

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Elazar Leibovich
The standard deviation can give you an estimation of the minimal running time. (99. of the samples are within X standard deviations below the average. Pick a high enough relative to the number of times you'll run the software, and you'll get an estimation of the minimum running time you'll

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Shlomi Fish
On Sunday 18 Jul 2010 11:56:33 Elazar Leibovich wrote: > While you're on to it. > I expect to read in a benchmark report, the number of time the software was > executed, the mean running time, and the standard deviation. Running and > timing it once can hide a pretty large error. > Perhaps you're

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Shlomi Fish
On Sunday 18 Jul 2010 11:37:17 Shlomi Fish wrote: > Hi Nadav, > > On Sunday 18 Jul 2010 10:03:32 Nadav Har'El wrote: > > On Sat, Jul 17, 2010, Shlomi Fish wrote about "New Freecell Solver > > gcc-4.5.0 > > vs. LLVM+clang Benchmark": > > > On the other hand, with gcc-4.5.0 with "-flto" and "-fwhol

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Elazar Leibovich
While you're on to it. I expect to read in a benchmark report, the number of time the software was executed, the mean running time, and the standard deviation. Running and timing it once can hide a pretty large error. On Sun, Jul 18, 2010 at 11:37 AM, Shlomi Fish wrote: > Hi Nadav, > > On Sunday

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Shlomi Fish
Hi Nadav, On Sunday 18 Jul 2010 10:03:32 Nadav Har'El wrote: > On Sat, Jul 17, 2010, Shlomi Fish wrote about "New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > > On the other hand, with gcc-4.5.0 with "-flto" and "-fwhole-program" > > Freecell Solver ran at 85.1303749084473 seconds. > >

Re: New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark

2010-07-18 Thread Nadav Har'El
On Sat, Jul 17, 2010, Shlomi Fish wrote about "New Freecell Solver gcc-4.5.0 vs. LLVM+clang Benchmark": > On the other hand, with gcc-4.5.0 with "-flto" and "-fwhole-program" Freecell > Solver ran at 85.1303749084473 seconds. > > I admit that I ran the gcc benchmark with a good renice and only