It's possible that a proper benchmark should not count the time the engine takes to instantiate nor the time it takes to warmup. In that case, it Ideally would run 1,000,000 tests to assure the engine has had the opportunity to optimize and reoptimize and further optimize to the best it can do. Then the test would actually run the real tests to get a "fair" measurement.
On the other hand, in a browser, nobody writes front facing JavaScript that runs 1,000,000 loops to warm up the JS engine. So it may well be fair to look at those real world situations as well. On Jul 10, 2013, at 2:04 AM, Jakob Kummerow <[email protected]> wrote: > > On Wed, Jul 10, 2013 at 3:32 AM, QingHai Hu <[email protected]> wrote: > Thank you for your reply . > I'm sorry I did not say it clearly. > The compile-time that I said is the JavaScript compilation time, such as > benchmark/racards.js compilation time. And the run-time that I said > is the JavaScript' run-time,the same example is the benchmark/racards.js's > run-time. > Does the final score contain both of them? > > As I said, that depends on the JavaScript engine. It is possible to write a > JS engine that doesn't compile anything at all -- a couple of years ago that > was the usual situation. If an engine does compile JS to machine code, it can > freely decide when to do that (e.g. immediately up front, or on demand, or > after x calls to a given function, etc.; for modern engines there's > additionally the distinction between unoptimized and optimized compilation). > There is no way a benchmark could explicitly measure this. > > In the special case of V8, some compilation time will be part of the score, > and some compilation will already happen during the warmup phase (which does > not count towards the score). I would expect other modern JS engines to > behave similarly. > > In addition, I want to know how to transform the compilation time and > run-time into the final score. > > Several runs are performed and timed, then a reference value is divided by > the average time per run. See > https://code.google.com/p/v8/source/browse/branches/bleeding_edge/benchmarks/base.js#177. > > > Yours Hu. > > 在 2013年7月9日星期二UTC+8下午7时19分20秒，Jakob Kummerow写道： > On Tue, Jul 9, 2013 at 12:09 PM, QingHai Hu <[email protected]> wrote: > Hi,guys > I want to know the meaning of the v8 benchmark's score. > > Higher is better. > > Does it depend on the compile-time and the run-time ? Or only dependent on > run-time? > > I'm not sure what you mean. > C++ compilation is definitely not part of the score. > Whether JavaScript compilation time is part of the score depends on the test > being run and the JS engine. There is a warmup phase, so JIT-compiling > engines will probably compile at least some of the source during this warmup > run, but this is very much an implementation detail. > > Also note that the V8 benchmark suite has been superseded by the Octane > benchmark, which includes some more tests. > > > -- > -- > v8-users mailing list > [email protected] > http://groups.google.com/group/v8-users > --- > You received this message because you are subscribed to the Google Groups > "v8-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > -- -- v8-users mailing list [email protected] http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
