Thorbjørn Ravn Andersen wrote:
There is the minor thing that you are not using the result returned by
the format routines. Hence the JIT _MAY_ have done optimization tricks
influencing the numbers you see. Also you have not shown that the
output numbers are statistically representative (i.e. close to the mean
of several runs) and how the numbers are if you test slf4J before log4mf
(as HotSpot optimizations may kick in while testLogMF is being run).
Is the resolution of your nanotime function high enough that the
differences over a single format()-invocation is usable for summarizing,
i.e. can you inject the measurement code in a real life scenario?
I have ran the tests at least three or four times. The results are
stable which imho obviates the need to compute averages of the
runs. Modifying the test to collect the results in a public field of
the test does not modify the results but prevents the JIT from short
circuiting the method invocation. Inversing the order of the tests or
disabling one test and then the order does not change the results
either. (I have actually performed all of the above checks.)
Moreover, as you point out, the nano time function may be playing
tricks on me, but if it is, it is doing so very consistently. :-)
When I say irrefutable proof, I mean within the context of log4j and
this discussion. If human lives were at stake, then it would be wise
to repeat the tests on at least 5 other machines running different
operating systems and different JVMs by different providers (IBM, Sun,
Bea). Asking independent parties to repeat the experiment would also
be warranted.
The test I provided directly compares the formatting functions of
SLF4J and LogMF. One can argue that the performance difference does
not matter, as Curt has. However, one can no longer argue that there
is no performance difference.
Nevertheless, I agree with you that care is required when dealing with
performance measurements which can be sometimes misleading. In that
spirit, I encourage you to repeat the experiment and share your
results.
--
Ceki Gülcü
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
