Igor,
We at Symas Corporation completely agree with you. Our benchmarks are
intended to be indicative. As data points, they help us understand
the relative performance among various products, platforms, releases
and options. They are factual, rigorous, accurate, and, in the end,
only indicative.
Your original simple and elegant statement is, in fact, correct. A
benchmark which reflects your installation's actual usage (schema,
mix of entries, mix of transaction activity) is superior for your
decision making to anyone's generalized results. We are proud of our
benchmarks but expect others to do their own as they make these kinds
of decisions.
On Jul 14, 2005, at 7:03 AM, Igor Brezac wrote:
On Wed, 13 Jul 2005, Quanah Gibson-Mount wrote:
--On Wednesday, July 13, 2005 12:48 PM -0400 Igor Brezac
<[EMAIL PROTECTED]> wrote:
Benchmark that matters is the one you perform.
Yes and no. Certainly, taking at look at how things perform for
what you plan to deploy can be quite useful.
However, if you set up your benchmarks where you document well
what you did, and make sure that your configuration settings are
public, you can set up benchmarks that others can duplicate, and
can be agreed upon as indicative of real meaning. Certainly
testing AD on a Pentium II with 1MB of RAM and OpenLDAP on a dual
CPU 3.2 GHZ Xeon with 16GB of RAM wouldn't be a fair comparison.
Qualitative statements about the general performance of any set of
given Directory Servers can be made if you make everything "the
same" as much as is feasibly possible.
I do not see this being realistic. You have fallen short in this
regard with your Symas benchmarks. Yes, benchmark results can be
used as a general performance indicator, but one should not assume
that their ldap deployment will perform the same. We all have our
own requirements, different machines, schema, data set size,
indexes, etc., and more than likely they are different than what is
used in benchmarks.
-Igor
--Quanah
--
Quanah Gibson-Mount
Product Engineer
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by
OpenLDAP:
<http://www.symas.com>
--
Igor
