it is hard to say how valid this comparison is given the different
hardware and the fact that casjobs is an end-user facility so the query
was competing with other jobs on the server at the time. it is a
heavily used system. also it uses a distributed execution where the
results are written to a non-local disk (the MyDB is on a different
server than the one on which the query is run).
this is always the problem with benchmarks done on different hardware
and software systems. in maria's original maxbcg comparison with TAM,
she included a factor to account for the different CPUs but of course
that was just a best guess as to how much difference it made.
besides, comparing an ad-hoc optimized file-based implementation is not
quite fair - you dont really want to develop such customized solutions
for every query or problem that comes your way. and this is indeed the
key reason why you would want to use a DBMS - so that you *dont* have to
write a customized C++ solution to every problem.
in addition to all the other goodies like a SQL query interface, data
organization and integrity, indexing, replication tools, etc., the point
is that a good DBMS will give you decent query perforamce across the
board. sure, you can always find a customized file-based solution to
beat it, but that defeats the purpose of the DBMS, which is to save you
all that trouble and provide a versatile data mining and discovery tool
for a large dataset serving a large and diverse user community. believe
me, you will have enough to do without getting into this kind of
development!
in the end, it is the performance on a typical LSST science query
workload combined with the other factors such as price, support, ease of
use, data ingest performance, and indexing options that will determine
the choice of DBMS, not comparison to file-based implementations.
if we're agreed that a file-based approach does not begin to provide the
features that we desire for our archive, then let's not waste time
comparing dbms performance to file-based implementations. cheers.
ani
On Tue, 21 Nov 2006, Leif Hendrik Wilden wrote:
> Jacek wrote:
>
> > Thanks, but I am not sure if we want to put any energy into evaluating
> > ROOT at this point. For a variety of reasons LSST would MUCH prefer to
> > use a real database. We don't even have sufficient resources to look
> > into major dbms players like DB2 and/or Oracle (even though they are
> > very much interested in helping us)
>
> Hi Jacek,
>
> I do not think that ROOT can do the job for LSST and I was certainly not
> suggesting that it should be used. ROOT completely lacks all the
> features LSST will need to dynamically maintain the database at the base
> center. There may be some use cases for ROOT in different areas - but
> this needs a lot more thought and discussion.
>
> My intention was to start thinking of how to evaulate the performance of
> a "real database". With the results from the benchmark I can now roughly
> estimate how well the SDSS SQL server implementation actually works. It
> works better than I thought considering experiences with the MS SQL
> server I had a few years ago, but it is still significantly slower than
> a well implemented file based approach.
>
> Sorry if that did not become clear.
>
> - Leif.
> _______________________________________________
> LSST-data mailing list
> [email protected]
> http://www.lsstmail.org/mailman/listinfo/lsst-data
>
--
Aniruddha R. Thakar, Research Scientist
Center for Astrophysical Sciences, JHU, Bloomberg 375
3701 San Martin Drive, Baltimore MD 21218-2695
410-516-4850, Fax: 410-516-5096
[EMAIL PROTECTED], http://www.sdss.jhu.edu/~thakar
-----------------------------------------------------------------------
Science, like virtue, is its own reward, and its only reward!
[J.L.Wallace]
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
