(this is a reply to Ani's comments. Ani's comment was not forwarded to my indox, and I do not know how to reply to an existing mail-man thread, without knowing the message-id...)
Ani, some factual info below: > it is hard to say how valid this comparison is given the different > hardware and the fact that casjobs is an end-user facility so the > query was competing with other jobs on the server at the time. it is > a heavily used system. The quoted run time was reproducable at various times. There were no other jobs in the queue. > also it uses a distributed execution where the results are written to > a non-local disk (the MyDB is on a different server than the one on > which the query is run). > > this is always the problem with benchmarks done on different hardware > and software systems. in maria's original maxbcg comparison with TAM, > she included a factor to account for the different CPUs but of course > that was just a best guess as to how much difference it made. I would have liked to include a rough hardware correction factor, but I did not have any knowledge of the CAS system's hardware. I pointed that out in my text. > besides, comparing an ad-hoc optimized file-based implementation is > not quite fair - you dont really want to develop such customized > solutions for every query or problem that comes your way. and this is > indeed the key reason why you would want to use a DBMS - so that you > *dont* have to write a customized C++ solution to every problem. There is a misconception here: The C++ implementation did not require more lines of code than the SQL implementation (a few less indeed). Not using an DBMS system does not mean writing everything from scratch. In fact the ROOT system provided me with everything I needed for this given task (but lacks in many other respects, see my text). So I just implemented the same algorithm in a different language. There were no specific optimizations involved concerning the database structure, algorithm, or anything else. Both implementation do exactly the same thing. > in addition to all the other goodies like a SQL query interface, data > organization and integrity, indexing, replication tools, etc., the > point is that a good DBMS will give you decent query perforamce across > the board. > > in the end, it is the performance on a typical LSST science query > workload combined with the other factors such as price, support, ease > of use, data ingest performance, and indexing options that will > determine the choice of DBMS, not comparison to file-based > implementations. Comparisons to file based approaches do give us an idea of how well or how bad a DBMS system is doing with respect to what we really need it to do. I think that is a valuable thing to know. And as an aside: It was not me who started comparing SQL server to a file based approach in the first place... > if we're agreed that a file-based approach does not begin to provide > the features that we desire for our archive, then let's not waste time > comparing dbms performance to file-based implementations. cheers. As always waste of time is a matter of opinion. Do you know the guy who set out to visit all Starbucks branches in the world? Now that is a real waste of time in my opinion! :-) Happy thanksgiving, - Leif. _______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
