On Feb 21, 2007, at 6:09 AM, Eugen Leitl wrote:
On Wed, Feb 21, 2007 at 08:59:51AM -0500, Mark Waser wrote:
do well. What are the special requirements/functionalities of the
indices
that you believe that enterprise DBs are not *optimized* to handle?
Relational databases are designed to do well at about anything,
sans further
tuning. Because of this relational databases perform very poorly
for selected
applications, many of them trivial, some complex, which are
typically much
better handled by custom code.
An example of where "enterprise" DBs fall flat onto their indices is
e.g. a multithreaded similiarity search on 5 million molecules.
There are tons of other examples.
This is a point that a great many people are obtuse to, and it is
valid far beyond databases. Relational databases in particular have
very narrowly optimized use cases; see the huge tradeoffs between row-
oriented versus column-oriented physical structure of the same data
for an example.
In short: The only index/search/retrieval patterns that are scalable
are those that are meaningfully reducible to a B-tree i.e. topologies
that are collapsible into a zero-dimensional object on a one-
dimensional line. By "scalable" I am referring to all three
algorithmic dimensions: time, space, and concurrency. Unfortunately,
there are many, many data types and spaces that collapse very poorly
(or not at all) into a usable B-tree analogue. This is an enormous
theoretical problem, and most people who work in software are
oblivious to its existence.
There is no scalable algorithm to represent e.g. spatio-temporal
data. Whoever devises such an algorithm stands to make a great deal
of money and/or fame, as it is *the* technological bottleneck for
ubiquitous geospatial and other applications. A related problem is
geospatial routing (a space I've been doing some important work in on
the side), which is semi-dependent on the solution to that problem.
Cheers,
J. Andrew Rogers
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
