>> There have been some comments from people who cannot imagine that a
>> large database might be valid. That simply shows a lack of
>> imagination. Here are examples.
>
>Looking back at earlier parts of this conversation, I see no-one
>contradicting the idea of large databases. CAD/CAM databases here are
>already in the 100's of Mb, the multi Gb database is not far away.
True enough.
>Some people have commented that it is not necessary to put an entire
>such dataset in one *FILE*. To suggest otherwise "simply shows a lack
>of imagination". The alternatives are to split the dataset over
>multiple files, or use a raw device.
Well, sort of. The raw datafiles are oftten best handled as one single
file. ("best" from users point of view). Being forced to split
the file into 10-1000 pieces for purpose of transport or casual
use is really, really annoying, and error prone.
I guess I am referring to an intermediate level of use where it
isn't worth it (or isn't desireable) to put the original raw data
into a database (whatever the implementation). Examples are
collecting the datasets off of tapes, and reorganizing them to
make logical sense, transporting the data from one site to another,
where the raw data is of interest, not the database version, etc.
In the case of Census data, it is a royal pain in the back side to
split the census data 50 ways (by state), or 10 ways to fit the
typical AFS volumes (200 meg) used here. This being only one
single dataset. With dozens of others of probable common interest,
the many thousands of pieces rapidly becomes unmanageable.
While users certainly can work their way around it, that misses
the point. The point is that the datasets are one unified lump
of stuff, and it is getting to the point that even single disk
drives can hold them. Why not put them in Unix filesystems?
There are a lot of possible solutions, obviously. But 2GB limits
on filesystems are going to cause problems, pretty much regardless
of what the "right" or "best" solutions are.
I can't speculate as to whether or not the user community size
that would benefit from such enhancements would justify many
changes to what is essentially a mainstream oriented product.
>Peter Lister [EMAIL PROTECTED]
>Computer Centre,
Paul Anderson
[EMAIL PROTECTED]
