Hi, I am new to metakit but what I have read about it, it seems to be a good database for what I want to do.
I want to use metakit mostly from python, but also from delphi (because I want to visualize things in delphi) on windows. I have been playing with sqlite, but it seems to have a real performance problem, when there are millions of rows in the database (and the database is not in the disk cache of the OS). In principle I want to model something like a file system with symbolic links. There are directories and files. Files have a 300-500 byte annotation. Most files are empty initially. And file content is generated over time. I have to deal with 1-10 million files of a total of 10Gb-100Gb. There are also sets of files (directories) that I want to efficiently access together. If those files are spread on the disk, the access disk time gets the limiting factor. If they are close to each other and I can access them in the order they are on disk, then the access is much faster compared to a random distribution on disk. Actually, files accessed together are usually created together. If I would use a auto-increment-integer-key-field, there is a high probability that I want to access files with keys that are within a small range. What I essentially need is: - a huge index (a few million keys) with md5 keys (preferably a hash table) - the index points to some data structure with meta info (meta_data) (about 10 attributes with 300-500 bytes per row) - most of those data structures point to a ~3kb blob field. - some of those data structures point to a big file like data structure (a blob with varying sizes 20kb-10MB) - some data structure to maintain tree and graph like structures Questions: - Can metakit deal with such data (if 50 GB is to big, I could move the blob like data into a flat file)? - Can I quickly sort on some columns of the meta_data? - How would you model a tree/graph like data structure in metakit? - Can I access item in the order they are on disk. Say: I give metakit 100 or 1000 keys, can it fetch the data efficiently (in disk order)? - Can I store binary data with metakit (null characters in strings)? - How stable is metakit (in case of power failure, can the database be corrupted so that it is unrecoverable)? I don't care if I loose the last (not committed) entries. Michael _______________________________________________ metakit mailing list - [EMAIL PROTECTED] http://www.equi4.com/mailman/listinfo/metakit
