Hi all,
I don't want to drive anyone away from SQLite (don't think that I can anyway
:-)) but a good solution for storing large amounts of data is HDF5.
HTH
-- ds
2007/11/15, Roger Binns <[EMAIL PROTECTED]>:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Asif Lodhi wrote:
> >
Hello John,
this is extremely helpful. Thanks a lot!!!
Dimitris
In the sense that the legacy code produces files ~100MB. The collection is
not legacy, that's what I am trying to setup. Unless I don't understand what
you mean
2007/3/19, guenther <[EMAIL PROTECTED]>:
On Sun, 2007-03-18 at 23:51 +0200, Dimitris Servis wrote:
> in my wilde
Hello Guenther,
in my wildest dreams... if you read carefully, *each* file is about
100-200MB. I now end up wit ha collection of 100-200 of them and need to
bundle in one file
BR
dimitris
2007/3/18, guenther <[EMAIL PROTECTED]>:
Well, actually I did not mean to post at this stage but
Hello John,
You do not have to load the entire file into memory. The best way is to
memory map it and use the returned pointer to copy it into the RDBMS.
You can retrieve it to a file in a similar way. It helps if you store
the file size in the DB so that you can create a file the correct
Hello Daniel,
Personally I think that files should be save like files on the filesystem.
Personally I think that each tool should be used for the purpose it has been
created, just to generalize what you said above. Nevertheless, there are
situations like mine, where you need the good old
Hello Eduardo,
this is one of the alternatives, for sure. It would bundle many files into
one very effectively, and even without compression, you have a filesystem.
However, my real problem is that I don't want to develop software for
handling file access, locking, concurrency etc myself. What
Hello John,
thanks for the valuable piece of advice. The idea is that either
1) I store data in tabular form and work with them
2) I create a table of blobs and each blob is the binary content of a file
(2) is my method in question, for (1) we all know it works. So I turned to
SQLite just
Hello Martin,
If it was me I'd "investigate" the problem by doing the "right" thing in
the first place, by which time I'd know enough to knock up the "wrong"
solution for the doubters before presenting the "proper" solution as a
fait accompli.
That's already been done. It is more or less
That's not a bad idea at all and I'll check it out. However, since the data
is written from a client, I can only do arbitrary chopping without
separating them in a sensible manner. Maybe I don't need it though, as I
could use it for setting up a pageing system in memory. Thanks!!!
2007/3/18, Teg
Hello Richard,
I have to admit you're right. Probably not any DBMS is the right tool for
that... However I have to prove it can be done, though I of course favor a
relational tables solution.
thanks a lot
-- dimitris
2007/3/18, [EMAIL PROTECTED] <[EMAIL PROTECTED]>:
"Dimitris P. Servis"
11 matches
Mail list logo
