On Mon, 2007-03-19 at 01:46 +0200, Dimitris Servis wrote:
> 2007/3/19, guenther <[EMAIL PROTECTED]>:
> > On Sun, 2007-03-18 at 23:51 +0200, Dimitris Servis wrote:
> > > in my wildest dreams... if you read carefully, *each* file is about
> > > 100-200MB. I now end up wit ha collection of 100-200
Hello John,
this is extremely helpful. Thanks a lot!!!
Dimitris
In the sense that the legacy code produces files ~100MB. The collection is
not legacy, that's what I am trying to setup. Unless I don't understand what
you mean
2007/3/19, guenther <[EMAIL PROTECTED]>:
On Sun, 2007-03-18 at 23:51 +0200, Dimitris Servis wrote:
> in my wildest dreams... if
On Sun, 2007-03-18 at 23:51 +0200, Dimitris Servis wrote:
> in my wildest dreams... if you read carefully, *each* file is about
> 100-200MB. I now end up wit ha collection of 100-200 of them and need to
> bundle in one file
Yes, I did read carefully. 100 (source) files, each 100 MByte,
Hello Guenther,
in my wildest dreams... if you read carefully, *each* file is about
100-200MB. I now end up wit ha collection of 100-200 of them and need to
bundle in one file
BR
dimitris
2007/3/18, guenther <[EMAIL PROTECTED]>:
Well, actually I did not mean to post at this stage but
Well, actually I did not mean to post at this stage but resort to
lurking and learning, since I am still doing some rather basic
experimenting with SQLite. Anyway, I followed this thread and it strikes
me as a crack idea. But aren't these the most fun to hack on? ;)
On Sun, 2007-03-18 at 01:06
Dimitris Servis wrote:
Hello John,
You do not have to load the entire file into memory. The best way is to
memory map it and use the returned pointer to copy it into the RDBMS.
You can retrieve it to a file in a similar way. It helps if you store
the file size in the DB so that you can
Hello John,
You do not have to load the entire file into memory. The best way is to
memory map it and use the returned pointer to copy it into the RDBMS.
You can retrieve it to a file in a similar way. It helps if you store
the file size in the DB so that you can create a file the correct
I agree! My statement was meant "in general", hence my proposal of the
sqlite3_bind_file-functions that I think would be a nice feature in SQLite.
Dimitris Servis wrote:
Hello Daniel,
Personally I think that files should be save like files on the
filesystem.
Personally I think that each
You do not have to load the entire file into memory. The best way is to
memory map it and use the returned pointer to copy it into the RDBMS.
You can retrieve it to a file in a similar way. It helps if you store
the file size in the DB so that you can create a file the correct size
to act as
Your BLOBs are big for rapid access as linked stored pages in an RDBMS.
Individual files give processing advantages but have the downside of
not being just one file like Sqlite.
Design an experiment. You will find the balance between simplicity and
speed which suits your application.
Hello Daniel,
Personally I think that files should be save like files on the filesystem.
Personally I think that each tool should be used for the purpose it has been
created, just to generalize what you said above. Nevertheless, there are
situations like mine, where you need the good old
Hello Eduardo,
this is one of the alternatives, for sure. It would bundle many files into
one very effectively, and even without compression, you have a filesystem.
However, my real problem is that I don't want to develop software for
handling file access, locking, concurrency etc myself. What
The questions about saving files in a database is a reoccurring subject
of this mailing list.
How about adding a feature request for something like a
sqlite3_bind_file() to load a file into a statement and maybe a
sqlite3_column_savefile(). I guess this could solve some things like not
loading
At 19:00 18/03/2007, you wrote:
Hello John,
thanks for the valuable piece of advice. The idea is that either
1) I store data in tabular form and work with them
2) I create a table of blobs and each blob is the binary content of a file
(2) is my method in question, for (1) we all know it
Hello John,
thanks for the valuable piece of advice. The idea is that either
1) I store data in tabular form and work with them
2) I create a table of blobs and each blob is the binary content of a file
(2) is my method in question, for (1) we all know it works. So I turned to
SQLite just
A word of warning if you use the traditional method, an RDBMS table with
descriptive data and a reference to the name of the file storing the
binary data. If you store a lot of files in a directory you can get
into trouble. A robust design uses some form of tree structure of
directories to
Hello Martin,
If it was me I'd "investigate" the problem by doing the "right" thing in
the first place, by which time I'd know enough to knock up the "wrong"
solution for the doubters before presenting the "proper" solution as a
fait accompli.
That's already been done. It is more or less
That's not a bad idea at all and I'll check it out. However, since the data
is written from a client, I can only do arbitrary chopping without
separating them in a sensible manner. Maybe I don't need it though, as I
could use it for setting up a pageing system in memory. Thanks!!!
2007/3/18, Teg
Dimitris P. Servis wrote:
I have to provide evidence that such an anorthodox solution is also
feasible
If it was me I'd "investigate" the problem by doing the "right" thing in
the first place, by which time I'd know enough to knock up the "wrong"
solution for the doubters before presenting
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Dimitris P. Servis wrote:
> 2) The file would be portable and movable (i.e. copy-paste will do, no
> special arrangement to move around)
You can do it if you drop that requirement and make it a single
directory. The easiest way of dealing with large
Hello Dimitris,
If I was going to do this, I'd chop up the binary file in some
manageable length like, say 1 meg, insert each chunk as a blob,including
an index record for each chunk so, you can select them in order and
re-assemble them piece by piece as you enumerate through the records.
In that
as far as we all know, it can be done, in that, yes, blobs can be
stored in SQLite.
As you yourself have realized, and as the creator of SQLite has
confirmed, SQLite is not the right tool for that. That said, why don't
you just do it and see what problem you encounter. That itself might
be valid
Hello Richard,
I have to admit you're right. Probably not any DBMS is the right tool for
that... However I have to prove it can be done, though I of course favor a
relational tables solution.
thanks a lot
-- dimitris
2007/3/18, [EMAIL PROTECTED] <[EMAIL PROTECTED]>:
"Dimitris P. Servis"
"Dimitris P. Servis" <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I want to do the following; save a set of 100-200 (or even more) binary
> files into a single DB file. The binary files are of 100-200 (or a bit
> more :-) ) MB each.
SQLite is not really the right tool for that.
--
D. Richard Hipp
Hi all,
I want to do the following; save a set of 100-200 (or even more) binary
files into a single DB file. The binary files are of 100-200 (or a bit
more :-) ) MB each. My requirements are:
1) Many clients should be able to connect to the database to save their
file. Clients are actually
26 matches
Mail list logo
