Re: [sqlite] Loading database from memory buffer

2008-11-27 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Brown, Daniel wrote:
> What will happen if I don't implement the locking functionality does
> that mean that SQLite would no longer be thread safe?

It means that concurrent access (from different sqlite3 pointers in the
same or different processes) will trash the database contents since
there won't be locking of the database itself.

If you need different sqlite3 pointers then you can use proxy locking.
Effectively this does the actual locking operations on a different file
than the database.  http://www.sqlite.org/cvstrac/wiki?p=ProxyLocking

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkkvE48ACgkQmOOfHg372QSX9ACgziGqbK4sUBBTTD19M4Xt+3pQ
0OYAoIzQN4IEZS4c03ZX8DKFpmi/vzR2
=W6co
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Loading database from memory buffer

2008-11-27 Thread Brown, Daniel
What will happen if I don't implement the locking functionality does
that mean that SQLite would no longer be thread safe?

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Roger Binns
Sent: Thursday, November 27, 2008 12:05 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Loading database from memory buffer

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Brown, Daniel wrote:
> However writing a new VFS would
> seem to be quite an time consuming solution, 

If you don't have to worry about implementing the locking functionality
(ie multi-threaded/process access) then doing your own VFS is very easy.
   You can just reuse most of the methods from the default VFS.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkku/V0ACgkQmOOfHg372QSQ0QCfeYqlQwCtfCQh2fvik9Qnq0yr
AeAAn07M/mos+yHS3QkSvaWzRCKjWuhj
=4OfS
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Loading database from memory buffer

2008-11-27 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Brown, Daniel wrote:
> However writing a new VFS would
> seem to be quite an time consuming solution, 

If you don't have to worry about implementing the locking functionality
(ie multi-threaded/process access) then doing your own VFS is very easy.
   You can just reuse most of the methods from the default VFS.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkku/V0ACgkQmOOfHg372QSQ0QCfeYqlQwCtfCQh2fvik9Qnq0yr
AeAAn07M/mos+yHS3QkSvaWzRCKjWuhj
=4OfS
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] Loading database from memory buffer

2008-11-26 Thread Brown, Daniel
Good afternoon list,

Is there any way to open a SQLite databases out of an in memory buffer
rather than an operating system file (via a VFS)?

The runtime environment of one of our platforms does not have local
storage that can be accessed via a normal operating system VFS but via
an interface that accesses very slow secure storage.  This interface
only supports passing files in and out via memory buffers and is
hideously slow.  On our other platforms we load our database into an
completely memory resident database (:memory:) from an database file on
read only storage which we attach to during start up and then detach
once we've finished copying our data into memory.  This approach has
worked quite well so far, as it avoids SQLite attempting to access
storage (which is read only) except during set-up.  So we would quite
like to keep using this approach if possible but are unsure about how to
solve this new problem with working with the buffers coming from or
going into secure storage.

One solution would seem to be to write a new VFS that handles this
reading and writing of memory buffers.  However writing a new VFS would
seem to be quite an time consuming solution, especially as current
solution of using :memory: databases seems to be working except for this
one issue.  It would seem the easier solution would be to copy the data
out of the buffer and into SQLite's memory pages when setting up and to
copy the data from SQLite to the buffer during shutdown but I am unsure
if this functionality already exists or if it does not exist how to
begin implementing it.

Any advice on potential solutions would be most welcome!

Cheers,

Daniel Brown | Software Engineer
"The best laid schemes o' mice an' men, gang aft agley"


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users