Re: [sqlite] Reg: In Memory Database Using SQLite

[ca44]

Yes, please do send it to me. 

Thx 

-C 


- Original Message - 
From: "Subhadeep Ghosh" <subhadeep.gh...@uniken.com> 
To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org> 
Sent: Saturday, July 10, 2010 7:42:03 AM 
Subject: Re: [sqlite] Reg: In Memory Database Using SQLite 

Hi Chris, 

I had attached the source code but it mysteriously vanished. I'm guessing 
that this mailing list does not support mail attachments. However, if you 
want I could mail it to you on your personal mail ID. 

Thank you and regards, 
Subhadeep. 

On Sat, Jul 10, 2010 at 8:09 PM, <c...@comcast.net> wrote: 

> 
> 
> Hi Subhadeep, 
> 
> I'm not seeing the source code included anywhere in your email. 
> 
> 
> 
> Thx 
> 
> -Chris 
> 
> 
> - Original Message - 
> From: "Subhadeep Ghosh" <subhadeep.gh...@uniken.com> 
> To: sqlite-users@sqlite.org 
> Sent: Wednesday, July 7, 2010 1:51:28 PM 
> Subject: [sqlite] Reg: In Memory Database Using SQLite 
> 
> Hello People, 
> 
> I finally managed to create a wrapper around the SQLite core to support the 
> creation of in-memory databases. The wrapper comprises of three functions - 
> one to serialize the database, one to de-serialize a database and the third 
> one to do the cleanup job. 
> 
> The function which serializes a database, copies the entire contents of the 
> database to a block of memory. The function which de-serializes takes a 
> block of memory as input and creates a database object out of it. 
> 
> I am including the source code for the interested souls and I've not put 
> comments as I hate doing it. 
> 
> I hope you guys find the code useful or at-least interesting. 
> 
> Thank you and regards, 
> Subhadeep Ghosh. 
> 
> ___ 
> sqlite-users mailing list 
> sqlite-users@sqlite.org 
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 
> ___ 
> sqlite-users mailing list 
> sqlite-users@sqlite.org 
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 
> 
___ 
sqlite-users mailing list 
sqlite-users@sqlite.org 
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[Subhadeep Ghosh]

Hi Chris,

I had attached the source code but it mysteriously vanished. I'm guessing
that this mailing list does not support mail attachments. However, if you
want I could mail it to you on your personal mail ID.

Thank you and regards,
Subhadeep.

On Sat, Jul 10, 2010 at 8:09 PM,  wrote:

>
>
> Hi Subhadeep,
>
> I'm not seeing the source code included anywhere in your email.
>
>
>
> Thx
>
> -Chris
>
>
> - Original Message -
> From: "Subhadeep Ghosh" 
> To: sqlite-users@sqlite.org
> Sent: Wednesday, July 7, 2010 1:51:28 PM
> Subject: [sqlite] Reg: In Memory Database Using SQLite
>
> Hello People,
>
> I finally managed to create a wrapper around the SQLite core to support the
> creation of in-memory databases. The wrapper comprises of three functions -
> one to serialize the database, one to de-serialize a database and the third
> one to do the cleanup job.
>
> The function which serializes a database, copies the entire contents of the
> database to a block of memory. The function which de-serializes takes a
> block of memory as input and creates a database object out of it.
>
> I am including the source code for the interested souls and I've not put
> comments as I hate doing it.
>
> I hope you guys find the code useful or at-least interesting.
>
> Thank you and regards,
> Subhadeep Ghosh.
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[ca44]

Hi Subhadeep, 

I'm not seeing the source code included anywhere in your email. 



Thx 

-Chris 


- Original Message - 
From: "Subhadeep Ghosh"  
To: sqlite-users@sqlite.org 
Sent: Wednesday, July 7, 2010 1:51:28 PM 
Subject: [sqlite] Reg: In Memory Database Using SQLite 

Hello People, 

I finally managed to create a wrapper around the SQLite core to support the 
creation of in-memory databases. The wrapper comprises of three functions - 
one to serialize the database, one to de-serialize a database and the third 
one to do the cleanup job. 

The function which serializes a database, copies the entire contents of the 
database to a block of memory. The function which de-serializes takes a 
block of memory as input and creates a database object out of it. 

I am including the source code for the interested souls and I've not put 
comments as I hate doing it. 

I hope you guys find the code useful or at-least interesting. 

Thank you and regards, 
Subhadeep Ghosh. 

___ 
sqlite-users mailing list 
sqlite-users@sqlite.org 
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[Eric Smith]

Subhadeep Ghosh wrote: 

> I am aware of the functionality which you mentioned.  But the reason why 
> I ended up coding the feature was because of a very specific requirement.  
> My requirement was such that some of my client applications were running 
> on disk-less systems where I was not using any kind of network storage.  
> But my applications used SQLite.  So I coded the below mentioned feature 
> such that the database was being stored on the server and when the 
> disk-less clients wanted to use them, they fetched the database from the 
> server over the network and directly create a database out of the fetched 
> data (basically serializing and de-serializing the database).  
> 
> I am guessing that SQLite does not support such functionality.  The 
> closest thing would have been to fetch the database over the network, 
> store it in a file (which in this case is not possible) and then open the 
> database.  
> 
> I maybe wrong, but if such a feature already exists then I would be more 
> than happy to adopt it in my code.  

I see.  

Firstly, you may be better off using a client/server db, since that 
seems to map more naturally into your use-case.  But let's forget about 
that for the moment.  

Had I written your application, I first would have considered dumping 
the original database to raw SQL text and reading it into the in-mem 
db on the other end.  That would be pretty trivial to do, and if it 
suits your needs perf-wise, you're golden.  This also gives the extra 
advantage of having a textual communication format between the master 
and the slave, which can be a useful debugging tool.  

I would then have considered using a RAM-backed filesystem on the 
slave.  You can copy the sqlite db as-is to the slave and open it 
using normal sqlite api calls.  SQLite and your app think the db is 
disk-backed because the OS is faking the existence of a disk.  Whether 
this option works well depends on how easy it is to get a ram-backed fs 
up and running on your slave.  (In linux this is very easy, not sure 
about your deployment oS.)  

Eric 

-- 
Eric A. Smith

The people can always be brought to the bidding of the leaders. That 
is easy. All you have to do is tell them they are being attacked and 
denounce the pacifists for lack of patriotism and exposing the 
country to danger. It works the same way in any country. 
-- Herman Goering, at the Nuremberg trials
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[Subhadeep Ghosh]

Hello Eric,

I am aware of the functionality which you mentioned. But the reason why I
ended up coding the feature was because of a very specific requirement. My
requirement was such that some of my client applications were running on
disk-less systems where I was not using any kind of network storage. But my
applications used SQLite. So I coded the below mentioned feature such that
the database was being stored on the server and when the disk-less clients
wanted to use them, they fetched the database from the server over the
network and directly create a database out of the fetched data (basically
serializing and de-serializing the database).

I am guessing that SQLite does not support such functionality. The closest
thing would have been to fetch the database over the network, store it in a
file (which in this case is not possible) and then open the database.

I maybe wrong, but if such a feature already exists then I would be more
than happy to adopt it in my code.

Thank you and regards,
Subhadeep Ghosh.

On Thu, Jul 8, 2010 at 6:40 PM, Eric Smith  wrote:

> Subhadeep Ghosh wrote:
>
> > I finally managed to create a wrapper around the SQLite core to support
> > the creation of in-memory databases.  The wrapper comprises of three
> > functions - one to serialize the database, one to de-serialize a database
> > and the third one to do the cleanup job.
> >
> > The function which serializes a database, copies the entire contents of
> > the database to a block of memory.  The function which de-serializes
> takes
> > a block of memory as input and creates a database object out of it.
> >
> > I am including the source code for the interested souls and I've not put
> > comments as I hate doing it.
> >
> > I hope you guys find the code useful or at-least interesting.
>
> I hope you didn't spend too much time on that.  SQLite supports
> in-memory databases natively: if you open the file called ":memory:" you
> get an in-memory database that dies when the connection is closed:
>
> http://sqlite.org/c3ref/open.html
>
> --
> Eric A. Smith
>
> More people would come here if it weren't so crowded.
>-- Yogi Berra
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[Eric Smith]

Subhadeep Ghosh wrote: 

> I finally managed to create a wrapper around the SQLite core to support 
> the creation of in-memory databases.  The wrapper comprises of three 
> functions - one to serialize the database, one to de-serialize a database 
> and the third one to do the cleanup job.  
> 
> The function which serializes a database, copies the entire contents of 
> the database to a block of memory.  The function which de-serializes takes 
> a block of memory as input and creates a database object out of it.  
> 
> I am including the source code for the interested souls and I've not put 
> comments as I hate doing it.  
> 
> I hope you guys find the code useful or at-least interesting.  

I hope you didn't spend too much time on that.  SQLite supports 
in-memory databases natively: if you open the file called ":memory:" you
get an in-memory database that dies when the connection is closed:

http://sqlite.org/c3ref/open.html

-- 
Eric A. Smith

More people would come here if it weren't so crowded.
-- Yogi Berra
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reg: In Memory Database Using SQLite

[Jean-Christophe Deschamps]

>I'm aware that SQLite offers the functionality of in-memory databases, 
>but I
>don't know how to read the data and send it over the wire to the server or
>how to push the data into the in-memory database of the server.

The backup API included in SQLite offers the facility to backup/restore 
(in fact = copy) disk-based or memory databases to/from disk or 
memory.  So you can launch a backup operation all in one shunk to 
create a memory DB, work on it and then backup it back to the 
server.  I use this everyday and it works like a charm.



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users