HI everyone,
First, i would like to thank you for your comments.
Then, for those who run linux, the following works:
1- copy your db.sqlite to /dev/shm/db.sqlite (thus in ram fs)
2- play with it through sqlalchemy
3- backup it to disk when needed
src = apsw.Connection(ram_db_file)
dest = apsw.Connection(disk_db_file)
backup = dest.backup("main", src, "main")
try:
while not backup.done:
time.sleep(0.1) # let other processes access
the source db
backup.step(5) # number of pages (1 kbyte each)
finally:
backup.finish()
subprocess.call(["sync"])
HTH,
Thanks again,
Pierre
On Friday, August 30, 2013 6:22:06 PM UTC+2, Richard Gerd Kuesters wrote:
>
> Well, what Jeff wrote is also true.
>
> I do hot-copy of databases because we have a set of products that have
> full automated builds and, to increase performance, I made the build
> generate the SQLite database on memory and then dump it to the filesystem.
>
>
> Cheers,
> Richard.
>
> Hi,
>
>
>
> I'm currently running several python applications (each app using
> sqlalchemy) accessing (read/write) a single SQLite database stored on disk.
>
>
>
> For performance reasons, I would like to store this db file in RAM memory
> (ie, in my /dev/shm)
>
>
>
> The applications would then access a shared in-memory db through
> SQLAlchemy and a backup application would periodically make a hot copy of
> the in-memory db to disk. Then, on a [power off, power on] sequence, this
> backup app would copy the backed up db file from disk to RAM before
> launching the other apps.
>
>
>
> Before starting, I would like to know if you think it is feaseable. My
> questions are:
>
> 1- Has SQLALchemy an API to do a hot copy? (based on
> http://sqlite.org/backup.html for this particular db type)
>
> 2- If so, is this an actual hot copy, ie: the other apps will still run
> without waiting for the backup app to finish he backup?
>
> 3- Is there a particular configuration in SQLAlchemy that enables
> sharing an in-momory db from different apps (python processes)?
>
>
>
> Thanks a lot for your feedback,
>
>
>
> Pierre
>
> --
>
>
>
> Pierre,
>
> While I do think this is feasible, I would discourage going down this
> path unless you have a really good reason. It sounds like you need a real
> database engine like postgres here. It basically does everything you
> describe out of the box, is easy to set up, and will likely be more
> reliable than anything you could come up with on your own. It will also
> perform better when dealing with many simultaneous transactions.
>
>
>
> If you still run into performance problems you could look into introducing
> a caching layer such as memcached, but I wouldn’t cross that bridge until
> I had thoroughly tweaked my db settings and identified real bottlenecks
> in my application(s).
>
>
>
> Jeff Peck
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected]<javascript:>
> .
> Visit this group at http://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/groups/opt_out.
