Roger Binns <rog...@rogerbinns.com> writes:
> On 04/10/2010 03:06 PM, Nikolaus Rath wrote:
>> However, I noticed that if I dump the entire database into a text file
>> with the SQLite shell and then compress the text file, the result is
>> significantly smaller than the "stripped" compressed database:
>
> Have you tried different page sizes?
Yes, but the effect isn't very big:
Page Size: DB: Compressed DB:
512 152 MB 21 MB
1024 147 MB 19 MB
4096 143 MB 18 MB
10240 147 MB 19 MB
> You could also do the transmission using rsync with compression which may
> turn out to transfer even less.
>
> It may also be worthwhile looking to other compression mechanisms. For
> example smaz works well on English text because the compression dictionary
> is prebuilt rather than generated for the specific data. A prebuilt
> dictionary type mechanism may work well for you. You may also find a PPM
> compressor even better.
Using rsync is unfortunately not possible, I'm stuck with HTTP and FTP.
I tried a few PPM compressors, but even though the compression ratio is
amazing, I'm not desperate enough to invest *that* much CPU time yet.
I may take at smaz though, thanks for the pointer.
>> Alternatively, is there an easy way to dump and recover the DB using the
>> standard API rather than the SQLite shell? (Obviously I could write a
>> dump program myself, but I'd be nice if there is a solution that
>> requires less work).
>
> You are using Python and APSW IIRC. APSW includes a Python based shell
> that has dumping and restore code and can be used programmatically or
> interactively.
Yes, you remember correctly. Actually that would be a perfect solution.
But how do I use it? It seems to me that I need to pass some argument to
Shell.command_dump(), because the following just produces an empty file:
import apsw
ofh = open('dump.txt', 'w')
db = apsw.Connection('test.db')
s = apsw.Shell(db=db, stdout=ofh)
s.command_dump('.dump')
Thanks,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
