On Oct 16, Elefterios Stamatogiannakis <[EMAIL PROTECTED]> wrote:
> By putting transactions (using CURS.execute('BEGIN TRANSACTION;')
> and CURS.execute('COMMIT;') ) around db inserts made by imdbpy2sql,
> i have managed to lower the time needed to import the IMDB into
> sqlite v3.4.2, from 35 hours to 48 minutes (+ the time needed to
> build the indexes).
As said, I've used the recently introduced support for custom SQL
queries to speed-up SQLite; as you say its performaces are now
_amazing_, at least to insert the data: building the indexes
appears to be somewhat slow.
In the CVS there's a version of the imdbpy2sql.py script with the
"--sqlite-transactions" command line option, to do exactly what
you're suggesting.
> I expect that the transactions will speedup all database engines
> supported by SQLobjects .
Unfortunately, as supposed by Jesper, other high-end databases are
unaffected by this change (and I won't use it by default, since -
for instance - MySQL with MyISAM tables doesn't support transactions).
I'm not even sure it depends on SQLite: I suspect this behavior
derives from the Python bindings for SQLite (but I may be wrong).
Thank you very much, I've added your name to the CREDITS!
--
Davide Alberani <[EMAIL PROTECTED]> [PGP KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Imdbpy-devel mailing list
Imdbpy-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-devel
