Hello Everyone,
Thank you all for your really helpful comments. Using the batch
loading tool of SQLite (someone mentioned the batch loading tools of
Oracle and MySQL) we got a 720x speed improvement (from 30 days to 1
hour).
Definitely going to keep the fancy MySQL stuff (partitioned tables,
clustering, archive storage engine) in mind.
Tomorrow we will be looking at selects instead of inserts ;-)
Wish us luck and thank you all again!
ryan
On Jan 24, 2007, at 3:47 PM, A. Pagaltzis wrote:
* Lee Standen <[EMAIL PROTECTED]> [2007-01-24 08:50]:
Step 1:
Ditch SQLite - it's not optimized for large databases (to
the best of my knowledge)
I always hear people saying that, and none of them has ever
been able to substantiate it.
What SQLite isn’t designed for is a lot of concurrent access,
particularly write access. It copes with huge databases just
fine, however.
Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
_______________________________________________
List: http://lists.rawmode.org/cgi-bin/mailman/listinfo/dbix-class
Wiki: http://dbix-class.shadowcatsystems.co.uk/
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/trunk/DBIx-Class/
Searchable Archive: http://www.mail-archive.com/dbix-
[EMAIL PROTECTED]/
_______________________________________________
List: http://lists.rawmode.org/cgi-bin/mailman/listinfo/dbix-class
Wiki: http://dbix-class.shadowcatsystems.co.uk/
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/trunk/DBIx-Class/
Searchable Archive: http://www.mail-archive.com/[email protected]/
