On Sat, Feb 11, 2012 at 22:06, Emmanuel Tabard <m...@webitup.fr> wrote: > > Looks nice !!!
Well, it's not exactly nice & clean code... let's hope it works. > It seems that you load all the datas in memory before storing it in the temp > databases. > "cls.select(ISNOTNULL(cls.q.imdbID))" > > Maybe you should save the imdbids by batch of 10000entries ? If you believe at the theory, the return value of a 'select' method is a lazy object, and so it should not consume much memory. It's true that it's then cycled over, one item at a time, and so it may be slow (in which case, we can directly use a cursors and process the results in batches). > Tell me if you need the complete database dump to test with tons of datas ! It won't hurt... :) A dump of "SELECT md5sum, imdb_id" from the 'name', 'title', 'char_name' and 'company_name' will be enough (not on the public list, please :)) Bye, -- Davide Alberani <davide.alber...@gmail.com> [PGP KeyID: 0x465BFD47] http://www.mimante.net/ ------------------------------------------------------------------------------ Virtualization & Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ _______________________________________________ Imdbpy-devel mailing list Imdbpy-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-devel
