On Tue, Aug 04, 2009 at 06:58:26PM -0700, C. Titus Brown wrote: -> Next, I debugged the memory consumption & load-time problem that -> Namshin was experiencing (2 or more seconds on load!). There were two -> different problems: -> -> a) screed was loading the entire index of records into memory twice. -> This was basically braindead behavior that I fixed with some ifstream -> magic. -> -> b) screed is loading the entire index of records into memory. This -> behavior is basically a tradeoff between memory and speed: if you don't -> do this, you have to do two disk seeks in order to retrieve a record; -> if you DO preload the index, you only have to do one.
Sorry: you can toggle this behavior with PRELOAD_INDEX at the top of src/dbread_lib.cpp. With PRELOAD_INDEX set to 1, the record index will be preloaded when the database is opened, taking time & consuming (8 bytes * # sequences). With PRELOAD_INDEX set to 0, seeks will be performed on the record index file to retrieve the record offset for each record upon request. Memory use will be constant (and small), but the amount of disk access will be dramatically increased. cheers, -titus -- C. Titus Brown, [email protected] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pygr-dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pygr-dev?hl=en -~----------~----~----~----~------~----~------~--~---
