On Tue, 1 Sep 2009 18:59:35 +0900 Namshin Kim <[email protected]> wrote:
> I ran the megatest with checkFirstID=False option, and it still uses > almost 2.5GB memory. Maybe it would be the minimum memory > requirements. This was sort of expected, as looking at the code of the megatest in questions shows all keys are iterated over even if the AnnotationDB constructor does not (which is what checkFirstID does) - so it is quite likely this is indeed simply how much memory performing this particular operation needs. This is another situation when it would be useful to be able to profile in detail Python's memory usage. Do you think you could probe into this using Heapy, Namshin? MySQLdb uses an extension library so it won't be a one hundred percent accurate measurement (as I mentioned a few weeks ago in a dedicated thread, extension code needs to be specially modified for Heapy to correctly measure its memory consumption) but maybe we'll be lucky and the source of the problem lies in the Python bits. It would also be good to have this tested against a different SQL-based back-end (preferably one as similar to MySQLdb in its general mode of operation as possible, one for Postgres for instance - I wonder how difficult it would be to add support for it to Pygr...). Fortunately we're going to implement SQLite support pretty soon (i.e. within the next two months) so we may have just the thing. -- MS --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pygr-dev" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pygr-dev?hl=en -~----------~----~----~----~------~----~------~--~---
