-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
... > > The profile showed 16 seconds (of 25 up to the kill-point) going into > storm object deserialisation. 30000 objects. (Thats 3000 rows, 10 > objects per row). > One possibility comes to mind here. Deserialisation tends to be the time when you are creating a lot of python objects, correct? Creating lots of objects without getting rid of them triggers GC runs. GC runs tend to depend on the total size of all objects in the VM, not restricted to just your current thread, etc. (In theory, the generational GC only depends on 'new enough' objects, but once you trigger enough level 1 runs, you do get a level 2 run, etc.) 30k objects isn't all that many, but depending on the objects, they may be multi-layered. (A plain Python instance-of-a-class is the object + its dict). This was only really noticeable in bzr when we had >1M tuple objects around, and I don't know the internals of launchpad servers. Note also that python2.7 is supposed to help with the quadratic tuple + dict behavior. Once GC runs, if it sees that a given tuple only points at non-gc objects (strings, or other 'fixed' tuples) then it removes the tuple (or dict) from GC. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkyGX8wACgkQJdeBCYSNAAO0FgCgpLgE5UcKIf2QYkhsjcJL8k5l /tQAoJ/xfk0Xsoin8Zeoe90BdHI6YTTT =3EdE -----END PGP SIGNATURE----- _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
