Cursor arraysize is the number of rows that psycopg2 fetches each time fetchmany() is called. Storm's Result.__iter__() rests upon that, and ResultSet rests upon that, so arraysize ought to bear on performance.
check-teamparticipation.py now slurps lots from the database, and one thing I tweaked to make it quicker is to hack the cursor arraysize to be much larger. I didn't take any measurements specifically about this because I was iterating other things too, but it appeared to have made slurping quicker. I'm kicking myself for not having taken proper isolated measurements, but they wouldn't be too hard to do now. Perhaps unbelievably, psycopg2 sets arraysize to 1. Storm attempts to salvage things by setting it to 10 (unless it has already been changed to something other than 1). I wonder if we could help Launchpad by setting it to, say, 100, or even just 20 as a first step? Unless someone can say it's pointless I'll try and get some measurements about the performance impact. Gavin. _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
