On 2/20/13, Jure Zitnik <[email protected]> wrote: > Hi, > :)
> On 2/20/13 9:58 AM, Olemis Lang wrote: >> Nonetheless the issue is definitely related to this . I've been >> tracking down this with PyDev and the source of all evil lays inside >> `MultiProductSystem.upgrade_environment` method and is quite bizarre . > [...] > > Thanks for pointing that out, that is in fact the actual source of the > problem. According to [1] and [2], sqlite creates temporary database on > the filesystem and keeps it there until the database connection is > closed. That happens despite the fact that the main database that we're > using is in-memory. > > r1448062 fixes the issue by not creating temporary tables but rather > tables in the main database when migrating the tables. > /me trying it ... will update spreadsheet afterwards as if this was yesterdays result ;) [...] >> >> It strikes to me that this statement is inside a loop but new fds >> allocated just once inside the loop , but a new pair once again after >> a new method call . That's the sustained +2 fd rate I mentioned before >> . > > The statement is in the loop as the loop is executed for all tables that > need to be migrated ;) > yes , that's pretty obvious ;) I was just trying to make my point in that this is related to a new method call and apparently another SQLite connection created . > >> @jure : even after fixing this I recommend to remove MP upgrade inside >> EnvironmentStub.__init__ > > The leaking fd issue is fixed. I toast to that ! > Multi product upgrade inside > EnvironmentStub.__init__ was added to make sure that the database is > always upgraded to multi-product schema as the SQL translator is > installed in any case. Not upgrading the schema would cause translated > SQLs to hit invalid schema. > I've been running MP test suite by removing that upgrade in __init__ and everything seems to be fine provided that upgrade will be performed in TC setUp method . Keeping it there might cause some trouble , especially considering the env override mechanism I've been using to convert test cases (i.e. env property with nop setter ;) . Notice that some env stub objects are deliberately ignored and dropped as a consequence . -- Regards, Olemis.
