| zhuyifei1999 added a comment. |
...which is not surprising:
>>> len(bot.history.historyDict.keys()) 119193 >>> bot.history.historyDict.keys()[:10] [u'http://www.pref.akita.lg.jp/www/toppage/0000000000000/APM03000.html', u'http://www.worldwarships.com/class/samar-0', u'http://www.fphil.uniba.sk/index.php?id=3294', u'http://www.hokej.cz/index.php?lng=CZ&view=clanek&id=52180', u'http://laser.zcu.cz/cz/opticke-vlastnosti/metody/snhrrt', u'http://www.3in.biz/pls/acm/load?p_name=T7010_2406446_941/Osobnosti_Stepan.html', u'http://www.lportala.net/players/137.html', u'http://www.johnbsebastian.com/bio.html', u'http://www.tanks-encyclopedia.com/OLD/ww2/soviet/soviet_T34-76.php', u'http://www.youngartistawards.org/noms23A.htm'] >>> bot.history.historyDict.values()[:10] [[(u'Prefektura Akita', 1518027628.9147513, u'404'), (u'Prefektura Akita', 1518102371.6200852, u'404')], [(u'T\u0159\xedda Samar', 1518397748.7366626, u'508')], [(u'Alexander Avenarius', 1516462213.1042707, u'404'), (u'Alexander Avenarius', 1516717920.382955, u'404')], [(u'HC Ocel\xe1\u0159i T\u0159inec v \u010desk\xe9 hokejov\xe9 extralize 2010/2011', 1516698848.8886993, u'404'), (u'\u010cesk\xe1 hokejov\xe1 extraliga 2010/2011', 1518550830.2320974, u'404')], [(u'Emisivita', 1516623109.8608327, u'404'), (u'Absorpce z\xe1\u0159en\xed', 1516710675.2672064, u'404'), (u'Emisivita', 1517057700.7524137, u'404'), (u'Odrazivost', 1517932349.4918804, u'404')], [(u'Miroslav \u0160t\u011bp\xe1n', 1517744892.8110282, u'404')], [(u'Todor Jonov', 1518382319.6734998, u'404')], [(u'John Sebastian', 1517212814.478413, u'404')], [(u'BT (tank)', 1516501027.0819583, u'404'), (u'BT (tank)', 1516743253.580716, u'404'), (u'BT (tank)', 1516770024.0754175, u'404'), (u'T-34', 1518305440.8585296, u'404'), (u'T-34', 1518344610.1407645, u'404'), (u'T-34', 1518365360.8237803, u'404')], [(u'Tyler Posey', 1518394011.70306, u'404')]]
Does the entire database have to persist in memory? If not I'll submit a patch to use dbm (py3 py2) instead. Yes it will be slower, but the gain is, the memory consumption can be much smaller and avoids OOM.
TASK DETAIL
EMAIL PREFERENCES
To: zhuyifei1999
Cc: gerritbot, Dalba, Xqt, Zoranzoki21, zhuyifei1999, Aklapper, pywikibot-bugs-list, Dvorapa, Giuliamocci, Adrian1985, Cpaulf30, Baloch007, Darkminds3113, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Magul, Tbscho, rafidaslam, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, jayvdb, Masti, Alchimista, Rxy
Cc: gerritbot, Dalba, Xqt, Zoranzoki21, zhuyifei1999, Aklapper, pywikibot-bugs-list, Dvorapa, Giuliamocci, Adrian1985, Cpaulf30, Baloch007, Darkminds3113, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Magul, Tbscho, rafidaslam, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, jayvdb, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
