[Pywikipedia-bugs] [Maniphest] [Commented On] T185561: weblinkchecker.py slows down (itself, OS) to freeze after a while reaching 100% of RAM

[zhuyifei1999]

zhuyifei1999 added a comment.

Reproduction code (simulating weblinkchecker.py#L200):

strace python3 -c 'import datetime, memento_client; mc = memento_client.MementoClient(); mc.timegate_uri = "http://web.archive.org/web/"; mc.get_memento_info("http://www.icann.org/en/tlds/agreements/cat/cat-agreement-23sep05.htm", datetime.datetime.now())'

Last few syscalls before hang (happens on both ubuntu 16.04 with python 3.6.4 and ubuntu 17.10 with python 3.6.3):

getsockopt(4, SOL_SOCKET, SO_TYPE, [1], [4]) = 0
ioctl(4, FIONBIO, [0])                  = 0
getpeername(4, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("192.0.32.7")}, [16]) = 0
write(4, "\26\3\1\2\0\1\0\1\374\3\3\205C\261\201\340\350,\177\n]\250\241b\244g\260\222\217\204\322\265"..., 517) = 517
read(4, "\26\3\3\0Q\2\0", 7)            = 7
read(4, "\0M\3\3t\7\304mI\202\23/,\333\20\335p\251\255\177\f\32\f\242\313\327\323\254\274+te"..., 79) = 79
read(4, "\26\3\3\vV", 5)                = 5
read(4, "\v\0\vR\0\vO\0\6\2240\202\6\2200\202\5x\240\3\2\1\2\2\20\2\363V\234\324\266\352"..., 2902) = 2709
read(4, "\312Yq}\364\4\226k\323YX=\376\3231%\\\0308\204\243\346\237\202\375\214[\2301N\315x"..., 193) = 193
brk(0x1ae5000)                          = 0x1ae5000
read(4, "\26\3\3\0\4", 5)               = 5
read(4, "\16\0\0\0", 4)                 = 4
write(4, "\26\3\3\2\6\20\0\2\2\2\0\206\24\33\214|\2363\213\246\307V\30\335_2a&\330\357i\307"..., 574) = 574
read(4, "\24\3\3\0\1", 5)               = 5
read(4, "\1", 1)                        = 1
read(4, "\26\3\3\0(", 5)                = 5
read(4, "\3\0\0\0\0\0\0\0001\223z\362]\334\317\371\35\2502g\314\261\341\347\6\365-\37\370\274N\215"..., 40) = 40
ioctl(4, FIONBIO, [0])                  = 0
write(4, "\27\3\3\0\331\"_\23\256\220\27\324e\241\36.\226dt\21\3103e\311\207\n\210\377\241\267\f\354"..., 222) = 222
ioctl(4, FIONBIO, [0])                  = 0
read(4,

Unexpectedly, the same hang happens for python 3.5.2 on ubuntu 16.04.

TASK DETAIL

https://phabricator.wikimedia.org/T185561

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: zhuyifei1999
Cc: gerritbot, Dalba, Xqt, Zoranzoki21, zhuyifei1999, Aklapper, pywikibot-bugs-list, Dvorapa, Magul, Tbscho, rafidaslam, MayS, Mdupont, JJMC89, Avicennasis, jayvdb, Masti, Alchimista, Rxy

_______________________________________________
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs