|zhuyifei1999 added a comment.|
That definitely sounds like an OOM condition.
Linux typically overcommits memory (see /proc/sys/vm/overcommit* in proc(5)), allowing more memory to be allocated via malloc(3) (which would return NULL if overcommit ratio is exceeded, or if overcommit is disabled and the system is out of memory) than the system actually has. This allows many programs to allocate more memory than the system actually has, and the system would not fail if a lot of memory is only allocated, but not used; otherwise the system would fail malloc(3) quite frequently due to many programs over-allocating.
The problem comes when huge chunks of memory have been allocated, exceeding the physical memory limit, and then large chunks of memory are actually used, exceeding the physical memory limit. As far as I understand it (may contain misunderstandings), when such condition occurs, and a program starts to use a new memory virtual address that is not backed by any physical address, a page fault occurs and the exception is sent to the kernel interrupt handler, which will put the currently running task into D-state (uninterruptable /IO wait sleep), while the kernel waits for more memory being available to allow the virtual address being written. This is kept as a task in the run queue until the memory is available and therefore load increases.
Typically, the linux OOM killer will start to ask eventually, freeing some memory to keep the system running. So far, OOM killer on my machine is rarely triggered automatically, but most of the time by me manually triggering via SysRq. The default OOM killer may sometimes have trouble doing the analysis of what to kill using the oom scores, and an alternative is to kill whichever task that triggers OOM (see /proc/sys/vm/oom_kill_allocating_task in proc(5)).
Cc: gerritbot, Dalba, Xqt, Zoranzoki21, zhuyifei1999, Aklapper, pywikibot-bugs-list, Dvorapa, Giuliamocci, Adrian1985, Cpaulf30, Baloch007, Darkminds3113, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Magul, Tbscho, rafidaslam, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, jayvdb, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list email@example.com https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs