Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
> If you actually tried to use the memory it says in MemAvailable, you > may very well already get bad side effects as the kernel needs to > reclaim memory used for other purposes (file caches, mmap'ed > executables, heap, …). Depending on the workload, this may already > cause the system to start thrashing. memavaild[1] is yet another tool to improve responsiveness during heavy swapping. How it works: slices are swapped out when MemAvailable is low by reducing memory.max (values change dynamically). Effects: performance increases in tasks that requires heavy swapping, especially in io-bound tasks. At the same time, tasks are performed at less io and memory pressure. It may be used out of the box with any DE. With combination of prelockd and memavaild GUI remains responsive with high memory and io pressure (pressure=85[2] with system and swap on HDD, I taked screenshot, GUI was not freezed). This proves that high memory pressure (by PSI) does not always correspond to system hang (psi-monitor uses mem pressure=10 as a thigger, for example). [1] https://github.com/hakavlad/memavaild [2] https://ibb.co/hKGy0bZ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
> So with > applications started, you might get higher. I think we should protect only basic GUI. On computers with 16G+ RAM locking 1G memory with apps should not be a problem if it helps to improve responsiveness. > Was that with or without swap? It was without swap, but with swap it has the same effect (faster killer coming and system reclaiming) + responsiveness was improved during heavy swapping. > I feel that uresourced is just nicer conceptually. > So uresourced is > going to be ineffective for KDE for the time being. However, the advantage of prelockd is that may be used on old systems with any DE and any file system and an init. On Debian 9 Mate locking 150-200M takes very good effect. On Fedora with LXDE 200M is also good value. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
> how much memory that amounts to in the usual scenarios 700M on F32 without any apps started. Largest file: (207.9M) /usr/lib/locale/locale-archive Files list with its sizes: https://pastebin.com/Hpr6D3Sv Locking even 250M-300M takes good effect. For example, demo: https://www.youtube.com/watch?v=vykUrP1UvcI - no freezes with prelockd and freeze without prelockd. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>What software in the default image leads to low memory issues? Web browsers? For example, browsers, electron-based apps, blender, compilation, VM, openings files, opening file manager (once I came across this: when I opened the file manager, an uncontrolled leak occurred somewhere in the thumbnail.so, and tand the system froze). It can leak anywhere, and it can happen unexpectedly. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>Linux handles low memory situations just fine, but it's much better if you >have an appropriately sized swap partition and let the kernel do its job No, by default Linux can hang at low memory condition. Huge swap space will not help you if a leak occurs. With a large swap space, the hang can happen later, but it can still happen. Another point is that the swap space is slow. With fast leaks and slow swap space, freezing is possible throughout the entire swap filling time. A typical problem: "once all the ram is used up the whole system freezes as swap starts getting full, but really the whole system is completely locked-up. I thought my swapfile was too small so I made it match my ram (12 GB) but the system still gets frozen" [1]. >If you didn't mean for the program to use as much memory as it tried to, the >correct solution would be to use cgroups. 1. This is not configured by default. 2. This can be inconvenient even for advanced users. 3. Quick leaks can happen unexpectedly. [1] https://github.com/hakavlad/nohang/issues/85#issue-564348496 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>It is also unclear that it can prevent full OOM (both >RAM and swap completely full) in all cases Kernel's killer is also cannot prevent full OOM in all cases. earlyoom prevent full OOM (and situations close to OOM) much more effective - faster (selects victim in 5-50ms, monitoring interval is 0.1-1s) and softer (sends SIGTERM first). >fail >to kill processes when it would be necessary to prevent swap thrashing OK, earlyoom does not prevent thrashing, but kernel also does not prevent it. Thrashing may be reduced with killers that supports response to PSI: oomd [1], nohang [2], psi-monitor [3] (psi-monitor is used by default in Endless OS). >I strongly believe that this kernel problem can only be solved within the >kernel, by a synchronous (hence race-free) hook on all allocations. It's already solved in the kernel: just set `vm.overcommit_memory=2` and `vm.overcommit_ratio=50`: "When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory." [4] 1. https://github.com/facebookincubator/oomd 2. https://github.com/hakavlad/nohang 3. https://github.com/endlessm/eos-boot-helper/tree/master/psi-monitor 4. https://www.kernel.org/doc/Documentation/sysctl/vm.txt ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>it should be disabled so it doesn't kill our software What should people who suffer from the fact that the kernel's OOM killer does not work, and they are forced to hard reboot (and lose unsaved data) the computer when it freezes? This is a serious and very common problem that exists for a long time and has not been fixed out of the box. What do you suggest? Should we do nothing? Of course, we can do nothing and wait for the inclusion of the system-oomd in the systemd [7]. Just wait. Also look at these discussions: - Why are low memory conditions handled so badly? [1] - Memory management "more effective" on Windows than Linux? (in preventing total system lockup) [2] - Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure [3] [4] [5] - How do I prevent Linux from freezing when out of memory? Today I (accidentally) ran some program on my Linux box that quickly used a lot of memory. My system froze, became unresponsive and thus I was unable to kill the offender. How can I prevent this in the future? Can't it at least keep a responsive core or something running? [6] 1. https://www.reddit.com/r/linux/comments/56r4xj/why_are_low_memory_conditions_handled_so_badly/ 2. https://www.reddit.com/r/linux/comments/aqd9mh/memory_management_more_effective_on_windows_than/ 3. https://lkml.org/lkml/2019/8/4/15 4. https://www.reddit.com/r/linux/comments/cmg48b/lets_talk_about_the_elephant_in_the_room_the/ 5. https://news.ycombinator.com/item?id=20620545 6. https://serverfault.com/questions/390623/how-do-i-prevent-linux-from-freezing-when-out-of-memory 7. https://github.com/systemd/systemd/pull/15206 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>What about /usr/lib64/qt5/libexec/QtWebEngineProcess processes? These processes get oom_score_adj=300 out of the box, see https://pastebin.com/AFVU9U8X ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>EarlyOOM being a >userspace process that races with the memory-consuming processes and that >may end up not getting scheduled due to the very impending OOM condition >that it is trying to prevent. earlyoom consumes 1 MiB VmRSS and all memory is locked by mlockall(). earlyoom works pretty fast and selects victim in 5-50 ms even with intense swapping. >It is also unclear that it can prevent full OOM (both >RAM and swap completely full) It's clear: earlyoom forks pretty fast and well. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Enable EarlyOOM on Fedora KDE - Fedora 33 Self-Contained Change proposal
>10% and 5% to 1% and 0% Default values is already changed to 4% (but not more than 400 MiB) and 2% (but not more than 200 MiB). A nonzero threshold helps maintain disk cache and speeds up system recovery after correction. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
