Re: Better interactivity in low-memory situations

2019-09-13 Thread Daniel Xu
Our team at FB is working on a similar (but more generic) solution. All of our work is open source / upstreamed into the linux kernel and we're running it in production on quite a large scale already. Results are very promising. We'll be presenting about it at All Systems Go (multiple talks)

Re: Better interactivity in low-memory situations

2019-09-13 Thread Daniel Xu
> On Mo, 12.08.19 09:40, Chris Murphy (lists(a)colorremedies.com) wrote: > > > Ideally, GNOME would run all its apps as systemd --user services. We > could then set DefaultMemoryHigh= globally for the systemd --user > instance to some percentage value (which is taken relative to the > physical

Re: Better interactivity in low-memory situations

2019-09-04 Thread Kyle Marek
On 9/4/19 3:38 PM, Chris Murphy wrote: > On Wed, Sep 4, 2019 at 1:03 PM Chris Murphy wrote: >> I'm skeptical of swapfiles for this use case, because it likely >> increases the chance for file system disk contention, see section >> 11.17 >>

Re: Better interactivity in low-memory situations

2019-09-04 Thread Kyle Marek
On 9/4/19 3:16 PM, Florian Weimer wrote: > * Kyle Marek: > >> I'm finding that 32G is not necessarily sufficient for compiling clang >> itself. Similarly I've had a hard time compiling UnrealEngine from >> source. I usually see ld using up to 12G of memory to link each >> artifact. Using

Re: Better interactivity in low-memory situations

2019-09-04 Thread Chris Murphy
On Wed, Sep 4, 2019 at 1:03 PM Chris Murphy wrote: > > I'm skeptical of swapfiles for this use case, because it likely > increases the chance for file system disk contention, see section > 11.17 > https://www.kernel.org/doc/gorman/html/understand/understand014.html Actually that looks outdated

Re: Better interactivity in low-memory situations

2019-09-04 Thread Chris Murphy
per cgroup swap file support https://lwn.net/Articles/591495/ This might be interesting. What happens if swap is only assigned to new unprivileged tasks? Your compile might take three days but it doesn't make the GUI fall over? *shrug* --- Chris Murphy

Re: Better interactivity in low-memory situations

2019-09-04 Thread Florian Weimer
* Kyle Marek: > I'm finding that 32G is not necessarily sufficient for compiling clang > itself. Similarly I've had a hard time compiling UnrealEngine from > source. I usually see ld using up to 12G of memory to link each > artifact. Using -j$(nproc) on a 16 VCPU system amplifies the issue. I >

Re: Better interactivity in low-memory situations

2019-09-04 Thread Chris Murphy
On Wed, Sep 4, 2019 at 12:01 PM Kyle Marek wrote: > > I'm finding that 32G is not necessarily sufficient for compiling clang > itself. Similarly I've had a hard time compiling UnrealEngine from > source. I usually see ld using up to 12G of memory to link each > artifact. Using -j$(nproc) on a 16

Re: Better interactivity in low-memory situations

2019-09-04 Thread Kyle Marek
On 9/3/19 12:30 AM, John Harris wrote: > On Tuesday, August 20, 2019 10:48:06 PM MST John Harris wrote: >> On Sunday, August 18, 2019 4:33:47 AM MST Gordan Bobic wrote: >> >>> On Sun, Aug 11, 2019 at 10:36 AM wrote: >>> This seems like a distraction from the real goal here, which is to

Re: Better interactivity in low-memory situations

2019-09-03 Thread Dan Čermák
Chris Murphy writes: > On Mon, Aug 12, 2019 at 5:47 PM Emery Berger wrote: >> >> For what it's worth, my research group attacked basically exactly this >> problem some time ago. We built a modified Linux kernel that we called >> Redline that was utterly resilient to fork bombs, malloc bombs,

Re: Better interactivity in low-memory situations

2019-09-02 Thread John Harris
On Tuesday, August 20, 2019 10:48:06 PM MST John Harris wrote: > On Sunday, August 18, 2019 4:33:47 AM MST Gordan Bobic wrote: > > > On Sun, Aug 11, 2019 at 10:36 AM wrote: > > > > > This seems like a distraction from the real goal here, which is to > > > ensure Fedora remains responsive under

Re: Better interactivity in low-memory situations

2019-09-01 Thread Chris Murphy
On Mon, Aug 12, 2019 at 5:47 PM Emery Berger wrote: > > For what it's worth, my research group attacked basically exactly this > problem some time ago. We built a modified Linux kernel that we called > Redline that was utterly resilient to fork bombs, malloc bombs, and so on. No > process

Re: Better interactivity in low-memory situations

2019-08-20 Thread John Harris
On Sunday, August 18, 2019 4:33:47 AM MST Gordan Bobic wrote: > On Sun, Aug 11, 2019 at 10:36 AM wrote: > > This seems like a distraction from the real goal here, which is to > > ensure Fedora remains responsive under heavy memory pressure, > > I think this is an overwhelmingly important point,

Re: Better interactivity in low-memory situations

2019-08-20 Thread Chris Murphy
On Tue, Aug 20, 2019 at 2:15 AM Lennart Poettering wrote: > > On Mo, 19.08.19 13:58, Chris Murphy (li...@colorremedies.com) wrote: > > > I'm skeptical as well. But to further explore this: > > > > 1. Does the kernel know better than to write a hibernation image (all > > or part) to a /dev/zram

Re: Better interactivity in low-memory situations

2019-08-20 Thread Lennart Poettering
On Mo, 19.08.19 13:58, Chris Murphy (li...@colorremedies.com) wrote: > I'm skeptical as well. But to further explore this: > > 1. Does the kernel know better than to write a hibernation image (all > or part) to a /dev/zram device? e.g. a system with: 8GiB RAM, 8GiB > swap on ZRAM, 8GiB swap

Re: Better interactivity in low-memory situations

2019-08-19 Thread Chris Murphy
On Mon, Aug 12, 2019 at 10:20 AM Lennart Poettering wrote: > > On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote: > > > Right now the only lever to avoid swap, is to not create a swap > > partition at installation time. Or create a smaller one instead of 1:1 > > ratio with RAM.

Re: Better interactivity in low-memory situations

2019-08-19 Thread Gordan Bobic
This seems very reminiscent of what is often referred to as "swapping insanity", often in the context of MySQL. The root cause there is NUMA with memory allocation only happening on the node that requests the memory, resulting in the possibility of there being plenty of free memory on one node but

Re: Better interactivity in low-memory situations

2019-08-19 Thread Florian Weimer
* Gordan Bobic: > That may be so, but this thread started off with memory pressure also > being an issue for regular desktop x86 use. I think the problem there is that the system has sufficient reclaimable memory, but cannot make that memory available to applications in a timely fashion.

Re: Better interactivity in low-memory situations

2019-08-18 Thread Emery Berger
For what it's worth, my research group attacked basically exactly this problem quite some time ago. We built a modified Linux kernel that we called Redline that was impervious to fork bombs, malloc bombs, and so on. No process could take down the system, much less unprivileged ones. I think some

Re: Better interactivity in low-memory situations

2019-08-18 Thread Chris Murphy
On Sun, Aug 18, 2019 at 2:55 PM Gordan Bobic wrote: > > On Sun, Aug 18, 2019 at 9:07 PM Kevin Kofler wrote: >> >> Gordan Bobic wrote: >> > Right, but is it better that _everything_ else suffers with more memory >> > pressure for the handful of relatively infrequent use cases for which >> >

Re: Better interactivity in low-memory situations

2019-08-18 Thread Gordan Bobic
On Sun, Aug 18, 2019 at 9:07 PM Kevin Kofler wrote: > Gordan Bobic wrote: > > Right, but is it better that _everything_ else suffers with more memory > > pressure for the handful of relatively infrequent use cases for which > > ulimit can be used to explicitly raise the limit? > > Well, as I

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler
Gordan Bobic wrote: > Right, but is it better that _everything_ else suffers with more memory > pressure for the handful of relatively infrequent use cases for which > ulimit can be used to explicitly raise the limit? Well, as I wrote, a lower limit might actually make sense on ARM. But modern

Re: Better interactivity in low-memory situations

2019-08-18 Thread Gordan Bobic
On Sun, Aug 18, 2019 at 8:51 PM Kevin Kofler wrote: > Gordan Bobic wrote: > > It may be simpler to approach the question from the other side, i.e. is > > there anything that actually ever needs more than 1MB of stack space? If > > there is, I haven't seen it in the decade since I've been using

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler
Gordan Bobic wrote: > Adding -ffunction-sections -fdata-sections to defaults can help > considerably in producing smaller binaries, and is not the default. > Linking with -Wl,--gc-sections helps a lot and is not the default Well, -ffunction-sections -fdata-sections -Wl,--gc-sections mostly helps

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler
Gordan Bobic wrote: > It may be simpler to approach the question from the other side, i.e. is > there anything that actually ever needs more than 1MB of stack space? If > there is, I haven't seen it in the decade since I've been using this tweak > with various Fedora derived distributions. I've

Re: Better interactivity in low-memory situations

2019-08-18 Thread Dridi Boukelmoune
> It may be simpler to approach the question from the other side, i.e. is there > anything that actually ever needs more than 1MB of stack space? If there is, > I haven't seen it in the decade since I've been using this tweak with various > Fedora derived distributions. Any application

Re: Better interactivity in low-memory situations

2019-08-18 Thread Gordan Bobic
On Sun, Aug 18, 2019 at 2:33 PM Hans de Goede wrote: > > > > Adding -ffunction-sections -fdata-sections to defaults can help > considerably in producing smaller binaries, and is not the default. > > > Linking with -Wl,--gc-sections helps a lot and is not the default > > > > These

Re: Better interactivity in low-memory situations

2019-08-18 Thread Hans de Goede
Hi, On 18-08-19 15:25, Gordan Bobic wrote: On Sun, Aug 18, 2019 at 2:06 PM Hans de Goede mailto:hdego...@redhat.com>> wrote: Hi, On 18-08-19 13:33, Gordan Bobic wrote: > On Sun, Aug 11, 2019 at 10:36 AM http://gnome.org> > wrote: >  > This seems like a

Re: Better interactivity in low-memory situations

2019-08-18 Thread Gordan Bobic
On Sun, Aug 18, 2019 at 2:06 PM Hans de Goede wrote: > Hi, > > On 18-08-19 13:33, Gordan Bobic wrote: > > On Sun, Aug 11, 2019 at 10:36 AM http://gnome.org/>> wrote: > > > This seems like a distraction from the real goal here, which is to > > > ensure Fedora remains responsive under heavy

Re: Better interactivity in low-memory situations

2019-08-18 Thread Hans de Goede
Hi, On 18-08-19 13:33, Gordan Bobic wrote: On Sun, Aug 11, 2019 at 10:36 AM http://gnome.org/>> wrote: > This seems like a distraction from the real goal here, which is to > ensure Fedora remains responsive under heavy memory pressure, I think this is an overwhelmingly important point, and

Re: Better interactivity in low-memory situations

2019-08-18 Thread Gordan Bobic
On Sun, Aug 11, 2019 at 10:36 AM wrote: > This seems like a distraction from the real goal here, which is to > ensure Fedora remains responsive under heavy memory pressure, I think this is an overwhelmingly important point, and as somebody regularly working with ARM machines with tiny amounts of

Re: Better interactivity in low-memory situations

2019-08-15 Thread David Airlie
On Fri, Aug 16, 2019 at 7:48 AM Chris Murphy wrote: > > On Thu, Aug 15, 2019 at 2:19 PM Chris Murphy wrote: > > > > [ 718.068633] fmac.local kernel: SLUB: Unable to allocate memory on > > node -1, gfp=0x900(GFP_NOWAIT|__GFP_ZERO) > > [ 718.068636] fmac.local kernel: cache: page->ptl, object

Re: Better interactivity in low-memory situations

2019-08-15 Thread Chris Murphy
On Thu, Aug 15, 2019 at 2:19 PM Chris Murphy wrote: > > [ 718.068633] fmac.local kernel: SLUB: Unable to allocate memory on > node -1, gfp=0x900(GFP_NOWAIT|__GFP_ZERO) > [ 718.068636] fmac.local kernel: cache: page->ptl, object size: 72, > buffer size: 72, default order: 0, min order: 0 > [

Re: Better interactivity in low-memory situations

2019-08-15 Thread Chris Murphy
On Thu, Aug 15, 2019 at 1:51 AM Artem Tim wrote: > > BFQ scheduler help a lot with this issue. Using it on Fedora since 4.19 > kernel. Also there was previous discussion about make it default for > Workstation >

Re: Better interactivity in low-memory situations

2019-08-15 Thread Artem Tim
BFQ scheduler help a lot with this issue. Using it on Fedora since 4.19 kernel. Also there was previous discussion about make it default for Workstation https://lists.fedoraproject.org/archives/list/ker...@lists.fedoraproject.org/message/I2OZWDD4QCDYUXJ5NHYTMGNAB4KLJN2K/

Re: Better interactivity in low-memory situations

2019-08-14 Thread S.
(Oops, sorry, re-post because I messed up the threading.) I'm not a developer, nor do I pretend to understand the nuances of memory management. But I signed up for this list just to say "thanks" to all the devs and others that are finally discussing what I consider to be one of the biggest

re: Better interactivity in low-memory situations

2019-08-14 Thread S.
I'm not a developer, nor do I pretend to understand the nuances of memory management. But I signed up for this list just to say "thanks" to all the devs and others that are finally discussing what I consider to be one of the biggest problems with Linux on the desktop. My experience with

Re: Better interactivity in low-memory situations

2019-08-13 Thread Simon Farnsworth
> On 10 Aug 2019, at 17:56, Georg Sauthoff wrote: > > On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote: > [..] >> Problem and thesis statement: >> Certain workloads, such as building webkitGTK from source, results in >> heavy swap usage eventually leading to the system becoming

Re: Better interactivity in low-memory situations

2019-08-12 Thread Chris Murphy
On Mon, Aug 12, 2019 at 6:31 PM David Airlie wrote: > > On Sun, Aug 11, 2019 at 2:57 AM Georg Sauthoff wrote: > > > > On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote: > > [..] > > > Problem and thesis statement: > > > Certain workloads, such as building webkitGTK from source,

Re: Better interactivity in low-memory situations

2019-08-12 Thread David Airlie
On Sun, Aug 11, 2019 at 2:57 AM Georg Sauthoff wrote: > > On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote: > [..] > > Problem and thesis statement: > > Certain workloads, such as building webkitGTK from source, results in > > heavy swap usage eventually leading to the system becoming

Re: Better interactivity in low-memory situations

2019-08-12 Thread Chris Murphy
On Mon, Aug 12, 2019 at 11:07 AM Benjamin Kircher wrote: > > (… I definitely need to play around with Silverblue to learn what they are > doing.) I'm pretty sure Silverblue will be rebased on Fedora CoreOS which recently released a preview. I'm not sure what the time frame for that is, but

Re: Better interactivity in low-memory situations

2019-08-12 Thread Lennart Poettering
On Mo, 12.08.19 19:06, Benjamin Kircher (benjamin.kirc...@gmail.com) wrote: > > > > On 12. Aug 2019, at 18:16, Lennart Poettering wrote: > > > > On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote: > > > >> How to do this automatically? Could there be a mechanism for the > >>

Re: Better interactivity in low-memory situations

2019-08-12 Thread Benjamin Kircher
> On 12. Aug 2019, at 18:16, Lennart Poettering wrote: > > On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote: > >> How to do this automatically? Could there be a mechanism for the >> system and the requesting application to negotiate resources? > > Ideally, GNOME would run

Re: Better interactivity in low-memory situations

2019-08-12 Thread Benjamin Kircher
> On 12. Aug 2019, at 17:40, Chris Murphy wrote: > > If I just run the example program, let's say systemd MemoryLimit is > set to /proc/meminfo MemAvailable, the program is still going to try > and bust out of that and fail. The failure reason is also non-obvious. > Yes this is definitely an

Re: Better interactivity in low-memory situations

2019-08-12 Thread Lennart Poettering
On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote: > How to do this automatically? Could there be a mechanism for the > system and the requesting application to negotiate resources? Ideally, GNOME would run all its apps as systemd --user services. We could then set

Re: Better interactivity in low-memory situations

2019-08-12 Thread Chris Murphy
On Mon, Aug 12, 2019 at 1:01 AM Florian Weimer wrote: > > * Chris Murphy: > > > Summary of findings (restated, but basically the same as found at [2]): > > Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM, > > Samsung SSD 840 EVO, Fedora Rawhide Workstation. > > Do you use the

Re: Better interactivity in low-memory situations

2019-08-12 Thread Chris Murphy
On Mon, Aug 12, 2019 at 12:30 AM Benjamin Kircher wrote: > > > > > On 11. Aug 2019, at 23:05, Chris Murphy wrote: > > > > I think the point at which the mouse pointer has frozen, the user has > > no practical means of controlling or interacting with the system, it's > > a failure. > > > > In the

Re: Better interactivity in low-memory situations

2019-08-12 Thread Florian Weimer
* Petr Pisar: > On 2019-08-12, Florian Weimer wrote: >> Do you use the built-in Intel graphics? Can you test with something >> else? >> > Does it have any effect? It happens to me even with a discrete GPU. I expect that the GEM shrinker (or rather, the reason why it is needed) radically

Re: Better interactivity in low-memory situations

2019-08-12 Thread Petr Pisar
On 2019-08-12, Florian Weimer wrote: > Do you use the built-in Intel graphics? Can you test with something > else? > Does it have any effect? It happens to me even with a discrete GPU. As far as I know integrated graphics arrays do not share physical memory from point of view of the CPU address

Re: Better interactivity in low-memory situations

2019-08-12 Thread Florian Weimer
* Chris Murphy: > Summary of findings (restated, but basically the same as found at [2]): > Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM, > Samsung SSD 840 EVO, Fedora Rawhide Workstation. Do you use the built-in Intel graphics? Can you test with something else? Thanks,

Re: Better interactivity in low-memory situations

2019-08-12 Thread Benjamin Kircher
> On 11. Aug 2019, at 23:05, Chris Murphy wrote: > > I think the point at which the mouse pointer has frozen, the user has > no practical means of controlling or interacting with the system, it's > a failure. > > In the short term, is it reasonable and possible, to get the oom > killer to

Re: Better interactivity in low-memory situations

2019-08-11 Thread Chris Murphy
On Sun, Aug 11, 2019 at 1:02 PM Jan Kratochvil wrote: > > On Sun, 11 Aug 2019 20:54:28 +0200, Chris Murphy wrote: > > and likely experiences data loss and possibly even file system > > corruption as a direct consequence of having to force power off on the > > machine because for all practical

Re: Better interactivity in low-memory situations

2019-08-11 Thread Jan Kratochvil
On Sun, 11 Aug 2019 20:54:28 +0200, Chris Murphy wrote: > and likely experiences data loss and possibly even file system > corruption as a direct consequence of having to force power off on the > machine because for all practical purposes normal control has been > lost. Not really, this is what

Re: Better interactivity in low-memory situations

2019-08-11 Thread Chris Murphy
On Sun, Aug 11, 2019 at 10:36 AM wrote: > > On Sun, Aug 11, 2019 at 10:50 AM, Chris Murphy > wrote: > > Let's take another argument. If the user manually specifies 'ninja -j > > 64' on this same system, is that sabotage? I'd say it is. And > > therefore why isn't it sabotage that the ninja

Re: Better interactivity in low-memory situations

2019-08-11 Thread Chris Murphy
On Sun, Aug 11, 2019 at 11:21 AM Jan Kratochvil wrote: > > On Sun, 11 Aug 2019 17:50:17 +0200, Chris Murphy wrote: > > I don't follow. You're saying RelWithDebInfo is never suitable for a > > local build? > > Most of the time. What is your use case for it? My use case is testing the

Re: Better interactivity in low-memory situations

2019-08-11 Thread Jan Kratochvil
On Sun, 11 Aug 2019 17:50:17 +0200, Chris Murphy wrote: > I don't follow. You're saying RelWithDebInfo is never suitable for a > local build? Most of the time. What is your use case for it? > isn't relevant to getting a successful build. With powerful enough machine everything is possible.

Re: Better interactivity in low-memory situations

2019-08-11 Thread mcatanzaro
On Sun, Aug 11, 2019 at 10:50 AM, Chris Murphy wrote: Let's take another argument. If the user manually specifies 'ninja -j 64' on this same system, is that sabotage? I'd say it is. And therefore why isn't it sabotage that the ninja default computes N jobs as nrcpus + 2? And also doesn't take

Re: Better interactivity in low-memory situations

2019-08-11 Thread Chris Murphy
On Sat, Aug 10, 2019 at 3:07 AM Jan Kratochvil wrote: > > On Fri, 09 Aug 2019 23:50:43 +0200, Chris Murphy wrote: > > $ cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja > > RelWithDebInfo is -O2 -g build. That is not suitable for debugging, for > debugging you should use

Re: Better interactivity in low-memory situations

2019-08-10 Thread Georg Sauthoff
On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote: [..] > Problem and thesis statement: > Certain workloads, such as building webkitGTK from source, results in > heavy swap usage eventually leading to the system becoming totally > unresponsive. Look into switching from disk based swap,

Re: Better interactivity in low-memory situations

2019-08-10 Thread Jan Kratochvil
On Fri, 09 Aug 2019 23:50:43 +0200, Chris Murphy wrote: > $ cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja RelWithDebInfo is -O2 -g build. That is not suitable for debugging, for debugging you should use -DCMAKE_BUILD_TYPE=Debug (that is -g). RelWithDebInfo is useful for final rpm

Re: Better interactivity in low-memory situations

2019-08-09 Thread Chris Murphy
Just in case anyone wants to try to reproduce this particular example: 1. Grab latest stable from here and untar it https://webkitgtk.org/releases/ 2. Run this included script, which is dnf aware, to install dependencies ./Tools/gtk/install-dependencies 3. Additional packages I had to install to

Re: Better interactivity in low-memory situations

2019-08-09 Thread Omair Majid
Hi, Chris Murphy writes: > Certain workloads, such as building webkitGTK from source, results in > heavy swap usage eventually leading to the system becoming totally > unresponsive. Look into switching from disk based swap, to swap on a > ZRAM device. It sounds like the same issue that has