subject:"Better interactivity in low\-memory situations"

On Sun, Aug 18, 2019 at 9:07 PM Kevin Kofler  wrote:

> Gordan Bobic wrote:
> > Right, but is it better that _everything_ else suffers with more memory
> > pressure for the handful of relatively infrequent use cases for which
> > ulimit can be used to explicitly raise the limit?
>
> Well, as I wrote, a lower limit might actually make sense on ARM. But
> modern
> x86 computers have gigabytes of RAM, so 1 MiB is ridiculously small there.
> So this would have to be an architecture-specific setting for ARM.

That may be so, but this thread started off with memory pressure also being
an issue for regular desktop x86 use.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler

Gordan Bobic wrote:
> Right, but is it better that _everything_ else suffers with more memory
> pressure for the handful of relatively infrequent use cases for which
> ulimit can be used to explicitly raise the limit?

Well, as I wrote, a lower limit might actually make sense on ARM. But modern 
x86 computers have gigabytes of RAM, so 1 MiB is ridiculously small there. 
So this would have to be an architecture-specific setting for ARM.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 18, 2019 at 8:51 PM Kevin Kofler  wrote:

> Gordan Bobic wrote:
> > It may be simpler to approach the question from the other side, i.e. is
> > there anything that actually ever needs more than 1MB of stack space? If
> > there is, I haven't seen it in the decade since I've been using this
> tweak
> > with various Fedora derived distributions.
>
> I've more than once had Java applications crash with a StackOverflowError
> because Java has such a retarded 1 MiB default stack size independently of
> the amount of available RAM. (You have to explicitly use the -Xss
> parameter
> to get more.) It happened at least once in Java code and at least once in
> C++ code interfaced through JNI.
>
> So I don't think 1 MiB is a reasonable default stack size for
> general-purpose computers, though it might make sense on ARM.
>

Right, but is it better that _everything_ else suffers with more memory
pressure for the handful of relatively infrequent use cases for which
ulimit can be used to explicitly raise the limit?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler

Gordan Bobic wrote:
> Adding -ffunction-sections -fdata-sections to defaults can help
> considerably in producing smaller binaries, and is not the default.
> Linking with -Wl,--gc-sections helps a lot and is not the default

Well, -ffunction-sections -fdata-sections -Wl,--gc-sections mostly helps if 
the binary contains a lot of unused code. This is not often the case in 
dynamically linked executables or shared libraries. Where it typically 
happens is statically linked binaries (where the static libraries contain 
APIs not used by the statically linked executable), and even there only if 
the library is not actually designed for static linking (in which case it 
would put separate APIs into separate compilation units to begin with). And 
static linking is a bad idea for RAM consumption anyway, at least on 
GNU/Linux where dynamically linked executables actually share the code 
sections of shared libraries in RAM too, not just on disk.

So I'm afraid you will probably find those options to be less useful in 
practice than you expect.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Kevin Kofler

Gordan Bobic wrote:
> It may be simpler to approach the question from the other side, i.e. is
> there anything that actually ever needs more than 1MB of stack space? If
> there is, I haven't seen it in the decade since I've been using this tweak
> with various Fedora derived distributions.

I've more than once had Java applications crash with a StackOverflowError 
because Java has such a retarded 1 MiB default stack size independently of 
the amount of available RAM. (You have to explicitly use the -Xss parameter 
to get more.) It happened at least once in Java code and at least once in 
C++ code interfaced through JNI.

So I don't think 1 MiB is a reasonable default stack size for
general-purpose computers, though it might make sense on ARM.

Kevin Kofler

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Dridi Boukelmoune

> It may be simpler to approach the question from the other side, i.e. is there 
> anything that actually ever needs more than 1MB of stack space? If there is, 
> I haven't seen it in the decade since I've been using this tweak with various 
> Fedora derived distributions.

Any application allowing arbitrary regular expressions and/or regex
input using libpcre...

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 18, 2019 at 2:33 PM Hans de Goede  wrote:

>
> >  > Adding -ffunction-sections -fdata-sections to defaults can help
> considerably in producing smaller binaries, and is not the default.
> >  > Linking with -Wl,--gc-sections helps a lot and is not the default
> >
> > These OTOH are interesting I know that e.g. uboot combines these and
> it helps a lot to get smaller binaries,
> > and this should help with RAM size too, since if a page of a binary
> contains mostly unused things and 1 symbol
> > which is actually used it will still get paged in.
> >
> > Can you perhaps start a new devel list thread about just this ?
> Maybe with some binary size numbers for
> > some apps / libs build with and without these options?
> >
> >
> > It's pretty well documented in various articles, e.g.:
> > https://wiki.wxwidgets.org/Reducing_Executable_Size
> > It also covers how much difference -Os can make.
>
> Interesting, thank you for that link.
>
>
> >  > Extensive stripping seems to already be the default
> (--strip-unneeded, removal of .comment and .note sections)
> >  >
> >  > 2) Runtime condiguration
> >  > Default stack size is 8192 (ulimit -s). This unnecessarily eats a
> considerably amount of memory. I have yet to see anything that actually
> experiences problems with 1M.
> >
> > Actually ulimit -s is the *maximum* stack size, I'm pretty sure the
> stack will start much smaller and
> > grow dynamically. So changing this is not saving any RAM and it will
> makes apps which do have high
> > stack usage crash when they hit the new lower limit.
> >
> >
> > Either way, it makes a noticeable difference to memory consumption on a
> very memory constrained system without any other obvious adverse effects.
>
> Interesting unless I'm reading the manpage wrong, "ulimit -s" sets the
> maximum stack-size.
> Maybe that also influences the initial sizing of the stack ?
>

I believe it does. Or at least that is the only explanation I can come up
with for the observation.


>
> Can someone who knows more about this shed some light on this? Is there a
> way to go with
> a smaller initial stack-size without changing the maximum size?
>

It may be simpler to approach the question from the other side, i.e. is
there anything that actually ever needs more than 1MB of stack space? If
there is, I haven't seen it in the decade since I've been using this tweak
with various Fedora derived distributions.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Hans de Goede

Hi,

On 18-08-19 15:25, Gordan Bobic wrote:

On Sun, Aug 18, 2019 at 2:06 PM Hans de Goede mailto:hdego...@redhat.com>> wrote:

Hi,

On 18-08-19 13:33, Gordan Bobic wrote:
 > On Sun, Aug 11, 2019 at 10:36 AM http://gnome.org> 
> wrote:
 >  > This seems like a distraction from the real goal here, which is to
 >  > ensure Fedora remains responsive under heavy memory pressure,
 >
 > I think this is an overwhelmingly important point, and as somebody 
regularly working with ARM machines with tiny amounts of RAM, it is of 
considerable interest to me.
 > I typically use CentOS because stability is important to me, but most 
worthwhile things filter to there, so I hope what I'm about to say is not _too_ 
deprecated.
 >
 > 1) Compile options
 >  From what I can tell from rpm macro options, default on C7 seems to be 
-O2. -Os seems to help in most cases.

I don't think it is likely that Fedora will switch to -Os

It is not my place to argue about whether it will. The thread was asking for 
things that might contribute toward alleviating the memory pressure problem. 
This can make a fairly dramatic difference and it would contribute toward 
alleviating the problem because smaller binaries mean less to mmap().

 > Adding -ffunction-sections -fdata-sections to defaults can help 
considerably in producing smaller binaries, and is not the default.
 > Linking with -Wl,--gc-sections helps a lot and is not the default

These OTOH are interesting I know that e.g. uboot combines these and it 
helps a lot to get smaller binaries,
and this should help with RAM size too, since if a page of a binary 
contains mostly unused things and 1 symbol
which is actually used it will still get paged in.

Can you perhaps start a new devel list thread about just this ? Maybe with 
some binary size numbers for
some apps / libs build with and without these options?

It's pretty well documented in various articles, e.g.:
https://wiki.wxwidgets.org/Reducing_Executable_Size
It also covers how much difference -Os can make.

Interesting, thank you for that link.

 > Extensive stripping seems to already be the default (--strip-unneeded, 
removal of .comment and .note sections)
 >
 > 2) Runtime condiguration
 > Default stack size is 8192 (ulimit -s). This unnecessarily eats a 
considerably amount of memory. I have yet to see anything that actually 
experiences problems with 1M.

Actually ulimit -s is the *maximum* stack size, I'm pretty sure the stack 
will start much smaller and
grow dynamically. So changing this is not saving any RAM and it will makes 
apps which do have high
stack usage crash when they hit the new lower limit.

Either way, it makes a noticeable difference to memory consumption on a very 
memory constrained system without any other obvious adverse effects.

Interesting unless I'm reading the manpage wrong, "ulimit -s" sets the maximum 
stack-size.
Maybe that also influences the initial sizing of the stack ?

Can someone who knows more about this shed some light on this? Is there a way 
to go with
a smaller initial stack-size without changing the maximum size?

Regards,

Hans
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 18, 2019 at 2:06 PM Hans de Goede  wrote:

> Hi,
>
> On 18-08-19 13:33, Gordan Bobic wrote:
> > On Sun, Aug 11, 2019 at 10:36 AM  http://gnome.org/>> wrote:
> >  > This seems like a distraction from the real goal here, which is to
> >  > ensure Fedora remains responsive under heavy memory pressure,
> >
> > I think this is an overwhelmingly important point, and as somebody
> regularly working with ARM machines with tiny amounts of RAM, it is of
> considerable interest to me.
> > I typically use CentOS because stability is important to me, but most
> worthwhile things filter to there, so I hope what I'm about to say is not
> _too_ deprecated.
> >
> > 1) Compile options
> >  From what I can tell from rpm macro options, default on C7 seems to be
> -O2. -Os seems to help in most cases.
>
> I don't think it is likely that Fedora will switch to -Os
>

It is not my place to argue about whether it will. The thread was asking
for things that might contribute toward alleviating the memory pressure
problem. This can make a fairly dramatic difference and it would contribute
toward alleviating the problem because smaller binaries mean less to mmap().

> > Adding -ffunction-sections -fdata-sections to defaults can help
> considerably in producing smaller binaries, and is not the default.
> > Linking with -Wl,--gc-sections helps a lot and is not the default
>
> These OTOH are interesting I know that e.g. uboot combines these and it
> helps a lot to get smaller binaries,
> and this should help with RAM size too, since if a page of a binary
> contains mostly unused things and 1 symbol
> which is actually used it will still get paged in.
>
> Can you perhaps start a new devel list thread about just this ? Maybe with
> some binary size numbers for
> some apps / libs build with and without these options?
>

It's pretty well documented in various articles, e.g.:
https://wiki.wxwidgets.org/Reducing_Executable_Size
It also covers how much difference -Os can make.

> > Extensive stripping seems to already be the default (--strip-unneeded,
> removal of .comment and .note sections)
> >
> > 2) Runtime condiguration
> > Default stack size is 8192 (ulimit -s). This unnecessarily eats a
> considerably amount of memory. I have yet to see anything that actually
> experiences problems with 1M.
>
> Actually ulimit -s is the *maximum* stack size, I'm pretty sure the stack
> will start much smaller and
> grow dynamically. So changing this is not saving any RAM and it will makes
> apps which do have high
> stack usage crash when they hit the new lower limit.

Either way, it makes a noticeable difference to memory consumption on a
very memory constrained system without any other obvious adverse effects.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-18 Thread Hans de Goede

Hi,

On 18-08-19 13:33, Gordan Bobic wrote:

On Sun, Aug 11, 2019 at 10:36 AM http://gnome.org/>> 
wrote:
 > This seems like a distraction from the real goal here, which is to
 > ensure Fedora remains responsive under heavy memory pressure,

I think this is an overwhelmingly important point, and as somebody regularly 
working with ARM machines with tiny amounts of RAM, it is of considerable 
interest to me.
I typically use CentOS because stability is important to me, but most 
worthwhile things filter to there, so I hope what I'm about to say is not _too_ 
deprecated.

1) Compile options
 From what I can tell from rpm macro options, default on C7 seems to be -O2. 
-Os seems to help in most cases.

I don't think it is likely that Fedora will switch to -Os

Adding -ffunction-sections -fdata-sections to defaults can help considerably in 
producing smaller binaries, and is not the default.
Linking with -Wl,--gc-sections helps a lot and is not the default

These OTOH are interesting I know that e.g. uboot combines these and it helps a 
lot to get smaller binaries,
and this should help with RAM size too, since if a page of a binary contains 
mostly unused things and 1 symbol
which is actually used it will still get paged in.

Can you perhaps start a new devel list thread about just this ? Maybe with some 
binary size numbers for
some apps / libs build with and without these options?

Extensive stripping seems to already be the default (--strip-unneeded, removal 
of .comment and .note sections)

2) Runtime condiguration
Default stack size is 8192 (ulimit -s). This unnecessarily eats a considerably 
amount of memory. I have yet to see anything that actually experiences problems 
with 1M.

Actually ulimit -s is the *maximum* stack size, I'm pretty sure the stack will 
start much smaller and
grow dynamically. So changing this is not saving any RAM and it will makes apps 
which do have high
stack usage crash when they hit the new lower limit.

Regards,

Hans
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 11, 2019 at 10:36 AM  wrote:
> This seems like a distraction from the real goal here, which is to
> ensure Fedora remains responsive under heavy memory pressure,

I think this is an overwhelmingly important point, and as somebody
regularly working with ARM machines with tiny amounts of RAM, it is of
considerable interest to me.
I typically use CentOS because stability is important to me, but most
worthwhile things filter to there, so I hope what I'm about to say is not
_too_ deprecated.

1) Compile options
>From what I can tell from rpm macro options, default on C7 seems to be -O2.
-Os seems to help in most cases.
Adding -ffunction-sections -fdata-sections to defaults can help
considerably in producing smaller binaries, and is not the default.
Linking with -Wl,--gc-sections helps a lot and is not the default
Extensive stripping seems to already be the default (--strip-unneeded,
removal of .comment and .note sections)

2) Runtime condiguration
Default stack size is 8192 (ulimit -s). This unnecessarily eats a
considerably amount of memory. I have yet to see anything that actually
experiences problems with 1M.

3) zram
This was mentioned earlier in the thread, and on most of my systems, memory
constrained or otherwise, unless I have an overwhelming reason not to, I
run with zram swap equal in size to RAM with lz4 compression and
vm.swappiness=100. I typically see compression ratios between 2:1 and 3:1
in zram, so on a system with, say, 10GB of RAM, it would provide 10GB of
very fast swap at a cost of 3-5GB of RAM. This seems like a favourable
trade off, especially on systems with extremely constrained RAM (e.g. ARM
devices with 512MB of RAM).

I'm sure there is more that can be done, but this seems like a good start
as far as the cost / benefit is concerned.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-15 Thread David Airlie

On Fri, Aug 16, 2019 at 7:48 AM Chris Murphy  wrote:
>
> On Thu, Aug 15, 2019 at 2:19 PM Chris Murphy  wrote:
> >
> > [  718.068633] fmac.local kernel: SLUB: Unable to allocate memory on
> > node -1, gfp=0x900(GFP_NOWAIT|__GFP_ZERO)
> > [  718.068636] fmac.local kernel:   cache: page->ptl, object size: 72,
> > buffer size: 72, default order: 0, min order: 0
> > [  718.068639] fmac.local kernel:   node 0: slabs: 296, objs: 16576, free: 0
> > [  718.068704] fmac.local kernel: chronyd: page allocation failure:
> > order:0, mode:0x800(GFP_NOWAIT),
> > nodemask=(null),cpuset=/,mems_allowed=0
> >
> > Not sure what to make of that.
>
> Asked on #fedora-kernel, it's a known issue with 5.3.0-rc4 and drm.
>

Nope it's not that.

Something has leaked all your memory (not drm).

Dave.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-15 Thread Chris Murphy

On Thu, Aug 15, 2019 at 2:19 PM Chris Murphy  wrote:
>
> [  718.068633] fmac.local kernel: SLUB: Unable to allocate memory on
> node -1, gfp=0x900(GFP_NOWAIT|__GFP_ZERO)
> [  718.068636] fmac.local kernel:   cache: page->ptl, object size: 72,
> buffer size: 72, default order: 0, min order: 0
> [  718.068639] fmac.local kernel:   node 0: slabs: 296, objs: 16576, free: 0
> [  718.068704] fmac.local kernel: chronyd: page allocation failure:
> order:0, mode:0x800(GFP_NOWAIT),
> nodemask=(null),cpuset=/,mems_allowed=0
>
> Not sure what to make of that.

Asked on #fedora-kernel, it's a known issue with 5.3.0-rc4 and drm.


-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-15 Thread Chris Murphy

On Thu, Aug 15, 2019 at 1:51 AM Artem Tim  wrote:
>
> BFQ scheduler help a lot with this issue. Using it on Fedora since 4.19 
> kernel. Also there was previous discussion about make it default for 
> Workstation
> https://lists.fedoraproject.org/archives/list/ker...@lists.fedoraproject.org/message/I2OZWDD4QCDYUXJ5NHYTMGNAB4KLJN2K/

It's mentioned in the workstation issue as having no effect in this case.
https://pagure.io/fedora-workstation/issue/98

I just switched to it and repeating the test case and the GUI still
hangs, is unresponsive, even without substantial pressure on the SSD,
and swap isn't even 1/2 used.
https://drive.google.com/open?id=13_5XIBMu01HfOdzGVH-4qTgd-PLpFsaN

But I am getting something new in kernel messages:


542sysrq+t during a GUI freeze that lasted over 1 minute, and then:

[  718.068633] fmac.local kernel: SLUB: Unable to allocate memory on
node -1, gfp=0x900(GFP_NOWAIT|__GFP_ZERO)
[  718.068636] fmac.local kernel:   cache: page->ptl, object size: 72,
buffer size: 72, default order: 0, min order: 0
[  718.068639] fmac.local kernel:   node 0: slabs: 296, objs: 16576, free: 0
[  718.068704] fmac.local kernel: chronyd: page allocation failure:
order:0, mode:0x800(GFP_NOWAIT),
nodemask=(null),cpuset=/,mems_allowed=0

Not sure what to make of that. Complete 'journalctl -k' is here:
https://drive.google.com/open?id=1Z1jAjMrmdXAxuSELdFfd4IKdceeufmVu

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-15 Thread Artem Tim

BFQ scheduler help a lot with this issue. Using it on Fedora since 4.19 kernel. 
Also there was previous discussion about make it default for Workstation
https://lists.fedoraproject.org/archives/list/ker...@lists.fedoraproject.org/message/I2OZWDD4QCDYUXJ5NHYTMGNAB4KLJN2K/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-14 Thread S.


(Oops, sorry, re-post because I messed up the threading.)

I'm not a developer, nor do I pretend to understand the nuances of memory management. But 
I signed up for this list just to say "thanks" to all the devs and others that 
are finally discussing what I consider to be one of the biggest problems with Linux on 
the desktop.

My experience with desktop Linux distros with SSDs when a few processes start 
to leak memory, or if I launch a new program when my system is right at the 
limits, is a full system hang where only the mouse occasionally moves jerkily, 
and I can't switch to a virtual terminal. I recently learned the SysRq trick to 
evoke the OOM killer, but I personally think that the kernel should deal with 
that, not the user. As unfortunate as it is for the OOM killer to have to 
randomly kill something, I am of the opinion that the OS should *never* lock 
up, period. I would strongly prefer that one application get killed instead of 
losing all my applications and working data because of a necessary hard reboot.

I don't know if this helps or not, but anecdotally I started see this issue 
*after* SSDs became more common, i.e. I don't think I ever experienced it with 
spinning rust. Maybe something to do with the vastly faster I/O of an SSD, 
which allows it to more quickly saturate the RAM before the OOM killer has time 
to react?

Also, I've had relatively low memory KVM guests running on a VPS under very high load, 
and they never lockup. The OOM killer does occasionally kick in, but the affected daemon 
or systemd service restarts and it's amazingly undramatic. It appears that this issue 
only occurs with Xorg (and I imagine Wayland) and "desktop" usage.

As for the problem of the randomness of the OOM killer, couldn't it be made to 
take into account the PID and/or how long the process has been running? 
Normally Xorg (and I assume Wayland stuff) gets started before the other 
desktop programs that tend to consume a lot of memory. So if it's a higher PID 
and/or has been running for less time, give it a higher score for killability.

In my experience on a system with 8GB of RAM and an SSD, the amount of swap 
space makes no difference. I've tried with no swap space, with 2GB, with 8GB, 
etc, and it still hangs under high memory usage. I've also tried tuning a lot 
of sysctl parameters such as vm.swappiness, vm.vfs_cache_pressure, and 
vm.min_free_kbytes, to no avail.

Don't know if this helps, but here are some additional discussions of Linux 
unresponsiveness under low memory situations from a layman's perspective:
- osnews.com/story/130117/kde-usability-and-productivity-are-we-there-yet/ (in 
the comments)
- 
unix.stackexchange.com/questions/373312/oom-killer-doesnt-work-properly-leads-to-a-frozen-os
- bbs.archlinux.org/viewtopic.php?id=233843
- 
askubuntu.com/questions/432809/why-is-kswapd0-running-on-a-computer-with-no-swap/432827#432827
- 
unix.stackexchange.com/questions/24625/how-to-completely-disable-swap/24646#24646

Thanks again to everyone for looking into this!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

re: Better interactivity in low-memory situations

2019-08-14 Thread S.

I'm not a developer, nor do I pretend to understand the nuances of memory management. But
I signed up for this list just to say "thanks" to all the devs and others that
are finally discussing what I consider to be one of the biggest problems with Linux on
the desktop.

My experience with desktop Linux distros with SSDs when a few processes start
to leak memory, or if I launch a new program when my system is right at the
limits, is a full system hang where only the mouse occasionally moves jerkily,
and I can't switch to a virtual terminal. I recently learned the SysRq trick to
evoke the OOM killer, but I personally think that the kernel should deal with
that, not the user. As unfortunate as it is for the OOM killer to have to
randomly kill something, I am of the opinion that the OS should *never* lock
up, period. I would strongly prefer that one application get killed instead of
losing all my applications and working data because of a necessary hard reboot.

I don't know if this helps or not, but anecdotally I started see this issue
*after* SSDs became more common, i.e. I don't think I ever experienced it with
spinning rust. Maybe something to do with the vastly faster I/O of an SSD,
which allows it to more quickly saturate the RAM before the OOM killer has time
to react?

Also, I've had relatively low memory KVM guests running on a VPS under very high load,
and they never lockup. The OOM killer does occasionally kick in, but the affected daemon
or systemd service restarts and it's amazingly undramatic. It appears that this issue
only occurs with Xorg (and I imagine Wayland) and "desktop" usage.

As for the problem of the randomness of the OOM killer, couldn't it be made to
take into account the PID and/or how long the process has been running?
Normally Xorg (and I assume Wayland stuff) gets started before the other
desktop programs that tend to consume a lot of memory. So if it's a higher PID
and/or has been running for less time, give it a higher score for killability.

In my experience on a system with 8GB of RAM and an SSD, the amount of swap
space makes no difference. I've tried with no swap space, with 2GB, with 8GB,
etc, and it still hangs under high memory usage. I've also tried tuning a lot
of sysctl parameters such as vm.swappiness, vm.vfs_cache_pressure, and
vm.min_free_kbytes, to no avail.

Don't know if this helps, but here are some additional discussions of Linux
unresponsiveness under low memory situations from a layman's perspective:
- osnews.com/story/130117/kde-usability-and-productivity-are-we-there-yet/ (in
the comments)
-
unix.stackexchange.com/questions/373312/oom-killer-doesnt-work-properly-leads-to-a-frozen-os
- bbs.archlinux.org/viewtopic.php?id=233843
-
askubuntu.com/questions/432809/why-is-kswapd0-running-on-a-computer-with-no-swap/432827#432827
-
unix.stackexchange.com/questions/24625/how-to-completely-disable-swap/24646#24646

Thanks again to everyone for looking into this!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-13 Thread Simon Farnsworth

> On 10 Aug 2019, at 17:56, Georg Sauthoff  wrote:
> 
> On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote:
> [..]
>> Problem and thesis statement:
>> Certain workloads, such as building webkitGTK from source, results in
>> heavy swap usage eventually leading to the system becoming totally
>> unresponsive. Look into switching from disk based swap, to swap on a
>> ZRAM device.
>> 
>> Summary of findings (restated, but basically the same as found at [2]):
>> Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
>> Samsung SSD 840 EVO, Fedora Rawhide Workstation.
>> Test case, build WebKitGTK from source.
> [..]
> 
> To avoid such issues I disable swap on my machines. I really don't see
> the point of having a swap partition if you have 16 or 32 GiB RAM. Even
> with 8 GiB I disable swap.
> 
https://chrisdown.name/2018/01/02/in-defence-of-swap.html 
 is worth a read - 
TL;DR the kernel used (pre 4.0) to be awful about swap, but modern kernels use 
it to avoid paging executable (file-backed) pages in low memory. If any paging 
is needed, lack of swap means that the kernel will page out active code before 
it gets as far as an OOM kill, resulting in a longer time to recover from 
memory contention (regardless of whether there's an OOM kill or the system 
recovers naturally).

Further, a sensible amount of swap (say 2 GiB or so) means that unused 
anonymous pages (e.g. data that's left over from initialization, or data that 
will only be needed when a process exits) can be swapped out and left on disk, 
freeing up valuable RAM for useful work.

Basically, a sane amount of swap is healthy - old advice about large amounts of 
swap is not.

> With - say - 8 GiB the build of a large project might fail (e.g. llvm,
> e.g. during linking) but it then fails fast and I can just restart it
> with `ninja -j2` or something like that.
> 
> Another source of IO related unresponsiveness is buffer bloat - I thus
> apply this configuration on my machines:
> 
>$ cat /etc/sysctl.d/01-disk-bufferbloat.conf
>vm.dirty_background_bytes=107374182
>vm.dirty_bytes=214748364
> 
> Best regards
> Georg
> -- 
> 'Time your programs before making claims about efficiency'
>  (Bjarne Stroustrup, The C++ Programming Language, 4th ed., p. 132, 2013)
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Mon, Aug 12, 2019 at 6:31 PM David Airlie  wrote:
>
> On Sun, Aug 11, 2019 at 2:57 AM Georg Sauthoff  wrote:
> >
> > On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote:
> > [..]
> > > Problem and thesis statement:
> > > Certain workloads, such as building webkitGTK from source, results in
> > > heavy swap usage eventually leading to the system becoming totally
> > > unresponsive. Look into switching from disk based swap, to swap on a
> > > ZRAM device.
> > >
> > > Summary of findings (restated, but basically the same as found at [2]):
> > > Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
> > > Samsung SSD 840 EVO, Fedora Rawhide Workstation.
> > > Test case, build WebKitGTK from source.
> > [..]
> >
> > To avoid such issues I disable swap on my machines. I really don't see
> > the point of having a swap partition if you have 16 or 32 GiB RAM. Even
> > with 8 GiB I disable swap.
>
> Disabling swap doesn't avoid the issues, it can in fact make them worse.
>
> If you have apps allocate memory they don't always OOM before the
> kernel tries to evict text pages, but since SSDs are fast it then
> tries to pull back in those text pages before realising (that is what
> most of the latest rounds of articles has been about). Something like
> firefox runs with no swap, starts to need more memory than the system
> has, parts of firefox executable get paged out, but then are needed
> for firefox to use the RAM, and round in circles it goes.
>
> Having swap is still in this day and age better for your system that
> not having it.

I agree that it's better to have swap for incidental swap purposes,
rather than random things just getting abruptly hit with oom. I say
random, because I see the oom_score_adj is the same for every process
other than systemd-udev, auditd, sshd, and dbus. Plausibly the shell
could get oom killed without warning, taking out the entire user
session, all apps, and all the build processes.

I just discovered in the log from yesterday, that iotop was subject to
oom killer, rather than one of the large cc1plus processes, which is
what I'd previously consistently witnessed. So iotop and cc1plus must
be in the ballpark oom score wise and oom killer just so happens to
pick one or the other. iotop going away relieved just enough memory
that nothing else was subject to oom killer, and yet processes were
clearly resource starved nevertheless: the GUI was frozen, but then
also other processes had already been dying due to timeouts, for
example:

Aug 11 18:26:57 fmac.local systemd[1]: sssd-kcm.service: Control
process exited, code=killed, status=15/TERM
Aug 11 18:26:57 fmac.local systemd[1]: sssd-kcm.service: Failed with
result 'timeout'.

Aug 11 18:27:00 fmac.local systemd[1]: systemd-journald.service: State
'stop-sigterm' timed out. Killing.
Aug 11 18:27:00 fmac.local systemd[1]: systemd-journald.service:
Killing process 31010 (systemd-journal) with signal SIGKILL.
Aug 11 18:27:00 fmac.local systemd[1]: systemd-journald.service: Main
process exited, code=killed, status=9/KILL

This is like a train wreck where there are all sorts of interesting
sub failures happening. At one point I think, well we need better oom
scores so the truly lowest important process is killed off. But upon
big picture scrutiny, the system is failing before oom killer has been
triggered. Processes are dying with timeouts. The GUI including the
mouse pointer is frozen, even when swap is half full. Practically
speaking, it's a goner the moment the mouse pointer froze the very
first time. I might tolerate some stuttering here and there, but
minutes of frozen state? Nah - not interested in seeing if this is
another 5 minutes of choke, or 5 days.

And that's the bad side of swap is when the system is more than
incidentally using it, and is depending on it. And apparently nothing
is on a deadline timer if things can just start timing out on their
own, including the system journal! That was a surprise to see. If it
was that hung up, maybe I can't trust the journal entry times or
order, maybe important entries were lost.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread David Airlie

On Sun, Aug 11, 2019 at 2:57 AM Georg Sauthoff  wrote:
>
> On Fri, Aug 09, 2019 at 03:50:43PM -0600, Chris Murphy wrote:
> [..]
> > Problem and thesis statement:
> > Certain workloads, such as building webkitGTK from source, results in
> > heavy swap usage eventually leading to the system becoming totally
> > unresponsive. Look into switching from disk based swap, to swap on a
> > ZRAM device.
> >
> > Summary of findings (restated, but basically the same as found at [2]):
> > Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
> > Samsung SSD 840 EVO, Fedora Rawhide Workstation.
> > Test case, build WebKitGTK from source.
> [..]
>
> To avoid such issues I disable swap on my machines. I really don't see
> the point of having a swap partition if you have 16 or 32 GiB RAM. Even
> with 8 GiB I disable swap.

Disabling swap doesn't avoid the issues, it can in fact make them worse.

If you have apps allocate memory they don't always OOM before the
kernel tries to evict text pages, but since SSDs are fast it then
tries to pull back in those text pages before realising (that is what
most of the latest rounds of articles has been about). Something like
firefox runs with no swap, starts to need more memory than the system
has, parts of firefox executable get paged out, but then are needed
for firefox to use the RAM, and round in circles it goes.

Having swap is still in this day and age better for your system that
not having it.

Dave.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Mon, Aug 12, 2019 at 11:07 AM Benjamin Kircher
 wrote:
>
> (… I definitely need to play around with Silverblue to learn what they are 
> doing.)

I'm pretty sure Silverblue will be rebased on Fedora CoreOS which
recently released a preview. I'm not sure what the time frame for that
is, but maybe that work will be concurrent with work on a release
version of Fedora CoreOS. The central means of installing/uninstalling
and running applications on a future immutable system is flatpak. But
you don't need to commit a system to Silverblue to use and test
flatpak applications on Fedora 29/30 Workstation. Containerization is
an option not a requirement of flatpaks, as is running it as a systemd
--user instance.

Since layering is permitted with rpm-ostree based systems, using
overlayfs, there still needs to be some way for the per-user service
manager to enforce limits on unprivileged programs. The use of the
word "limit" might be misleading. Perhaps instead it should be on
defining and preserving the user interface responsiveness, whether
that's CLI or GUI, so that control isn't lost. i.e. the unprivileged
program gets the leftover resources, it's not a peer with the user
interface. Promoting the active user interfaces relative to the
unprivileged task would provide a way of effectively containing the
unprivileged tasks, by one always being able to preempt the other.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Lennart Poettering

On Mo, 12.08.19 19:06, Benjamin Kircher (benjamin.kirc...@gmail.com) wrote:

>
>
> > On 12. Aug 2019, at 18:16, Lennart Poettering  wrote:
> >
> > On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote:
> >
> >> How to do this automatically? Could there be a mechanism for the
> >> system and the requesting application to negotiate resources?
> >
> > Ideally, GNOME would run all its apps as systemd --user services. We
> > could then set DefaultMemoryHigh= globally for the systemd --user
> > instance to some percentage value (which is taken relative to the
> > physical RAM size). This would then mean every user app individually
> > could use — let's say — 75% of the physical RAM size and when it wants
> > more it would be penalized during reclaim compared to apps using less.
> >
> > If GNOME would run all apps as user services we could do various other
> > nice things too. For example, it could dynamically assign the fg app
> > more CPU/IO weight than the bg apps, if the system is starved of
> > both.
>
> I really like the ideas. Why isn’t this done this way anyway?

Well, let's just say certain popular container managers blocked
switching to cgroupsv2, and only in cgroupsv2 delegating cgroup
subtrees to unprivileged users is safe. Hence doing this kind of
resource management wasn't really doable without ugly hacks.

But as it appears cgroupsv2 has a chance of becoming a reality on
Fedora now, so this opens a lot of doors.

> I don’t have a GNOME desktop at hand right now to investigate how
> GNOME starts applications and so on but aren’t new processes started
> by the user — GNOME or not — always children of the user.slice? Is
> there a difference if I start a GNOME application or a normal
> process from my shell?

Well, "user.slice" is a concept of the *system* service manager, but
desktop apps are if anything a concept of the *per-user* service
manager.

> And for the beginning, wouldn’t it be enough to differentiate
> between user slices and system slice and set DefaultMemoryHigh= in a
> way to make sure there is always some headroom left for the system?

From the system service manager's PoV all user apps together make up
the user's 'user@.service' instance, it doesn#t look below.

i.e. cgroups is hierarchial, and various components can manage their
own subtrees. PID 1 manages the top of the tree, and the per-user
service manager a subtree of it that is below it and arranges per-user
apps below that. But from PID1's PoV each of those per-user subtrees
is opaque and it won't do resource management beneath that
boundary. It's the job of the per-user service manager to do resource
management there.

Lennart

--
Lennart Poettering, Berlin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Benjamin Kircher



> On 12. Aug 2019, at 18:16, Lennart Poettering  wrote:
> 
> On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote:
> 
>> How to do this automatically? Could there be a mechanism for the
>> system and the requesting application to negotiate resources?
> 
> Ideally, GNOME would run all its apps as systemd --user services. We
> could then set DefaultMemoryHigh= globally for the systemd --user
> instance to some percentage value (which is taken relative to the
> physical RAM size). This would then mean every user app individually
> could use — let's say — 75% of the physical RAM size and when it wants
> more it would be penalized during reclaim compared to apps using less.
> 
> If GNOME would run all apps as user services we could do various other
> nice things too. For example, it could dynamically assign the fg app
> more CPU/IO weight than the bg apps, if the system is starved of
> both.

I really like the ideas. Why isn’t this done this way anyway?

I don’t have a GNOME desktop at hand right now to investigate how GNOME starts 
applications and so on but aren’t new processes started by the user — GNOME or 
not — always children of the user.slice? Is there a difference if I start a 
GNOME application or a normal process from my shell?

And for the beginning, wouldn’t it be enough to differentiate between user 
slices and system slice and set DefaultMemoryHigh= in a way to make sure there 
is always some headroom left for the system?

BK

(… I definitely need to play around with Silverblue to learn what they are 
doing.)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Benjamin Kircher

> On 12. Aug 2019, at 17:40, Chris Murphy  wrote:
> 
> If I just run the example program, let's say systemd MemoryLimit is
> set to /proc/meminfo MemAvailable, the program is still going to try
> and bust out of that and fail. The failure reason is also non-obvious.
> Yes this is definitely an improvement in that the system isn't taken
> down.
> 
> How to do this automatically? Could there be a mechanism for the
> system and the requesting application to negotiate resources?

Honestly, right now, doing this automatically is not possible.

Instead, we anticipate the workload or the nature of the work. Like as when we 
connect remotely to a box and start some long running process, we anticipate 
trouble with the network and use a terminal multiplexer, right? Same thing with 
resource intensive processes.

But in future, I could imagine that this whole control group mechanism really 
pays off in a way where we distribute system resources automatically.

Isn’t that what Silverblue is all about? Having a base system and on top of 
that, everything is run in a container that could be potentially resource 
constraint?

BK
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Lennart Poettering

On Mo, 12.08.19 09:40, Chris Murphy (li...@colorremedies.com) wrote:

> How to do this automatically? Could there be a mechanism for the
> system and the requesting application to negotiate resources?

Ideally, GNOME would run all its apps as systemd --user services. We
could then set DefaultMemoryHigh= globally for the systemd --user
instance to some percentage value (which is taken relative to the
physical RAM size). This would then mean every user app individually
could use — let's say — 75% of the physical RAM size and when it wants
more it would be penalized during reclaim compared to apps using less.

If GNOME would run all apps as user services we could do various other
nice things too. For example, it could dynamically assign the fg app
more CPU/IO weight than the bg apps, if the system is starved of
both.

> Right now the only lever to avoid swap, is to not create a swap
> partition at installation time. Or create a smaller one instead of 1:1
> ratio with RAM. Or use a 1/4 RAM sized swap on ZRAM. A consequence of
> each of these alternatives, is hibernation can't be used. Fedora
> already explicitly does not support hibernation, but strictly that
> means we don't block release on hibernation related bugs. Fedora does
> still create a swap that meets the minimum size for hibernation, and
> also inserts the required 'resume' kernel  parameter to locate the
> hibernation image at the next boot. So we kinda sorta do support it.

We could add a mode to systemd's hibernation support to only "swapon"
a swap partition immediately before hibernating, and "swapoff" it
right after coming back. This has been proposed before, but noone so
far did the work on it. But quite frankly this feels just like taping
over the fact that the Linux kernel is rubbish when it comes to
swapping...

> Another reality is, the example program, also doesn't have a good way
> of estimating the resources it needs. It has some levers, that just
> aren't being used by default, including -l option which reads "do not
> start new jobs if the load average is greater than N". But that's
> different than "tell me the box sizes you can use" and then the system
> supplying a matching box, and for the program to work within it.

As suggested above, I think DefaultMemoryHigh=75% would be an OK
approach which would allow us adjust to the "beefiness" of a machine
automatically.

Lennart

--
Lennart Poettering, Berlin
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Mon, Aug 12, 2019 at 1:01 AM Florian Weimer  wrote:
>
> * Chris Murphy:
>
> > Summary of findings (restated, but basically the same as found at [2]):
> > Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
> > Samsung SSD 840 EVO, Fedora Rawhide Workstation.
>
> Do you use the built-in Intel graphics?  Can you test with something
> else?

Only intel graphics. The AMD GPU on the test system is
non-functional/defective. Other systems only have Intel graphics. I
have tested this in a VM which I think is qxl graphics (?), and I get
the same results, with minimal sample size. It seems like the oom
happens more often and sooner on the VM, but that might because the VM
is necessarily even more resource constrained than the host. But I
have reproduced the total and seemingly indefinite hang. The results
aren't completely deterministic, whether baremetal or VM. They're all
"failures" in one form or another, but how they fail does differ run
to run. And that's expected because to what degree I'm simultaneously
browsing in Firefox, how many tabs are open, other programs being
used, the user is a cause of that non-determinism and is a relevant
factor.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Mon, Aug 12, 2019 at 12:30 AM Benjamin Kircher
 wrote:
>
>
>
> > On 11. Aug 2019, at 23:05, Chris Murphy  wrote:
> >
> > I think the point at which the mouse pointer has frozen, the user has
> > no practical means of controlling or interacting with the system, it's
> > a failure.
> >
> > In the short term, is it reasonable and possible, to get the oom
> > killer to trigger sooner and thereby avoid the system becoming
> > unresponsive in the first place? The oom score for most all processes
> > is 0, and niced processes have their oom score increased. I'm not
> > seeing levers to control how aggressive it is, only a way of hinting
> > at which processes can be more readily subject to being killed. In
> > fact, a requirement of oom killer is that swap is completely consumed,
> > which if swap is on anything other than a fast SSD, swapping creates
> > its own performance problems way before oom can be a rescuer. I think
> > I just argued against my own question.
>
> Yes you just did :-)
>
> From what I understand from this LKML thread [1] fast swap on NVMe is only 
> part of the issue (or adds to the issue). The kernel really really tries hard 
> not to OOM kill anything and keep the system going. And this overcommitment 
> is where it eventually gets unresponsive to the extend that the machine needs 
> to be hard rebooted.
>
> The LKML thread also mentions that user-space OOM handling could help.
>
> But what about cgroups? Isn’t there a systemd utility that helps me wrap 
> processes in resource constrained groups? Something along the line
>
> $ systemd-run -p MemoryLimit=1G firefox
>
> (Not tested.) I imagine that a well-behaved program will handle a bad malloc 
> by ending itself?
>
> BTW, this happens not only on Linux. I’m used to deal with quite big files 
> during my day job and if you accidentally write some… em… very 
> unsophisticated code that attempts to read the entire file into memory at 
> once you can experience the same behavior on a recent macOS, too. You’re left 
> with nothing else than force rebooting your machine.
>
> [1] https://lkml.org/lkml/2019/8/4/15
>

If I just run the example program, let's say systemd MemoryLimit is
set to /proc/meminfo MemAvailable, the program is still going to try
and bust out of that and fail. The failure reason is also non-obvious.
Yes this is definitely an improvement in that the system isn't taken
down.

How to do this automatically? Could there be a mechanism for the
system and the requesting application to negotiate resources?

One reality is, the system isn't a good estimator of system
responsiveness from the user's point of view. Anytime swap is under
significant pressure (what's the definition of significant?) the
system is effectively lost at that point, *if* this is a desktop
system (includes laptops). In the example case, once swap is being
heavily used on either the SSD, or on ZRAM, the mouse pointer is
frozen variably 50%-90% of the time. It's not a usable system, well
before swap is full. How does the system learn that a light swap rate
is OK, but a heavy swap rate will lead to an angry user? And even
heavy swap might be OK on NVMe, or on a server.

Right now the only lever to avoid swap, is to not create a swap
partition at installation time. Or create a smaller one instead of 1:1
ratio with RAM. Or use a 1/4 RAM sized swap on ZRAM. A consequence of
each of these alternatives, is hibernation can't be used. Fedora
already explicitly does not support hibernation, but strictly that
means we don't block release on hibernation related bugs. Fedora does
still create a swap that meets the minimum size for hibernation, and
also inserts the required 'resume' kernel  parameter to locate the
hibernation image at the next boot. So we kinda sorta do support it.

Another reality is, the example program, also doesn't have a good way
of estimating the resources it needs. It has some levers, that just
aren't being used by default, including -l option which reads "do not
start new jobs if the load average is greater than N". But that's
different than "tell me the box sizes you can use" and then the system
supplying a matching box, and for the program to work within it.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Florian Weimer

* Petr Pisar:

> On 2019-08-12, Florian Weimer  wrote:
>> Do you use the built-in Intel graphics?  Can you test with something
>> else?
>>

> Does it have any effect? It happens to me even with a discrete GPU.

I expect that the GEM shrinker (or rather, the reason why it is needed)
radically alters kernel memory management.

Thanks,
Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Petr Pisar

On 2019-08-12, Florian Weimer  wrote:
> Do you use the built-in Intel graphics?  Can you test with something
> else?
>
Does it have any effect? It happens to me even with a discrete GPU.

As far as I know integrated graphics arrays do not share physical memory
from point of view of the CPU address space. The physical memory is
split between GPU and CPU regions and CPU never see the GPU's physical
memory. IOMMU can be asked for mapping GPU's memory into CPU's virtual
space as can be done with any PCI card, but the physical memory is
always separated. (Although it lives in the same memory chip.) Some
BIOSes allows to define the UMA split (ratio beteen GPU and CPU memory).
But that is out of control of an operating system and cannot be change
until reset.

What actually happens is that some CPU physical memory is used for a GUI
program text and some CPU memory for a block device I/O cache. Both
purposes are handled uniformly by Linux. When the physical memory is
exhausted, a memory allocator starts paging to a swap device. The evil
thing is how memory pages are selected to be swapped out. The algorithm
is to swap out the least recently used ones. And that is often the
program text. Not the block cache. As a result your GUI becomes
unresponsive because all the physical memory is filled with a block
cache and the program text has to be reloaded from a block device. And
what's worse, this happens even without swap space because program text
pages are backed by a file and thus can dropped and loaded from a file
system later. I.e. program text is always swapable.

A cure would be more fair memory allocator that could magically
discover that a user is more interested in the few megabytes of his
window manager than the gigabytes of a transfered file. The issue is
that the allocator does not discriminate. A process can actully provide
some hints using madvise(2) and mlock(2), but that does not apply to
the program text, neither to the block cache in the kernel space. And
even if processes provided hints, there always could be some adversarial
program abusing others. Maybe if ulimit were augmented with a block
cache maximal usage and an I/O scheduler accounted for that. That could
help.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-12 Thread Florian Weimer

* Chris Murphy:

> Summary of findings (restated, but basically the same as found at [2]):
> Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
> Samsung SSD 840 EVO, Fedora Rawhide Workstation.

Do you use the built-in Intel graphics?  Can you test with something
else?

Thanks,
Florian
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-11 Thread Benjamin Kircher

> On 11. Aug 2019, at 23:05, Chris Murphy  wrote:
> 
> I think the point at which the mouse pointer has frozen, the user has
> no practical means of controlling or interacting with the system, it's
> a failure.
> 
> In the short term, is it reasonable and possible, to get the oom
> killer to trigger sooner and thereby avoid the system becoming
> unresponsive in the first place? The oom score for most all processes
> is 0, and niced processes have their oom score increased. I'm not
> seeing levers to control how aggressive it is, only a way of hinting
> at which processes can be more readily subject to being killed. In
> fact, a requirement of oom killer is that swap is completely consumed,
> which if swap is on anything other than a fast SSD, swapping creates
> its own performance problems way before oom can be a rescuer. I think
> I just argued against my own question.

Yes you just did :-)

From what I understand from this LKML thread [1] fast swap on NVMe is only part 
of the issue (or adds to the issue). The kernel really really tries hard not to 
OOM kill anything and keep the system going. And this overcommitment is where 
it eventually gets unresponsive to the extend that the machine needs to be hard 
rebooted.

The LKML thread also mentions that user-space OOM handling could help.

But what about cgroups? Isn’t there a systemd utility that helps me wrap 
processes in resource constrained groups? Something along the line

$ systemd-run -p MemoryLimit=1G firefox

(Not tested.) I imagine that a well-behaved program will handle a bad malloc by 
ending itself?

BTW, this happens not only on Linux. I’m used to deal with quite big files 
during my day job and if you accidentally write some… em… very unsophisticated 
code that attempts to read the entire file into memory at once you can 
experience the same behavior on a recent macOS, too. You’re left with nothing 
else than force rebooting your machine.

[1] https://lkml.org/lkml/2019/8/4/15

BK
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 11, 2019 at 1:02 PM Jan Kratochvil
 wrote:
>
> On Sun, 11 Aug 2019 20:54:28 +0200, Chris Murphy wrote:
> > and likely experiences data loss and possibly even file system
> > corruption as a direct consequence of having to force power off on the
> > machine because for all practical purposes normal control has been
> > lost.
>
> Not really, this is what journaling filesystem is there for.

Successful journal replay obviates the need for fsck, it has nothing
to do with avoiding corruption. And in any case, anything the user is
working on that isn't already saved and committed to stable media,
isn't going to survive the poweroff.

> But then there still can be an application-level data corruptions if an
> application does not handle its sudden termination properly.
> Which should be rare but IIRC I did see it for example with Firefox.

I think the point at which the mouse pointer has frozen, the user has
no practical means of controlling or interacting with the system, it's
a failure.

In the short term, is it reasonable and possible, to get the oom
killer to trigger sooner and thereby avoid the system becoming
unresponsive in the first place? The oom score for most all processes
is 0, and niced processes have their oom score increased. I'm not
seeing levers to control how aggressive it is, only a way of hinting
at which processes can be more readily subject to being killed. In
fact, a requirement of oom killer is that swap is completely consumed,
which if swap is on anything other than a fast SSD, swapping creates
its own performance problems way before oom can be a rescuer. I think
I just argued against my own question.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-11 Thread Jan Kratochvil

On Sun, 11 Aug 2019 20:54:28 +0200, Chris Murphy wrote:
> and likely experiences data loss and possibly even file system
> corruption as a direct consequence of having to force power off on the
> machine because for all practical purposes normal control has been
> lost.

Not really, this is what journaling filesystem is there for.

But then there still can be an application-level data corruptions if an
application does not handle its sudden termination properly.
Which should be rare but IIRC I did see it for example with Firefox.


Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 11, 2019 at 10:36 AM  wrote:
>
> On Sun, Aug 11, 2019 at 10:50 AM, Chris Murphy
>  wrote:
> > Let's take another argument. If the user manually specifies 'ninja -j
> > 64' on this same system, is that sabotage? I'd say it is. And
> > therefore why isn't it sabotage that the ninja default computes N jobs
> > as nrcpus + 2?  And also doesn't take available memory into account
> > when deciding what resources to demand? I can build linux all day long
> > on this system with its defaults and never run into a concurrent
> > usability problem.
> >
> > There does seem to be a dual responsibility, somehow, between the
> > operating system and the application, to make sure sane requests are
> > made and honored.
>
> This seems like a distraction from the real goal here, which is to
> ensure Fedora remains responsive under heavy memory pressure, and to
> ensure unprivileged processes cannot take down the system by allocating
> large amounts of memory. Fixing ninja and make to dynamically scale the
> number of parallel build processes based on memory pressure would be
> wonderful, but it's not going to solve the underlying issue here, which
> is that random user processes should never be able to hang the system.

That's fair.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

On Sun, Aug 11, 2019 at 11:21 AM Jan Kratochvil
 wrote:
>
> On Sun, 11 Aug 2019 17:50:17 +0200, Chris Murphy wrote:
> > I don't follow. You're saying RelWithDebInfo is never suitable for a
> > local build?
>
> Most of the time. What is your use case for it?

My use case is testing the responsiveness of Fedora Workstation under
CPU and memory pressure, as experienced by an ordinary user.

> > In file included from Source/JavaScriptCore/config.h:32,
> >  from 
> > Source/JavaScriptCore/llint/LLIntSettingsExtractor.cpp:26:
> > Source/JavaScriptCore/runtime/JSExportMacros.h:32:10: fatal error: 
> > wtf/ExportMacros.h: No such file or directory
>
> You are reinventing the wheel Fedora packager has already done for this
> package.

That's out of scope.

I said from the outset this is an example. The central topic is that
an unprivileged program is able to ask for resources that do not
exist, and the operating system tries and fails to supply those
resources, resulting not only in task failure, but the entire system
is lost. In this example the user is doing other things concurrently
and likely experiences data loss and possibly even file system
corruption as a direct consequence of having to force power off on the
machine because for all practical purposes normal control has been
lost.

> > Let's take another argument. If the user manually specifies 'ninja -j
> > 64' on this same system, is that sabotage?
>
> For untrusted users Linux has given up for that, it is too big can of worms.
> Use virtual machine (KVM) with specified resources (memory size).  Nowadays it
> should be also possible with less overhead by using Docker containers.
>
> If you mean some local builds of your own causing runaway then
> (1) Turn off swap as RAM is cheap enough today.
> If something really runs out of the RAM it gets killed by kernel OOM.
> (2) Have the swap on NVMe, it from my experience does not kill the machine.
> (3) Use some reasonable ulimits in your ~/.bash_profile.
> (4) When the machine is really unresponsible login there from a different box
> and kill the culprits. From my own experience the machine is still able to
> accept new SSH connection, despite a bit slowly.
> But yes, I agree this problem has AFAIK no perfect solution.

I don't think it's acceptable in 2019 that an unpriviledged task takes
out the entire operating system. As I mention in the very first post,
remote ssh was not responsive for 30 minutes, at which point I gave up
and forced power off. It's a bit of a trap though to suggest the user
needs the ability and skill to remote ssh to kill off runaway
programs, I refuse that premise.

It's completely sane for an ordinary user to consider that control of
the system has been lost immediately upon experiencing a frozen mouse
arrow.

-- 
Chris Murphy
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-11 Thread Jan Kratochvil

On Sun, 11 Aug 2019 17:50:17 +0200, Chris Murphy wrote:
> I don't follow. You're saying RelWithDebInfo is never suitable for a
> local build?

Most of the time. What is your use case for it?

> isn't relevant to getting a successful build.

With powerful enough machine everything is possible.  Just be aware
RelWithDebInfo is the most resource demanding option compared to Release and
Debug and at the same time it is the least useful one for local builds.

> In file included from Source/JavaScriptCore/config.h:32,
>  from 
> Source/JavaScriptCore/llint/LLIntSettingsExtractor.cpp:26:
> Source/JavaScriptCore/runtime/JSExportMacros.h:32:10: fatal error: 
> wtf/ExportMacros.h: No such file or directory

You are reinventing the wheel Fedora packager has already done for this
package.  I guess you are missing some dependency.  If you have a problem
stick to the proven build (unless it is temporarily FTBFS which this package
is not now).  I think Fedora recommends mock for such rebuild but I find mock
inconvenient for local development so I use (I have some scripts for that):
dnf download --source webkit2gtk3
mkdir webkit2gtk3-2.24.3-1.fc30.src
cd webkit2gtk3-2.24.3-1.fc30.src
rpm2cpio ../webkit2gtk3-2.24.3-1.fc30.src.rpm|cpio -id
function rpmbuildlocal { time MAKEFLAGS= rpmbuild --define "_topdir 
$PWD" --define "_builddir $PWD" --define "_rpmdir $PWD" --define "_sourcedir 
$PWD" --define "_specdir $PWD" --define "_srcrpmdir $PWD" --define 
"_build_name_fmt %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm" "$@"; rmdir 
&>/dev/null BUILDROOT; }
# Is the .src.rpm rebuild still needed?  
https://bugzilla.redhat.com/show_bug.cgi?id=1210276
rpmbuildlocal -bs *.spec
sudo dnf builddep webkit2gtk3-2.24.3-1.fc30.src.rpm
rm webkit2gtk3-2.24.3-1.fc30.src.rpm
rpmbuildlocal -bc webkit2gtk3.spec 2>&1|tee log
# or -bb or what do you want.
It has built fine for me here now.

> Let's take another argument. If the user manually specifies 'ninja -j
> 64' on this same system, is that sabotage?

For untrusted users Linux has given up for that, it is too big can of worms.
Use virtual machine (KVM) with specified resources (memory size).  Nowadays it
should be also possible with less overhead by using Docker containers.

If you mean some local builds of your own causing runaway then
(1) Turn off swap as RAM is cheap enough today.
If something really runs out of the RAM it gets killed by kernel OOM.
(2) Have the swap on NVMe, it from my experience does not kill the machine.
(3) Use some reasonable ulimits in your ~/.bash_profile.
(4) When the machine is really unresponsible login there from a different box
and kill the culprits. From my own experience the machine is still able to
accept new SSH connection, despite a bit slowly.
But yes, I agree this problem has AFAIK no perfect solution.

Jan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations

2019-08-11 Thread mcatanzaro

On Sun, Aug 11, 2019 at 10:50 AM, Chris Murphy 
 wrote:

Let's take another argument. If the user manually specifies 'ninja -j
64' on this same system, is that sabotage? I'd say it is. And
therefore why isn't it sabotage that the ninja default computes N jobs
as nrcpus + 2?  And also doesn't take available memory into account
when deciding what resources to demand? I can build linux all day long
on this system with its defaults and never run into a concurrent
usability problem.

There does seem to be a dual responsibility, somehow, between the
operating system and the application, to make sure sane requests are
made and honored.


This seems like a distraction from the real goal here, which is to 
ensure Fedora remains responsive under heavy memory pressure, and to 
ensure unprivileged processes cannot take down the system by allocating 
large amounts of memory. Fixing ninja and make to dynamically scale the 
number of parallel build processes based on memory pressure would be 
wonderful, but it's not going to solve the underlying issue here, which 
is that random user processes should never be able to hang the system.


Michael

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Better interactivity in low-memory situations