[PATCH 5.10 148/290] sparc32: Limit memblock allocation to low memory

2021-03-15 Thread gregkh
together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc

[PATCH 5.11 144/306] sparc32: Limit memblock allocation to low memory

2021-03-15 Thread gregkh
together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc

[PATCH 4.19 049/120] sparc32: Limit memblock allocation to low memory

2021-03-15 Thread gregkh
together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc

[PATCH 5.4 075/168] sparc32: Limit memblock allocation to low memory

2021-03-15 Thread gregkh
together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc

Re: [PATCH AUTOSEL 4.14 06/13] sparc32: Limit memblock allocation to low memory

2021-03-12 Thread Sasha Levin
, but also made high memory available via memblock allocation which does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike

Re: [PATCH AUTOSEL 4.4 4/8] sparc32: Limit memblock allocation to low memory

2021-03-03 Thread Andreas Larsson
does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin

Re: [PATCH AUTOSEL 4.9 05/10] sparc32: Limit memblock allocation to low memory

2021-03-03 Thread Andreas Larsson
does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin

Re: [PATCH AUTOSEL 4.14 06/13] sparc32: Limit memblock allocation to low memory

2021-03-03 Thread Andreas Larsson
does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin

[PATCH AUTOSEL 4.4 4/8] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 5.4 16/33] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 4.14 06/13] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 4.9 05/10] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 4.19 09/21] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 5.10 23/47] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH AUTOSEL 5.11 27/52] sparc32: Limit memblock allocation to low memory

2021-03-02 Thread Sasha Levin
and can lead to kernel panic. This changes back to only low memory being allocatable in the early stages, now using memblock allocation. Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport Signed-off-by: David S. Miller Signed-off-by: Sasha Levin --- arch/sparc/mm/init_32.c | 3 +++ 1

[PATCH v2] sparc32: Limit memblock allocation to low memory

2021-02-05 Thread Andreas Larsson
Commit cca079ef8ac29a7c02192d2bad2ffe4c0c5ffdd0 changed sparc32 to use memblocks instead of bootmem, but also made high memory available via memblock allocation which does not work together with e.g. phys_to_virt and can lead to kernel panic. This changes back to only low memory being allocatable

Re: [PATCH] sparc32: Limit memblock allocation to low memory

2021-02-04 Thread Mike Rapoport
d > leads to crashes. > > This changes back to only low memory being allocatable in the early > stages, now using memblock allocation. > > Signed-off-by: Andreas Larsson Acked-by: Mike Rapoport > --- > arch/sparc/mm/init_32.c | 3 +++ > 1 file changed, 3 insertions(+) >

[PATCH] sparc32: Limit memblock allocation to low memory

2021-02-04 Thread Andreas Larsson
Commit cca079ef8ac29a7c02192d2bad2ffe4c0c5ffdd0 changed sparc32 to use memblocks instead of bootmem, but also made high memory available via memblock allocation which does work together with e.g. phys_to_virt and leads to crashes. This changes back to only low memory being allocatable

[PATCH v2] erofs: force inplace I/O under low memory scenario

2020-12-09 Thread Gao Xiang
From: Gao Xiang Try to forcely switch to inplace I/O under low memory scenario in order to avoid direct memory reclaim due to cached page allocation. Reviewed-by: Chao Yu Signed-off-by: Gao Xiang --- v2: refine the gfp definition. fs/erofs/compress.h | 3 +++ fs/erofs/zdata.c| 48

Re: [PATCH] erofs: force inplace I/O under low memory scenario

2020-12-09 Thread Gao Xiang
Hi Chao, On Wed, Dec 09, 2020 at 06:07:08PM +0800, Chao Yu wrote: > On 2020/12/8 13:46, Gao Xiang wrote: ... > > bool standalone = true; > > + gfp_t gfp = mapping_gfp_constraint(mc, GFP_KERNEL) & > > ~__GFP_DIRECT_RECLAIM; > > Could be local as there is only one place uses it. This

Re: [PATCH] erofs: force inplace I/O under low memory scenario

2020-12-09 Thread Chao Yu
On 2020/12/8 13:46, Gao Xiang wrote: From: Gao Xiang Try to forcely switch to inplace I/O under low memory scenario in order to avoid direct memory reclaim due to cached page allocation. Signed-off-by: Gao Xiang --- This was commercially used internally for years, but due to customized page

[PATCH] erofs: force inplace I/O under low memory scenario

2020-12-07 Thread Gao Xiang
From: Gao Xiang Try to forcely switch to inplace I/O under low memory scenario in order to avoid direct memory reclaim due to cached page allocation. Signed-off-by: Gao Xiang --- This was commercially used internally for years, but due to customized page->mapping before, it cannot clea

Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-06-19 Thread chenzhou
Zhou wrote: >>>>> Add documentation for DT property used by arm64 kdump: >>>>> linux,low-memory-range. >>>>> "linux,low-memory-range" is an another memory region used for crash >>>>> dump kernel devices. >>>>> di

Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-05-29 Thread James Morse
Hi guys, On 26/05/2020 22:18, Rob Herring wrote: > On Fri, May 22, 2020 at 11:24:11AM +0800, chenzhou wrote: >> On 2020/5/21 21:29, Rob Herring wrote: >>> On Thu, May 21, 2020 at 3:35 AM Chen Zhou wrote: >>>> Add documentation for DT property used by arm64 kdum

Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-05-26 Thread Rob Herring
m64 kdump: > >> linux,low-memory-range. > >> "linux,low-memory-range" is an another memory region used for crash > >> dump kernel devices. > >> > >> Signed-off-by: Chen Zhou > >> --- > >> Documentation/devicetree/bindings

Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-05-21 Thread chenzhou
Hi Rob, On 2020/5/21 21:29, Rob Herring wrote: > On Thu, May 21, 2020 at 3:35 AM Chen Zhou wrote: >> Add documentation for DT property used by arm64 kdump: >> linux,low-memory-range. >> "linux,low-memory-range" is an another memory region used for crash >>

Re: [PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-05-21 Thread Rob Herring
On Thu, May 21, 2020 at 3:35 AM Chen Zhou wrote: > > Add documentation for DT property used by arm64 kdump: > linux,low-memory-range. > "linux,low-memory-range" is an another memory region used for crash > dump kernel devices. > > Signed-off-by: Chen Zhou &g

[PATCH v8 5/5] dt-bindings: chosen: Document linux,low-memory-range for arm64 kdump

2020-05-21 Thread Chen Zhou
Add documentation for DT property used by arm64 kdump: linux,low-memory-range. "linux,low-memory-range" is an another memory region used for crash dump kernel devices. Signed-off-by: Chen Zhou --- Documentation/devicetree/bindings/chosen.txt | 25 1 file c

[PATCH v8 3/5] arm64: kdump: add memory for devices by DT property, low-memory-range

2020-05-21 Thread Chen Zhou
If we want to reserve crashkernel above 4G, we could use parameters "crashkernel=X crashkernel=Y,low", in this case, specified size low memory is reserved for crash dump kernel devices and never mapped by the first kernel. This memory range is advertised to crash dump kernel via DT prop

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-09-02 Thread Pavel Machek
Hi! > > > > And if there is a meaningful way to make the kernel behave better, that > > would > > obviously be of huge value too. > > > > Thanks > > Daniel > > Hi, > > Is there a way for an application to be told that there is a memory > pressure situation? > For example, say I do a "make

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-29 Thread Michal Hocko
On Wed 21-08-19 22:42:29, James Courtier-Dutton wrote: > On Tue, 20 Aug 2019 at 07:47, Daniel Drake wrote: > > > > Hi, > > > > And if there is a meaningful way to make the kernel behave better, that > > would > > obviously be of huge value too. > > > > Thanks > > Daniel > > Hi, > > Is there a

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-22 Thread Daniel Drake
On Fri, Aug 23, 2019 at 9:54 AM ndrw wrote: > That's obviously a lot better than hard freezes but I wouldn't call such > system lock-ups an excellent result. PSI-triggered OOM killer would have > indeed been very useful as an emergency brake, and IMHO such mechanism > should be built in the

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-22 Thread ndrw
On 20/08/2019 07:46, Daniel Drake wrote: To share our results so far, despite this daemon being a quick initial implementation, we find that it is bringing excellent results, no more memory pressure hangs. The system recovers in less than 30 seconds, usually in more like 10-15 seconds. That's

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-21 Thread James Courtier-Dutton
On Tue, 20 Aug 2019 at 07:47, Daniel Drake wrote: > > Hi, > > And if there is a meaningful way to make the kernel behave better, that would > obviously be of huge value too. > > Thanks > Daniel Hi, Is there a way for an application to be told that there is a memory pressure situation? For

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-20 Thread Daniel Drake
loyed as a standard part of the Linux desktop. https://gitlab.freedesktop.org/hadess/low-memory-monitor/ And if there is a meaningful way to make the kernel behave better, that would obviously be of huge value too. Thanks Daniel

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-13 Thread Vlastimil Babka
On 8/9/19 7:31 PM, Johannes Weiner wrote: >> It made a difference, but not enough, it seems. Before the patch I could >> observe "io:full avg10" around 75% and "memory:full avg10" around 20%, >> after the patch, "memory:full avg10" went to around 45%, while io stayed >> the same (BTW should the

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-12 Thread Michal Hocko
On Sat 10-08-19 13:34:06, ndrw wrote: > On 09/08/2019 11:50, Michal Hocko wrote: > > We try to protect low amount of cache. Have a look at get_scan_count > > function. But the exact amount of the cache to be protected is really > > hard to know wihtout a crystal ball or understanding of the

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-10 Thread ndrw
On 09/08/2019 09:57, Michal Hocko wrote: This is a useful feedback! What was your workload? Which kernel version? With 16GB zram swap and swappiness=60 I get the avg10 memory PSI numbers of about 10 when swap is half filled and ~30 immediately before the freeze. Swapping with zram has less

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-10 Thread ndrw
On 09/08/2019 11:50, Michal Hocko wrote: We try to protect low amount of cache. Have a look at get_scan_count function. But the exact amount of the cache to be protected is really hard to know wihtout a crystal ball or understanding of the workload. The kernel doesn't have neither of the two.

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread Johannes Weiner
On Fri, Aug 09, 2019 at 04:56:28PM +0200, Vlastimil Babka wrote: > On 8/8/19 7:27 PM, Johannes Weiner wrote: > > On Thu, Aug 08, 2019 at 04:47:18PM +0200, Vlastimil Babka wrote: > >> On 8/7/19 10:51 PM, Johannes Weiner wrote: > >>> From 9efda85451062dea4ea287a886e515efefeb1545 Mon Sep 17 00:00:00

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread Vlastimil Babka
On 8/8/19 7:27 PM, Johannes Weiner wrote: > On Thu, Aug 08, 2019 at 04:47:18PM +0200, Vlastimil Babka wrote: >> On 8/7/19 10:51 PM, Johannes Weiner wrote: >>> From 9efda85451062dea4ea287a886e515efefeb1545 Mon Sep 17 00:00:00 2001 >>> From: Johannes Weiner >>> Date: Mon, 5 Aug 2019 13:15:16 -0400

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread Pintu Agarwal
[...] Hi, This is an interesting topic for me so I would like to join the conversation. I will be glad if I can be of any help here either in testing PSI, or verifying some scenarios and observation. I have some experience working with low memory embedded devices, like RAM as low as 128MB

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread Michal Hocko
On Fri 09-08-19 11:09:33, ndrw wrote: > On 09/08/2019 09:57, Michal Hocko wrote: > > We already do have a reserve (min_free_kbytes). That gives kswapd some > > room to perform reclaim in the background without obvious latencies to > > allocating tasks (well CPU still be used so there is still some

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread ndrw
On 09/08/2019 09:57, Michal Hocko wrote: We already do have a reserve (min_free_kbytes). That gives kswapd some room to perform reclaim in the background without obvious latencies to allocating tasks (well CPU still be used so there is still some effect). I tried this option in the past.

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-09 Thread Michal Hocko
On Thu 08-08-19 22:59:32, ndrw wrote: > On 08/08/2019 19:59, Michal Hocko wrote: > > Well, I am afraid that implementing anything like that in the kernel > > will lead to many regressions and bug reports. People tend to have very > > different opinions on when it is suitable to kill a potentially

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread ndrw
On 08/08/2019 19:59, Michal Hocko wrote: Well, I am afraid that implementing anything like that in the kernel will lead to many regressions and bug reports. People tend to have very different opinions on when it is suitable to kill a potentially important part of a workload just because memory

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread Michal Hocko
On Thu 08-08-19 18:57:02, ndrw...@redhazel.co.uk wrote: > > > On 8 August 2019 17:32:28 BST, Michal Hocko wrote: > > > >> Would it be possible to reserve a fixed (configurable) amount of RAM > >for caches, > > > >I am afraid there is nothing like that available and I would even argue > >it

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread ndrw . xf
On 8 August 2019 17:32:28 BST, Michal Hocko wrote: > >> Would it be possible to reserve a fixed (configurable) amount of RAM >for caches, > >I am afraid there is nothing like that available and I would even argue >it doesn't make much sense either. What would you consider to be a >cache? A

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread Johannes Weiner
On Thu, Aug 08, 2019 at 04:47:18PM +0200, Vlastimil Babka wrote: > On 8/7/19 10:51 PM, Johannes Weiner wrote: > > From 9efda85451062dea4ea287a886e515efefeb1545 Mon Sep 17 00:00:00 2001 > > From: Johannes Weiner > > Date: Mon, 5 Aug 2019 13:15:16 -0400 > > Subject: [PATCH] psi: trigger the OOM

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread Michal Hocko
On Thu 08-08-19 16:10:07, ndrw...@redhazel.co.uk wrote: > > > On 8 August 2019 12:48:26 BST, Michal Hocko wrote: > >> > >> Per default, the OOM killer will engage after 15 seconds of at least > >> 80% memory pressure. These values are tunable via sysctls > >> vm.thrashing_oom_period and

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread ndrw . xf
On 8 August 2019 12:48:26 BST, Michal Hocko wrote: >> >> Per default, the OOM killer will engage after 15 seconds of at least >> 80% memory pressure. These values are tunable via sysctls >> vm.thrashing_oom_period and vm.thrashing_oom_level. > >As I've said earlier I would be somehow more

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread Vlastimil Babka
On 8/7/19 10:51 PM, Johannes Weiner wrote: > From 9efda85451062dea4ea287a886e515efefeb1545 Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Mon, 5 Aug 2019 13:15:16 -0400 > Subject: [PATCH] psi: trigger the OOM killer on severe thrashing Thanks a lot, perhaps finally we are going to eat

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-08 Thread Michal Hocko
On Wed 07-08-19 16:51:38, Johannes Weiner wrote: [...] > >From 9efda85451062dea4ea287a886e515efefeb1545 Mon Sep 17 00:00:00 2001 > From: Johannes Weiner > Date: Mon, 5 Aug 2019 13:15:16 -0400 > Subject: [PATCH] psi: trigger the OOM killer on severe thrashing > > Over the last few years we have

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-07 Thread Johannes Weiner
On Wed, Aug 07, 2019 at 02:01:30PM -0700, Andrew Morton wrote: > On Wed, 7 Aug 2019 16:51:38 -0400 Johannes Weiner wrote: > > > However, eb414681d5a0 ("psi: pressure stall information for CPU, > > memory, and IO") introduced a memory pressure metric that quantifies > > the share of wallclock

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-07 Thread Johannes Weiner
On Wed, Aug 07, 2019 at 04:51:42PM -0400, Johannes Weiner wrote: > Per default, the OOM killer will engage after 15 seconds of at least > 80% memory pressure. These values are tunable via sysctls > vm.thrashing_oom_period and vm.thrashing_oom_level. Let's go with this: Per default, the OOM

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-07 Thread Andrew Morton
On Wed, 7 Aug 2019 16:51:38 -0400 Johannes Weiner wrote: > However, eb414681d5a0 ("psi: pressure stall information for CPU, > memory, and IO") introduced a memory pressure metric that quantifies > the share of wallclock time in which userspace waits on reclaim, > refaults, swapins. By using

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-07 Thread Johannes Weiner
On Wed, Aug 07, 2019 at 09:59:27AM +0200, Michal Hocko wrote: > On Tue 06-08-19 18:01:50, Johannes Weiner wrote: > > On Tue, Aug 06, 2019 at 09:27:05AM -0700, Suren Baghdasaryan wrote: > [...] > > > > > I'm not sure 10s is the perfect value here, but I do think the kernel > > > > > should try to

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-07 Thread Michal Hocko
On Tue 06-08-19 18:01:50, Johannes Weiner wrote: > On Tue, Aug 06, 2019 at 09:27:05AM -0700, Suren Baghdasaryan wrote: [...] > > > > I'm not sure 10s is the perfect value here, but I do think the kernel > > > > should try to get out of such a state, where interacting with the > > > > system is

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Johannes Weiner
On Tue, Aug 06, 2019 at 09:27:05AM -0700, Suren Baghdasaryan wrote: > On Tue, Aug 6, 2019 at 7:36 AM Michal Hocko wrote: > > > > On Tue 06-08-19 10:27:28, Johannes Weiner wrote: > > > On Tue, Aug 06, 2019 at 11:36:48AM +0200, Vlastimil Babka wrote: > > > > On 8/6/19 3:08 AM, Suren Baghdasaryan

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread James Courtier-Dutton
On Tue, 6 Aug 2019 at 02:09, Suren Baghdasaryan wrote: > > 80% of the last 10 seconds spent in full stall would definitely be a > problem. If the system was already low on memory (which it probably > is, or we would not be reclaiming so hard and registering such a big > stall) then oom-killer

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Remi Gauvin
Sorry, I don't have the original message to reply to.. But to those interested, I have found a solution to the kernel's complete inability to allocate more memory when it needs to swap out. Increase the /proc/sys/vm/watermark_scale_factor from the default 10 to 500 It will make a huge

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Florian Weimer
* Artem S. Tashkinov: > There's this bug which has been bugging many people for many years > already and which is reproducible in less than a few minutes under the > latest and greatest kernel, 5.2.6. All the kernel parameters are set to > defaults. > > Steps to reproduce: > > 1) Boot with mem=4G

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Suren Baghdasaryan
On Tue, Aug 6, 2019 at 7:36 AM Michal Hocko wrote: > > On Tue 06-08-19 10:27:28, Johannes Weiner wrote: > > On Tue, Aug 06, 2019 at 11:36:48AM +0200, Vlastimil Babka wrote: > > > On 8/6/19 3:08 AM, Suren Baghdasaryan wrote: > > > >> @@ -1280,3 +1285,50 @@ static int __init psi_proc_init(void) > >

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Michal Hocko
On Tue 06-08-19 10:27:28, Johannes Weiner wrote: > On Tue, Aug 06, 2019 at 11:36:48AM +0200, Vlastimil Babka wrote: > > On 8/6/19 3:08 AM, Suren Baghdasaryan wrote: > > >> @@ -1280,3 +1285,50 @@ static int __init psi_proc_init(void) > > >> return 0; > > >> } > > >>

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Johannes Weiner
On Tue, Aug 06, 2019 at 11:36:48AM +0200, Vlastimil Babka wrote: > On 8/6/19 3:08 AM, Suren Baghdasaryan wrote: > >> @@ -1280,3 +1285,50 @@ static int __init psi_proc_init(void) > >> return 0; > >> } > >> module_init(psi_proc_init); > >> + > >> +#define OOM_PRESSURE_LEVEL 80 > >>

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Vlastimil Babka
On 8/6/19 3:08 AM, Suren Baghdasaryan wrote: >> @@ -1280,3 +1285,50 @@ static int __init psi_proc_init(void) >> return 0; >> } >> module_init(psi_proc_init); >> + >> +#define OOM_PRESSURE_LEVEL 80 >> +#define OOM_PRESSURE_PERIOD(10 * NSEC_PER_SEC) > > 80% of the last 10 seconds

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Michal Hocko
On Mon 05-08-19 14:55:42, Johannes Weiner wrote: > On Mon, Aug 05, 2019 at 03:31:19PM +0200, Michal Hocko wrote: > > On Mon 05-08-19 14:13:16, Vlastimil Babka wrote: > > > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > > > Hello, > > > > > > > > There's this bug which has been bugging many

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-06 Thread Johannes Buchner
> On Mon, Aug 5, 2019 at 12:31 PM Johannes Weiner wrote: >> >> On Mon, Aug 05, 2019 at 02:13:16PM +0200, Vlastimil Babka wrote: >> > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: >> > > Hello, >> > > >> > > There's this bug which has been bugging many people for many years >> > > already and

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Suren Baghdasaryan
On Mon, Aug 5, 2019 at 12:31 PM Johannes Weiner wrote: > > On Mon, Aug 05, 2019 at 02:13:16PM +0200, Vlastimil Babka wrote: > > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > > Hello, > > > > > > There's this bug which has been bugging many people for many years > > > already and which is

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Johannes Weiner
On Mon, Aug 05, 2019 at 02:13:16PM +0200, Vlastimil Babka wrote: > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > Hello, > > > > There's this bug which has been bugging many people for many years > > already and which is reproducible in less than a few minutes under the > > latest and greatest

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Johannes Weiner
On Mon, Aug 05, 2019 at 03:31:19PM +0200, Michal Hocko wrote: > On Mon 05-08-19 14:13:16, Vlastimil Babka wrote: > > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > > Hello, > > > > > > There's this bug which has been bugging many people for many years > > > already and which is reproducible in

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Suren Baghdasaryan
On Mon, Aug 5, 2019 at 6:31 AM Michal Hocko wrote: > > On Mon 05-08-19 14:13:16, Vlastimil Babka wrote: > > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > > Hello, > > > > > > There's this bug which has been bugging many people for many years > > > already and which is reproducible in less

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Michal Hocko
On Mon 05-08-19 14:13:16, Vlastimil Babka wrote: > On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > > Hello, > > > > There's this bug which has been bugging many people for many years > > already and which is reproducible in less than a few minutes under the > > latest and greatest kernel, 5.2.6.

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Vlastimil Babka
On 8/4/19 11:23 AM, Artem S. Tashkinov wrote: > Hello, > > There's this bug which has been bugging many people for many years > already and which is reproducible in less than a few minutes under the > latest and greatest kernel, 5.2.6. All the kernel parameters are set to > defaults. > > Steps

Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-05 Thread Artem S. Tashkinov
On 8/5/19 9:05 AM, Hillf Danton wrote: On Sun, 4 Aug 2019 09:23:17 + "Artem S. Tashkinov" wrote: Hello, There's this bug which has been bugging many people for many years already and which is reproducible in less than a few minutes under the latest and greatest kernel, 5.2.6. All the

Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

2019-08-04 Thread Artem S. Tashkinov
Hello, There's this bug which has been bugging many people for many years already and which is reproducible in less than a few minutes under the latest and greatest kernel, 5.2.6. All the kernel parameters are set to defaults. Steps to reproduce: 1) Boot with mem=4G 2) Disable swap to make

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-16 Thread Oleg Nesterov
On 05/15, Sultan Alsawaf wrote: > > On Wed, May 15, 2019 at 04:58:32PM +0200, Oleg Nesterov wrote: > > Could you explain in detail what exactly did you do and what do you see in > > dmesg? > > > > Just in case, lockdep complains only once, print_circular_bug() does > > debug_locks_off() > > so

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-15 Thread Steven Rostedt
On Wed, 15 May 2019 11:52:57 -0700 Sultan Alsawaf wrote: > On Wed, May 15, 2019 at 02:32:48PM -0400, Steven Rostedt wrote: > > I'm confused why you did this? > > Oleg said that debug_locks_off() could've been called and thus prevented > lockdep complaints about simple_lmk from appearing. To

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-15 Thread Sultan Alsawaf
On Wed, May 15, 2019 at 02:32:48PM -0400, Steven Rostedt wrote: > I'm confused why you did this? Oleg said that debug_locks_off() could've been called and thus prevented lockdep complaints about simple_lmk from appearing. To eliminate any possibility of that, I disabled debug_locks_off(). Oleg

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-15 Thread Steven Rostedt
On Wed, 15 May 2019 10:27:28 -0700 Sultan Alsawaf wrote: > On Wed, May 15, 2019 at 04:58:32PM +0200, Oleg Nesterov wrote: > > Could you explain in detail what exactly did you do and what do you see in > > dmesg? > > > > Just in case, lockdep complains only once, print_circular_bug() does > >

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-15 Thread Oleg Nesterov
On 05/13, Sultan Alsawaf wrote: > > On Fri, May 10, 2019 at 05:10:25PM +0200, Oleg Nesterov wrote: > > I am starting to think I am ;) > > > > If you have task1 != task2 this code > > > > task_lock(task1); > > task_lock(task2); > > > > should trigger print_deadlock_bug(), task1->alloc_lock

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-14 Thread Sultan Alsawaf
On Tue, May 14, 2019 at 12:44:53PM -0400, Steven Rostedt wrote: > OK, this has gotten my attention. > > This thread is quite long, do you have a git repo I can look at, and > also where is the first task_lock() taken before the > find_lock_task_mm()? > > -- Steve Hi Steve, This is the git repo

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-14 Thread Steven Rostedt
On Mon, 13 May 2019 09:45:55 -0700 Sultan Alsawaf wrote: > On Fri, May 10, 2019 at 05:10:25PM +0200, Oleg Nesterov wrote: > > I am starting to think I am ;) > > > > If you have task1 != task2 this code > > > > task_lock(task1); > > task_lock(task2); > > > > should trigger

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-13 Thread Sultan Alsawaf
On Fri, May 10, 2019 at 05:10:25PM +0200, Oleg Nesterov wrote: > I am starting to think I am ;) > > If you have task1 != task2 this code > > task_lock(task1); > task_lock(task2); > > should trigger print_deadlock_bug(), task1->alloc_lock and task2->alloc_lock > are > the "same"

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-10 Thread Oleg Nesterov
On 05/09, Sultan Alsawaf wrote: > > On Thu, May 09, 2019 at 05:56:46PM +0200, Oleg Nesterov wrote: > > Impossible ;) I bet lockdep should report the deadlock as soon as > > find_victims() > > calls find_lock_task_mm() when you already have a locked victim. > > I hope you're not a betting man ;)

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-09 Thread Sultan Alsawaf
On Thu, May 09, 2019 at 05:56:46PM +0200, Oleg Nesterov wrote: > Impossible ;) I bet lockdep should report the deadlock as soon as > find_victims() > calls find_lock_task_mm() when you already have a locked victim. I hope you're not a betting man ;) With the following configured:

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-09 Thread Oleg Nesterov
On 05/07, Sultan Alsawaf wrote: > > On Tue, May 07, 2019 at 05:31:54PM +0200, Oleg Nesterov wrote: > > > Did you test this patch with lockdep enabled? > > > > If I read the patch correctly, lockdep should complain. vtsk_is_duplicate() > > ensures that we do not take the same ->alloc_lock twice or

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Suren Baghdasaryan
From: Sultan Alsawaf Date: Tue, May 7, 2019 at 9:53 AM To: Suren Baghdasaryan Cc: Christian Brauner, Greg Kroah-Hartman, open list:ANDROID DRIVERS, Daniel Colascione, Todd Kjos, Kees Cook, Peter Zijlstra, Martijn Coenen, LKML, Tim Murray, Michal Hocko, linux-mm, Arve Hjønnevåg, Ingo Molnar,

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Joel Fernandes
t; > feasible. One of the things that I have been working on for quite a > > while is the whole file descriptor for processes thing that is important > > for LMKD (Even though I never thought about this use-case when I started > > pitching this.). Joel and Daniel have joined in an

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Greg Kroah-Hartman
On Tue, May 07, 2019 at 10:17:11AM -0700, Sultan Alsawaf wrote: > On Tue, May 07, 2019 at 01:09:21PM +0200, Greg Kroah-Hartman wrote: > > > It's even more odd that although a userspace solution is touted as the > > > proper > > > way to go on LKML, almost no Android OEMs are using it, and even in

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Sultan Alsawaf
r LMKD (Even though I never thought about this use-case when I started > pitching this.). Joel and Daniel have joined in and are working on > making LMKD possible. > What I find odd is that every couple of weeks different solutions to the > low memory problem are pitched. There is si

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Sultan Alsawaf
On Tue, May 07, 2019 at 09:28:47AM -0700, Suren Baghdasaryan wrote: > Hi Sultan, > Looks like you are posting this patch for devices that do not use > userspace LMKD solution due to them using older kernels or due to > their vendors sticking to in-kernel solution. If so, I see couple > logistical

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Christian Brauner
e solution and it is a > process to address all concerns but we are getting there. > > > > Regardless, even if PSI were backported, a full-fledged LMKD using it has > > > yet to > > > be made, so it wouldn't be of much use now. > > > > This i

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Sultan Alsawaf
On Tue, May 07, 2019 at 05:31:54PM +0200, Oleg Nesterov wrote: > I am not going to comment the intent, but to be honest I am skeptical too. The general sentiment has been that this is a really bad idea, but I'm just a frustrated Android user who wants his phone to not require mountains of zRAM

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Suren Baghdasaryan
; > yet to > > be made, so it wouldn't be of much use now. > > This is work that is ongoing and requires kernel changes to make it > feasible. One of the things that I have been working on for quite a > while is the whole file descriptor for processes thing that is importa

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Oleg Nesterov
I am not going to comment the intent, but to be honest I am skeptical too. On 05/06, Sultan Alsawaf wrote: > > +static unsigned long find_victims(struct victim_info *varr, int *vindex, > + int vmaxlen, int min_adj, int max_adj) > +{ > + unsigned long pages_found

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Michal Hocko
On Mon 06-05-19 19:16:22, Sultan Alsawaf wrote: > This is a complete low memory killer solution for Android that is small > and simple. Processes are killed according to the priorities that > Android gives them, so that the least important processes are always > killed first. Processe

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Greg Kroah-Hartman
On Tue, May 07, 2019 at 01:12:36AM -0700, Sultan Alsawaf wrote: > On Tue, May 07, 2019 at 09:43:34AM +0200, Greg Kroah-Hartman wrote: > > Given that any "new" android device that gets shipped "soon" should be > > using 4.9.y or newer, is this a real issue? > > It's certainly a real issue for

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Christian Brauner
ver thought about this use-case when I started pitching this.). Joel and Daniel have joined in and are working on making LMKD possible. What I find odd is that every couple of weeks different solutions to the low memory problem are pitched. There is simple_lkml, there is LMKD, and there was a patc

Re: [RFC] simple_lmk: Introduce Simple Low Memory Killer for Android

2019-05-07 Thread Sultan Alsawaf
On Tue, May 07, 2019 at 09:43:34AM +0200, Greg Kroah-Hartman wrote: > Given that any "new" android device that gets shipped "soon" should be > using 4.9.y or newer, is this a real issue? It's certainly a real issue for those who can't buy brand new Android devices without software bugs every six

  1   2   3   4   5   6   7   >