Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Fri 02-10-20 09:53:05, Rik van Riel wrote: > On Fri, 2020-10-02 at 09:03 +0200, Michal Hocko wrote: > > On Thu 01-10-20 18:18:10, Sebastiaan Meijer wrote: > > > (Apologies for messing up the mailing list thread, Gmail had fooled > > > me into > > > believing that it properly picked up the

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Fri, Oct 02, 2020 at 09:53:05AM -0400, Rik van Riel wrote: > On Fri, 2020-10-02 at 09:03 +0200, Michal Hocko wrote: > > On Thu 01-10-20 18:18:10, Sebastiaan Meijer wrote: > > > (Apologies for messing up the mailing list thread, Gmail had fooled > > > me into > > > believing that it properly

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Fri, 2020-10-02 at 09:03 +0200, Michal Hocko wrote: > On Thu 01-10-20 18:18:10, Sebastiaan Meijer wrote: > > (Apologies for messing up the mailing list thread, Gmail had fooled > > me into > > believing that it properly picked up the thread) > > > > On Thu, 1 Oct 2020 at 14:30, Michal Hocko

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Fri, Oct 02, 2020 at 09:03:33AM +0200, Michal Hocko wrote: > > > My recollection of the particular patch is dimm but I do remember it > > > tried to add more kswapd threads which would just paper over the problem > > > you are seein rather than solve it. > > > > Yeah, that's exactly what it

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Thu 01-10-20 18:18:10, Sebastiaan Meijer wrote: > (Apologies for messing up the mailing list thread, Gmail had fooled me into > believing that it properly picked up the thread) > > On Thu, 1 Oct 2020 at 14:30, Michal Hocko wrote: > > > > On Wed 30-09-20 21:27:12, Sebastiaan Meijer wrote: > >

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

(Apologies for messing up the mailing list thread, Gmail had fooled me into believing that it properly picked up the thread) On Thu, 1 Oct 2020 at 14:30, Michal Hocko wrote: > > On Wed 30-09-20 21:27:12, Sebastiaan Meijer wrote: > > > yes it shows the bottleneck but it is quite artificial. Read

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Wed 30-09-20 21:27:12, Sebastiaan Meijer wrote: > > yes it shows the bottleneck but it is quite artificial. Read data is > > usually processed and/or written back and that changes the picture a > > lot. > Apologies for reviving an ancient thread (and apologies in advance for my lack > of

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> yes it shows the bottleneck but it is quite artificial. Read data is > usually processed and/or written back and that changes the picture a > lot. Apologies for reviving an ancient thread (and apologies in advance for my lack of knowledge on how mailing lists work), but I'd like to offer up

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Mon 16-04-18 20:02:22, Buddy Lumpkin wrote: > > > On Apr 12, 2018, at 6:16 AM, Michal Hocko wrote: [...] > > But once you hit a wall with > > hard-to-reclaim pages then I would expect multiple threads will simply > > contend more (e.g. on fs locks in shrinkers etc…). > >

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Mon 16-04-18 20:02:22, Buddy Lumpkin wrote: > > > On Apr 12, 2018, at 6:16 AM, Michal Hocko wrote: [...] > > But once you hit a wall with > > hard-to-reclaim pages then I would expect multiple threads will simply > > contend more (e.g. on fs locks in shrinkers etc…). > > If that is the case,

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 12, 2018, at 6:16 AM, Michal Hocko wrote: > > On Tue 03-04-18 12:41:56, Buddy Lumpkin wrote: >> >>> On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: >>> >>> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: Page replacement is handled in the

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 12, 2018, at 6:16 AM, Michal Hocko wrote: > > On Tue 03-04-18 12:41:56, Buddy Lumpkin wrote: >> >>> On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: >>> >>> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: Page replacement is handled in the Linux Kernel in one of two ways:

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue 10-04-18 20:10:24, Buddy Lumpkin wrote: [...] > > Also please note that the direct reclaim is a way to throttle overly > > aggressive memory consumers. The more we do in the background context > > the easier for them it will be to allocate faster. So I am not really > > sure that more

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue 10-04-18 20:10:24, Buddy Lumpkin wrote: [...] > > Also please note that the direct reclaim is a way to throttle overly > > aggressive memory consumers. The more we do in the background context > > the easier for them it will be to allocate faster. So I am not really > > sure that more

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue 03-04-18 12:41:56, Buddy Lumpkin wrote: > > > On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > >> Page replacement is handled in the Linux Kernel in one of two ways: > >> > >> 1) Asynchronously via kswapd > >> 2)

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue 03-04-18 12:41:56, Buddy Lumpkin wrote: > > > On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > >> Page replacement is handled in the Linux Kernel in one of two ways: > >> > >> 1) Asynchronously via kswapd > >> 2) Synchronously, via

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>>

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>> makes sense to use as

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 12:07 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: >> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >>> The presence of direct reclaims 10 years ago was a fairly reliable >>> indicator that too much was

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 12:07 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: >> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >>> The presence of direct reclaims 10 years ago was a fairly reliable >>> indicator that too much was being asked of a Linux

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >> Page replacement is handled in the Linux Kernel in one of two ways: >> >> 1) Asynchronously via kswapd >> 2) Synchronously, via direct reclaim >> >> At page allocation

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >> Page replacement is handled in the Linux Kernel in one of two ways: >> >> 1) Asynchronously via kswapd >> 2) Synchronously, via direct reclaim >> >> At page allocation time the allocating

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>>

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>> makes sense to use as

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>>

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 2:12 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: >>> Yes, very much this. If you have a single-threaded workload which is >>> using the entirety of memory and would like to use even more, then it >>> makes sense to use as

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: > > Yes, very much this. If you have a single-threaded workload which is > > using the entirety of memory and would like to use even more, then it > > makes sense to use as many CPUs as necessary getting memory out of its > > way. If

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue, Apr 03, 2018 at 01:49:25PM -0700, Buddy Lumpkin wrote: > > Yes, very much this. If you have a single-threaded workload which is > > using the entirety of memory and would like to use even more, then it > > makes sense to use as many CPUs as necessary getting memory out of its > > way. If

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 12:07 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: >> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >>> The presence of direct reclaims 10 years ago was a fairly reliable >>> indicator that too much was

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

> On Apr 3, 2018, at 12:07 PM, Matthew Wilcox wrote: > > On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: >> On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >>> The presence of direct reclaims 10 years ago was a fairly reliable >>> indicator that too much was being asked of a Linux

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

Very sorry, I forgot to send my last response as plain text. > On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >> Page replacement is handled in the Linux Kernel in one of two ways: >> >> 1) Asynchronously via kswapd >> 2)

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

Very sorry, I forgot to send my last response as plain text. > On Apr 3, 2018, at 6:31 AM, Michal Hocko wrote: > > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: >> Page replacement is handled in the Linux Kernel in one of two ways: >> >> 1) Asynchronously via kswapd >> 2) Synchronously, via

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > > The presence of direct reclaims 10 years ago was a fairly reliable > > indicator that too much was being asked of a Linux system. Kswapd was > > likely wasting time scanning pages

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Tue, Apr 03, 2018 at 03:31:15PM +0200, Michal Hocko wrote: > On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > > The presence of direct reclaims 10 years ago was a fairly reliable > > indicator that too much was being asked of a Linux system. Kswapd was > > likely wasting time scanning pages

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > Page replacement is handled in the Linux Kernel in one of two ways: > > 1) Asynchronously via kswapd > 2) Synchronously, via direct reclaim > > At page allocation time the allocating task is immediately given a page > from the zone free list

Re: [RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

On Mon 02-04-18 09:24:22, Buddy Lumpkin wrote: > Page replacement is handled in the Linux Kernel in one of two ways: > > 1) Asynchronously via kswapd > 2) Synchronously, via direct reclaim > > At page allocation time the allocating task is immediately given a page > from the zone free list

[RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

Page replacement is handled in the Linux Kernel in one of two ways: 1) Asynchronously via kswapd 2) Synchronously, via direct reclaim At page allocation time the allocating task is immediately given a page from the zone free list allowing it to go right back to work doing whatever it was doing;

[RFC PATCH 1/1] vmscan: Support multiple kswapd threads per node

Page replacement is handled in the Linux Kernel in one of two ways: 1) Asynchronously via kswapd 2) Synchronously, via direct reclaim At page allocation time the allocating task is immediately given a page from the zone free list allowing it to go right back to work doing whatever it was doing;