Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-16 Thread Jerome Glisse
On Thu, Aug 16, 2018 at 04:58:49PM +0200, Oscar Salvador wrote: > On Thu, Aug 09, 2018 at 12:58:21PM -0400, Jerome Glisse wrote: > > I agree, i never thought about that before. Looking at existing resource > > management i think the simplest solution would be to use a refcount on the > > resources

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-16 Thread Oscar Salvador
On Thu, Aug 09, 2018 at 12:58:21PM -0400, Jerome Glisse wrote: > I agree, i never thought about that before. Looking at existing resource > management i think the simplest solution would be to use a refcount on the > resources instead of the IORESOURCE_BUSY flags. > > So when you release resource

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Oscar Salvador
On Thu, Aug 09, 2018 at 12:58:21PM -0400, Jerome Glisse wrote: > > I would really prefer to be explicit about these requirements rather > > than having subtle side effects quite deep in the memory hotplug code > > and checks for zone device sprinkled at places for special handling. > > I agree, i

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Jerome Glisse
On Thu, Aug 09, 2018 at 05:09:50PM +0200, Michal Hocko wrote: > On Thu 09-08-18 10:27:09, Jerome Glisse wrote: > > On Thu, Aug 09, 2018 at 10:24:15AM +0200, Michal Hocko wrote: > > > On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > > > > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote:

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Michal Hocko
On Thu 09-08-18 10:27:09, Jerome Glisse wrote: > On Thu, Aug 09, 2018 at 10:24:15AM +0200, Michal Hocko wrote: > > On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > > > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > > > > On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > > > > > On T

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Jerome Glisse
On Thu, Aug 09, 2018 at 10:24:15AM +0200, Michal Hocko wrote: > On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > > > On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > > > > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote:

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Michal Hocko
On Wed 08-08-18 12:58:15, Jerome Glisse wrote: > On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > > On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > > > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > > > > On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > > > > > On T

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Oscar Salvador
On Thu, Aug 09, 2018 at 09:50:55AM +0200, Oscar Salvador wrote: > On Wed, Aug 08, 2018 at 11:29:08PM +0200, Oscar Salvador wrote: > > On Wed, Aug 08, 2018 at 01:55:59PM -0400, Jerome Glisse wrote: > > > Note that Dan did post patches that already go in that direction (unifying > > > code between de

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-09 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 11:29:08PM +0200, Oscar Salvador wrote: > On Wed, Aug 08, 2018 at 01:55:59PM -0400, Jerome Glisse wrote: > > Note that Dan did post patches that already go in that direction (unifying > > code between devm and HMM). I think they are in Andrew queue, looks for > > > > mm: Re

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 01:55:59PM -0400, Jerome Glisse wrote: > Note that Dan did post patches that already go in that direction (unifying > code between devm and HMM). I think they are in Andrew queue, looks for > > mm: Rework hmm to use devm_memremap_pages and other fixes Thanks for pointing t

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 12:58:15PM -0400, Jerome Glisse wrote: > > If the former then I do not see any reason why we couldn't simply > > refactor the code to expect a failure and drop the warning in that path. > > Referring to newer case ie calling release_mem_region_adjustable() for > ZONE_DEVICE

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Jerome Glisse
On Wed, Aug 08, 2018 at 03:42:33PM +0200, Oscar Salvador wrote: > On Wed, Aug 08, 2018 at 10:08:41AM +0200, David Hildenbrand wrote: > > Then it is maybe time to cleary distinguish both types of memory, as > > they are fundamentally different when it comes to online/offline behavior. > > > > Ordin

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Jerome Glisse
On Wed, Aug 08, 2018 at 11:45:02AM +0200, Oscar Salvador wrote: > On Tue, Aug 07, 2018 at 11:18:10AM -0400, Jerome Glisse wrote: > > Correct, you should not call release_mem_region_adjustable() the device > > region is not part of regular iomem resource as it might not necessarily > > be enumerated

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Jerome Glisse
On Wed, Aug 08, 2018 at 08:47:58AM +0200, Michal Hocko wrote: > On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > > > On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > > > > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadven

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 10:08:41AM +0200, David Hildenbrand wrote: > Then it is maybe time to cleary distinguish both types of memory, as > they are fundamentally different when it comes to online/offline behavior. > > Ordinary ram: > add_memory ... > online_pages ... > offline_pages > remove_

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Tue, Aug 07, 2018 at 11:18:10AM -0400, Jerome Glisse wrote: > Correct, you should not call release_mem_region_adjustable() the device > region is not part of regular iomem resource as it might not necessarily > be enumerated through known ways to the kernel (ie only the device driver > can disco

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread David Hildenbrand
On 08.08.2018 09:56, Oscar Salvador wrote: > On Wed, Aug 08, 2018 at 09:45:37AM +0200, David Hildenbrand wrote: >> On 08.08.2018 09:38, Oscar Salvador wrote: >>> On Tue, Aug 07, 2018 at 06:13:45PM -0400, Jerome Glisse wrote: > And since we know for sure that memhotplug-code cannot call it with

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 09:51:50AM +0200, David Hildenbrand wrote: > > I am pretty sure this is a dumb question, but why HMM/devm path > > do not call online_pages/offline_pages? > > I think mainly because onlining/offlining (wild guesses) > > - calls memory notifiers > - works with memory blocks

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Wed, Aug 08, 2018 at 09:45:37AM +0200, David Hildenbrand wrote: > On 08.08.2018 09:38, Oscar Salvador wrote: > > On Tue, Aug 07, 2018 at 06:13:45PM -0400, Jerome Glisse wrote: > >>> And since we know for sure that memhotplug-code cannot call it with > >>> ZONE_DEVICE, > >>> I think this can be

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread David Hildenbrand
On 08.08.2018 09:38, Oscar Salvador wrote: > On Tue, Aug 07, 2018 at 06:13:45PM -0400, Jerome Glisse wrote: >>> And since we know for sure that memhotplug-code cannot call it with >>> ZONE_DEVICE, >>> I think this can be done easily. >> >> This might change down road but for now this is correct. T

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread David Hildenbrand
On 08.08.2018 09:38, Oscar Salvador wrote: > On Tue, Aug 07, 2018 at 06:13:45PM -0400, Jerome Glisse wrote: >>> And since we know for sure that memhotplug-code cannot call it with >>> ZONE_DEVICE, >>> I think this can be done easily. >> >> This might change down road but for now this is correct. T

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-08 Thread Oscar Salvador
On Tue, Aug 07, 2018 at 06:13:45PM -0400, Jerome Glisse wrote: > > And since we know for sure that memhotplug-code cannot call it with > > ZONE_DEVICE, > > I think this can be done easily. > > This might change down road but for now this is correct. They are > talks to enumerate device memory thr

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Michal Hocko
On Tue 07-08-18 11:18:10, Jerome Glisse wrote: > On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > > On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > > > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net > > > wrote: > > > > From: Oscar Salvador > > > > > > [..

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Jerome Glisse
On Tue, Aug 07, 2018 at 10:48:34PM +0200, Oscar Salvador wrote: > On Tue, Aug 07, 2018 at 04:54:57PM +0200, David Hildenbrand wrote: > > I wonder if we could instead forward from the callers whether we are > > dealing with ZONE_DEVICE memory (is_device ...), at least that seems > > feasible in hmm

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Oscar Salvador
On Tue, Aug 07, 2018 at 04:54:57PM +0200, David Hildenbrand wrote: > I wonder if we could instead forward from the callers whether we are > dealing with ZONE_DEVICE memory (is_device ...), at least that seems > feasible in hmm code. Not having looked at details yet. Yes, this looks like the most s

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread David Hildenbrand
On 07.08.2018 17:19, Jerome Glisse wrote: > On Tue, Aug 07, 2018 at 04:54:57PM +0200, David Hildenbrand wrote: >> On 07.08.2018 15:52, Jerome Glisse wrote: >>> On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net >>> wrote: From: Oscar Salvador >>> >>> [...] >>> diff -

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Jerome Glisse
On Tue, Aug 07, 2018 at 04:54:57PM +0200, David Hildenbrand wrote: > On 07.08.2018 15:52, Jerome Glisse wrote: > > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net > > wrote: > >> From: Oscar Salvador > > > > [...] > > > >> diff --git a/mm/memory_hotplug.c b/mm/memory_hotp

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Jerome Glisse
On Tue, Aug 07, 2018 at 04:59:00PM +0200, Michal Hocko wrote: > On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net > > wrote: > > > From: Oscar Salvador > > > > [...] > > > > > diff --git a/mm/memory_hotplug.c b/mm/memory_ho

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Michal Hocko
On Tue 07-08-18 09:52:21, Jerome Glisse wrote: > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net wrote: > > From: Oscar Salvador > > [...] > > > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > > index 9bd629944c91..e33555651e46 100644 > > --- a/mm/memory_hotplug.c

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread David Hildenbrand
On 07.08.2018 15:52, Jerome Glisse wrote: > On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net wrote: >> From: Oscar Salvador > > [...] > >> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c >> index 9bd629944c91..e33555651e46 100644 >> --- a/mm/memory_hotplug.c >> +++ b

Re: [RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread Jerome Glisse
On Tue, Aug 07, 2018 at 03:37:56PM +0200, osalva...@techadventures.net wrote: > From: Oscar Salvador [...] > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index 9bd629944c91..e33555651e46 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c [...] > /** > * __remove_page

[RFC PATCH 2/3] mm/memory_hotplug: Create __shrink_pages and move it to offline_pages

2018-08-07 Thread osalvador
From: Oscar Salvador Currently, we decrement zone/node spanned_pages when we __remove__ the memory. This is not really great. Incrementing of spanned pages is done in online_pages() path, decrementing spanned pages should be moved to offline_pages(). This, besides making the core more consisten