Currently lock_memory_hotplug()/unlock_memory_hotplug() are used to protect totalram_pages and zone->managed_pages. Other than the memory hotplug driver, totalram_pages and zone->managed_pages may be modified by Xen balloon, virtio_balloon etc at runtime. For those case, memory hotplug lock is a little too heavy, so introduce a dedicated lock to protect them.
Now the locking rules for totalram_pages and zone->managed_pages have been simpilied as: 1) no locking for read accesses because they are unsigned long. 2) no locking for write access at boot time in single-threaded context. 3) serialize write accesses at run time by managed_page_count_lock. Also adjust zone->managed_pages when dealing with reserved pages. Signed-off-by: Jiang Liu <[email protected]> Cc: Andrew Morton <[email protected]> Cc: Mel Gorman <[email protected]> Cc: Michel Lespinasse <[email protected]> Cc: Rik van Riel <[email protected]> Cc: Minchan Kim <[email protected]> Cc: [email protected] (open list:MEMORY MANAGEMENT) Cc: [email protected] (open list) --- include/linux/mm.h | 6 +----- include/linux/mmzone.h | 14 ++++++++++---- mm/page_alloc.c | 19 +++++++++++++++++++ 3 files changed, 30 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 4d1509b..f9cc7f0 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1299,14 +1299,10 @@ extern void free_initmem(void); #ifdef CONFIG_HIGHMEM extern void free_highmem_page(struct page *page); #endif +extern void adjust_managed_page_count(struct page *page, long count); extern unsigned long free_reserved_area(unsigned long start, unsigned long end, int poison, char *s); -static inline void adjust_managed_page_count(struct page *page, long count) -{ - totalram_pages += count; -} - static inline void __free_reserved_page(struct page *page) { ClearPageReserved(page); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index ede2749..bc58fef 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -474,10 +474,16 @@ struct zone { * frequently read in proximity to zone->lock. It's good to * give them a chance of being in the same cacheline. * - * Write access to present_pages and managed_pages at runtime should - * be protected by lock_memory_hotplug()/unlock_memory_hotplug(). - * Any reader who can't tolerant drift of present_pages and - * managed_pages should hold memory hotplug lock to get a stable value. + * Write access to present_pages at runtime should be protected by + * lock_memory_hotplug()/unlock_memory_hotplug(). Any reader who can't + * tolerant drift of present_pages should hold memory hotplug lock to + * get a stable value. + * + * Read access to managed_pages should be safe because it's unsigned + * long. Write access to zone->managed_pages and totalram_pages are + * protected by managed_page_count_lock at runtime. Basically only + * adjust_managed_page_count() should be used instead of directly + * touching zone->managed_pages and totalram_pages. */ unsigned long spanned_pages; unsigned long present_pages; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8106aa5..2692931 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -99,6 +99,9 @@ nodemask_t node_states[NR_NODE_STATES] __read_mostly = { }; EXPORT_SYMBOL(node_states); +/* Protect totalram_pages and zone->managed_pages */ +static DEFINE_SPINLOCK(managed_page_count_lock); + unsigned long totalram_pages __read_mostly; unsigned long totalreserve_pages __read_mostly; /* @@ -5113,6 +5116,22 @@ early_param("movablecore", cmdline_parse_movablecore); #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */ +void adjust_managed_page_count(struct page *page, long count) +{ + bool lock = (system_state != SYSTEM_BOOTING); + + /* No need to acquire the lock during boot */ + if (lock) + spin_lock(&managed_page_count_lock); + + page_zone(page)->managed_pages += count; + totalram_pages += count; + + if (lock) + spin_unlock(&managed_page_count_lock); +} +EXPORT_SYMBOL(adjust_managed_page_count); + unsigned long free_reserved_area(unsigned long start, unsigned long end, int poison, char *s) { -- 1.7.9.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
