Re: [PATCH -V8 00/21] swap: Swapout/swapin THP in one piece

2018-12-06 Thread Huang, Ying
Huang Ying writes: > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the > swap part of the patchset? Especially [02/21], [03/21], [04/21], > [05/21], [06/21], [07/21], [08

[PATCH -V8 02/21] swap: Add __swap_duplicate_locked()

2018-12-06 Thread Huang Ying
refactoring, there is no any functional change in this patch. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horig

[PATCH -V8 05/21] swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-12-06 Thread Huang Ying
be freed. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- arch/s39

[PATCH -V8 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-12-06 Thread Huang Ying
a THP, add it into the swap cache. So later the contents of the huge swap cluster can be read into the THP. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V8 18/21] swap: Support PMD swap mapping in mincore()

2018-12-06 Thread Huang Ying
During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V8 12/21] swap: Support PMD swap mapping in swapoff

2018-12-06 Thread Huang Ying
is split already, we will split the PMD swap mapping and unuse the PTEs. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Nao

[PATCH -V8 13/21] swap: Support PMD swap mapping in madvise_free()

2018-12-06 Thread Huang Ying
mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen

[PATCH -V8 16/21] swap: Free PMD swap mapping when zap_huge_pmd()

2018-12-06 Thread Huang Ying
For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V8 14/21] swap: Support to move swap account for PMD swap mapping

2018-12-06 Thread Huang Ying
to PTE processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan

[PATCH -V8 11/21] swap: Add sysfs interface to configure THP swapin

2018-12-06 Thread Huang Ying
is disabled, the huge swap cluster and the PMD swap mapping will be split and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Ri

[PATCH -V8 17/21] swap: Support PMD swap mapping for MADV_WILLNEED

2018-12-06 Thread Huang Ying
During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutem

[PATCH -V8 20/21] swap: create PMD swap mapping when unmap the THP

2018-12-06 Thread Huang Ying
swapping is used, so that we can take full advantage of THP including its high performance for swapout/swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V8 19/21] swap: Support PMD swap mapping in common path

2018-12-06 Thread Huang Ying
Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel

[PATCH -V8 21/21] swap: Update help of CONFIG_THP_SWAP

2018-12-06 Thread Huang Ying
The help of CONFIG_THP_SWAP is updated to reflect the latest progress of THP (Tranparent Huge Page) swap optimization. Signed-off-by: "Huang, Ying" Reviewed-by: Dan Williams Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua

[PATCH -V8 10/21] swap: Support to count THP swapin and its fallback

2018-12-06 Thread Huang Ying
2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device in one piece and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Ho

[PATCH -V8 15/21] swap: Support to copy PMD swap mapping when fork()

2018-12-06 Thread Huang Ying
continuation failed to allocate a page with GFP_ATOMIC, we need to unlock the spinlock and try again with GFP_KERNEL. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minc

[PATCH -V8 00/21] swap: Swapout/swapin THP in one piece

2018-12-06 Thread Huang Ying
Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21], [12/21], [20/21],

[PATCH -V8 04/21] swap: Support PMD swap mapping in put_swap_page()

2018-12-06 Thread Huang Ying
to PMD swap mappings to the corresponding swap cluster. So when clearing the SWAP_HAS_CACHE flag, the huge swap cluster will only be split if the PMD swap mapping count is 0. Otherwise, we will keep it as the huge swap cluster. So that we can swapin a THP in one piece later. Signed-off-by: "

[PATCH -V8 01/21] swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-12-06 Thread Huang Ying
ead. Some functions enabled by CONFIG_ARCH_ENABLE_THP_MIGRATION are for page migration only, they are still enabled only for that. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickin

[PATCH -V8 06/21] swap: Support PMD swap mapping when splitting huge PMD

2018-12-06 Thread Huang Ying
by misaligned address processing issue in __split_huge_swap_pmd(). Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horig

[PATCH -V8 09/21] swap: Swapin a THP in one piece

2018-12-06 Thread Huang Ying
to fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel

[PATCH -V8 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-12-06 Thread Huang Ying
will be split if its PMD swap mapping count is 0. The first parameter of swap_duplicate() is changed to return the swap entry to call add_swap_count_continuation() for. Because we may need to call it for a swap entry in the middle of a huge swap cluster. Signed-off-by: "Huang, Ying" C

[PATCH -V8 07/21] swap: Support PMD swap mapping in split_swap_cluster()

2018-12-06 Thread Huang Ying
is changed to before unlocking sub-pages. So that all sub-pages will be kept locked from the THP has been split to the huge swap cluster is split. This makes the code much easier to be reasoned. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

Re: [PATCH -V7 RESEND 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-12-03 Thread Huang, Ying
Daniel Jordan writes: > On Sat, Dec 01, 2018 at 08:34:06AM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> > What do you think? >> >> I think that swapoff() which is the main user of try_to_unuse() isn't a >> common operation in practical. S

Re: [PATCH -V7 RESEND 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-11-30 Thread Huang, Ying
Hi, Daniel, Daniel Jordan writes: > Hi Ying, > > On Tue, Nov 20, 2018 at 04:54:36PM +0800, Huang Ying wrote: >> diff --git a/mm/swap_state.c b/mm/swap_state.c >> index 97831166994a..1eedbc0aede2 100644 >> --- a/mm/swap_state.c >> +++ b/mm/swap_state.c >

Re: [LKP] [mm] ac5b2c1891: vm-scalability.throughput -61.3% regression

2018-11-27 Thread Huang, Ying
is commit, the first process exited at 203s. That is, this commit makes memory allocation more fair among processes, so that processes proceeded at more similar speed. But this raises system memory footprint too, so triggered much more swap, thus lower benchmark score. In general, memory allocation fairness among processes should be a good thing. So I think the report should have been a "performance improvement" instead of "performance regression". Best Regards, Huang, Ying

[PATCH -V7 RESEND 11/21] swap: Add sysfs interface to configure THP swapin

2018-11-20 Thread Huang Ying
is disabled, the huge swap cluster and the PMD swap mapping will be split and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Ri

[PATCH -V7 RESEND 21/21] swap: Update help of CONFIG_THP_SWAP

2018-11-20 Thread Huang Ying
The help of CONFIG_THP_SWAP is updated to reflect the latest progress of THP (Tranparent Huge Page) swap optimization. Signed-off-by: "Huang, Ying" Reviewed-by: Dan Williams Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua

[PATCH -V7 RESEND 19/21] swap: Support PMD swap mapping in common path

2018-11-20 Thread Huang Ying
Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel

[PATCH -V7 RESEND 20/21] swap: create PMD swap mapping when unmap the THP

2018-11-20 Thread Huang Ying
swapping is used, so that we can take full advantage of THP including its high performance for swapout/swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V7 RESEND 16/21] swap: Free PMD swap mapping when zap_huge_pmd()

2018-11-20 Thread Huang Ying
For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V7 RESEND 15/21] swap: Support to copy PMD swap mapping when fork()

2018-11-20 Thread Huang Ying
continuation failed to allocate a page with GFP_ATOMIC, we need to unlock the spinlock and try again with GFP_KERNEL. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minc

[PATCH -V7 RESEND 14/21] swap: Support to move swap account for PMD swap mapping

2018-11-20 Thread Huang Ying
to PTE processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan

[PATCH -V7 RESEND 18/21] swap: Support PMD swap mapping in mincore()

2018-11-20 Thread Huang Ying
During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V7 RESEND 17/21] swap: Support PMD swap mapping for MADV_WILLNEED

2018-11-20 Thread Huang Ying
During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutem

[PATCH -V7 RESEND 01/21] swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-11-20 Thread Huang Ying
ead. Some functions enabled by CONFIG_ARCH_ENABLE_THP_MIGRATION are for page migration only, they are still enabled only for that. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickin

[PATCH -V7 RESEND 06/21] swap: Support PMD swap mapping when splitting huge PMD

2018-11-20 Thread Huang Ying
by misaligned address processing issue in __split_huge_swap_pmd(). Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horig

[PATCH -V7 RESEND 13/21] swap: Support PMD swap mapping in madvise_free()

2018-11-20 Thread Huang Ying
mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen

[PATCH -V7 RESEND 12/21] swap: Support PMD swap mapping in swapoff

2018-11-20 Thread Huang Ying
is split already, we will split the PMD swap mapping and unuse the PTEs. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Nao

[PATCH -V7 RESEND 10/21] swap: Support to count THP swapin and its fallback

2018-11-20 Thread Huang Ying
2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device in one piece and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Ho

[PATCH -V7 RESEND 02/21] swap: Add __swap_duplicate_locked()

2018-11-20 Thread Huang Ying
refactoring, there is no any functional change in this patch. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horig

[PATCH -V7 00/21] swap: Swapout/swapin THP in one piece

2018-11-20 Thread Huang Ying
Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21], [12/21], [20/21],

[PATCH -V7 RESEND 05/21] swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-11-20 Thread Huang Ying
be freed. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- arch/s39

[PATCH -V7 RESEND 09/21] swap: Swapin a THP in one piece

2018-11-20 Thread Huang Ying
to fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel

[PATCH -V7 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-11-20 Thread Huang Ying
will be split if its PMD swap mapping count is 0. The first parameter of swap_duplicate() is changed to return the swap entry to call add_swap_count_continuation() for. Because we may need to call it for a swap entry in the middle of a huge swap cluster. Signed-off-by: "Huang, Ying" C

[PATCH -V7 RESEND 07/21] swap: Support PMD swap mapping in split_swap_cluster()

2018-11-20 Thread Huang Ying
is changed to before unlocking sub-pages. So that all sub-pages will be kept locked from the THP has been split to the huge swap cluster is split. This makes the code much easier to be reasoned. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V7 RESEND 04/21] swap: Support PMD swap mapping in put_swap_page()

2018-11-20 Thread Huang Ying
to PMD swap mappings to the corresponding swap cluster. So when clearing the SWAP_HAS_CACHE flag, the huge swap cluster will only be split if the PMD swap mapping count is 0. Otherwise, we will keep it as the huge swap cluster. So that we can swapin a THP in one piece later. Signed-off-by: "

[PATCH -V7 RESEND 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-11-20 Thread Huang Ying
a THP, add it into the swap cache. So later the contents of the huge swap cluster can be read into the THP. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

Re: [PATCH 1/2] mm: use kvzalloc for swap_info_struct allocation

2018-11-04 Thread Huang, Ying
Vasily Averin writes: > On 11/5/18 3:50 AM, Huang, Ying wrote: >> Vasily Averin writes: >> >>> commit a2468cc9bfdf ("swap: choose swap device according to numa node") >>> increased size of swap_info_struct up to 44 Kbytes, now it requires >>&g

Re: [PATCH 2/2] mm: avoid unnecessary swap_info_struct allocation

2018-11-04 Thread Huang, Ying
. Best Regards, Huang, Ying > Signed-off-by: Vasily Averin > --- > mm/swapfile.c | 18 +- > 1 file changed, 13 insertions(+), 5 deletions(-) > > diff --git a/mm/swapfile.c b/mm/swapfile.c > index 8688ae65ef58..53ec2f0cdf26 100644 > --- a/mm/swapfile.c &g

Re: [PATCH 1/2] mm: use kvzalloc for swap_info_struct allocation

2018-11-04 Thread Huang, Ying
'avail_lists' field could be tens KB? If so, I think it's fair to use kvzalloc(). Can you add one line comment? Because struct swap_info_struct is quite small in default configuration. Best Regards, Huang, Ying > Switch to kvzmalloc allows to avoid unexpected allocation failures. > >

Re: [PATCH -V6 14/21] swap: Support to move swap account for PMD swap mapping

2018-10-24 Thread Huang, Ying
Daniel Jordan writes: > On Wed, Oct 10, 2018 at 03:19:17PM +0800, Huang Ying wrote: >> +static struct page *mc_handle_swap_pmd(struct vm_area_struct *vma, >> +pmd_t pmd, swp_entry_t *entry) >> +{ > > Got > /home/dbbench/linux/mm

Re: [PATCH -V6 06/21] swap: Support PMD swap mapping when splitting huge PMD

2018-10-24 Thread Huang, Ying
Daniel Jordan writes: > On Wed, Oct 10, 2018 at 03:19:09PM +0800, Huang Ying wrote: >> +#ifdef CONFIG_THP_SWAP >> +/* >> + * The corresponding page table shouldn't be changed under us, that >> + * is, the page table lock should be held. >> + */ >> +int s

Re: [PATCH -V6 00/21] swap: Swapout/swapin THP in one piece

2018-10-24 Thread Huang, Ying
Daniel Jordan writes: > On Wed, Oct 24, 2018 at 11:31:42AM +0800, Huang, Ying wrote: >> Hi, Daniel, >> >> Daniel Jordan writes: >> >> > On Wed, Oct 10, 2018 at 03:19:03PM +0800, Huang Ying wrote: >> >> And for all, Any comment is welcome! >

Re: [PATCH -V6 00/21] swap: Swapout/swapin THP in one piece

2018-10-23 Thread Huang, Ying
Hi, Daniel, Daniel Jordan writes: > On Wed, Oct 10, 2018 at 03:19:03PM +0800, Huang Ying wrote: >> And for all, Any comment is welcome! >> >> This patchset is based on the 2018-10-3 head of mmotm/master. > > There seems to be some infrequent memory corruption with TH

[PATCH -V6 01/21] swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-10-10 Thread Huang Ying
ead. Some functions enabled by CONFIG_ARCH_ENABLE_THP_MIGRATION are for page migration only, they are still enabled only for that. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickin

[PATCH -V6 14/21] swap: Support to move swap account for PMD swap mapping

2018-10-10 Thread Huang Ying
to PTE processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan

[PATCH -V6 05/21] swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-10-10 Thread Huang Ying
be freed. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- arch/s39

[PATCH -V6 00/21] swap: Swapout/swapin THP in one piece

2018-10-10 Thread Huang Ying
Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21], [12/21], [20/21],

[PATCH -V6 12/21] swap: Support PMD swap mapping in swapoff

2018-10-10 Thread Huang Ying
is split already, we will split the PMD swap mapping and unuse the PTEs. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Nao

[PATCH -V6 06/21] swap: Support PMD swap mapping when splitting huge PMD

2018-10-10 Thread Huang Ying
swap cluster too. If the PMD swap mapping count becomes 0, the huge swap cluster will be split. Notice: is_huge_zero_pmd() and pmd_page() doesn't work well with swap PMD, so pmd_present() check is called before them. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc:

[PATCH -V6 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-10-10 Thread Huang Ying
will be split if its PMD swap mapping count is 0. The first parameter of swap_duplicate() is changed to return the swap entry to call add_swap_count_continuation() for. Because we may need to call it for a swap entry in the middle of a huge swap cluster. Signed-off-by: "Huang, Ying" C

[PATCH -V6 11/21] swap: Add sysfs interface to configure THP swapin

2018-10-10 Thread Huang Ying
is disabled, the huge swap cluster and the PMD swap mapping will be split and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Ri

[PATCH -V6 21/21] swap: Update help of CONFIG_THP_SWAP

2018-10-10 Thread Huang Ying
The help of CONFIG_THP_SWAP is updated to reflect the latest progress of THP (Tranparent Huge Page) swap optimization. Signed-off-by: "Huang, Ying" Reviewed-by: Dan Williams Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua

[PATCH -V6 02/21] swap: Add __swap_duplicate_locked()

2018-10-10 Thread Huang Ying
refactoring, there is no any functional change in this patch. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horig

[PATCH -V6 16/21] swap: Free PMD swap mapping when zap_huge_pmd()

2018-10-10 Thread Huang Ying
For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V6 17/21] swap: Support PMD swap mapping for MADV_WILLNEED

2018-10-10 Thread Huang Ying
During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutem

[PATCH -V6 19/21] swap: Support PMD swap mapping in common path

2018-10-10 Thread Huang Ying
Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel

[PATCH -V6 18/21] swap: Support PMD swap mapping in mincore()

2018-10-10 Thread Huang Ying
During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V6 15/21] swap: Support to copy PMD swap mapping when fork()

2018-10-10 Thread Huang Ying
continuation failed to allocate a page with GFP_ATOMIC, we need to unlock the spinlock and try again with GFP_KERNEL. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minc

[PATCH -V6 13/21] swap: Support PMD swap mapping in madvise_free()

2018-10-10 Thread Huang Ying
mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen

[PATCH -V6 20/21] swap: create PMD swap mapping when unmap the THP

2018-10-10 Thread Huang Ying
swapping is used, so that we can take full advantage of THP including its high performance for swapout/swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V6 04/21] swap: Support PMD swap mapping in put_swap_page()

2018-10-10 Thread Huang Ying
to PMD swap mappings to the corresponding swap cluster. So when clearing the SWAP_HAS_CACHE flag, the huge swap cluster will only be split if the PMD swap mapping count is 0. Otherwise, we will keep it as the huge swap cluster. So that we can swapin a THP in one piece later. Signed-off-by: "

[PATCH -V6 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-10-10 Thread Huang Ying
a THP, add it into the swap cache. So later the contents of the huge swap cluster can be read into the THP. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V6 07/21] swap: Support PMD swap mapping in split_swap_cluster()

2018-10-10 Thread Huang Ying
is changed to before unlocking sub-pages. So that all sub-pages will be kept locked from the THP has been split to the huge swap cluster is split. This makes the code much easier to be reasoned. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V6 09/21] swap: Swapin a THP in one piece

2018-10-10 Thread Huang Ying
to fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel

[PATCH -V6 10/21] swap: Support to count THP swapin and its fallback

2018-10-10 Thread Huang Ying
2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device in one piece and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Ho

Re: [PATCH -V5 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-09-28 Thread Huang, Ying
Daniel Jordan writes: > On Fri, Sep 28, 2018 at 04:19:03PM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> > One way is to change >> > copy_one_pte's return to int so we can just pass the error code back to >> > copy_pte_range so it knows wh

Re: [PATCH -V5 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-09-28 Thread Huang, Ying
Daniel Jordan writes: > On Thu, Sep 27, 2018 at 09:34:36AM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> > On Wed, Sep 26, 2018 at 08:55:59PM +0800, Huang, Ying wrote: >> >> Daniel Jordan writes: >> >> > On Tue, Sep 2

Re: [PATCH -V5 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-09-26 Thread Huang, Ying
Daniel Jordan writes: > On Wed, Sep 26, 2018 at 08:55:59PM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> > On Tue, Sep 25, 2018 at 03:13:30PM +0800, Huang Ying wrote: >> >> /* >> >> * Increase reference count of swap entry by 1. &

Re: [PATCH -V5 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-09-26 Thread Huang, Ying
Daniel Jordan writes: > On Tue, Sep 25, 2018 at 03:13:30PM +0800, Huang Ying wrote: >> @@ -3487,35 +3521,66 @@ static int __swap_duplicate_locked(struct >> swap_info_struct *p, >> } >> >> /* >> - * Verify that a swap entry is valid and incr

[PATCH -V5 RESEND 09/21] swap: Swapin a THP in one piece

2018-09-25 Thread Huang Ying
to fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel

[PATCH -V5 RESEND 11/21] swap: Add sysfs interface to configure THP swapin

2018-09-25 Thread Huang Ying
is disabled, the huge swap cluster and the PMD swap mapping will be split and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Ri

[PATCH -V5 RESEND 10/21] swap: Support to count THP swapin and its fallback

2018-09-25 Thread Huang Ying
2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device in one piece and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Ho

[PATCH -V5 RESEND 13/21] swap: Support PMD swap mapping in madvise_free()

2018-09-25 Thread Huang Ying
mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen

[PATCH -V5 RESEND 21/21] swap: Update help of CONFIG_THP_SWAP

2018-09-25 Thread Huang Ying
The help of CONFIG_THP_SWAP is updated to reflect the latest progress of THP (Tranparent Huge Page) swap optimization. Signed-off-by: "Huang, Ying" Reviewed-by: Dan Williams Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua

[PATCH -V5 RESEND 07/21] swap: Support PMD swap mapping in split_swap_cluster()

2018-09-25 Thread Huang Ying
is changed to before unlocking sub-pages. So that all sub-pages will be kept locked from the THP has been split to the huge swap cluster is split. This makes the code much easier to be reasoned. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V5 RESEND 12/21] swap: Support PMD swap mapping in swapoff

2018-09-25 Thread Huang Ying
is split already, we will split the PMD swap mapping and unuse the PTEs. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Nao

[PATCH -V5 RESEND 05/21] swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-09-25 Thread Huang Ying
be freed. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- arch/s39

[PATCH -V5 RESEND 14/21] swap: Support to move swap account for PMD swap mapping

2018-09-25 Thread Huang Ying
to PTE processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan

[PATCH -V5 RESEND 15/21] swap: Support to copy PMD swap mapping when fork()

2018-09-25 Thread Huang Ying
continuation failed to allocate a page with GFP_ATOMIC, we need to unlock the spinlock and try again with GFP_KERNEL. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minc

[PATCH -V5 RESEND 16/21] swap: Free PMD swap mapping when zap_huge_pmd()

2018-09-25 Thread Huang Ying
For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V5 RESEND 18/21] swap: Support PMD swap mapping in mincore()

2018-09-25 Thread Huang Ying
During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli

[PATCH -V5 RESEND 20/21] swap: create PMD swap mapping when unmap the THP

2018-09-25 Thread Huang Ying
swapping is used, so that we can take full advantage of THP including its high performance for swapout/swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V5 RESEND 08/21] swap: Support to read a huge swap cluster for swapin a THP

2018-09-25 Thread Huang Ying
a THP, add it into the swap cache. So later the contents of the huge swap cluster can be read into the THP. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -V5 RESEND 03/21] swap: Support PMD swap mapping in swap_duplicate()

2018-09-25 Thread Huang Ying
will be split if its PMD swap mapping count is 0. The first parameter of swap_duplicate() is changed to return the swap entry to call add_swap_count_continuation() for. Because we may need to call it for a swap entry in the middle of a huge swap cluster. Signed-off-by: "Huang, Ying" C

[PATCH -V5 RESEND 19/21] swap: Support PMD swap mapping in common path

2018-09-25 Thread Huang Ying
Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel

[PATCH -V5 RESEND 06/21] swap: Support PMD swap mapping when splitting huge PMD

2018-09-25 Thread Huang Ying
swap cluster too. If the PMD swap mapping count becomes 0, the huge swap cluster will be split. Notice: is_huge_zero_pmd() and pmd_page() doesn't work well with swap PMD, so pmd_present() check is called before them. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc:

[PATCH -V5 RESEND 01/21] swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-09-25 Thread Huang Ying
ead. Some functions enabled by CONFIG_ARCH_ENABLE_THP_MIGRATION are for page migration only, they are still enabled only for that. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickin

[PATCH -V5 RESEND 17/21] swap: Support PMD swap mapping for MADV_WILLNEED

2018-09-25 Thread Huang Ying
During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutem

  1   2   3   4   5   6   7   8   9   10   >