Re: [PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-07-02 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 22, 2018 at 11:51:38AM +0800, Huang, Ying wrote: >> +++ b/mm/swap_state.c >> @@ -426,33 +447,37 @@ struct page *__read_swap_cache_async(swp_entry_t >> entry, gfp_t gfp_mask, >> /* >> * call radix

Re: [PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-07-02 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 22, 2018 at 11:51:38AM +0800, Huang, Ying wrote: >> +++ b/mm/swap_state.c >> @@ -426,33 +447,37 @@ struct page *__read_swap_cache_async(swp_entry_t >> entry, gfp_t gfp_mask, >> /* >> * call radix

Re: [PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-07-01 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 22, 2018 at 11:51:33AM +0800, Huang, Ying wrote: >> +++ b/mm/swap_state.c >> @@ -433,7 +433,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, >> gfp_t gfp_mask, >> /* >> * Swap entry may

Re: [PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-07-01 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 22, 2018 at 11:51:33AM +0800, Huang, Ying wrote: >> +++ b/mm/swap_state.c >> @@ -433,7 +433,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, >> gfp_t gfp_mask, >> /* >> * Swap entry may

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-07-01 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 29, 2018 at 09:17:16AM +0800, Huang, Ying wrote: >> Matthew Wilcox writes: >> > I'll take a look. Honestly, my biggest problem with this patch set is >> > overuse of tagging: >> > >> > 59832 Jun 22 Huang,

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-07-01 Thread Huang, Ying
Matthew Wilcox writes: > On Fri, Jun 29, 2018 at 09:17:16AM +0800, Huang, Ying wrote: >> Matthew Wilcox writes: >> > I'll take a look. Honestly, my biggest problem with this patch set is >> > overuse of tagging: >> > >> > 59832 Jun 22 Huang,

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-28 Thread Huang, Ying
Matthew Wilcox writes: > On Wed, Jun 27, 2018 at 11:18:39PM -0700, Andrew Morton wrote: >> On Thu, 28 Jun 2018 13:35:15 +0800 "Huang\, Ying" >> wrote: >> > No problem. I will rebase the patchset on your latest -mm tree, or the >> > next versi

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-28 Thread Huang, Ying
Matthew Wilcox writes: > On Wed, Jun 27, 2018 at 11:18:39PM -0700, Andrew Morton wrote: >> On Thu, 28 Jun 2018 13:35:15 +0800 "Huang\, Ying" >> wrote: >> > No problem. I will rebase the patchset on your latest -mm tree, or the >> > next versi

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-27 Thread Huang, Ying
Andrew Morton writes: > On Thu, 28 Jun 2018 13:29:09 +0800 "Huang\, Ying" > wrote: > >> Andrew Morton writes: >> >> > On Fri, 22 Jun 2018 11:51:30 +0800 "Huang, Ying" >> > wrote: >> > >> >> This is the fin

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-27 Thread Huang, Ying
Andrew Morton writes: > On Thu, 28 Jun 2018 13:29:09 +0800 "Huang\, Ying" > wrote: > >> Andrew Morton writes: >> >> > On Fri, 22 Jun 2018 11:51:30 +0800 "Huang, Ying" >> > wrote: >> > >> >> This is the fin

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-27 Thread Huang, Ying
Andrew Morton writes: > On Fri, 22 Jun 2018 11:51:30 +0800 "Huang, Ying" wrote: > >> This is the final step of THP (Transparent Huge Page) swap >> optimization. After the first and second step, the splitting huge >> page is delayed from almost the first step

Re: [PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-27 Thread Huang, Ying
Andrew Morton writes: > On Fri, 22 Jun 2018 11:51:30 +0800 "Huang, Ying" wrote: > >> This is the final step of THP (Transparent Huge Page) swap >> optimization. After the first and second step, the splitting huge >> page is delayed from almost the first step

[PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-21 Thread Huang, Ying
From: Huang Ying To support to swapin the THP as a whole, we need to create PMD swap mapping during swapout, and maintain PMD swap mapping count. This patch implements the support to increase the PMD swap mapping count (for swapout, fork, etc.) and set SWAP_HAS_CACHE flag (for swapin, etc

[PATCH -mm -v4 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-21 Thread Huang, Ying
From: Huang Ying To support to swapin the THP as a whole, we need to create PMD swap mapping during swapout, and maintain PMD swap mapping count. This patch implements the support to increase the PMD swap mapping count (for swapout, fork, etc.) and set SWAP_HAS_CACHE flag (for swapin, etc

[PATCH -mm -v4 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD

2018-06-21 Thread Huang, Ying
From: Huang Ying A huge PMD need to be split when zap a part of the PMD mapping etc. If the PMD mapping is a swap mapping, we need to split it too. This patch implemented the support for this. This is similar as splitting the PMD page mapping, except we need to decrease the PMD swap mapping

[PATCH -mm -v4 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD

2018-06-21 Thread Huang, Ying
From: Huang Ying A huge PMD need to be split when zap a part of the PMD mapping etc. If the PMD mapping is a swap mapping, we need to split it too. This patch implemented the support for this. This is similar as splitting the PMD page mapping, except we need to decrease the PMD swap mapping

[PATCH -mm -v4 10/21] mm, THP, swap: Support to count THP swapin and its fallback

2018-06-21 Thread Huang, Ying
From: Huang Ying 2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device as a whole and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangel

[PATCH -mm -v4 09/21] mm, THP, swap: Swapin a THP as a whole

2018-06-21 Thread Huang, Ying
From: Huang Ying With this patch, when page fault handler find a PMD swap mapping, it will swap in a THP as a whole. This avoids the overhead of splitting/collapsing before/after the THP swapping. And improves the swap performance greatly for reduced page fault count etc

[PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-06-21 Thread Huang, Ying
From: Huang Ying To swapin a THP as a whole, we need to read a huge swap cluster from the swap device. This patch revised the __read_swap_cache_async() and its callers and callees to support this. If __read_swap_cache_async() find the swap cluster of the specified swap entry is huge

[PATCH -mm -v4 10/21] mm, THP, swap: Support to count THP swapin and its fallback

2018-06-21 Thread Huang, Ying
From: Huang Ying 2 new /proc/vmstat fields are added, "thp_swapin" and "thp_swapin_fallback" to count swapin a THP from swap device as a whole and fallback to normal page swapin. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangel

[PATCH -mm -v4 09/21] mm, THP, swap: Swapin a THP as a whole

2018-06-21 Thread Huang, Ying
From: Huang Ying With this patch, when page fault handler find a PMD swap mapping, it will swap in a THP as a whole. This avoids the overhead of splitting/collapsing before/after the THP swapping. And improves the swap performance greatly for reduced page fault count etc

[PATCH -mm -v4 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-06-21 Thread Huang, Ying
From: Huang Ying To swapin a THP as a whole, we need to read a huge swap cluster from the swap device. This patch revised the __read_swap_cache_async() and its callers and callees to support this. If __read_swap_cache_async() find the swap cluster of the specified swap entry is huge

[PATCH -mm -v4 07/21] mm, THP, swap: Support PMD swap mapping in split_swap_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying When splitting a THP in swap cache or failing to allocate a THP when swapin a huge swap cluster, the huge swap cluster will be split. In addition to clear the huge flag of the swap cluster, the PMD swap mapping count recorded in cluster_count() will be set to 0. But we

[PATCH -mm -v4 07/21] mm, THP, swap: Support PMD swap mapping in split_swap_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying When splitting a THP in swap cache or failing to allocate a THP when swapin a huge swap cluster, the huge swap cluster will be split. In addition to clear the huge flag of the swap cluster, the PMD swap mapping count recorded in cluster_count() will be set to 0. But we

[PATCH -mm -v4 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED

2018-06-21 Thread Huang, Ying
From: Huang Ying During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" C

[PATCH -mm -v4 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED

2018-06-21 Thread Huang, Ying
From: Huang Ying During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" C

[PATCH -mm -v4 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously the huge swap cluster will be split after the THP is swapout. Now, to support to swapin the THP as a whole, the huge swap cluster will not be split after the THP is reclaimed. So in memcg, we need to move the swap account for PMD swap mappings in the process's page

[PATCH -mm -v4 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously the huge swap cluster will be split after the THP is swapout. Now, to support to swapin the THP as a whole, the huge swap cluster will not be split after the THP is reclaimed. So in memcg, we need to move the swap account for PMD swap mappings in the process's page

[PATCH -mm -v4 12/21] mm, THP, swap: Support PMD swap mapping in swapoff

2018-06-21 Thread Huang, Ying
From: Huang Ying During swapoff, for a huge swap cluster, we need to allocate a THP, read its contents into the THP and unuse the PMD and PTE swap mappings to it. If failed to allocate a THP, the huge swap cluster will be split. During unuse, if it is found that the swap cluster mapped

[PATCH -mm -v4 12/21] mm, THP, swap: Support PMD swap mapping in swapoff

2018-06-21 Thread Huang, Ying
From: Huang Ying During swapoff, for a huge swap cluster, we need to allocate a THP, read its contents into the THP and unuse the PMD and PTE swap mappings to it. If failed to allocate a THP, the huge swap cluster will be split. During unuse, if it is found that the swap cluster mapped

[PATCH -mm -v4 21/21] mm, THP: Avoid to split THP when reclaim MADV_FREE THP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, to reclaim MADV_FREE THP, the THP will be split firstly, then reclaim each sub-pages. This wastes cycles to split THP and unmap and free each sub-pages, and split THP even if it has been written since MADV_FREE. We have to do this because MADV_FREE THP reclaiming

[PATCH -mm -v4 19/21] mm, THP, swap: Support PMD swap mapping in common path

2018-06-21 Thread Huang, Ying
From: Huang Ying Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -mm -v4 21/21] mm, THP: Avoid to split THP when reclaim MADV_FREE THP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, to reclaim MADV_FREE THP, the THP will be split firstly, then reclaim each sub-pages. This wastes cycles to split THP and unmap and free each sub-pages, and split THP even if it has been written since MADV_FREE. We have to do this because MADV_FREE THP reclaiming

[PATCH -mm -v4 19/21] mm, THP, swap: Support PMD swap mapping in common path

2018-06-21 Thread Huang, Ying
From: Huang Ying Original code is only for PMD migration entry, it is revised to support PMD swap mapping. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim

[PATCH -mm -v4 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin

2018-06-21 Thread Huang, Ying
From: Huang Ying Swapin a THP as a whole isn't desirable at some situations. For example, for random access pattern, swapin a THP as a whole will inflate the reading greatly. So a sysfs interface: /sys/kernel/mm/transparent_hugepage/swapin_enabled is added to configure it. Three options

[PATCH -mm -v4 18/21] mm, THP, swap: Support PMD swap mapping in mincore()

2018-06-21 Thread Huang, Ying
From: Huang Ying During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc:

[PATCH -mm -v4 20/21] mm, THP, swap: create PMD swap mapping when unmap the THP

2018-06-21 Thread Huang, Ying
From: Huang Ying This is the final step of the THP swapin support. When reclaiming a anonymous THP, after allocating the huge swap cluster and add the THP into swap cache, the PMD page mapping will be changed to the mapping to the swap space. Previously, the PMD page mapping will be split

[PATCH -mm -v4 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin

2018-06-21 Thread Huang, Ying
From: Huang Ying Swapin a THP as a whole isn't desirable at some situations. For example, for random access pattern, swapin a THP as a whole will inflate the reading greatly. So a sysfs interface: /sys/kernel/mm/transparent_hugepage/swapin_enabled is added to configure it. Three options

[PATCH -mm -v4 18/21] mm, THP, swap: Support PMD swap mapping in mincore()

2018-06-21 Thread Huang, Ying
From: Huang Ying During mincore(), for PMD swap mapping, swap cache will be looked up. If the resulting page isn't compound page, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc:

[PATCH -mm -v4 20/21] mm, THP, swap: create PMD swap mapping when unmap the THP

2018-06-21 Thread Huang, Ying
From: Huang Ying This is the final step of the THP swapin support. When reclaiming a anonymous THP, after allocating the huge swap cluster and add the THP into swap cache, the PMD page mapping will be changed to the mapping to the swap space. Previously, the PMD page mapping will be split

[PATCH -mm -v4 16/21] mm, THP, swap: Free PMD swap mapping when zap_huge_pmd()

2018-06-21 Thread Huang, Ying
From: Huang Ying For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov"

[PATCH -mm -v4 15/21] mm, THP, swap: Support to copy PMD swap mapping when fork()

2018-06-21 Thread Huang, Ying
From: Huang Ying During fork, the page table need to be copied from parent to child. A PMD swap mapping need to be copied too and the swap reference count need to be increased. When the huge swap cluster has been split already, we need to split the PMD swap mapping and fallback to PTE copying

[PATCH -mm -v4 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called

[PATCH -mm -v4 16/21] mm, THP, swap: Free PMD swap mapping when zap_huge_pmd()

2018-06-21 Thread Huang, Ying
From: Huang Ying For a PMD swap mapping, zap_huge_pmd() will clear the PMD and call free_swap_and_cache() to decrease the swap reference count and maybe free or split the huge swap cluster and the THP in swap cache. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov"

[PATCH -mm -v4 15/21] mm, THP, swap: Support to copy PMD swap mapping when fork()

2018-06-21 Thread Huang, Ying
From: Huang Ying During fork, the page table need to be copied from parent to child. A PMD swap mapping need to be copied too and the swap reference count need to be increased. When the huge swap cluster has been split already, we need to split the PMD swap mapping and fallback to PTE copying

[PATCH -mm -v4 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called

[PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When a PMD swap mapping is removed from a huge swap cluster, for example, unmap a memory range mapped with PMD swap mapping, etc, free_swap_and_cache() will be called to decrease the reference count to the huge swap cluster. free_swap_and_cache() may also free or split the huge

[PATCH -mm -v4 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, the PMD swap operations are only enabled for CONFIG_ARCH_ENABLE_THP_MIGRATION. Because they are only used by the THP migration support. We will support PMD swap mapping to the huge swap cluster and swapin the THP as a whole. That will be enabled via

[PATCH -mm -v4 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-06-21 Thread Huang, Ying
From: Huang Ying When a PMD swap mapping is removed from a huge swap cluster, for example, unmap a memory range mapped with PMD swap mapping, etc, free_swap_and_cache() will be called to decrease the reference count to the huge swap cluster. free_swap_and_cache() may also free or split the huge

[PATCH -mm -v4 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, the PMD swap operations are only enabled for CONFIG_ARCH_ENABLE_THP_MIGRATION. Because they are only used by the THP migration support. We will support PMD swap mapping to the huge swap cluster and swapin the THP as a whole. That will be enabled via

[PATCH -mm -v4 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, during swapout, all PMD page mapping will be split and replaced with PTE swap mapping. And when clearing the SWAP_HAS_CACHE flag for the huge swap cluster in swapcache_free_cluster(), the huge swap cluster will be split. Now, during swapout, the PMD page mapping

[PATCH -mm -v4 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying It's unreasonable to optimize swapping for THP without basic swapping support. And this will cause build errors when THP_SWAP functions are defined in swapfile.c and called elsewhere. The comments are fixed too to reflect the latest progress. Signed-off-by: "Huang, Ying

[PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-21 Thread Huang, Ying
From: Huang Ying Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21

[PATCH -mm -v4 04/21] mm, THP, swap: Support PMD swap mapping in swapcache_free_cluster()

2018-06-21 Thread Huang, Ying
From: Huang Ying Previously, during swapout, all PMD page mapping will be split and replaced with PTE swap mapping. And when clearing the SWAP_HAS_CACHE flag for the huge swap cluster in swapcache_free_cluster(), the huge swap cluster will be split. Now, during swapout, the PMD page mapping

[PATCH -mm -v4 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-06-21 Thread Huang, Ying
From: Huang Ying It's unreasonable to optimize swapping for THP without basic swapping support. And this will cause build errors when THP_SWAP functions are defined in swapfile.c and called elsewhere. The comments are fixed too to reflect the latest progress. Signed-off-by: "Huang, Ying

[PATCH -mm -v4 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-21 Thread Huang, Ying
From: Huang Ying Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [02/21], [03/21], [04/21], [05/21], [06/21], [07/21], [08/21], [09/21], [10/21], [11/21

Re: [lkp-robot] [Kbuild] 050e9baa9d: netperf.Throughput_total_tps -5.6% regression (FYI)

2018-06-21 Thread Huang, Ying
t;> CONFIG_STACKPROTECTOR=y >> CONFIG_STACKPROTECTOR_STRONG=y >> CONFIG_CC_HAS_SANE_STACKPROTECTOR=y > > so you're testing the "no overhead" case to the "worst overhead" case. Do you have interest in some other comparison? Best Regards, Huang, Ying

Re: [lkp-robot] [Kbuild] 050e9baa9d: netperf.Throughput_total_tps -5.6% regression (FYI)

2018-06-21 Thread Huang, Ying
t;> CONFIG_STACKPROTECTOR=y >> CONFIG_STACKPROTECTOR_STRONG=y >> CONFIG_CC_HAS_SANE_STACKPROTECTOR=y > > so you're testing the "no overhead" case to the "worst overhead" case. Do you have interest in some other comparison? Best Regards, Huang, Ying

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-12 Thread Huang, Ying
Daniel Jordan writes: > On Tue, Jun 12, 2018 at 09:23:19AM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> >> +#else >> >> +static inline int __swap_duplicate_cluster(swp_entry_t *entry, >> > >> > This doesn't need inline. >&g

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-12 Thread Huang, Ying
Daniel Jordan writes: > On Tue, Jun 12, 2018 at 09:23:19AM +0800, Huang, Ying wrote: >> Daniel Jordan writes: >> >> +#else >> >> +static inline int __swap_duplicate_cluster(swp_entry_t *entry, >> > >> > This doesn't need inline. >&g

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-11 Thread Huang, Ying
"Huang, Ying" writes: >> On Wed, May 23, 2018 at 04:26:07PM +0800, Huang, Ying wrote: >>> @@ -3516,11 +3512,39 @@ static int __swap_duplicate(swp_entry_t entry, >>> unsigned char usage) >> >> Two comments about this part of __swap_duplicate as long

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-11 Thread Huang, Ying
"Huang, Ying" writes: >> On Wed, May 23, 2018 at 04:26:07PM +0800, Huang, Ying wrote: >>> @@ -3516,11 +3512,39 @@ static int __swap_duplicate(swp_entry_t entry, >>> unsigned char usage) >> >> Two comments about this part of __swap_duplicate as long

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-11 Thread Huang, Ying
t; - err = swapcache_prepare(entry); > + err = swapcache_prepare(entry, false); > if (err == -EEXIST) { > radix_tree_preload_end(); > /* Thanks for pointing this out! Will change in the next version

Re: [PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-06-11 Thread Huang, Ying
t; - err = swapcache_prepare(entry); > + err = swapcache_prepare(entry, false); > if (err == -EEXIST) { > radix_tree_preload_end(); > /* Thanks for pointing this out! Will change in the next version

Re: [PATCH -mm -V3 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-04 Thread Huang, Ying
Daniel Jordan writes: > On Wed, May 23, 2018 at 04:26:04PM +0800, Huang, Ying wrote: >> And for all, Any comment is welcome! >> >> This patchset is based on the 2018-05-18 head of mmotm/master. > > Trying to review this and it doesn't apply to mmotm-2018-05-18-16-4

Re: [PATCH -mm -V3 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-04 Thread Huang, Ying
Daniel Jordan writes: > On Wed, May 23, 2018 at 04:26:04PM +0800, Huang, Ying wrote: >> And for all, Any comment is welcome! >> >> This patchset is based on the 2018-05-18 head of mmotm/master. > > Trying to review this and it doesn't apply to mmotm-2018-05-18-16-4

Re: [PATCH -mm -V3 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-01 Thread Huang, Ying
Naoya Horiguchi writes: > On Wed, May 23, 2018 at 04:26:04PM +0800, Huang, Ying wrote: >> From: Huang Ying >> >> Hi, Andrew, could you help me to check whether the overall design is >> reasonable? >> >> Hi, Hugh, Shaohua, Minchan and Rik, could

Re: [PATCH -mm -V3 00/21] mm, THP, swap: Swapout/swapin THP in one piece

2018-06-01 Thread Huang, Ying
Naoya Horiguchi writes: > On Wed, May 23, 2018 at 04:26:04PM +0800, Huang, Ying wrote: >> From: Huang Ying >> >> Hi, Andrew, could you help me to check whether the overall design is >> reasonable? >> >> Hi, Hugh, Shaohua, Minchan and Rik, could

[PATCH -V2 -mm 3/4] mm, hugetlbfs: Rename address to haddr in hugetlb_cow()

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To take better advantage of general huge page copying optimization, the target subpage address will be passed to hugetlb_cow(), then copy_user_huge_page(). So we will use both target subpage address and huge page size aligned address in huget

[PATCH -V2 -mm 2/4] mm, huge page: Copy target sub-page last when copy huge page

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Huge page helps to reduce TLB miss rate, but it has higher cache footprint, sometimes this may cause some issue. For example, when copying huge page on x86_64 platform, the cache footprint is 4M. But on a Xeon E5 v3 2699 CPU, there are 18 cor

[PATCH -V2 -mm 3/4] mm, hugetlbfs: Rename address to haddr in hugetlb_cow()

2018-05-23 Thread Huang, Ying
From: Huang Ying To take better advantage of general huge page copying optimization, the target subpage address will be passed to hugetlb_cow(), then copy_user_huge_page(). So we will use both target subpage address and huge page size aligned address in hugetlb_cow(). To distinguish between

[PATCH -V2 -mm 2/4] mm, huge page: Copy target sub-page last when copy huge page

2018-05-23 Thread Huang, Ying
From: Huang Ying Huge page helps to reduce TLB miss rate, but it has higher cache footprint, sometimes this may cause some issue. For example, when copying huge page on x86_64 platform, the cache footprint is 4M. But on a Xeon E5 v3 2699 CPU, there are 18 cores, 36 threads, and only 45M LLC

[PATCH -V2 -mm 4/4] mm, hugetlbfs: Pass fault address to cow handler

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> This is to take better advantage of the general huge page copying optimization. Where, the target subpage will be copied last to avoid the cache lines of target subpage to be evicted when copying other subpages. This works better if the a

[PATCH -V2 -mm 1/4] mm, clear_huge_page: Move order algorithm into a separate function

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> In commit c79b57e462b5d ("mm: hugetlb: clear target sub-page last when clearing huge page"), to keep the cache lines of the target subpage hot, the order to clear the subpages in the huge page in clear_huge_page() is changed to clearing t

[PATCH -V2 -mm 4/4] mm, hugetlbfs: Pass fault address to cow handler

2018-05-23 Thread Huang, Ying
From: Huang Ying This is to take better advantage of the general huge page copying optimization. Where, the target subpage will be copied last to avoid the cache lines of target subpage to be evicted when copying other subpages. This works better if the address of the target subpage

[PATCH -V2 -mm 1/4] mm, clear_huge_page: Move order algorithm into a separate function

2018-05-23 Thread Huang, Ying
From: Huang Ying In commit c79b57e462b5d ("mm: hugetlb: clear target sub-page last when clearing huge page"), to keep the cache lines of the target subpage hot, the order to clear the subpages in the huge page in clear_huge_page() is changed to clearing the subpage which is fur

[PATCH -V2 -mm 0/4] mm, huge page: Copy target sub-page last when copy huge page

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Huge page helps to reduce TLB miss rate, but it has higher cache footprint, sometimes this may cause some issue. For example, when copying huge page on x86_64 platform, the cache footprint is 4M. But on a Xeon E5 v3 2699 CPU, there are 18 cor

[PATCH -V2 -mm 0/4] mm, huge page: Copy target sub-page last when copy huge page

2018-05-23 Thread Huang, Ying
From: Huang Ying Huge page helps to reduce TLB miss rate, but it has higher cache footprint, sometimes this may cause some issue. For example, when copying huge page on x86_64 platform, the cache footprint is 4M. But on a Xeon E5 v3 2699 CPU, there are 18 cores, 36 threads, and only 45M LLC

[PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To support to swapin the THP as a whole, we need to create PMD swap mapping during swapout, and maintain PMD swap mapping count. This patch implements the support to increase the PMD swap mapping count (for swapout, fork, etc.) and set SWAP_HAS_CACH

[PATCH -mm -V3 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Previously, the PMD swap operations are only enabled for CONFIG_ARCH_ENABLE_THP_MIGRATION. Because they are only used by the THP migration support. We will support PMD swap mapping to the huge swap cluster and swapin the THP as a

[PATCH -mm -V3 03/21] mm, THP, swap: Support PMD swap mapping in swap_duplicate()

2018-05-23 Thread Huang, Ying
From: Huang Ying To support to swapin the THP as a whole, we need to create PMD swap mapping during swapout, and maintain PMD swap mapping count. This patch implements the support to increase the PMD swap mapping count (for swapout, fork, etc.) and set SWAP_HAS_CACHE flag (for swapin, etc

[PATCH -mm -V3 01/21] mm, THP, swap: Enable PMD swap operations for CONFIG_THP_SWAP

2018-05-23 Thread Huang, Ying
From: Huang Ying Previously, the PMD swap operations are only enabled for CONFIG_ARCH_ENABLE_THP_MIGRATION. Because they are only used by the THP migration support. We will support PMD swap mapping to the huge swap cluster and swapin the THP as a whole. That will be enabled via

[PATCH -mm -V3 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> When a PMD swap mapping is removed from a huge swap cluster, for example, unmap a memory range mapped with PMD swap mapping, etc, free_swap_and_cache() will be called to decrease the reference count to the huge swap cluster. free_swap_and_cache() ma

[PATCH -mm -V3 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> It's unreasonable to optimize swapping for THP without basic swapping support. And this will cause build errors when THP_SWAP functions are defined in swapfile.c and called elsewhere. The comments are fixed too to reflect the latest progress. Sign

[PATCH -mm -V3 05/21] mm, THP, swap: Support PMD swap mapping in free_swap_and_cache()/swap_free()

2018-05-23 Thread Huang, Ying
From: Huang Ying When a PMD swap mapping is removed from a huge swap cluster, for example, unmap a memory range mapped with PMD swap mapping, etc, free_swap_and_cache() will be called to decrease the reference count to the huge swap cluster. free_swap_and_cache() may also free or split the huge

[PATCH -mm -V3 02/21] mm, THP, swap: Make CONFIG_THP_SWAP depends on CONFIG_SWAP

2018-05-23 Thread Huang, Ying
From: Huang Ying It's unreasonable to optimize swapping for THP without basic swapping support. And this will cause build errors when THP_SWAP functions are defined in swapfile.c and called elsewhere. The comments are fixed too to reflect the latest progress. Signed-off-by: "Huang, Ying

[PATCH -mm -V3 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> To swapin a THP as a whole, we need to read a huge swap cluster from the swap device. This patch revised the __read_swap_cache_async() and its callers and callees to support this. If __read_swap_cache_async() find the swap cluster of the specifie

[PATCH -mm -V3 08/21] mm, THP, swap: Support to read a huge swap cluster for swapin a THP

2018-05-23 Thread Huang, Ying
From: Huang Ying To swapin a THP as a whole, we need to read a huge swap cluster from the swap device. This patch revised the __read_swap_cache_async() and its callers and callees to support this. If __read_swap_cache_async() find the swap cluster of the specified swap entry is huge

[PATCH -mm -V3 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> A huge PMD need to be split when zap a part of the PMD mapping etc. If the PMD mapping is a swap mapping, we need to split it too. This patch implemented the support for this. This is similar as splitting the PMD page mapping, except we need to de

[PATCH -mm -V3 09/21] mm, THP, swap: Swapin a THP as a whole

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> With this patch, when page fault handler find a PMD swap mapping, it will swap in a THP as a whole. This avoids the overhead of splitting/collapsing before/after the THP swapping. And improves the swap performance greatly for reduced page fault cou

[PATCH -mm -V3 06/21] mm, THP, swap: Support PMD swap mapping when splitting huge PMD

2018-05-23 Thread Huang, Ying
From: Huang Ying A huge PMD need to be split when zap a part of the PMD mapping etc. If the PMD mapping is a swap mapping, we need to split it too. This patch implemented the support for this. This is similar as splitting the PMD page mapping, except we need to decrease the PMD swap mapping

[PATCH -mm -V3 09/21] mm, THP, swap: Swapin a THP as a whole

2018-05-23 Thread Huang, Ying
From: Huang Ying With this patch, when page fault handler find a PMD swap mapping, it will swap in a THP as a whole. This avoids the overhead of splitting/collapsing before/after the THP swapping. And improves the swap performance greatly for reduced page fault count etc

[PATCH -mm -V3 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Swapin a THP as a whole isn't desirable at some situations. For example, for random access pattern, swapin a THP as a whole will inflate the reading greatly. So a sysfs interface: /sys/kernel/mm/transparent_hugepage/swapin_enabled is added to con

[PATCH -mm -V3 11/21] mm, THP, swap: Add sysfs interface to configure THP swapin

2018-05-23 Thread Huang, Ying
From: Huang Ying Swapin a THP as a whole isn't desirable at some situations. For example, for random access pattern, swapin a THP as a whole will inflate the reading greatly. So a sysfs interface: /sys/kernel/mm/transparent_hugepage/swapin_enabled is added to configure it. Three options

[PATCH -mm -V3 12/21] mm, THP, swap: Support PMD swap mapping in swapoff

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> During swapoff, for a huge swap cluster, we need to allocate a THP, read its contents into the THP and unuse the PMD and PTE swap mappings to it. If failed to allocate a THP, the huge swap cluster will be split. During unuse, if it is found that th

[PATCH -mm -V3 12/21] mm, THP, swap: Support PMD swap mapping in swapoff

2018-05-23 Thread Huang, Ying
From: Huang Ying During swapoff, for a huge swap cluster, we need to allocate a THP, read its contents into the THP and unuse the PMD and PTE swap mappings to it. If failed to allocate a THP, the huge swap cluster will be split. During unuse, if it is found that the swap cluster mapped

[PATCH -mm -V3 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> Previously the huge swap cluster will be split after the THP is swapout. Now, to support to swapin the THP as a whole, the huge swap cluster will not be split after the THP is reclaimed. So in memcg, we need to move the swap account for PMD swap ma

[PATCH -mm -V3 14/21] mm, cgroup, THP, swap: Support to move swap account for PMD swap mapping

2018-05-23 Thread Huang, Ying
From: Huang Ying Previously the huge swap cluster will be split after the THP is swapout. Now, to support to swapin the THP as a whole, the huge swap cluster will not be split after the THP is reclaimed. So in memcg, we need to move the swap account for PMD swap mappings in the process's page

[PATCH -mm -V3 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED

2018-05-23 Thread Huang, Ying
From: Huang Ying <ying.hu...@intel.com> During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by:

[PATCH -mm -V3 17/21] mm, THP, swap: Support PMD swap mapping for MADV_WILLNEED

2018-05-23 Thread Huang, Ying
From: Huang Ying During MADV_WILLNEED, for a PMD swap mapping, if THP swapin is enabled for the VMA, the whole swap cluster will be swapin. Otherwise, the huge swap cluster and the PMD swap mapping will be split and fallback to PTE swap mapping. Signed-off-by: "Huang, Ying" C

<    4   5   6   7   8   9   10   11   12   13   >