Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Shaohua Li writes: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: >> Rik van Riel writes: >> >> > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> >. >> >> > - It will

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Shaohua Li writes: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: >> Rik van Riel writes: >> >> > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> >. >> >> > - It will help the memory fragmentation,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Sun, Sep 25, 2016 at 12:18:49PM -0700, Shaohua Li wrote: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > > Rik van Riel writes: > > > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote:

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Sun, Sep 25, 2016 at 12:18:49PM -0700, Shaohua Li wrote: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > > Rik van Riel writes: > > > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > >> >  > > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > Rik van Riel writes: > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >> >  > >> > - It will help the memory fragmentation, especially

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > Rik van Riel writes: > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >> >  > >> > - It will help the memory fragmentation, especially when the THP is > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Rik van Riel writes: > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >  >> > - It will help the memory fragmentation, especially when the THP is >> >   heavily used by the applications.  The 2M continuous

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Rik van Riel writes: > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >  >> > - It will help the memory fragmentation, especially when the THP is >> >   heavily used by the applications.  The 2M continuous pages will >> >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Shaohua, Thanks for comments! Shaohua Li writes: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> The advantages of the THP swap support include: Sorry for confusing. This is the advantages of the final goal, that is, avoid splitting/collapsing the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Shaohua, Thanks for comments! Shaohua Li writes: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> The advantages of the THP swap support include: Sorry for confusing. This is the advantages of the final goal, that is, avoid splitting/collapsing the THP during swap

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >  > > - It will help the memory fragmentation, especially when the THP is > >   heavily used by the applications.  The 2M continuous pages will > > be > >   free up after THP

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >  > > - It will help the memory fragmentation, especially when the THP is > >   heavily used by the applications.  The 2M continuous pages will > > be > >   free up after THP

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

"Chen, Tim C" writes: >> >>So this is impossible without THP swapin. While 2M swapout makes a lot of >>sense, I doubt 2M swapin is really useful. What kind of application is >>'optimized' >>to do sequential memory access? Anything that touches regions larger than 4K and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

"Chen, Tim C" writes: >> >>So this is impossible without THP swapin. While 2M swapout makes a lot of >>sense, I doubt 2M swapin is really useful. What kind of application is >>'optimized' >>to do sequential memory access? Anything that touches regions larger than 4K and we want to do the

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

> >So this is impossible without THP swapin. While 2M swapout makes a lot of >sense, I doubt 2M swapin is really useful. What kind of application is >'optimized' >to do sequential memory access? We waste a lot of cpu cycles to re-compact 4K pages back to a large page under THP. Swapping it back

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

> >So this is impossible without THP swapin. While 2M swapout makes a lot of >sense, I doubt 2M swapin is really useful. What kind of application is >'optimized' >to do sequential memory access? We waste a lot of cpu cycles to re-compact 4K pages back to a large page under THP. Swapping it back

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > The advantages of the THP swap support include: > > - Batch the swap operations for the THP to reduce lock > acquiring/releasing, including allocating/freeing the swap space, > adding/deleting to/from the swap cache, and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > The advantages of the THP swap support include: > > - Batch the swap operations for the THP to reduce lock > acquiring/releasing, including allocating/freeing the swap space, > adding/deleting to/from the swap cache, and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: > Hi, Minchan, > > Minchan Kim writes: > > Hi Huang, > > > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > On Tue, Sep 13, 2016 at

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: > Hi, Minchan, > > Minchan Kim writes: > > Hi Huang, > > > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Hi Huang, >> > >> > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> >> Minchan Kim

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Hi Huang, >> > >> > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > On Tue, Sep 13, 2016 at

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, 7 Sep 2016, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, 7 Sep 2016, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik, could you help me to

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Mon, 2016-09-19 at 16:11 +0900, Minchan Kim wrote: > Hi Tim, > > On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > > > > > > > > > > > > > > > > > - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > Yes, if you want, please give us how bad it is. > > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Mon, 2016-09-19 at 16:11 +0900, Minchan Kim wrote: > Hi Tim, > > On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > > > > > > > > > > > > > > > > > - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > Yes, if you want, please give us how bad it is. > > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Tim, On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > >> > >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > >Yes, if you want, please give us how bad it is. > > > > It could be pretty bad. In an experiment with THP turned on and we > enter swap, 50%

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Tim, On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > >> > >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > >Yes, if you want, please give us how bad it is. > > > > It could be pretty bad. In an experiment with THP turned on and we > enter swap, 50%

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> >> Minchan Kim writes: > >> >> >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > Hi Huang, >> >> > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > Hi Huang, >> >> > >> >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

>> >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > >Yes, if you want, please give us how bad it is. > It could be pretty bad. In an experiment with THP turned on and we enter swap, 50% of the cpu are spent in the page compaction path. So if we could deal with units of

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

>> >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > >Yes, if you want, please give us how bad it is. > It could be pretty bad. In an experiment with THP turned on and we enter swap, 50% of the cpu are spent in the page compaction path. So if we could deal with units of

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hello, On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > I am glad to discuss my final goal, that is, swapping out/in the full > THP without splitting. Why I want to do that is copied as below, I think that is a fine objective. It wasn't implemented initially just to keep things

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hello, On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > I am glad to discuss my final goal, that is, swapping out/in the full > THP without splitting. Why I want to do that is copied as below, I think that is a fine objective. It wasn't implemented initially just to keep things

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > Hi Huang, > >> > > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > Hi Huang, > >> > > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > >> > > >> > < snip > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: >> > >> > < snip > >> > >> >> >> Recently, the performance

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: >> > >> > < snip > >> > >> >> >> Recently, the performance of the storage devices improved so fast

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > Hi Huang, > > > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > > > < snip > > > > >> >> Recently, the performance of the storage devices improved so fast that > >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > Hi Huang, > > > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > > > < snip > > > > >> >> Recently, the performance of the storage devices improved so fast that > >> >> we cannot saturate

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > < snip > > >> >> Recently, the performance of the storage devices improved so fast that >> >> we cannot saturate the disk bandwidth when do page swap out even on a >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > < snip > > >> >> Recently, the performance of the storage devices improved so fast that >> >> we cannot saturate the disk bandwidth when do page swap out even on a >> >> high-end server machine.

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: < snip > > >> Recently, the performance of the storage devices improved so fast that > >> we cannot saturate the disk bandwidth when do page swap out even on a > >> high-end server machine. Because the performance of the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: < snip > > >> Recently, the performance of the storage devices improved so fast that > >> we cannot saturate the disk bandwidth when do page swap out even on a > >> high-end server machine. Because the performance of the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> From: Huang Ying >> >> This patchset is to optimize the performance of Transparent Huge Page >> (THP) swap. >> >> Hi, Andrew, could you

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> From: Huang Ying >> >> This patchset is to optimize the performance of Transparent Huge Page >> (THP) swap. >> >> Hi, Andrew, could you help me to check whether the overall design

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, 2016-09-09 at 14:43 +0900, Minchan Kim wrote: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > > > From: Huang Ying > > > > This patchset is to optimize the performance of Transparent Huge Page > > (THP) swap. > > > > Hi, Andrew,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, 2016-09-09 at 14:43 +0900, Minchan Kim wrote: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > > > From: Huang Ying > > > > This patchset is to optimize the performance of Transparent Huge Page > > (THP) swap. > > > > Hi, Andrew, could you help me to

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua,

[PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

From: Huang Ying This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the

[PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

From: Huang Ying This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially