Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Shaohua Li writes: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: >> Rik van Riel writes: >> >> > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> >. >> >> > - It will

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Shaohua Li writes: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: >> Rik van Riel writes: >> >> > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> >. >> >> > - It will help the memory fragmentation,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Sun, Sep 25, 2016 at 12:18:49PM -0700, Shaohua Li wrote: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > > Rik van Riel writes: > > > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote:

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Sun, Sep 25, 2016 at 12:18:49PM -0700, Shaohua Li wrote: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > > Rik van Riel writes: > > > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > >> >  > > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > Rik van Riel writes: > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >> >  > >> > - It will help the memory fragmentation, especially

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > Rik van Riel writes: > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >> >  > >> > - It will help the memory fragmentation, especially when the THP is > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Rik van Riel writes: > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >  >> > - It will help the memory fragmentation, especially when the THP is >> >   heavily used by the applications.  The 2M continuous

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Rik van Riel writes: > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >  >> > - It will help the memory fragmentation, especially when the THP is >> >   heavily used by the applications.  The 2M continuous pages will >> >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Shaohua, Thanks for comments! Shaohua Li writes: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> The advantages of the THP swap support include: Sorry for confusing. This is the advantages of the final goal, that is, avoid splitting/collapsing the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Shaohua, Thanks for comments! Shaohua Li writes: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> The advantages of the THP swap support include: Sorry for confusing. This is the advantages of the final goal, that is, avoid splitting/collapsing the THP during swap

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >  > > - It will help the memory fragmentation, especially when the THP is > >   heavily used by the applications.  The 2M continuous pages will > > be > >   free up after THP

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >  > > - It will help the memory fragmentation, especially when the THP is > >   heavily used by the applications.  The 2M continuous pages will > > be > >   free up after THP

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

"Chen, Tim C" writes: >> >>So this is impossible without THP swapin. While 2M swapout makes a lot of >>sense, I doubt 2M swapin is really useful. What kind of application is >>'optimized' >>to do sequential memory access? Anything that touches regions larger than 4K and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

"Chen, Tim C" writes: >> >>So this is impossible without THP swapin. While 2M swapout makes a lot of >>sense, I doubt 2M swapin is really useful. What kind of application is >>'optimized' >>to do sequential memory access? Anything that touches regions larger than 4K and we want to do the

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

> >So this is impossible without THP swapin. While 2M swapout makes a lot of >sense, I doubt 2M swapin is really useful. What kind of application is >'optimized' >to do sequential memory access? We waste a lot of cpu cycles to re-compact 4K pages back to a large page under THP. Swapping it back

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

> >So this is impossible without THP swapin. While 2M swapout makes a lot of >sense, I doubt 2M swapin is really useful. What kind of application is >'optimized' >to do sequential memory access? We waste a lot of cpu cycles to re-compact 4K pages back to a large page under THP. Swapping it back

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > The advantages of the THP swap support include: > > - Batch the swap operations for the THP to reduce lock > acquiring/releasing, including allocating/freeing the swap space, > adding/deleting to/from the swap cache, and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > The advantages of the THP swap support include: > > - Batch the swap operations for the THP to reduce lock > acquiring/releasing, including allocating/freeing the swap space, > adding/deleting to/from the swap cache, and

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: > Hi, Minchan, > > Minchan Kim writes: > > Hi Huang, > > > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > On Tue, Sep 13, 2016 at

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: > Hi, Minchan, > > Minchan Kim writes: > > Hi Huang, > > > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Hi Huang, >> > >> > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> >> Minchan Kim

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Hi Huang, >> > >> > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > On Tue, Sep 13, 2016 at

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, 7 Sep 2016, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Wed, 7 Sep 2016, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik, could you help me to

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Mon, 2016-09-19 at 16:11 +0900, Minchan Kim wrote: > Hi Tim, > > On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > > > > > > > > > > > > > > > > > - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > Yes, if you want, please give us how bad it is. > > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Mon, 2016-09-19 at 16:11 +0900, Minchan Kim wrote: > Hi Tim, > > On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > > > > > > > > > > > > > > > > > - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > Yes, if you want, please give us how bad it is. > > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Tim, On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > >> > >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > >Yes, if you want, please give us how bad it is. > > > > It could be pretty bad. In an experiment with THP turned on and we > enter swap, 50%

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Tim, On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > >> > >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > >Yes, if you want, please give us how bad it is. > > > > It could be pretty bad. In an experiment with THP turned on and we > enter swap, 50%

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> >> Minchan Kim writes: > >> >> >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > Hi Huang, >> >> > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > Hi Huang, >> >> > >> >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

>> >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > >Yes, if you want, please give us how bad it is. > It could be pretty bad. In an experiment with THP turned on and we enter swap, 50% of the cpu are spent in the page compaction path. So if we could deal with units of

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

>> >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > >Yes, if you want, please give us how bad it is. > It could be pretty bad. In an experiment with THP turned on and we enter swap, 50% of the cpu are spent in the page compaction path. So if we could deal with units of

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hello, On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > I am glad to discuss my final goal, that is, swapping out/in the full > THP without splitting. Why I want to do that is copied as below, I think that is a fine objective. It wasn't implemented initially just to keep things

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hello, On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > I am glad to discuss my final goal, that is, swapping out/in the full > THP without splitting. Why I want to do that is copied as below, I think that is a fine objective. It wasn't implemented initially just to keep things

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > Hi Huang, > >> > > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > Hi Huang, > >> > > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > >> > > >> > < snip > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: >> > >> > < snip > >> > >> >> >> Recently, the performance

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: >> > >> > < snip > >> > >> >> >> Recently, the performance of the storage devices improved so fast

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > Hi Huang, > > > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > > > < snip > > > > >> >> Recently, the performance of the storage devices improved so fast that > >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > Hi Huang, > > > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > > > < snip > > > > >> >> Recently, the performance of the storage devices improved so fast that > >> >> we cannot saturate

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > < snip > > >> >> Recently, the performance of the storage devices improved so fast that >> >> we cannot saturate the disk bandwidth when do page swap out even on a >> >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Minchan Kim writes: > Hi Huang, > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > < snip > > >> >> Recently, the performance of the storage devices improved so fast that >> >> we cannot saturate the disk bandwidth when do page swap out even on a >> >> high-end server machine.

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: < snip > > >> Recently, the performance of the storage devices improved so fast that > >> we cannot saturate the disk bandwidth when do page swap out even on a > >> high-end server machine. Because the performance of the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: < snip > > >> Recently, the performance of the storage devices improved so fast that > >> we cannot saturate the disk bandwidth when do page swap out even on a > >> high-end server machine. Because the performance of the

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> From: Huang Ying >> >> This patchset is to optimize the performance of Transparent Huge Page >> (THP) swap. >> >> Hi, Andrew, could you

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> From: Huang Ying >> >> This patchset is to optimize the performance of Transparent Huge Page >> (THP) swap. >> >> Hi, Andrew, could you help me to check whether the overall design

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, 2016-09-09 at 14:43 +0900, Minchan Kim wrote: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > > > From: Huang Ying > > > > This patchset is to optimize the performance of Transparent Huge Page > > (THP) swap. > > > > Hi, Andrew,

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

On Fri, 2016-09-09 at 14:43 +0900, Minchan Kim wrote: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > > > From: Huang Ying > > > > This patchset is to optimize the performance of Transparent Huge Page > > (THP) swap. > > > > Hi, Andrew, could you help me to

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

Hi Huang, On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua,