Re: [PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Huang, Ying]

Ming Lei  writes:

> On Thu, May 25, 2017 at 02:46:28PM +0800, Huang, Ying wrote:
>> From: Huang Ying 
>> 
>> In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
>> CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
>> THP (Transparent Huge Page) swap optimization.  Where the THP will be
>> write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
>> the various operations during swap.  And the page is likely to be
>> written to disk to free memory when system memory goes really low, the
>> memory pool need to be used to avoid deadlock.
>> 
>> Signed-off-by: "Huang, Ying" 
>> Cc: Johannes Weiner 
>> Cc: Minchan Kim 
>> Cc: Jens Axboe 
>> Cc: Ming Lei 
>> Cc: Shaohua Li 
>> Cc: linux-bl...@vger.kernel.org
>> ---
>>  include/linux/bio.h | 8 
>>  1 file changed, 8 insertions(+)
>> 
>> diff --git a/include/linux/bio.h b/include/linux/bio.h
>> index d1b04b0e99cf..314796486507 100644
>> --- a/include/linux/bio.h
>> +++ b/include/linux/bio.h
>> @@ -38,7 +38,15 @@
>>  #define BIO_BUG_ON
>>  #endif
>>  
>> +#ifdef CONFIG_THP_SWAP
>> +#if HPAGE_PMD_NR > 256
>> +#define BIO_MAX_PAGES   HPAGE_PMD_NR
>> +#else
>>  #define BIO_MAX_PAGES   256
>> +#endif
>> +#else
>> +#define BIO_MAX_PAGES   256
>> +#endif
>>  
>>  #define bio_prio(bio)   (bio)->bi_ioprio
>>  #define bio_set_prio(bio, prio) ((bio)->bi_ioprio = prio)
>
> Last time we discussed we should use multipage bvec for this usage.
>
> I will rebase the last post on v4.12-rc and kick if off again since
> the raid cleanup is just done on v4.11.
>
>   http://marc.info/?t=14845367902=1=2

Thanks for your information!  I will rebase my patchset on that after
they are merged.  From now on, this patch and the next one [07/13] is
only a temporary workaround for testing.

Best Regards,
Huang, Ying

> Thanks,
> Ming

Re: [PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Huang, Ying]

Ming Lei  writes:

> On Thu, May 25, 2017 at 02:46:28PM +0800, Huang, Ying wrote:
>> From: Huang Ying 
>> 
>> In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
>> CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
>> THP (Transparent Huge Page) swap optimization.  Where the THP will be
>> write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
>> the various operations during swap.  And the page is likely to be
>> written to disk to free memory when system memory goes really low, the
>> memory pool need to be used to avoid deadlock.
>> 
>> Signed-off-by: "Huang, Ying" 
>> Cc: Johannes Weiner 
>> Cc: Minchan Kim 
>> Cc: Jens Axboe 
>> Cc: Ming Lei 
>> Cc: Shaohua Li 
>> Cc: linux-bl...@vger.kernel.org
>> ---
>>  include/linux/bio.h | 8 
>>  1 file changed, 8 insertions(+)
>> 
>> diff --git a/include/linux/bio.h b/include/linux/bio.h
>> index d1b04b0e99cf..314796486507 100644
>> --- a/include/linux/bio.h
>> +++ b/include/linux/bio.h
>> @@ -38,7 +38,15 @@
>>  #define BIO_BUG_ON
>>  #endif
>>  
>> +#ifdef CONFIG_THP_SWAP
>> +#if HPAGE_PMD_NR > 256
>> +#define BIO_MAX_PAGES   HPAGE_PMD_NR
>> +#else
>>  #define BIO_MAX_PAGES   256
>> +#endif
>> +#else
>> +#define BIO_MAX_PAGES   256
>> +#endif
>>  
>>  #define bio_prio(bio)   (bio)->bi_ioprio
>>  #define bio_set_prio(bio, prio) ((bio)->bi_ioprio = prio)
>
> Last time we discussed we should use multipage bvec for this usage.
>
> I will rebase the last post on v4.12-rc and kick if off again since
> the raid cleanup is just done on v4.11.
>
>   http://marc.info/?t=14845367902=1=2

Thanks for your information!  I will rebase my patchset on that after
they are merged.  From now on, this patch and the next one [07/13] is
only a temporary workaround for testing.

Best Regards,
Huang, Ying

> Thanks,
> Ming

Re: [PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Ming Lei]

On Thu, May 25, 2017 at 02:46:28PM +0800, Huang, Ying wrote:
> From: Huang Ying 
> 
> In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
> CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
> THP (Transparent Huge Page) swap optimization.  Where the THP will be
> write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
> the various operations during swap.  And the page is likely to be
> written to disk to free memory when system memory goes really low, the
> memory pool need to be used to avoid deadlock.
> 
> Signed-off-by: "Huang, Ying" 
> Cc: Johannes Weiner 
> Cc: Minchan Kim 
> Cc: Jens Axboe 
> Cc: Ming Lei 
> Cc: Shaohua Li 
> Cc: linux-bl...@vger.kernel.org
> ---
>  include/linux/bio.h | 8 
>  1 file changed, 8 insertions(+)
> 
> diff --git a/include/linux/bio.h b/include/linux/bio.h
> index d1b04b0e99cf..314796486507 100644
> --- a/include/linux/bio.h
> +++ b/include/linux/bio.h
> @@ -38,7 +38,15 @@
>  #define BIO_BUG_ON
>  #endif
>  
> +#ifdef CONFIG_THP_SWAP
> +#if HPAGE_PMD_NR > 256
> +#define BIO_MAX_PAGESHPAGE_PMD_NR
> +#else
>  #define BIO_MAX_PAGES256
> +#endif
> +#else
> +#define BIO_MAX_PAGES256
> +#endif
>  
>  #define bio_prio(bio)(bio)->bi_ioprio
>  #define bio_set_prio(bio, prio)  ((bio)->bi_ioprio = prio)

Last time we discussed we should use multipage bvec for this usage.

I will rebase the last post on v4.12-rc and kick if off again since
the raid cleanup is just done on v4.11.

http://marc.info/?t=14845367902=1=2

Thanks,
Ming

Re: [PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Ming Lei]

On Thu, May 25, 2017 at 02:46:28PM +0800, Huang, Ying wrote:
> From: Huang Ying 
> 
> In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
> CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
> THP (Transparent Huge Page) swap optimization.  Where the THP will be
> write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
> the various operations during swap.  And the page is likely to be
> written to disk to free memory when system memory goes really low, the
> memory pool need to be used to avoid deadlock.
> 
> Signed-off-by: "Huang, Ying" 
> Cc: Johannes Weiner 
> Cc: Minchan Kim 
> Cc: Jens Axboe 
> Cc: Ming Lei 
> Cc: Shaohua Li 
> Cc: linux-bl...@vger.kernel.org
> ---
>  include/linux/bio.h | 8 
>  1 file changed, 8 insertions(+)
> 
> diff --git a/include/linux/bio.h b/include/linux/bio.h
> index d1b04b0e99cf..314796486507 100644
> --- a/include/linux/bio.h
> +++ b/include/linux/bio.h
> @@ -38,7 +38,15 @@
>  #define BIO_BUG_ON
>  #endif
>  
> +#ifdef CONFIG_THP_SWAP
> +#if HPAGE_PMD_NR > 256
> +#define BIO_MAX_PAGESHPAGE_PMD_NR
> +#else
>  #define BIO_MAX_PAGES256
> +#endif
> +#else
> +#define BIO_MAX_PAGES256
> +#endif
>  
>  #define bio_prio(bio)(bio)->bi_ioprio
>  #define bio_set_prio(bio, prio)  ((bio)->bi_ioprio = prio)

Last time we discussed we should use multipage bvec for this usage.

I will rebase the last post on v4.12-rc and kick if off again since
the raid cleanup is just done on v4.11.

http://marc.info/?t=14845367902=1=2

Thanks,
Ming

[PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Huang, Ying]

From: Huang Ying 

In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
THP (Transparent Huge Page) swap optimization.  Where the THP will be
write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
the various operations during swap.  And the page is likely to be
written to disk to free memory when system memory goes really low, the
memory pool need to be used to avoid deadlock.

Signed-off-by: "Huang, Ying" 
Cc: Johannes Weiner 
Cc: Minchan Kim 
Cc: Jens Axboe 
Cc: Ming Lei 
Cc: Shaohua Li 
Cc: linux-bl...@vger.kernel.org
---
 include/linux/bio.h | 8 
 1 file changed, 8 insertions(+)

diff --git a/include/linux/bio.h b/include/linux/bio.h
index d1b04b0e99cf..314796486507 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -38,7 +38,15 @@
 #define BIO_BUG_ON
 #endif
 
+#ifdef CONFIG_THP_SWAP
+#if HPAGE_PMD_NR > 256
+#define BIO_MAX_PAGES  HPAGE_PMD_NR
+#else
 #define BIO_MAX_PAGES  256
+#endif
+#else
+#define BIO_MAX_PAGES  256
+#endif
 
 #define bio_prio(bio)  (bio)->bi_ioprio
 #define bio_set_prio(bio, prio)((bio)->bi_ioprio = prio)
-- 
2.11.0

[PATCH -mm 06/13] block: Increase BIO_MAX_PAGES to PMD size if THP_SWAP enabled

[Huang, Ying]

From: Huang Ying 

In this patch, BIO_MAX_PAGES is changed from 256 to HPAGE_PMD_NR if
CONFIG_THP_SWAP is enabled and HPAGE_PMD_NR > 256.  This is to support
THP (Transparent Huge Page) swap optimization.  Where the THP will be
write to disk as a whole instead of HPAGE_PMD_NR normal pages to batch
the various operations during swap.  And the page is likely to be
written to disk to free memory when system memory goes really low, the
memory pool need to be used to avoid deadlock.

Signed-off-by: "Huang, Ying" 
Cc: Johannes Weiner 
Cc: Minchan Kim 
Cc: Jens Axboe 
Cc: Ming Lei 
Cc: Shaohua Li 
Cc: linux-bl...@vger.kernel.org
---
 include/linux/bio.h | 8 
 1 file changed, 8 insertions(+)

diff --git a/include/linux/bio.h b/include/linux/bio.h
index d1b04b0e99cf..314796486507 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -38,7 +38,15 @@
 #define BIO_BUG_ON
 #endif
 
+#ifdef CONFIG_THP_SWAP
+#if HPAGE_PMD_NR > 256
+#define BIO_MAX_PAGES  HPAGE_PMD_NR
+#else
 #define BIO_MAX_PAGES  256
+#endif
+#else
+#define BIO_MAX_PAGES  256
+#endif
 
 #define bio_prio(bio)  (bio)->bi_ioprio
 #define bio_set_prio(bio, prio)((bio)->bi_ioprio = prio)
-- 
2.11.0