Re: [V2] powerpc/Kconfig: Update config option based on page size.
On 17/09/16 01:16, Aneesh Kumar K.V wrote: > - range 9 13 if PPC64 && !PPC_64K_PAGES > + range 13 13 if PPC64 && !PPC_64K_PAGES > default "13" if PPC64 && !PPC_64K_PAGES Do we still want t to be 13 13 or 12 13? Looks like the lower side of the range can cause issues based on the values of Huge page size Balbir Singh.
Re: [V2] powerpc/Kconfig: Update config option based on page size.
Balbir Singh writes: > On 14/09/16 20:40, santhosh wrote: >> >>> Michael Ellerman writes: >>> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: > Currently on PPC64 changing kernel pagesize from 4K to 64K leaves > FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. > ... > So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages > and from 13-64 to 9-13 for 4K pages. > > Signed-off-by: Rashmica Gupta > Reviewed-by: Balbir Singh Applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >>> HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page >>> size. We do have the below constraint w.r.t hugetlb pages >>> >>> static inline bool hstate_is_gigantic(struct hstate *h) >>> { >>> return huge_page_order(h) >= MAX_ORDER; >>> } >>> >>> That require MAX_ORDER to be greater than 12. >>> > > 9 to 13 was done based on calculations you can find the commit > > > >>> Did we test hugetlbfs 4k config with this patch ? Will it work if we >>> start marking hugepage as gigantic page ? >>> >>> -aneesh >>> >> Hello Rashmica, >> >> With upstream linux kernel 4.8.0-rc1-6-gbae9cc6 compiled with linux 4k >> page size we are not able set hugepages, Aneesh had a look at the problem >> and he mentioned this commit is causing the issue. >> >> *Details:* >> We are using pkvm ubuntu 16.04 guest with upstream kernel >> [4.8.0-rc1-6-gbae9cc6] compiled with 4k page size >> >> o/p from guest: >> HugePages_Total: 0 >> HugePages_Free:0 >> HugePages_Rsvd:0 >> HugePages_Surp:0 >> Hugepagesize: 16384 kB >> >> Page sizes from device-tree: [dmesg] >> [0.00] base_shift=12: shift=12, sllp=0x, avpnm=0x, >> tlbiel=1, penc=0 >> [0.00] base_shift=12: shift=24, sllp=0x, avpnm=0x, >> tlbiel=1, penc=56 >> [0.00] base_shift=24: shift=24, sllp=0x0100, avpnm=0x0001, >> tlbiel=0, penc=0 >> >> while trying to configure the hugepages inside the guest it throws the below >> error: >> >> echo 100 > /proc/sys/vm/nr_hugepages >> -bash: echo: write error: Invalid argument >> >> *Note*: we do not see the problem when the linux page is 64k > > > Just to reiterate you are seeing this problem using 4k page size and 16M as > the hugepage size. > With FORCE_MAX_ZONEORDER set to 9 to 13 for 4k pages, you can do upto 32M if > FORCE_MAX_ZONEORDER > is 13 and same for 64K with FORCE_MAX_ZONEORDER set to 9. > > Basically the constraint is > > > FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT > > What is your value of FORCE_MAX_ZONEORDER in the .config? The problem is the reverse of why the orginal fix was done. ie, When you change the pae size from 64K to 4K using nconfig, we don't update the FORCE_MAX_ZONEORDER and hence we end up a value of 9. That results in the above error with hugetlb. So from a failed build when switching from 4k to 64K we now have a broken hugetlb when switching from 64K to 4K. As suggested in the review of the original patch, we should make the FORCE_MAX_ZONEORDER range such that it picks the right value that will get 16MB hugetlb to work. Something like the below. That range is strange, but without that it picks a value of 11. diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 927d2ab2ce08..792cb1768c8f 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -637,7 +637,7 @@ config FORCE_MAX_ZONEORDER int "Maximum zone order" range 8 9 if PPC64 && PPC_64K_PAGES default "9" if PPC64 && PPC_64K_PAGES - range 9 13 if PPC64 && !PPC_64K_PAGES + range 13 13 if PPC64 && !PPC_64K_PAGES default "13" if PPC64 && !PPC_64K_PAGES range 9 64 if PPC32 && PPC_16K_PAGES default "9" if PPC32 && PPC_16K_PAGES -aneesh
Re: [V2] powerpc/Kconfig: Update config option based on page size.
On 14/09/16 20:40, santhosh wrote: > >> Michael Ellerman writes: >> >>> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: Currently on PPC64 changing kernel pagesize from 4K to 64K leaves FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>> ... So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages and from 13-64 to 9-13 for 4K pages. Signed-off-by: Rashmica Gupta Reviewed-by: Balbir Singh >>> Applied to powerpc next, thanks. >>> >>> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >>> >> HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page >> size. We do have the below constraint w.r.t hugetlb pages >> >> static inline bool hstate_is_gigantic(struct hstate *h) >> { >> return huge_page_order(h) >= MAX_ORDER; >> } >> >> That require MAX_ORDER to be greater than 12. >> 9 to 13 was done based on calculations you can find the commit >> Did we test hugetlbfs 4k config with this patch ? Will it work if we >> start marking hugepage as gigantic page ? >> >> -aneesh >> > Hello Rashmica, > > With upstream linux kernel 4.8.0-rc1-6-gbae9cc6 compiled with linux 4k > page size we are not able set hugepages, Aneesh had a look at the problem and > he mentioned this commit is causing the issue. > > *Details:* > We are using pkvm ubuntu 16.04 guest with upstream kernel > [4.8.0-rc1-6-gbae9cc6] compiled with 4k page size > > o/p from guest: > HugePages_Total: 0 > HugePages_Free:0 > HugePages_Rsvd:0 > HugePages_Surp:0 > Hugepagesize: 16384 kB > > Page sizes from device-tree: [dmesg] > [0.00] base_shift=12: shift=12, sllp=0x, avpnm=0x, > tlbiel=1, penc=0 > [0.00] base_shift=12: shift=24, sllp=0x, avpnm=0x, > tlbiel=1, penc=56 > [0.00] base_shift=24: shift=24, sllp=0x0100, avpnm=0x0001, > tlbiel=0, penc=0 > > while trying to configure the hugepages inside the guest it throws the below > error: > > echo 100 > /proc/sys/vm/nr_hugepages > -bash: echo: write error: Invalid argument > > *Note*: we do not see the problem when the linux page is 64k Just to reiterate you are seeing this problem using 4k page size and 16M as the hugepage size. With FORCE_MAX_ZONEORDER set to 9 to 13 for 4k pages, you can do upto 32M if FORCE_MAX_ZONEORDER is 13 and same for 64K with FORCE_MAX_ZONEORDER set to 9. Basically the constraint is FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT What is your value of FORCE_MAX_ZONEORDER in the .config? Balbir Singh.
Re: [V2] powerpc/Kconfig: Update config option based on page size.
santhosh writes: >> Michael Ellerman writes: >> >>> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: Currently on PPC64 changing kernel pagesize from 4K to 64K leaves FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>> ... So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages and from 13-64 to 9-13 for 4K pages. >>> >>> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >>> >> HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page >> size. We do have the below constraint w.r.t hugetlb pages >> >> static inline bool hstate_is_gigantic(struct hstate *h) >> { >> return huge_page_order(h) >= MAX_ORDER; >> } >> >> That require MAX_ORDER to be greater than 12. So have you tried that fix? cheers
Re: [V2] powerpc/Kconfig: Update config option based on page size.
Michael Ellerman writes: On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: Currently on PPC64 changing kernel pagesize from 4K to 64K leaves FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. ... So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages and from 13-64 to 9-13 for 4K pages. Signed-off-by: Rashmica Gupta Reviewed-by: Balbir Singh Applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page size. We do have the below constraint w.r.t hugetlb pages static inline bool hstate_is_gigantic(struct hstate *h) { return huge_page_order(h) >= MAX_ORDER; } That require MAX_ORDER to be greater than 12. Did we test hugetlbfs 4k config with this patch ? Will it work if we start marking hugepage as gigantic page ? -aneesh Hello Rashmica, With upstream linux kernel 4.8.0-rc1-6-gbae9cc6 compiled with linux 4k page size we are not able set hugepages, Aneesh had a look at the problem and he mentioned this commit is causing the issue. *Details:* We are using pkvm ubuntu 16.04 guest with upstream kernel [4.8.0-rc1-6-gbae9cc6] compiled with 4k page size o/p from guest: HugePages_Total: 0 HugePages_Free:0 HugePages_Rsvd:0 HugePages_Surp:0 Hugepagesize: 16384 kB Page sizes from device-tree: [dmesg] [0.00] base_shift=12: shift=12, sllp=0x, avpnm=0x, tlbiel=1, penc=0 [0.00] base_shift=12: shift=24, sllp=0x, avpnm=0x, tlbiel=1, penc=56 [0.00] base_shift=24: shift=24, sllp=0x0100, avpnm=0x0001, tlbiel=0, penc=0 while trying to configure the hugepages inside the guest it throws the below error: echo 100 > /proc/sys/vm/nr_hugepages -bash: echo: write error: Invalid argument *Note*: we do not see the problem when the linux page is 64k Thanks, Santhosh G
Re: [V2] powerpc/Kconfig: Update config option based on page size.
On 20/04/16 00:59, Aneesh Kumar K.V wrote: > Michael Ellerman writes: > >> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >>> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >>> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>> >> ... >>> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >>> and from 13-64 to 9-13 for 4K pages. >>> >>> Signed-off-by: Rashmica Gupta >>> Reviewed-by: Balbir Singh >> >> Applied to powerpc next, thanks. >> >> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >> > > HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page > size. We do have the below constraint w.r.t hugetlb pages > > static inline bool hstate_is_gigantic(struct hstate *h) > { > return huge_page_order(h) >= MAX_ORDER; > } > > That require MAX_ORDER to be greater than 12. > The build will fail for MAX_ZONEORDER beyond the specified limits. MAX_ORDER > 12 for what page size? My understanding is this 1. gigantic refers to the fact the regular allocators cannot allocate this page 2. Use alloc_contig_range() with CONFIG_CMA for gigantic pages I could be wrong > Did we test hugetlbfs 4k config with this patch ? Will it work if we > start marking hugepage as gigantic page ? Nope.. I did not Thanks for the review! Balbir Singh ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [V2] powerpc/Kconfig: Update config option based on page size.
Michael Ellerman writes: > On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >> > ... >> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >> and from 13-64 to 9-13 for 4K pages. >> >> Signed-off-by: Rashmica Gupta >> Reviewed-by: Balbir Singh > > Applied to powerpc next, thanks. > > https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea > HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page size. We do have the below constraint w.r.t hugetlb pages static inline bool hstate_is_gigantic(struct hstate *h) { return huge_page_order(h) >= MAX_ORDER; } That require MAX_ORDER to be greater than 12. Did we test hugetlbfs 4k config with this patch ? Will it work if we start marking hugepage as gigantic page ? -aneesh ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [V2] powerpc/Kconfig: Update config option based on page size.
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: > Currently on PPC64 changing kernel pagesize from 4K to 64K leaves > FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. > ... > So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages > and from 13-64 to 9-13 for 4K pages. > > Signed-off-by: Rashmica Gupta > Reviewed-by: Balbir Singh Applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea cheers ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev