Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Wed, Mar 28, 2018 at 09:45:33AM +0800, Jia He wrote:
>
>
>On 3/28/2018 8:30 AM, Wei Yang Wrote:
>> On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
>> >
>> > On 3/27/2018 9:02 AM, Wei Yang Wrote:
>> > > On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>> > > > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> > > > where possible") tried to optimize the loop in memmap_init_zone(). But
>> > > > there is still some room for improvement.
>> > > >
>> > > > Patch 1 remain the memblock_next_valid_pfn when
>> > > > CONFIG_HAVE_ARCH_PFN_VALID
>> > > > is enabled
>> > > > Patch 2 optimizes the memblock_next_valid_pfn()
>> > > > Patch 3~5 optimizes the early_pfn_valid(), I have to split it into
>> > > > parts
>> > > > because the changes are located across subsystems.
>> > > >
>> > > > I tested the pfn loop process in memmap_init(), the same as before.
>> > > > As for the performance improvement, after this set, I can see the time
>> > > > overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>> > > > armv8a server(QDF2400 with 96G memory).
>> > > >
>> > > > Attached the memblock region information in my server.
>> > > > [ 86.956758] Zone ranges:
>> > > > [ 86.959452] DMA [mem 0x0020-0x]
>> > > > [ 86.966041] Normal [mem 0x0001-0x0017]
>> > > > [ 86.972631] Movable zone start for each node
>> > > > [ 86.977179] Early memory node ranges
>> > > > [ 86.980985] node 0: [mem 0x0020-0x0021]
>> > > > [ 86.987666] node 0: [mem 0x0082-0x0307]
>> > > > [ 86.994348] node 0: [mem 0x0308-0x0308]
>> > > > [ 87.001029] node 0: [mem 0x0309-0x031f]
>> > > > [ 87.007710] node 0: [mem 0x0320-0x033f]
>> > > > [ 87.014392] node 0: [mem 0x0341-0x0563]
>> > > > [ 87.021073] node 0: [mem 0x0564-0x0567]
>> > > > [ 87.027754] node 0: [mem 0x0568-0x056d]
>> > > > [ 87.034435] node 0: [mem 0x056e-0x086f]
>> > > > [ 87.041117] node 0: [mem 0x0870-0x0871]
>> > > > [ 87.047798] node 0: [mem 0x0872-0x0894]
>> > > > [ 87.054479] node 0: [mem 0x0895-0x08ba]
>> > > > [ 87.061161] node 0: [mem 0x08bb-0x08bc]
>> > > > [ 87.067842] node 0: [mem 0x08bd-0x08c4]
>> > > > [ 87.074524] node 0: [mem 0x08c5-0x08e2]
>> > > > [ 87.081205] node 0: [mem 0x08e3-0x08e4]
>> > > > [ 87.087886] node 0: [mem 0x08e5-0x08fc]
>> > > > [ 87.094568] node 0: [mem 0x08fd-0x0910]
>> > > > [ 87.101249] node 0: [mem 0x0911-0x092e]
>> > > > [ 87.107930] node 0: [mem 0x092f-0x0930]
>> > > > [ 87.114612] node 0: [mem 0x0931-0x0963]
>> > > > [ 87.121293] node 0: [mem 0x0964-0x0e61]
>> > > > [ 87.127975] node 0: [mem 0x0e62-0x0e64]
>> > > > [ 87.134657] node 0: [mem 0x0e65-0x0fff]
>> > > > [ 87.141338] node 0: [mem 0x1080-0x17fe]
>> > > > [ 87.148019] node 0: [mem 0x1c00-0x1c00]
>> > > > [ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>> > > > [ 87.161383] node 0: [mem 0x1c81-0x7efb]
>> > > > [ 87.168064] node 0: [mem 0x7efc-0x7efd]
>> > > > [ 87.174746] node 0: [mem 0x7efe-0x7efe]
>> > > > [ 87.181427] node 0: [mem 0x7eff-0x7eff]
>> > > > [ 87.188108] node 0: [mem 0x7f00-0x0017]
>> > > Hi, Jia
>> > >
>> > > I haven't taken a deep look into your code, just one curious question on
>> > > your
>> > > memory layout.
>> > >
>> > > The log above is printed out in free_area_init_nodes(), which iterates on
>> > > memblock.memory and prints them. If I am not wrong, memory regions added
>> > > to
>> > > memblock.memory are ordered and merged if possible.
>> > >
>> > > While from your log, I see many regions could be merged but are
>> > > isolated. For
>> > > example, the last two region:
>> > >
>> > > node 0: [mem 0x7eff-0x7eff]
>> > > node 0: [mem 0x7f00-0x0017]
>> > >
>> > > So I am curious why they are isolated instead of combined to one.
>> > >
>> > > >From the code, the possible reason is the region's flag differs from
>> > > >each
>> > > other. If you have time, would you mind taking a look into this?
>> > >
>> > Hi Wei
>> > I
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Wed, Mar 28, 2018 at 09:45:33AM +0800, Jia He wrote:
>
>
>On 3/28/2018 8:30 AM, Wei Yang Wrote:
>> On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
>> >
>> > On 3/27/2018 9:02 AM, Wei Yang Wrote:
>> > > On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>> > > > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> > > > where possible") tried to optimize the loop in memmap_init_zone(). But
>> > > > there is still some room for improvement.
>> > > >
>> > > > Patch 1 remain the memblock_next_valid_pfn when
>> > > > CONFIG_HAVE_ARCH_PFN_VALID
>> > > > is enabled
>> > > > Patch 2 optimizes the memblock_next_valid_pfn()
>> > > > Patch 3~5 optimizes the early_pfn_valid(), I have to split it into
>> > > > parts
>> > > > because the changes are located across subsystems.
>> > > >
>> > > > I tested the pfn loop process in memmap_init(), the same as before.
>> > > > As for the performance improvement, after this set, I can see the time
>> > > > overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>> > > > armv8a server(QDF2400 with 96G memory).
>> > > >
>> > > > Attached the memblock region information in my server.
>> > > > [ 86.956758] Zone ranges:
>> > > > [ 86.959452] DMA [mem 0x0020-0x]
>> > > > [ 86.966041] Normal [mem 0x0001-0x0017]
>> > > > [ 86.972631] Movable zone start for each node
>> > > > [ 86.977179] Early memory node ranges
>> > > > [ 86.980985] node 0: [mem 0x0020-0x0021]
>> > > > [ 86.987666] node 0: [mem 0x0082-0x0307]
>> > > > [ 86.994348] node 0: [mem 0x0308-0x0308]
>> > > > [ 87.001029] node 0: [mem 0x0309-0x031f]
>> > > > [ 87.007710] node 0: [mem 0x0320-0x033f]
>> > > > [ 87.014392] node 0: [mem 0x0341-0x0563]
>> > > > [ 87.021073] node 0: [mem 0x0564-0x0567]
>> > > > [ 87.027754] node 0: [mem 0x0568-0x056d]
>> > > > [ 87.034435] node 0: [mem 0x056e-0x086f]
>> > > > [ 87.041117] node 0: [mem 0x0870-0x0871]
>> > > > [ 87.047798] node 0: [mem 0x0872-0x0894]
>> > > > [ 87.054479] node 0: [mem 0x0895-0x08ba]
>> > > > [ 87.061161] node 0: [mem 0x08bb-0x08bc]
>> > > > [ 87.067842] node 0: [mem 0x08bd-0x08c4]
>> > > > [ 87.074524] node 0: [mem 0x08c5-0x08e2]
>> > > > [ 87.081205] node 0: [mem 0x08e3-0x08e4]
>> > > > [ 87.087886] node 0: [mem 0x08e5-0x08fc]
>> > > > [ 87.094568] node 0: [mem 0x08fd-0x0910]
>> > > > [ 87.101249] node 0: [mem 0x0911-0x092e]
>> > > > [ 87.107930] node 0: [mem 0x092f-0x0930]
>> > > > [ 87.114612] node 0: [mem 0x0931-0x0963]
>> > > > [ 87.121293] node 0: [mem 0x0964-0x0e61]
>> > > > [ 87.127975] node 0: [mem 0x0e62-0x0e64]
>> > > > [ 87.134657] node 0: [mem 0x0e65-0x0fff]
>> > > > [ 87.141338] node 0: [mem 0x1080-0x17fe]
>> > > > [ 87.148019] node 0: [mem 0x1c00-0x1c00]
>> > > > [ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>> > > > [ 87.161383] node 0: [mem 0x1c81-0x7efb]
>> > > > [ 87.168064] node 0: [mem 0x7efc-0x7efd]
>> > > > [ 87.174746] node 0: [mem 0x7efe-0x7efe]
>> > > > [ 87.181427] node 0: [mem 0x7eff-0x7eff]
>> > > > [ 87.188108] node 0: [mem 0x7f00-0x0017]
>> > > Hi, Jia
>> > >
>> > > I haven't taken a deep look into your code, just one curious question on
>> > > your
>> > > memory layout.
>> > >
>> > > The log above is printed out in free_area_init_nodes(), which iterates on
>> > > memblock.memory and prints them. If I am not wrong, memory regions added
>> > > to
>> > > memblock.memory are ordered and merged if possible.
>> > >
>> > > While from your log, I see many regions could be merged but are
>> > > isolated. For
>> > > example, the last two region:
>> > >
>> > > node 0: [mem 0x7eff-0x7eff]
>> > > node 0: [mem 0x7f00-0x0017]
>> > >
>> > > So I am curious why they are isolated instead of combined to one.
>> > >
>> > > >From the code, the possible reason is the region's flag differs from
>> > > >each
>> > > other. If you have time, would you mind taking a look into this?
>> > >
>> > Hi Wei
>> > I
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On 3/28/2018 8:30 AM, Wei Yang Wrote:
On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
On 3/27/2018 9:02 AM, Wei Yang Wrote:
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
Hi Wei
I thought these 2 have different flags
[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
MEMBLOCK_NOMAP
[ 0.00] node 0: [mem 0x7eff-0x7eff]
[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka MEMBLOCK_NONE
[ 0.00] node 0: [mem 0x7f00-0x0017]
Thanks.
Hmm, I am not that familiar with those flags, while they look like to indicate
the physical capability of this range.
MEMBLOCK_NONE no special
MEMBLOCK_HOTPLUGhotplug-able
MEMBLOCK_MIRROR high reliable
MEMBLOCK_NOMAP no direct map
While these flags are not there when they are first added into the memory
region. When you look at the
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On 3/28/2018 8:30 AM, Wei Yang Wrote:
On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
On 3/27/2018 9:02 AM, Wei Yang Wrote:
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
Hi Wei
I thought these 2 have different flags
[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
MEMBLOCK_NOMAP
[ 0.00] node 0: [mem 0x7eff-0x7eff]
[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka MEMBLOCK_NONE
[ 0.00] node 0: [mem 0x7f00-0x0017]
Thanks.
Hmm, I am not that familiar with those flags, while they look like to indicate
the physical capability of this range.
MEMBLOCK_NONE no special
MEMBLOCK_HOTPLUGhotplug-able
MEMBLOCK_MIRROR high reliable
MEMBLOCK_NOMAP no direct map
While these flags are not there when they are first added into the memory
region. When you look at the
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
>
>
>On 3/27/2018 9:02 AM, Wei Yang Wrote:
>> On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>> > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> > where possible") tried to optimize the loop in memmap_init_zone(). But
>> > there is still some room for improvement.
>> >
>> > Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
>> > is enabled
>> > Patch 2 optimizes the memblock_next_valid_pfn()
>> > Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
>> > because the changes are located across subsystems.
>> >
>> > I tested the pfn loop process in memmap_init(), the same as before.
>> > As for the performance improvement, after this set, I can see the time
>> > overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>> > armv8a server(QDF2400 with 96G memory).
>> >
>> > Attached the memblock region information in my server.
>> > [ 86.956758] Zone ranges:
>> > [ 86.959452] DMA [mem 0x0020-0x]
>> > [ 86.966041] Normal [mem 0x0001-0x0017]
>> > [ 86.972631] Movable zone start for each node
>> > [ 86.977179] Early memory node ranges
>> > [ 86.980985] node 0: [mem 0x0020-0x0021]
>> > [ 86.987666] node 0: [mem 0x0082-0x0307]
>> > [ 86.994348] node 0: [mem 0x0308-0x0308]
>> > [ 87.001029] node 0: [mem 0x0309-0x031f]
>> > [ 87.007710] node 0: [mem 0x0320-0x033f]
>> > [ 87.014392] node 0: [mem 0x0341-0x0563]
>> > [ 87.021073] node 0: [mem 0x0564-0x0567]
>> > [ 87.027754] node 0: [mem 0x0568-0x056d]
>> > [ 87.034435] node 0: [mem 0x056e-0x086f]
>> > [ 87.041117] node 0: [mem 0x0870-0x0871]
>> > [ 87.047798] node 0: [mem 0x0872-0x0894]
>> > [ 87.054479] node 0: [mem 0x0895-0x08ba]
>> > [ 87.061161] node 0: [mem 0x08bb-0x08bc]
>> > [ 87.067842] node 0: [mem 0x08bd-0x08c4]
>> > [ 87.074524] node 0: [mem 0x08c5-0x08e2]
>> > [ 87.081205] node 0: [mem 0x08e3-0x08e4]
>> > [ 87.087886] node 0: [mem 0x08e5-0x08fc]
>> > [ 87.094568] node 0: [mem 0x08fd-0x0910]
>> > [ 87.101249] node 0: [mem 0x0911-0x092e]
>> > [ 87.107930] node 0: [mem 0x092f-0x0930]
>> > [ 87.114612] node 0: [mem 0x0931-0x0963]
>> > [ 87.121293] node 0: [mem 0x0964-0x0e61]
>> > [ 87.127975] node 0: [mem 0x0e62-0x0e64]
>> > [ 87.134657] node 0: [mem 0x0e65-0x0fff]
>> > [ 87.141338] node 0: [mem 0x1080-0x17fe]
>> > [ 87.148019] node 0: [mem 0x1c00-0x1c00]
>> > [ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>> > [ 87.161383] node 0: [mem 0x1c81-0x7efb]
>> > [ 87.168064] node 0: [mem 0x7efc-0x7efd]
>> > [ 87.174746] node 0: [mem 0x7efe-0x7efe]
>> > [ 87.181427] node 0: [mem 0x7eff-0x7eff]
>> > [ 87.188108] node 0: [mem 0x7f00-0x0017]
>> Hi, Jia
>>
>> I haven't taken a deep look into your code, just one curious question on your
>> memory layout.
>>
>> The log above is printed out in free_area_init_nodes(), which iterates on
>> memblock.memory and prints them. If I am not wrong, memory regions added to
>> memblock.memory are ordered and merged if possible.
>>
>> While from your log, I see many regions could be merged but are isolated. For
>> example, the last two region:
>>
>>node 0: [mem 0x7eff-0x7eff]
>>node 0: [mem 0x7f00-0x0017]
>>
>> So I am curious why they are isolated instead of combined to one.
>>
>> >From the code, the possible reason is the region's flag differs from each
>> other. If you have time, would you mind taking a look into this?
>>
>Hi Wei
>I thought these 2 have different flags
>[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
>MEMBLOCK_NOMAP
>[ 0.00] node 0: [mem 0x7eff-0x7eff]
>[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka MEMBLOCK_NONE
>[ 0.00] node 0: [mem 0x7f00-0x0017]
Thanks.
Hmm, I am not that familiar with those flags, while they look like to indicate
the physical capability of this range.
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Tue, Mar 27, 2018 at 03:15:08PM +0800, Jia He wrote:
>
>
>On 3/27/2018 9:02 AM, Wei Yang Wrote:
>> On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>> > Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> > where possible") tried to optimize the loop in memmap_init_zone(). But
>> > there is still some room for improvement.
>> >
>> > Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
>> > is enabled
>> > Patch 2 optimizes the memblock_next_valid_pfn()
>> > Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
>> > because the changes are located across subsystems.
>> >
>> > I tested the pfn loop process in memmap_init(), the same as before.
>> > As for the performance improvement, after this set, I can see the time
>> > overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>> > armv8a server(QDF2400 with 96G memory).
>> >
>> > Attached the memblock region information in my server.
>> > [ 86.956758] Zone ranges:
>> > [ 86.959452] DMA [mem 0x0020-0x]
>> > [ 86.966041] Normal [mem 0x0001-0x0017]
>> > [ 86.972631] Movable zone start for each node
>> > [ 86.977179] Early memory node ranges
>> > [ 86.980985] node 0: [mem 0x0020-0x0021]
>> > [ 86.987666] node 0: [mem 0x0082-0x0307]
>> > [ 86.994348] node 0: [mem 0x0308-0x0308]
>> > [ 87.001029] node 0: [mem 0x0309-0x031f]
>> > [ 87.007710] node 0: [mem 0x0320-0x033f]
>> > [ 87.014392] node 0: [mem 0x0341-0x0563]
>> > [ 87.021073] node 0: [mem 0x0564-0x0567]
>> > [ 87.027754] node 0: [mem 0x0568-0x056d]
>> > [ 87.034435] node 0: [mem 0x056e-0x086f]
>> > [ 87.041117] node 0: [mem 0x0870-0x0871]
>> > [ 87.047798] node 0: [mem 0x0872-0x0894]
>> > [ 87.054479] node 0: [mem 0x0895-0x08ba]
>> > [ 87.061161] node 0: [mem 0x08bb-0x08bc]
>> > [ 87.067842] node 0: [mem 0x08bd-0x08c4]
>> > [ 87.074524] node 0: [mem 0x08c5-0x08e2]
>> > [ 87.081205] node 0: [mem 0x08e3-0x08e4]
>> > [ 87.087886] node 0: [mem 0x08e5-0x08fc]
>> > [ 87.094568] node 0: [mem 0x08fd-0x0910]
>> > [ 87.101249] node 0: [mem 0x0911-0x092e]
>> > [ 87.107930] node 0: [mem 0x092f-0x0930]
>> > [ 87.114612] node 0: [mem 0x0931-0x0963]
>> > [ 87.121293] node 0: [mem 0x0964-0x0e61]
>> > [ 87.127975] node 0: [mem 0x0e62-0x0e64]
>> > [ 87.134657] node 0: [mem 0x0e65-0x0fff]
>> > [ 87.141338] node 0: [mem 0x1080-0x17fe]
>> > [ 87.148019] node 0: [mem 0x1c00-0x1c00]
>> > [ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>> > [ 87.161383] node 0: [mem 0x1c81-0x7efb]
>> > [ 87.168064] node 0: [mem 0x7efc-0x7efd]
>> > [ 87.174746] node 0: [mem 0x7efe-0x7efe]
>> > [ 87.181427] node 0: [mem 0x7eff-0x7eff]
>> > [ 87.188108] node 0: [mem 0x7f00-0x0017]
>> Hi, Jia
>>
>> I haven't taken a deep look into your code, just one curious question on your
>> memory layout.
>>
>> The log above is printed out in free_area_init_nodes(), which iterates on
>> memblock.memory and prints them. If I am not wrong, memory regions added to
>> memblock.memory are ordered and merged if possible.
>>
>> While from your log, I see many regions could be merged but are isolated. For
>> example, the last two region:
>>
>>node 0: [mem 0x7eff-0x7eff]
>>node 0: [mem 0x7f00-0x0017]
>>
>> So I am curious why they are isolated instead of combined to one.
>>
>> >From the code, the possible reason is the region's flag differs from each
>> other. If you have time, would you mind taking a look into this?
>>
>Hi Wei
>I thought these 2 have different flags
>[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
>MEMBLOCK_NOMAP
>[ 0.00] node 0: [mem 0x7eff-0x7eff]
>[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka MEMBLOCK_NONE
>[ 0.00] node 0: [mem 0x7f00-0x0017]
Thanks.
Hmm, I am not that familiar with those flags, while they look like to indicate
the physical capability of this range.
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On 3/27/2018 9:02 AM, Wei Yang Wrote:
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
Hi Wei
I thought these 2 have different flags
[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
MEMBLOCK_NOMAP
[ 0.00] node 0: [mem 0x7eff-0x7eff]
[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka
MEMBLOCK_NONE
[ 0.00] node 0: [mem 0x7f00-0x0017]
--
Cheers,
Jia
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On 3/27/2018 9:02 AM, Wei Yang Wrote:
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
Hi Wei
I thought these 2 have different flags
[ 0.00] idx=30,region [7eff:1]flag=4 <--- aka
MEMBLOCK_NOMAP
[ 0.00] node 0: [mem 0x7eff-0x7eff]
[ 0.00] idx=31,region [7f00:8100]flag=0 <--- aka
MEMBLOCK_NONE
[ 0.00] node 0: [mem 0x7f00-0x0017]
--
Cheers,
Jia
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>where possible") tried to optimize the loop in memmap_init_zone(). But
>there is still some room for improvement.
>
>Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
>is enabled
>Patch 2 optimizes the memblock_next_valid_pfn()
>Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
>because the changes are located across subsystems.
>
>I tested the pfn loop process in memmap_init(), the same as before.
>As for the performance improvement, after this set, I can see the time
>overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>armv8a server(QDF2400 with 96G memory).
>
>Attached the memblock region information in my server.
>[ 86.956758] Zone ranges:
>[ 86.959452] DMA [mem 0x0020-0x]
>[ 86.966041] Normal [mem 0x0001-0x0017]
>[ 86.972631] Movable zone start for each node
>[ 86.977179] Early memory node ranges
>[ 86.980985] node 0: [mem 0x0020-0x0021]
>[ 86.987666] node 0: [mem 0x0082-0x0307]
>[ 86.994348] node 0: [mem 0x0308-0x0308]
>[ 87.001029] node 0: [mem 0x0309-0x031f]
>[ 87.007710] node 0: [mem 0x0320-0x033f]
>[ 87.014392] node 0: [mem 0x0341-0x0563]
>[ 87.021073] node 0: [mem 0x0564-0x0567]
>[ 87.027754] node 0: [mem 0x0568-0x056d]
>[ 87.034435] node 0: [mem 0x056e-0x086f]
>[ 87.041117] node 0: [mem 0x0870-0x0871]
>[ 87.047798] node 0: [mem 0x0872-0x0894]
>[ 87.054479] node 0: [mem 0x0895-0x08ba]
>[ 87.061161] node 0: [mem 0x08bb-0x08bc]
>[ 87.067842] node 0: [mem 0x08bd-0x08c4]
>[ 87.074524] node 0: [mem 0x08c5-0x08e2]
>[ 87.081205] node 0: [mem 0x08e3-0x08e4]
>[ 87.087886] node 0: [mem 0x08e5-0x08fc]
>[ 87.094568] node 0: [mem 0x08fd-0x0910]
>[ 87.101249] node 0: [mem 0x0911-0x092e]
>[ 87.107930] node 0: [mem 0x092f-0x0930]
>[ 87.114612] node 0: [mem 0x0931-0x0963]
>[ 87.121293] node 0: [mem 0x0964-0x0e61]
>[ 87.127975] node 0: [mem 0x0e62-0x0e64]
>[ 87.134657] node 0: [mem 0x0e65-0x0fff]
>[ 87.141338] node 0: [mem 0x1080-0x17fe]
>[ 87.148019] node 0: [mem 0x1c00-0x1c00]
>[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>[ 87.161383] node 0: [mem 0x1c81-0x7efb]
>[ 87.168064] node 0: [mem 0x7efc-0x7efd]
>[ 87.174746] node 0: [mem 0x7efe-0x7efe]
>[ 87.181427] node 0: [mem 0x7eff-0x7eff]
>[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
--
Wei Yang
Help you, Help me
Re: [PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
On Sun, Mar 25, 2018 at 08:02:14PM -0700, Jia He wrote:
>Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>where possible") tried to optimize the loop in memmap_init_zone(). But
>there is still some room for improvement.
>
>Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
>is enabled
>Patch 2 optimizes the memblock_next_valid_pfn()
>Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
>because the changes are located across subsystems.
>
>I tested the pfn loop process in memmap_init(), the same as before.
>As for the performance improvement, after this set, I can see the time
>overhead of memmap_init() is reduced from 41313 us to 24345 us in my
>armv8a server(QDF2400 with 96G memory).
>
>Attached the memblock region information in my server.
>[ 86.956758] Zone ranges:
>[ 86.959452] DMA [mem 0x0020-0x]
>[ 86.966041] Normal [mem 0x0001-0x0017]
>[ 86.972631] Movable zone start for each node
>[ 86.977179] Early memory node ranges
>[ 86.980985] node 0: [mem 0x0020-0x0021]
>[ 86.987666] node 0: [mem 0x0082-0x0307]
>[ 86.994348] node 0: [mem 0x0308-0x0308]
>[ 87.001029] node 0: [mem 0x0309-0x031f]
>[ 87.007710] node 0: [mem 0x0320-0x033f]
>[ 87.014392] node 0: [mem 0x0341-0x0563]
>[ 87.021073] node 0: [mem 0x0564-0x0567]
>[ 87.027754] node 0: [mem 0x0568-0x056d]
>[ 87.034435] node 0: [mem 0x056e-0x086f]
>[ 87.041117] node 0: [mem 0x0870-0x0871]
>[ 87.047798] node 0: [mem 0x0872-0x0894]
>[ 87.054479] node 0: [mem 0x0895-0x08ba]
>[ 87.061161] node 0: [mem 0x08bb-0x08bc]
>[ 87.067842] node 0: [mem 0x08bd-0x08c4]
>[ 87.074524] node 0: [mem 0x08c5-0x08e2]
>[ 87.081205] node 0: [mem 0x08e3-0x08e4]
>[ 87.087886] node 0: [mem 0x08e5-0x08fc]
>[ 87.094568] node 0: [mem 0x08fd-0x0910]
>[ 87.101249] node 0: [mem 0x0911-0x092e]
>[ 87.107930] node 0: [mem 0x092f-0x0930]
>[ 87.114612] node 0: [mem 0x0931-0x0963]
>[ 87.121293] node 0: [mem 0x0964-0x0e61]
>[ 87.127975] node 0: [mem 0x0e62-0x0e64]
>[ 87.134657] node 0: [mem 0x0e65-0x0fff]
>[ 87.141338] node 0: [mem 0x1080-0x17fe]
>[ 87.148019] node 0: [mem 0x1c00-0x1c00]
>[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
>[ 87.161383] node 0: [mem 0x1c81-0x7efb]
>[ 87.168064] node 0: [mem 0x7efc-0x7efd]
>[ 87.174746] node 0: [mem 0x7efe-0x7efe]
>[ 87.181427] node 0: [mem 0x7eff-0x7eff]
>[ 87.188108] node 0: [mem 0x7f00-0x0017]
Hi, Jia
I haven't taken a deep look into your code, just one curious question on your
memory layout.
The log above is printed out in free_area_init_nodes(), which iterates on
memblock.memory and prints them. If I am not wrong, memory regions added to
memblock.memory are ordered and merged if possible.
While from your log, I see many regions could be merged but are isolated. For
example, the last two region:
node 0: [mem 0x7eff-0x7eff]
node 0: [mem 0x7f00-0x0017]
So I am curious why they are isolated instead of combined to one.
>From the code, the possible reason is the region's flag differs from each
other. If you have time, would you mind taking a look into this?
--
Wei Yang
Help you, Help me
[PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
[ 87.194791] Initmem setup node 0 [mem 0x0020-0x0017]
Without this patchset:
[ 117.106153] Initmem setup node 0 [mem 0x0020-0x0017]
[ 117.113677] before memmap_init
[ 117.118195] after memmap_init
>>> memmap_init takes 4518 us
[ 117.121446] before memmap_init
[ 117.154992] after memmap_init
>>> memmap_init takes 33546 us
[ 117.158241] before memmap_init
[ 117.161490] after memmap_init
>>> memmap_init takes 3249 us
>>> totally takes 41313 us
With this patchset:
[ 87.194791] Initmem setup node 0 [mem 0x0020-0x0017]
[ 87.202314] before memmap_init
[ 87.206164] after memmap_init
>>> memmap_init takes 3850 us
[ 87.209416] before memmap_init
[ 87.226662] after memmap_init
>>> memmap_init takes 17246 us
[ 87.229911] before memmap_init
[ 87.233160] after memmap_init
>>> memmap_init takes 3249 us
>>> totally takes 24345 us
Changelog:
V3: - fix 2 issues reported by kbuild test robot
V2: - rebase to mmotm latest
- remain memblock_next_valid_pfn on arm64
- refine memblock_search_pfn_regions and pfn_valid_region
Jia He (5):
mm: page_alloc: remain memblock_next_valid_pfn() when
CONFIG_HAVE_ARCH_PFN_VALID is enable
mm: page_alloc: reduce unnecessary binary search in
memblock_next_valid_pfn()
mm/memblock: introduce memblock_search_pfn_regions()
arm64: introduce pfn_valid_region()
mm: page_alloc: reduce unnecessary binary search in early_pfn_valid()
arch/arm64/include/asm/page.h| 3 ++-
arch/arm64/mm/init.c | 25 ++-
arch/x86/include/asm/mmzone_32.h | 2 +-
include/linux/memblock.h | 6 +
include/linux/mmzone.h
[PATCH v3 0/5] optimize memblock_next_valid_pfn and early_pfn_valid
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") tried to optimize the loop in memmap_init_zone(). But
there is still some room for improvement.
Patch 1 remain the memblock_next_valid_pfn when CONFIG_HAVE_ARCH_PFN_VALID
is enabled
Patch 2 optimizes the memblock_next_valid_pfn()
Patch 3~5 optimizes the early_pfn_valid(), I have to split it into parts
because the changes are located across subsystems.
I tested the pfn loop process in memmap_init(), the same as before.
As for the performance improvement, after this set, I can see the time
overhead of memmap_init() is reduced from 41313 us to 24345 us in my
armv8a server(QDF2400 with 96G memory).
Attached the memblock region information in my server.
[ 86.956758] Zone ranges:
[ 86.959452] DMA [mem 0x0020-0x]
[ 86.966041] Normal [mem 0x0001-0x0017]
[ 86.972631] Movable zone start for each node
[ 86.977179] Early memory node ranges
[ 86.980985] node 0: [mem 0x0020-0x0021]
[ 86.987666] node 0: [mem 0x0082-0x0307]
[ 86.994348] node 0: [mem 0x0308-0x0308]
[ 87.001029] node 0: [mem 0x0309-0x031f]
[ 87.007710] node 0: [mem 0x0320-0x033f]
[ 87.014392] node 0: [mem 0x0341-0x0563]
[ 87.021073] node 0: [mem 0x0564-0x0567]
[ 87.027754] node 0: [mem 0x0568-0x056d]
[ 87.034435] node 0: [mem 0x056e-0x086f]
[ 87.041117] node 0: [mem 0x0870-0x0871]
[ 87.047798] node 0: [mem 0x0872-0x0894]
[ 87.054479] node 0: [mem 0x0895-0x08ba]
[ 87.061161] node 0: [mem 0x08bb-0x08bc]
[ 87.067842] node 0: [mem 0x08bd-0x08c4]
[ 87.074524] node 0: [mem 0x08c5-0x08e2]
[ 87.081205] node 0: [mem 0x08e3-0x08e4]
[ 87.087886] node 0: [mem 0x08e5-0x08fc]
[ 87.094568] node 0: [mem 0x08fd-0x0910]
[ 87.101249] node 0: [mem 0x0911-0x092e]
[ 87.107930] node 0: [mem 0x092f-0x0930]
[ 87.114612] node 0: [mem 0x0931-0x0963]
[ 87.121293] node 0: [mem 0x0964-0x0e61]
[ 87.127975] node 0: [mem 0x0e62-0x0e64]
[ 87.134657] node 0: [mem 0x0e65-0x0fff]
[ 87.141338] node 0: [mem 0x1080-0x17fe]
[ 87.148019] node 0: [mem 0x1c00-0x1c00]
[ 87.154701] node 0: [mem 0x1c01-0x1c7f]
[ 87.161383] node 0: [mem 0x1c81-0x7efb]
[ 87.168064] node 0: [mem 0x7efc-0x7efd]
[ 87.174746] node 0: [mem 0x7efe-0x7efe]
[ 87.181427] node 0: [mem 0x7eff-0x7eff]
[ 87.188108] node 0: [mem 0x7f00-0x0017]
[ 87.194791] Initmem setup node 0 [mem 0x0020-0x0017]
Without this patchset:
[ 117.106153] Initmem setup node 0 [mem 0x0020-0x0017]
[ 117.113677] before memmap_init
[ 117.118195] after memmap_init
>>> memmap_init takes 4518 us
[ 117.121446] before memmap_init
[ 117.154992] after memmap_init
>>> memmap_init takes 33546 us
[ 117.158241] before memmap_init
[ 117.161490] after memmap_init
>>> memmap_init takes 3249 us
>>> totally takes 41313 us
With this patchset:
[ 87.194791] Initmem setup node 0 [mem 0x0020-0x0017]
[ 87.202314] before memmap_init
[ 87.206164] after memmap_init
>>> memmap_init takes 3850 us
[ 87.209416] before memmap_init
[ 87.226662] after memmap_init
>>> memmap_init takes 17246 us
[ 87.229911] before memmap_init
[ 87.233160] after memmap_init
>>> memmap_init takes 3249 us
>>> totally takes 24345 us
Changelog:
V3: - fix 2 issues reported by kbuild test robot
V2: - rebase to mmotm latest
- remain memblock_next_valid_pfn on arm64
- refine memblock_search_pfn_regions and pfn_valid_region
Jia He (5):
mm: page_alloc: remain memblock_next_valid_pfn() when
CONFIG_HAVE_ARCH_PFN_VALID is enable
mm: page_alloc: reduce unnecessary binary search in
memblock_next_valid_pfn()
mm/memblock: introduce memblock_search_pfn_regions()
arm64: introduce pfn_valid_region()
mm: page_alloc: reduce unnecessary binary search in early_pfn_valid()
arch/arm64/include/asm/page.h| 3 ++-
arch/arm64/mm/init.c | 25 ++-
arch/x86/include/asm/mmzone_32.h | 2 +-
include/linux/memblock.h | 6 +
include/linux/mmzone.h
