Re: [PATCH v6 0/6] Introduce ZONE_CMA

[Joonsoo Kim]

On Mon, Nov 07, 2016 at 03:25:01PM +0900, Joonsoo Kim wrote:
> On Fri, Oct 14, 2016 at 12:03:10PM +0900, js1...@gmail.com wrote:
> > From: Joonsoo Kim 
> > 
> > Hello,
> > 
> > Changes from v5
> > o Add acked/reviewed-by tag from Vlastimil and Aneesh
> > o Rebase on next-20161013
> > o Cosmetic change on patch 1
> > o Optimize span of ZONE_CMA on multiple node system
> 
> Hello, Andrew.
> 
> I got some acked/reviewed-by tags from some of main MM developers who
> are actually familiar/associated with this change. Could you merge
> this patchset to your tree to get more test coverage?
> 
> If I need to do more things to merge this patchset, please let me know
> about it.

Hello, Andrew.

Could I get your answer about this patchset?

Thanks.

Re: [PATCH v6 0/6] Introduce ZONE_CMA

[Joonsoo Kim]

On Mon, Nov 07, 2016 at 03:25:01PM +0900, Joonsoo Kim wrote:
> On Fri, Oct 14, 2016 at 12:03:10PM +0900, js1...@gmail.com wrote:
> > From: Joonsoo Kim 
> > 
> > Hello,
> > 
> > Changes from v5
> > o Add acked/reviewed-by tag from Vlastimil and Aneesh
> > o Rebase on next-20161013
> > o Cosmetic change on patch 1
> > o Optimize span of ZONE_CMA on multiple node system
> 
> Hello, Andrew.
> 
> I got some acked/reviewed-by tags from some of main MM developers who
> are actually familiar/associated with this change. Could you merge
> this patchset to your tree to get more test coverage?
> 
> If I need to do more things to merge this patchset, please let me know
> about it.

Hello, Andrew.

Could I get your answer about this patchset?

Thanks.

Re: [PATCH v6 0/6] Introduce ZONE_CMA

[Joonsoo Kim]

On Fri, Oct 14, 2016 at 12:03:10PM +0900, js1...@gmail.com wrote:
> From: Joonsoo Kim 
> 
> Hello,
> 
> Changes from v5
> o Add acked/reviewed-by tag from Vlastimil and Aneesh
> o Rebase on next-20161013
> o Cosmetic change on patch 1
> o Optimize span of ZONE_CMA on multiple node system

Hello, Andrew.

I got some acked/reviewed-by tags from some of main MM developers who
are actually familiar/associated with this change. Could you merge
this patchset to your tree to get more test coverage?

If I need to do more things to merge this patchset, please let me know
about it.

Thanks.

Re: [PATCH v6 0/6] Introduce ZONE_CMA

[Joonsoo Kim]

On Fri, Oct 14, 2016 at 12:03:10PM +0900, js1...@gmail.com wrote:
> From: Joonsoo Kim 
> 
> Hello,
> 
> Changes from v5
> o Add acked/reviewed-by tag from Vlastimil and Aneesh
> o Rebase on next-20161013
> o Cosmetic change on patch 1
> o Optimize span of ZONE_CMA on multiple node system

Hello, Andrew.

I got some acked/reviewed-by tags from some of main MM developers who
are actually familiar/associated with this change. Could you merge
this patchset to your tree to get more test coverage?

If I need to do more things to merge this patchset, please let me know
about it.

Thanks.

[PATCH v6 0/6] Introduce ZONE_CMA

[js1304]

From: Joonsoo Kim 

Hello,

Changes from v5
o Add acked/reviewed-by tag from Vlastimil and Aneesh
o Rebase on next-20161013
o Cosmetic change on patch 1
o Optimize span of ZONE_CMA on multiple node system

Changes from v4
o Rebase on next-20160825
o Add general fix patch for lowmem reserve
o Fix lowmem reserve ratio
o Fix zone span optimizaion per Vlastimil
o Fix pageset initialization
o Change invocation timing on cma_init_reserved_areas()

Changes from v3
o Rebase on next-20160805
o Split first patch per Vlastimil
o Remove useless function parameter per Vlastimil
o Add code comment per Vlastimil
o Add following description on cover-letter

This is the 4th version of ZONE_CMA patchset. Most of changes are
due to rebase and there is no major change.

CMA has many problems and I mentioned them on the bottom of the
cover letter. These problems comes from limitation of CMA memory that
should be always migratable for device usage. I think that introducing
a new zone is the best approach to solve them. Here are the reasons.

Zone is introduced to solve some issues due to H/W addressing limitation.
MM subsystem is implemented to work efficiently with these zones.
Allocation/reclaim logic in MM consider this limitation very much.
What I did in this patchset is introducing a new zone and extending zone's
concept slightly. New concept is that zone can have not only H/W addressing
limitation but also S/W limitation to guarantee page migration.
This concept is originated from ZONE_MOVABLE and it works well
for a long time. So, ZONE_CMA should not be special at this moment.

There is a major concern from Mel that ZONE_MOVABLE which has
S/W limitation causes highmem/lowmem problem. Highmem/lowmem problem is
that some of memory cannot be usable for kernel memory due to limitation
of the zone. It causes to break LRU ordering and makes hard to find kernel
usable memory when memory pressure.

However, important point is that this problem doesn't come from
implementation detail (ZONE_MOVABLE/MIGRATETYPE). Even if we implement it
by MIGRATETYPE instead of by ZONE_MOVABLE, we cannot use that type of
memory for kernel allocation because it isn't migratable. So, it will cause
to break LRU ordering, too. We cannot avoid the problem in any case.
Therefore, we should focus on which solution is better for maintainance
and not intrusive for MM subsystem.

In this viewpoint, I think that zone approach is better. As mentioned
earlier, MM subsystem already have many infrastructures to deal with
zone's H/W addressing limitation. Adding S/W limitation on zone concept
and adding a new zone doesn't change anything. It will work by itself.
My patchset can remove many hooks related to CMA area management in MM
while solving the problems. More hooks are required to solve the problems
if we choose MIGRATETYPE approach.

Although Mel withdrew the review, Vlastimil expressed an agreement on this
new zone approach [6].

 "I realize I differ here from much more experienced mm guys, and will
 probably deservingly regret it later on, but I think that the ZONE_CMA
 approach could work indeed better than current MIGRATE_CMA pageblocks."

If anyone has a different opinion, please let me know.

Thanks.


Changes from v2
o Rebase on next-20160525
o No other changes except following description

There was a discussion with Mel [5] after LSF/MM 2016. I could summarise
it to help merge decision but it's better to read by yourself since
if I summarise it, it would be biased for me. But, if anyone hope
the summary, I will do it. :)

Anyway, Mel's position on this patchset seems to be neutral. He saids:
"I'm not going to outright NAK your series but I won't ACK it either"

We can fix the problems with any approach but I hope to go a new zone
approach because it is less error-prone. It reduces some corner case
handling for now and remove need for potential corner case handling to fix
problems.

Note that our company is already using ZONE_CMA and there is no problem.

If anyone has a different opinion, please let me know and let's discuss
together.

Andrew, if there is something to do for merge, please let me know.

Changes from v1
o Separate some patches which deserve to submit independently
o Modify description to reflect current kernel state
(e.g. high-order watermark problem disappeared by Mel's work)
o Don't increase SECTION_SIZE_BITS to make a room in page flags
(detailed reason is on the patch that adds ZONE_CMA)
o Adjust ZONE_CMA population code

This series try to solve problems of current CMA implementation.

CMA is introduced to provide physically contiguous pages at runtime
without exclusive reserved memory area. But, current implementation
works like as previous reserved memory approach, because freepages
on CMA region are used only if there is no movable freepage. In other
words, freepages on CMA region are only used as fallback. In that
situation where freepages on CMA region are used as fallback, kswapd
would be woken up 

[PATCH v6 0/6] Introduce ZONE_CMA

[js1304]

From: Joonsoo Kim 

Hello,

Changes from v5
o Add acked/reviewed-by tag from Vlastimil and Aneesh
o Rebase on next-20161013
o Cosmetic change on patch 1
o Optimize span of ZONE_CMA on multiple node system

Changes from v4
o Rebase on next-20160825
o Add general fix patch for lowmem reserve
o Fix lowmem reserve ratio
o Fix zone span optimizaion per Vlastimil
o Fix pageset initialization
o Change invocation timing on cma_init_reserved_areas()

Changes from v3
o Rebase on next-20160805
o Split first patch per Vlastimil
o Remove useless function parameter per Vlastimil
o Add code comment per Vlastimil
o Add following description on cover-letter

This is the 4th version of ZONE_CMA patchset. Most of changes are
due to rebase and there is no major change.

CMA has many problems and I mentioned them on the bottom of the
cover letter. These problems comes from limitation of CMA memory that
should be always migratable for device usage. I think that introducing
a new zone is the best approach to solve them. Here are the reasons.

Zone is introduced to solve some issues due to H/W addressing limitation.
MM subsystem is implemented to work efficiently with these zones.
Allocation/reclaim logic in MM consider this limitation very much.
What I did in this patchset is introducing a new zone and extending zone's
concept slightly. New concept is that zone can have not only H/W addressing
limitation but also S/W limitation to guarantee page migration.
This concept is originated from ZONE_MOVABLE and it works well
for a long time. So, ZONE_CMA should not be special at this moment.

There is a major concern from Mel that ZONE_MOVABLE which has
S/W limitation causes highmem/lowmem problem. Highmem/lowmem problem is
that some of memory cannot be usable for kernel memory due to limitation
of the zone. It causes to break LRU ordering and makes hard to find kernel
usable memory when memory pressure.

However, important point is that this problem doesn't come from
implementation detail (ZONE_MOVABLE/MIGRATETYPE). Even if we implement it
by MIGRATETYPE instead of by ZONE_MOVABLE, we cannot use that type of
memory for kernel allocation because it isn't migratable. So, it will cause
to break LRU ordering, too. We cannot avoid the problem in any case.
Therefore, we should focus on which solution is better for maintainance
and not intrusive for MM subsystem.

In this viewpoint, I think that zone approach is better. As mentioned
earlier, MM subsystem already have many infrastructures to deal with
zone's H/W addressing limitation. Adding S/W limitation on zone concept
and adding a new zone doesn't change anything. It will work by itself.
My patchset can remove many hooks related to CMA area management in MM
while solving the problems. More hooks are required to solve the problems
if we choose MIGRATETYPE approach.

Although Mel withdrew the review, Vlastimil expressed an agreement on this
new zone approach [6].

 "I realize I differ here from much more experienced mm guys, and will
 probably deservingly regret it later on, but I think that the ZONE_CMA
 approach could work indeed better than current MIGRATE_CMA pageblocks."

If anyone has a different opinion, please let me know.

Thanks.


Changes from v2
o Rebase on next-20160525
o No other changes except following description

There was a discussion with Mel [5] after LSF/MM 2016. I could summarise
it to help merge decision but it's better to read by yourself since
if I summarise it, it would be biased for me. But, if anyone hope
the summary, I will do it. :)

Anyway, Mel's position on this patchset seems to be neutral. He saids:
"I'm not going to outright NAK your series but I won't ACK it either"

We can fix the problems with any approach but I hope to go a new zone
approach because it is less error-prone. It reduces some corner case
handling for now and remove need for potential corner case handling to fix
problems.

Note that our company is already using ZONE_CMA and there is no problem.

If anyone has a different opinion, please let me know and let's discuss
together.

Andrew, if there is something to do for merge, please let me know.

Changes from v1
o Separate some patches which deserve to submit independently
o Modify description to reflect current kernel state
(e.g. high-order watermark problem disappeared by Mel's work)
o Don't increase SECTION_SIZE_BITS to make a room in page flags
(detailed reason is on the patch that adds ZONE_CMA)
o Adjust ZONE_CMA population code

This series try to solve problems of current CMA implementation.

CMA is introduced to provide physically contiguous pages at runtime
without exclusive reserved memory area. But, current implementation
works like as previous reserved memory approach, because freepages
on CMA region are used only if there is no movable freepage. In other
words, freepages on CMA region are only used as fallback. In that
situation where freepages on CMA region are used as fallback, kswapd
would be woken up easily since there is no