RE: [PATCH 08/10] xen/arm: introduce reserved_page_list
Hi Jan > -Original Message- > From: Jan Beulich > Sent: Tuesday, May 18, 2021 7:25 PM > To: Penny Zheng > Cc: Bertrand Marquis ; Wei Chen > ; nd ; xen-devel@lists.xenproject.org; > sstabell...@kernel.org; jul...@xen.org > Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list > > On 18.05.2021 10:38, Penny Zheng wrote: > > Hi Jan > > > >> -Original Message- > >> From: Jan Beulich > >> Sent: Tuesday, May 18, 2021 3:39 PM > >> To: Penny Zheng > >> Cc: Bertrand Marquis ; Wei Chen > >> ; nd ; xen- > de...@lists.xenproject.org; > >> sstabell...@kernel.org; jul...@xen.org > >> Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list > >> > >> On 18.05.2021 07:21, Penny Zheng wrote: > >>> Since page_list under struct domain refers to linked pages as gueast > >>> RAM allocated from heap, it should not include reserved pages of > >>> static > >> memory. > >>> > >>> The number of PGC_reserved pages assigned to a domain is tracked in > >>> a new 'reserved_pages' counter. Also introduce a new > >>> reserved_page_list to link pages of static memory. Let page_to_list > >>> return reserved_page_list, when flag is PGC_reserved. > >>> > >>> Later, when domain get destroyed or restarted, those new values will > >>> help relinquish memory to proper place, not been given back to heap. > >>> > >>> Signed-off-by: Penny Zheng > >>> --- > >>> xen/common/domain.c | 1 + > >>> xen/common/page_alloc.c | 7 +-- xen/include/xen/sched.h | 5 > >>> + > >>> 3 files changed, 11 insertions(+), 2 deletions(-) > >> > >> This contradicts the title's prefix: There's no Arm-specific change here at > all. > >> But imo the title is correct, and the changes should be Arm-specific. > >> There's no point having struct domain fields on e.g. x86 which aren't used > there at all. > >> > > > > Yep, you're right. > > I'll add ifdefs in the following changes. > > As necessary, please. Moving stuff to Arm-specific files would be preferable. > Sure, I'll add a new CONFIG_STATICMEM to include all related functions and variables. Thx > Jan Cheers Penny Zheng
RE: [PATCH 08/10] xen/arm: introduce reserved_page_list
Hi Julien
> -Original Message-
> From: Julien Grall
> Sent: Tuesday, May 18, 2021 7:02 PM
> To: Penny Zheng ; xen-devel@lists.xenproject.org;
> sstabell...@kernel.org
> Cc: Bertrand Marquis ; Wei Chen
> ; nd
> Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
>
> Hi Penny,
>
> On 18/05/2021 06:21, Penny Zheng wrote:
> > Since page_list under struct domain refers to linked pages as gueast
> > RAM
>
> s/gueast/guest/
>
Thx~
> > allocated from heap, it should not include reserved pages of static memory.
>
> You already have PGC_reserved to indicate they are "static memory". So why
> do you need yet another list?
>
> >
> > The number of PGC_reserved pages assigned to a domain is tracked in a
> > new 'reserved_pages' counter. Also introduce a new reserved_page_list
> > to link pages of static memory. Let page_to_list return
> > reserved_page_list, when flag is PGC_reserved.
> >
> > Later, when domain get destroyed or restarted, those new values will
> > help relinquish memory to proper place, not been given back to heap.
> >
> > Signed-off-by: Penny Zheng
> > ---
> > xen/common/domain.c | 1 +
> > xen/common/page_alloc.c | 7 +--
> > xen/include/xen/sched.h | 5 +
> > 3 files changed, 11 insertions(+), 2 deletions(-)
> >
> > diff --git a/xen/common/domain.c b/xen/common/domain.c index
> > 6b71c6d6a9..c38afd2969 100644
> > --- a/xen/common/domain.c
> > +++ b/xen/common/domain.c
> > @@ -578,6 +578,7 @@ struct domain *domain_create(domid_t domid,
> > INIT_PAGE_LIST_HEAD(>page_list);
> > INIT_PAGE_LIST_HEAD(>extra_page_list);
> > INIT_PAGE_LIST_HEAD(>xenpage_list);
> > +INIT_PAGE_LIST_HEAD(>reserved_page_list);
> >
> > spin_lock_init(>node_affinity_lock);
> > d->node_affinity = NODE_MASK_ALL; diff --git
> > a/xen/common/page_alloc.c b/xen/common/page_alloc.c index
> > f1f1296a61..e3f07ec6c5 100644
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -2410,7 +2410,7 @@ int assign_pages(
> >
> > for ( i = 0; i < nr_pfns; i++ )
> > {
> > -ASSERT(!(pg[i].count_info & ~PGC_extra));
> > +ASSERT(!(pg[i].count_info & ~(PGC_extra |
> > + PGC_reserved)));
> I think this change belongs to the previous patch.
>
Ok. It will be re-organized into previous commit of
"xen/arm: intruduce alloc_domstatic_pages"
> > if ( pg[i].count_info & PGC_extra )
> > extra_pages++;
> > }
> > @@ -2439,6 +2439,9 @@ int assign_pages(
> > }
> > }
> >
> > +if ( pg[0].count_info & PGC_reserved )
> > +d->reserved_pages += nr_pfns;
>
> reserved_pages doesn't seem to be ever read or decremented. So why do
> you need it?
>
Yeah, I may delete it from this Patch Serie.
Like I addressed in before commits:
"when implementing rebooting domain on static allocation, memory will be
relinquished
and right now, all shall be freed back to heap, which is not suitable for
static memory here.
` relinquish_memory(d, >page_list) --> put_page --> free_domheap_page`
For pages in PGC_reserved, now, I am considering that, other than giving it
back to heap,
maybe creating a new global `struct page_info*[DOMID]` value to hold.
So it is better to have a new field in struct page_info, as follows, to hold
such info.
/* Page is reserved. */
struct {
/*
* Reserved Owner of this page,
* if this page is reserved to a specific domain.
*/
domid_t reserved_owner;
} reserved;
"
So I will delete here, and re-import when implementing rebooting domain on
static allocation.
> > +
> > if ( !(memflags & MEMF_no_refcount) &&
> >unlikely(domain_adjust_tot_pages(d, nr_pfns) == nr_pfns) )
> > get_knownalive_domain(d);
> > @@ -2452,7 +2455,7 @@ int assign_pages(
> > page_set_reserved_owner([i], d);
> > smp_wmb(); /* Domain pointer must be visible before updating
> refcnt. */
> > pg[i].count_info =
> > -(pg[i].count_info & PGC_extra) | PGC_allocated | 1;
> > +(pg[i].count_info & (PGC_extra | PGC_reserved)) |
> > + PGC_allocated | 1;
>
> Same here.
I'll re-organize it to the previous commit.
>
> > page_list_add_tail([i], page_to_list(d, [i]));
> > }
> >
> > diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index
> > 3982167144..b6333ed8bb 1006
Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
On 18.05.2021 10:38, Penny Zheng wrote: > Hi Jan > >> -Original Message- >> From: Jan Beulich >> Sent: Tuesday, May 18, 2021 3:39 PM >> To: Penny Zheng >> Cc: Bertrand Marquis ; Wei Chen >> ; nd ; xen-devel@lists.xenproject.org; >> sstabell...@kernel.org; jul...@xen.org >> Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list >> >> On 18.05.2021 07:21, Penny Zheng wrote: >>> Since page_list under struct domain refers to linked pages as gueast >>> RAM allocated from heap, it should not include reserved pages of static >> memory. >>> >>> The number of PGC_reserved pages assigned to a domain is tracked in a >>> new 'reserved_pages' counter. Also introduce a new reserved_page_list >>> to link pages of static memory. Let page_to_list return >>> reserved_page_list, when flag is PGC_reserved. >>> >>> Later, when domain get destroyed or restarted, those new values will >>> help relinquish memory to proper place, not been given back to heap. >>> >>> Signed-off-by: Penny Zheng >>> --- >>> xen/common/domain.c | 1 + >>> xen/common/page_alloc.c | 7 +-- >>> xen/include/xen/sched.h | 5 + >>> 3 files changed, 11 insertions(+), 2 deletions(-) >> >> This contradicts the title's prefix: There's no Arm-specific change here at >> all. >> But imo the title is correct, and the changes should be Arm-specific. There's >> no point having struct domain fields on e.g. x86 which aren't used there at >> all. >> > > Yep, you're right. > I'll add ifdefs in the following changes. As necessary, please. Moving stuff to Arm-specific files would be preferable. Jan
Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
Hi Penny,
On 18/05/2021 06:21, Penny Zheng wrote:
Since page_list under struct domain refers to linked pages as gueast RAM
s/gueast/guest/
allocated from heap, it should not include reserved pages of static memory.
You already have PGC_reserved to indicate they are "static memory". So
why do you need yet another list?
The number of PGC_reserved pages assigned to a domain is tracked in
a new 'reserved_pages' counter. Also introduce a new reserved_page_list
to link pages of static memory. Let page_to_list return reserved_page_list,
when flag is PGC_reserved.
Later, when domain get destroyed or restarted, those new values will help
relinquish memory to proper place, not been given back to heap.
Signed-off-by: Penny Zheng
---
xen/common/domain.c | 1 +
xen/common/page_alloc.c | 7 +--
xen/include/xen/sched.h | 5 +
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6b71c6d6a9..c38afd2969 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -578,6 +578,7 @@ struct domain *domain_create(domid_t domid,
INIT_PAGE_LIST_HEAD(>page_list);
INIT_PAGE_LIST_HEAD(>extra_page_list);
INIT_PAGE_LIST_HEAD(>xenpage_list);
+INIT_PAGE_LIST_HEAD(>reserved_page_list);
spin_lock_init(>node_affinity_lock);
d->node_affinity = NODE_MASK_ALL;
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index f1f1296a61..e3f07ec6c5 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2410,7 +2410,7 @@ int assign_pages(
for ( i = 0; i < nr_pfns; i++ )
{
-ASSERT(!(pg[i].count_info & ~PGC_extra));
+ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
I think this change belongs to the previous patch.
if ( pg[i].count_info & PGC_extra )
extra_pages++;
}
@@ -2439,6 +2439,9 @@ int assign_pages(
}
}
+if ( pg[0].count_info & PGC_reserved )
+d->reserved_pages += nr_pfns;
reserved_pages doesn't seem to be ever read or decremented. So why do
you need it?
+
if ( !(memflags & MEMF_no_refcount) &&
unlikely(domain_adjust_tot_pages(d, nr_pfns) == nr_pfns) )
get_knownalive_domain(d);
@@ -2452,7 +2455,7 @@ int assign_pages(
page_set_reserved_owner([i], d);
smp_wmb(); /* Domain pointer must be visible before updating refcnt.
*/
pg[i].count_info =
-(pg[i].count_info & PGC_extra) | PGC_allocated | 1;
+(pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated |
1;
Same here.
page_list_add_tail([i], page_to_list(d, [i]));
}
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 3982167144..b6333ed8bb 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -368,6 +368,7 @@ struct domain
struct page_list_head page_list; /* linked list */
struct page_list_head extra_page_list; /* linked list (size extra_pages)
*/
struct page_list_head xenpage_list; /* linked list (size xenheap_pages) */
+struct page_list_head reserved_page_list; /* linked list (size reserved
pages) */
/*
* This field should only be directly accessed by
domain_adjust_tot_pages()
@@ -379,6 +380,7 @@ struct domain
unsigned int outstanding_pages; /* pages claimed but not possessed */
unsigned int max_pages; /* maximum value for
domain_tot_pages() */
unsigned int extra_pages; /* pages not included in
domain_tot_pages() */
+unsigned int reserved_pages;/* pages of static memory */
atomic_t shr_pages; /* shared pages */
atomic_t paged_pages; /* paged-out pages */
@@ -588,6 +590,9 @@ static inline struct page_list_head *page_to_list(
if ( pg->count_info & PGC_extra )
return >extra_page_list;
+if ( pg->count_info & PGC_reserved )
+return >reserved_page_list;
+
return >page_list;
}
Cheers,
--
Julien Grall
RE: [PATCH 08/10] xen/arm: introduce reserved_page_list
Hi Jan
> -Original Message-
> From: Jan Beulich
> Sent: Tuesday, May 18, 2021 3:39 PM
> To: Penny Zheng
> Cc: Bertrand Marquis ; Wei Chen
> ; nd ; xen-devel@lists.xenproject.org;
> sstabell...@kernel.org; jul...@xen.org
> Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
>
> On 18.05.2021 07:21, Penny Zheng wrote:
> > Since page_list under struct domain refers to linked pages as gueast
> > RAM allocated from heap, it should not include reserved pages of static
> memory.
> >
> > The number of PGC_reserved pages assigned to a domain is tracked in a
> > new 'reserved_pages' counter. Also introduce a new reserved_page_list
> > to link pages of static memory. Let page_to_list return
> > reserved_page_list, when flag is PGC_reserved.
> >
> > Later, when domain get destroyed or restarted, those new values will
> > help relinquish memory to proper place, not been given back to heap.
> >
> > Signed-off-by: Penny Zheng
> > ---
> > xen/common/domain.c | 1 +
> > xen/common/page_alloc.c | 7 +--
> > xen/include/xen/sched.h | 5 +
> > 3 files changed, 11 insertions(+), 2 deletions(-)
>
> This contradicts the title's prefix: There's no Arm-specific change here at
> all.
> But imo the title is correct, and the changes should be Arm-specific. There's
> no point having struct domain fields on e.g. x86 which aren't used there at
> all.
>
Yep, you're right.
I'll add ifdefs in the following changes.
> > --- a/xen/common/page_alloc.c
> > +++ b/xen/common/page_alloc.c
> > @@ -2410,7 +2410,7 @@ int assign_pages(
> >
> > for ( i = 0; i < nr_pfns; i++ )
> > {
> > -ASSERT(!(pg[i].count_info & ~PGC_extra));
> > +ASSERT(!(pg[i].count_info & ~(PGC_extra |
> > + PGC_reserved)));
> > if ( pg[i].count_info & PGC_extra )
> > extra_pages++;
> > }
> > @@ -2439,6 +2439,9 @@ int assign_pages(
> > }
> > }
> >
> > +if ( pg[0].count_info & PGC_reserved )
> > +d->reserved_pages += nr_pfns;
>
> I guess this again will fail to build on x86.
>
> > @@ -588,6 +590,9 @@ static inline struct page_list_head *page_to_list(
> > if ( pg->count_info & PGC_extra )
> > return >extra_page_list;
> >
> > +if ( pg->count_info & PGC_reserved )
> > +return >reserved_page_list;
>
> Same here.
>
> Jan
Thanks
Penny
Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
On 18.05.2021 07:21, Penny Zheng wrote:
> Since page_list under struct domain refers to linked pages as gueast RAM
> allocated from heap, it should not include reserved pages of static memory.
>
> The number of PGC_reserved pages assigned to a domain is tracked in
> a new 'reserved_pages' counter. Also introduce a new reserved_page_list
> to link pages of static memory. Let page_to_list return reserved_page_list,
> when flag is PGC_reserved.
>
> Later, when domain get destroyed or restarted, those new values will help
> relinquish memory to proper place, not been given back to heap.
>
> Signed-off-by: Penny Zheng
> ---
> xen/common/domain.c | 1 +
> xen/common/page_alloc.c | 7 +--
> xen/include/xen/sched.h | 5 +
> 3 files changed, 11 insertions(+), 2 deletions(-)
This contradicts the title's prefix: There's no Arm-specific change
here at all. But imo the title is correct, and the changes should
be Arm-specific. There's no point having struct domain fields on
e.g. x86 which aren't used there at all.
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2410,7 +2410,7 @@ int assign_pages(
>
> for ( i = 0; i < nr_pfns; i++ )
> {
> -ASSERT(!(pg[i].count_info & ~PGC_extra));
> +ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
> if ( pg[i].count_info & PGC_extra )
> extra_pages++;
> }
> @@ -2439,6 +2439,9 @@ int assign_pages(
> }
> }
>
> +if ( pg[0].count_info & PGC_reserved )
> +d->reserved_pages += nr_pfns;
I guess this again will fail to build on x86.
> @@ -588,6 +590,9 @@ static inline struct page_list_head *page_to_list(
> if ( pg->count_info & PGC_extra )
> return >extra_page_list;
>
> +if ( pg->count_info & PGC_reserved )
> +return >reserved_page_list;
Same here.
Jan
[PATCH 08/10] xen/arm: introduce reserved_page_list
Since page_list under struct domain refers to linked pages as gueast RAM
allocated from heap, it should not include reserved pages of static memory.
The number of PGC_reserved pages assigned to a domain is tracked in
a new 'reserved_pages' counter. Also introduce a new reserved_page_list
to link pages of static memory. Let page_to_list return reserved_page_list,
when flag is PGC_reserved.
Later, when domain get destroyed or restarted, those new values will help
relinquish memory to proper place, not been given back to heap.
Signed-off-by: Penny Zheng
---
xen/common/domain.c | 1 +
xen/common/page_alloc.c | 7 +--
xen/include/xen/sched.h | 5 +
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 6b71c6d6a9..c38afd2969 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -578,6 +578,7 @@ struct domain *domain_create(domid_t domid,
INIT_PAGE_LIST_HEAD(>page_list);
INIT_PAGE_LIST_HEAD(>extra_page_list);
INIT_PAGE_LIST_HEAD(>xenpage_list);
+INIT_PAGE_LIST_HEAD(>reserved_page_list);
spin_lock_init(>node_affinity_lock);
d->node_affinity = NODE_MASK_ALL;
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index f1f1296a61..e3f07ec6c5 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -2410,7 +2410,7 @@ int assign_pages(
for ( i = 0; i < nr_pfns; i++ )
{
-ASSERT(!(pg[i].count_info & ~PGC_extra));
+ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
if ( pg[i].count_info & PGC_extra )
extra_pages++;
}
@@ -2439,6 +2439,9 @@ int assign_pages(
}
}
+if ( pg[0].count_info & PGC_reserved )
+d->reserved_pages += nr_pfns;
+
if ( !(memflags & MEMF_no_refcount) &&
unlikely(domain_adjust_tot_pages(d, nr_pfns) == nr_pfns) )
get_knownalive_domain(d);
@@ -2452,7 +2455,7 @@ int assign_pages(
page_set_reserved_owner([i], d);
smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
pg[i].count_info =
-(pg[i].count_info & PGC_extra) | PGC_allocated | 1;
+(pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated |
1;
page_list_add_tail([i], page_to_list(d, [i]));
}
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 3982167144..b6333ed8bb 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -368,6 +368,7 @@ struct domain
struct page_list_head page_list; /* linked list */
struct page_list_head extra_page_list; /* linked list (size extra_pages) */
struct page_list_head xenpage_list; /* linked list (size xenheap_pages) */
+struct page_list_head reserved_page_list; /* linked list (size reserved
pages) */
/*
* This field should only be directly accessed by domain_adjust_tot_pages()
@@ -379,6 +380,7 @@ struct domain
unsigned int outstanding_pages; /* pages claimed but not possessed */
unsigned int max_pages; /* maximum value for
domain_tot_pages() */
unsigned int extra_pages; /* pages not included in
domain_tot_pages() */
+unsigned int reserved_pages;/* pages of static memory */
atomic_t shr_pages; /* shared pages */
atomic_t paged_pages; /* paged-out pages */
@@ -588,6 +590,9 @@ static inline struct page_list_head *page_to_list(
if ( pg->count_info & PGC_extra )
return >extra_page_list;
+if ( pg->count_info & PGC_reserved )
+return >reserved_page_list;
+
return >page_list;
}
--
2.25.1
