[virtio-dev] Re: [virtio-comment] Re: [virtio-dev] [PATCH v8 1/3] content: Document balloon feature free page hints
On Wed, Nov 11, 2020 at 11:45 PM Cornelia Huck wrote:
>
> On Wed, 11 Nov 2020 14:09:23 -0800
> Alexander Duyck wrote:
>
> > On Wed, Nov 11, 2020 at 10:03 AM Cornelia Huck wrote:
> > >
> > > On Wed, 16 Sep 2020 09:28:34 -0400
> > > "Michael S. Tsirkin" wrote:
> > >
> > > > On Mon, Sep 14, 2020 at 08:13:55AM +0200, Cornelia Huck wrote:
> > > > > On Fri, 11 Sep 2020 08:11:15 -0700
> > > > > Alexander Duyck wrote:
> > > > >
> > > > > > On Thu, Sep 10, 2020 at 10:28 PM Jan Kiszka
> > > > > > wrote:
> > > > > > >
> > > > > > > On 04.09.20 18:56, Alexander Duyck wrote:
> > > > > > > > On Fri, Sep 4, 2020 at 8:20 AM Jan Kiszka
> > > > > > > > wrote:
> > > > > > > >>
> > > > > > > >> On 25.08.20 16:45, Alexander Duyck wrote:
> > > > >
> > > > > > > >>> @@ -5042,13 +5049,17 @@ \subsection{Feature
> > > > > > > >>> bits}\label{sec:Device Types / Memory Balloon Device / Featu
> > > > > > > >>> VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> > > > > > > >>>
> > > > > > > >>> \subsection{Device configuration layout}\label{sec:Device
> > > > > > > >>> Types / Memory Balloon Device / Device configuration layout}
> > > > > > > >>> - Both fields of this configuration
> > > > > > > >>> - are always available.
> > > > > > > >>> + \field{num_pages} and \field{actual} are always available.
> > > > > > > >>> +
> > > > > > > >>> + \field{free_page_hint_cmd_id} is available if
> > > > > > > >>> +VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and
> > > > > > > >>> is read-only by
> > > > > > > >>> +the driver.
> > > > > > > >>
> > > > > > > >> This reads at least to me like "...if
> > > > > > > >> VIRTIO_BALLOON_F_FREE_PAGE_HINT
> > > > > > > >> ... is read-only by the driver". I suspect you rather meant
> > > > > > > >> "free_page_hint_cmd_id is read-only...". Maybe split up into
> > > > > > > >> two sentences?
> > > > > > > >
> > > > > > > > Yes, the intention is:
> > > > > > > > 1. free_page_hint_cmd_id is only available if
> > > > > > > > VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated
> > > > > > > > 2. free_page_hint_cmd_id is read only by the driver
> > > > > > > >
> > > > > > > > If needed I suppose we could break it up by splitting it into
> > > > > > > > two
> > > > > > > > sentences, or adding "the field" after the "and".
> > > > > > > >
> > > > > > >
> > > > > > > I'm fine with both options but please adjust this - on top
> > > > > > > (Michael just
> > > > > > > opened the voting for this version again due to the formal typo
> > > > > > > in round 1).
> > > > > > >
> > > > > > > Jan
> > > > > >
> > > > > > Since the patch set is being voted on is there a preferred method
> > > > > > for
> > > > > > making this sort of update? I'm just wondering if I should do an
> > > > > > additional incremental patch, just submit a replacement for this
> > > > > > patch, or make the change and resubmit the entire patch set?
> > > > >
> > > > > I think doing an additional patch on top and then doing another vote
> > > > > on
> > > > > that is the best way to handle this. (Or maybe it is minor enough to
> > > > > simply merge the incremental patch?)
> > > >
> > > > yes, this sounds good to me
> > >
> > > Which one of the options, actually? I'd be happy to merge a proper
> > > patch on top (which I'd consider a trivial fix.)
> >
> > I had submitted a 4/3 for this set that was meant to fix it, but
> > looking it over I don't think it would apply cleanly as I think I had
> > only applied the patch it fixed and didn't have the poison patch
> > applied when I generated it. If you need I can resubmit that patch as
> > a trivial fix since it is only a 2 line change.
>
> Thanks, that would be great.
No problem. I just submitted it as a standalone patch and added the
Reviewed-by you had added when I submitted before.
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH] content: Minor change to clarify free_page_hint_cmd_id
From: Alexander Duyck
The original wording was a bit unclear and could have been misinterpreted
as indicating that VIRTIO_BALLOON_FREE_PAGE_HINT was read-only instead of
the field free_page_hint_cmd_id. To clarify that break it up into two
sentences making it clear that the field is only available if the feature
is negotiated, and that the field is read-only.
Reviewed-by: Cornelia Huck
Signed-off-by: Alexander Duyck
---
content.tex |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/content.tex b/content.tex
index 21e0cee903b0..9c39831acc15 100644
--- a/content.tex
+++ b/content.tex
@@ -5206,8 +5206,8 @@ \subsection{Device configuration layout}\label{sec:Device
Types / Memory Balloon
\field{num_pages} and \field{actual} are always available.
\field{free_page_hint_cmd_id} is available if
-VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
-the driver.
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated. The field is
+read-only by the driver.
\field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
negotiated.
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] Re: [virtio-dev] [PATCH v8 1/3] content: Document balloon feature free page hints
On Wed, Nov 11, 2020 at 10:03 AM Cornelia Huck wrote:
>
> On Wed, 16 Sep 2020 09:28:34 -0400
> "Michael S. Tsirkin" wrote:
>
> > On Mon, Sep 14, 2020 at 08:13:55AM +0200, Cornelia Huck wrote:
> > > On Fri, 11 Sep 2020 08:11:15 -0700
> > > Alexander Duyck wrote:
> > >
> > > > On Thu, Sep 10, 2020 at 10:28 PM Jan Kiszka
> > > > wrote:
> > > > >
> > > > > On 04.09.20 18:56, Alexander Duyck wrote:
> > > > > > On Fri, Sep 4, 2020 at 8:20 AM Jan Kiszka
> > > > > > wrote:
> > > > > >>
> > > > > >> On 25.08.20 16:45, Alexander Duyck wrote:
> > >
> > > > > >>> @@ -5042,13 +5049,17 @@ \subsection{Feature
> > > > > >>> bits}\label{sec:Device Types / Memory Balloon Device / Featu
> > > > > >>> VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> > > > > >>>
> > > > > >>> \subsection{Device configuration layout}\label{sec:Device Types
> > > > > >>> / Memory Balloon Device / Device configuration layout}
> > > > > >>> - Both fields of this configuration
> > > > > >>> - are always available.
> > > > > >>> + \field{num_pages} and \field{actual} are always available.
> > > > > >>> +
> > > > > >>> + \field{free_page_hint_cmd_id} is available if
> > > > > >>> +VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is
> > > > > >>> read-only by
> > > > > >>> +the driver.
> > > > > >>
> > > > > >> This reads at least to me like "...if
> > > > > >> VIRTIO_BALLOON_F_FREE_PAGE_HINT
> > > > > >> ... is read-only by the driver". I suspect you rather meant
> > > > > >> "free_page_hint_cmd_id is read-only...". Maybe split up into two
> > > > > >> sentences?
> > > > > >
> > > > > > Yes, the intention is:
> > > > > > 1. free_page_hint_cmd_id is only available if
> > > > > > VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated
> > > > > > 2. free_page_hint_cmd_id is read only by the driver
> > > > > >
> > > > > > If needed I suppose we could break it up by splitting it into two
> > > > > > sentences, or adding "the field" after the "and".
> > > > > >
> > > > >
> > > > > I'm fine with both options but please adjust this - on top (Michael
> > > > > just
> > > > > opened the voting for this version again due to the formal typo in
> > > > > round 1).
> > > > >
> > > > > Jan
> > > >
> > > > Since the patch set is being voted on is there a preferred method for
> > > > making this sort of update? I'm just wondering if I should do an
> > > > additional incremental patch, just submit a replacement for this
> > > > patch, or make the change and resubmit the entire patch set?
> > >
> > > I think doing an additional patch on top and then doing another vote on
> > > that is the best way to handle this. (Or maybe it is minor enough to
> > > simply merge the incremental patch?)
> >
> > yes, this sounds good to me
>
> Which one of the options, actually? I'd be happy to merge a proper
> patch on top (which I'd consider a trivial fix.)
I had submitted a 4/3 for this set that was meant to fix it, but
looking it over I don't think it would apply cleanly as I think I had
only applied the patch it fixed and didn't have the poison patch
applied when I generated it. If you need I can resubmit that patch as
a trivial fix since it is only a 2 line change.
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v8 4/3] content: Minor change to clarify free_page_hint_cmd_id
From: Alexander Duyck
The original wording was a bit unclear and could have been misinterpreted
as indicating that VIRTIO_BALLOON_FREE_PAGE_HINT was read-only instead of
the field free_page_hint_cmd_id. To clarify that break it up into two
sentences making it clear that the field is only available if the feature
is negotiated, and that the field is read-only.
Signed-off-by: Alexander Duyck
---
So this patch addresses a minor nit that was pointed out after the ballot
had already been created. If need be I can either open a new bugzilla and
request a ballot to have it added.
content.tex |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/content.tex b/content.tex
index 76dfce919b97..f74090491534 100644
--- a/content.tex
+++ b/content.tex
@@ -5052,8 +5052,8 @@ \subsection{Device configuration layout}\label{sec:Device
Types / Memory Balloon
\field{num_pages} and \field{actual} are always available.
\field{free_page_hint_cmd_id} is available if
-VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
-the driver.
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated. The field is
+read-only by the driver.
\begin{lstlisting}
struct virtio_balloon_config {
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] Re: [virtio-dev] [PATCH v8 1/3] content: Document balloon feature free page hints
On Thu, Sep 10, 2020 at 10:28 PM Jan Kiszka wrote:
>
> On 04.09.20 18:56, Alexander Duyck wrote:
> > On Fri, Sep 4, 2020 at 8:20 AM Jan Kiszka wrote:
> >>
> >> On 25.08.20 16:45, Alexander Duyck wrote:
> >>> From: Alexander Duyck
> >>>
> >>> Free page hints allow the balloon driver to provide information on what
> >>> pages are not currently in use so that we can avoid the cost of copying
> >>> them in migration scenarios. Add a feature description for free page hints
> >>> describing basic functioning and requirements.
> >>>
> >>> Acked-by: Cornelia Huck
> >>> Reviewed-by: David Hildenbrand
> >>> Signed-off-by: Alexander Duyck
> >>> ---
> >>> conformance.tex |2 +
> >>> content.tex | 161
> >>> +--
> >>> 2 files changed, 157 insertions(+), 6 deletions(-)
> >>>
> >>> diff --git a/conformance.tex b/conformance.tex
> >>> index b6fdec090383..a14e26edfcb2 100644
> >>> --- a/conformance.tex
> >>> +++ b/conformance.tex
> >>> @@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> >>> Conformance Targets}
> >>> \item \ref{drivernormative:Device Types / Memory Balloon Device /
> >>> Feature bits}
> >>> \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> >>> Operation}
> >>> \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> >>> Operation / Memory Statistics}
> >>> +\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> >>> Operation / Free Page Hinting}
> >>> \end{itemize}
> >>>
> >>> \conformance{\subsection}{SCSI Host Driver
> >>> Conformance}\label{sec:Conformance / Driver Conformance / SCSI Host
> >>> Driver Conformance}
> >>> @@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> >>> Conformance Targets}
> >>> \item \ref{devicenormative:Device Types / Memory Balloon Device /
> >>> Feature bits}
> >>> \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> >>> Operation}
> >>> \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> >>> Operation / Memory Statistics}
> >>> +\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> >>> Operation / Free Page Hinting}
> >>> \end{itemize}
> >>>
> >>> \conformance{\subsection}{SCSI Host Device
> >>> Conformance}\label{sec:Conformance / Device Conformance / SCSI Host
> >>> Device Conformance}
> >>> diff --git a/content.tex b/content.tex
> >>> index 91735e3eb018..76dfce919b97 100644
> >>> --- a/content.tex
> >>> +++ b/content.tex
> >>> @@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types /
> >>> Memory Balloon Device / Virtque
> >>> \begin{description}
> >>> \item[0] inflateq
> >>> \item[1] deflateq
> >>> -\item[2] statsq.
> >>> +\item[2] statsq
> >>> +\item[3] free_page_vq
> >>> \end{description}
> >>>
> >>> - Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
> >>> + statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> >>> +
> >>> + free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >>>
> >>> \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device
> >>> / Feature bits}
> >>> \begin{description}
> >>> @@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> >>> Memory Balloon Device / Featu
> >>> memory statistics is present.
> >>> \item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
> >>> guest out of memory condition.
> >>> +\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for
> >>> free
> >>> +page hinting. A virtqueue for providing hints as to what memory is
> >>> +currently free is present. Configuration field
> >>> \field{free_page_hint_cmd_id}
> >>> +is valid.
> >>>
> >>> \end{description}
> >>>
> >>> @@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types
> >>&
[virtio-dev] Re: [PATCH v8 0/3] virtio-spec: Add documentation for recently added balloon features
On Tue, Sep 1, 2020 at 7:34 AM Michael S. Tsirkin wrote: > > On Tue, Aug 25, 2020 at 01:40:25PM -0700, Alexander Duyck wrote: > > On Tue, Aug 25, 2020 at 8:01 AM Cornelia Huck wrote: > > > > > > On Tue, 25 Aug 2020 07:44:55 -0700 > > > Alexander Duyck wrote: > > > > > > > This patch set is meant to add documentation for balloon features that > > > > have > > > > been recently added to the Linux kernel[1,2] and that were recently > > > > added > > > > to QEMU[3]. > > > > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 > > > > > > > > Changes since RFC: > > > > Incorporated suggestions from Cornelia Huck > > > > Fixed a few additional spelling errors > > > > > > > > Changes since v1: > > > > Incorporated additional suggestions from Cornelia Huck > > > > Dropped documentation referring to free page reporting from page poison > > > > patch > > > > > > > > Changes since v2: > > > > Rewrote multiple statements based on input from David Hildenbrand > > > > Dropped use of balloon and deflate from page hinting description > > > > Dropped use of free page reporting from page poison description > > > > Cleaned up several spots that didn't match RFC2119 style comments > > > > Added conformance links. > > > > Various other clean-ups. > > > > Updated balloon command IDs based on input from Cornelia Huck > > > > > > > > Changes since v3: > > > > Reordered patches to place free page hinting at end of patch set > > > > Moved contents out of patch to poison and free page reporting patches > > > > Updated patch description to document some known issues with feature > > > > Further clean-ups based on input from David Hildenbrand > > > > > > > > Changes since v4: > > > > Further clean-ups for page hinting based on input from David Hildenbrand > > > > > > > > Changes since v5: > > > > Reverted patcher order changes from v3 placing hinting as first patch > > > > Addressed comments and added Reviewed-by from Cornelia Huck > > > > Addressed further items I found in free page hinting > > > > Added requirement that output buffers be 4 bytes in size > > > > Replaced reference to descriptor with buffer > > > > Replaced references to free_page_hint_vq with free_page_vq > > > > > > > > Changes since v6: > > > > Fixed line that was left over from a merge conflict > > > > Added Reviewed-by from David Hildenbrand > > > > Created issue and added "Fixes" to cover page > > > > > > > > Changes since v7: > > > > Removed comments from patch 1 describing issues that we fixed or > > > > addressed > > > > Added Acked-by for patch 1 from Cornelia Huck > > > > > > > > [1]: > > > > https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ > > > > [2]: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc > > > > [3]: > > > > https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html > > > > > > > > --- > > > > > > > > Alexander Duyck (3): > > > > content: Document balloon feature free page hints > > > > content: Document balloon feature page poison > > > > content: Document balloon feature free page reporting > > > > > > > > > > > > conformance.tex |6 + > > > > content.tex | 286 > > > > ++- > > > > 2 files changed, 286 insertions(+), 6 deletions(-) > > > > > > > > -- > > > > > > > > > > I think this is good to vote on now. > > > > Thanks for the review feedback. If there are no other comments then I > > request a vote for inclusion. > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 > > > > - Alex > > Ballot created at https://www.oasis-open.org/committees/ballot.php?id=3521 > Once it concludes, would appreciate a reminder to merge the patch. > Thanks! So it looks like it was voted down due to a bad link in the ballot. The correct link that references the patch set is in the bugzilla. Do I need to resubmit the patch set or can a new ballot be created? Thanks. - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
Re: [virtio-dev] [PATCH v8 1/3] content: Document balloon feature free page hints
On Fri, Sep 4, 2020 at 8:20 AM Jan Kiszka wrote:
>
> On 25.08.20 16:45, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Free page hints allow the balloon driver to provide information on what
> > pages are not currently in use so that we can avoid the cost of copying
> > them in migration scenarios. Add a feature description for free page hints
> > describing basic functioning and requirements.
> >
> > Acked-by: Cornelia Huck
> > Reviewed-by: David Hildenbrand
> > Signed-off-by: Alexander Duyck
> > ---
> > conformance.tex |2 +
> > content.tex | 161
> > +--
> > 2 files changed, 157 insertions(+), 6 deletions(-)
> >
> > diff --git a/conformance.tex b/conformance.tex
> > index b6fdec090383..a14e26edfcb2 100644
> > --- a/conformance.tex
> > +++ b/conformance.tex
> > @@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Feature
> > bits}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > +\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Hinting}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Driver
> > Conformance}\label{sec:Conformance / Driver Conformance / SCSI Host Driver
> > Conformance}
> > @@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Feature
> > bits}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > +\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Hinting}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Device
> > Conformance}\label{sec:Conformance / Device Conformance / SCSI Host Device
> > Conformance}
> > diff --git a/content.tex b/content.tex
> > index 91735e3eb018..76dfce919b97 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \begin{description}
> > \item[0] inflateq
> > \item[1] deflateq
> > -\item[2] statsq.
> > +\item[2] statsq
> > +\item[3] free_page_vq
> > \end{description}
> >
> > - Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
> > + statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> > +
> > + free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > @@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > memory statistics is present.
> > \item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
> > guest out of memory condition.
> > +\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
> > +page hinting. A virtqueue for providing hints as to what memory is
> > +currently free is present. Configuration field
> > \field{free_page_hint_cmd_id}
> > +is valid.
> >
> > \end{description}
> >
> > @@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> >
> > \subsection{Device configuration layout}\label{sec:Device Types / Memory
> > Balloon Device / Device configuration layout}
> > - Both fields of this configuration
> > - are always available.
> > + \field{num_pages} and \field{actual} are always available.
> > +
> > + \field{free_page_hint_cmd_id} is available if
> > +VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
> > +the driver.
>
> This reads at least to me like "...if VIRTIO_BALLOON_F_FREE_PAGE_HINT
> ... is read-only by the driver". I suspect you rather meant
> "free_page_hint_cmd_id is read-only...". Maybe split up into two sentences?
Yes, the intention is:
1. free_page_hint_cmd_id is only available if
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated
2. free_page_hint_cmd_id is read only by the driver
If needed I suppose we could break it up by splitting it into two
sentences, or adding "the field" after the "and".
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH v8 0/3] virtio-spec: Add documentation for recently added balloon features
On Tue, Aug 25, 2020 at 8:01 AM Cornelia Huck wrote: > > On Tue, 25 Aug 2020 07:44:55 -0700 > Alexander Duyck wrote: > > > This patch set is meant to add documentation for balloon features that have > > been recently added to the Linux kernel[1,2] and that were recently added > > to QEMU[3]. > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 > > > > Changes since RFC: > > Incorporated suggestions from Cornelia Huck > > Fixed a few additional spelling errors > > > > Changes since v1: > > Incorporated additional suggestions from Cornelia Huck > > Dropped documentation referring to free page reporting from page poison > > patch > > > > Changes since v2: > > Rewrote multiple statements based on input from David Hildenbrand > > Dropped use of balloon and deflate from page hinting description > > Dropped use of free page reporting from page poison description > > Cleaned up several spots that didn't match RFC2119 style comments > > Added conformance links. > > Various other clean-ups. > > Updated balloon command IDs based on input from Cornelia Huck > > > > Changes since v3: > > Reordered patches to place free page hinting at end of patch set > > Moved contents out of patch to poison and free page reporting patches > > Updated patch description to document some known issues with feature > > Further clean-ups based on input from David Hildenbrand > > > > Changes since v4: > > Further clean-ups for page hinting based on input from David Hildenbrand > > > > Changes since v5: > > Reverted patcher order changes from v3 placing hinting as first patch > > Addressed comments and added Reviewed-by from Cornelia Huck > > Addressed further items I found in free page hinting > > Added requirement that output buffers be 4 bytes in size > > Replaced reference to descriptor with buffer > > Replaced references to free_page_hint_vq with free_page_vq > > > > Changes since v6: > > Fixed line that was left over from a merge conflict > > Added Reviewed-by from David Hildenbrand > > Created issue and added "Fixes" to cover page > > > > Changes since v7: > > Removed comments from patch 1 describing issues that we fixed or addressed > > Added Acked-by for patch 1 from Cornelia Huck > > > > [1]: > > https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ > > [2]: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc > > [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html > > > > --- > > > > Alexander Duyck (3): > > content: Document balloon feature free page hints > > content: Document balloon feature page poison > > content: Document balloon feature free page reporting > > > > > > conformance.tex |6 + > > content.tex | 286 > > ++- > > 2 files changed, 286 insertions(+), 6 deletions(-) > > > > -- > > > > I think this is good to vote on now. Thanks for the review feedback. If there are no other comments then I request a vote for inclusion. Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v8 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Reviewed-by: Cornelia Huck
Reviewed-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 46 ++
2 files changed, 48 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index a14e26edfcb2..5038b36324ac 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 76dfce919b97..0569d633e3ba 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field
\field{free_page_hint_cmd_id}
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5033,6 +5036,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5054,12 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
\field{free_page_hint_cmd_id} is available if
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5087,6 +5097,8 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated,
identify the free_page_vq.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
+ the \field{poison_val} configuration field.
\item DRIVER_OK is set: device operation begins.
@@ -5494,6 +5506,40 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
endian of the guest rather than (necessarily when not using the legacy
interface) little-endian.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host that the guest is initializing
+free
[virtio-dev] [PATCH v8 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that were recently added to QEMU[3]. Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand Changes since v4: Further clean-ups for page hinting based on input from David Hildenbrand Changes since v5: Reverted patcher order changes from v3 placing hinting as first patch Addressed comments and added Reviewed-by from Cornelia Huck Addressed further items I found in free page hinting Added requirement that output buffers be 4 bytes in size Replaced reference to descriptor with buffer Replaced references to free_page_hint_vq with free_page_vq Changes since v6: Fixed line that was left over from a merge conflict Added Reviewed-by from David Hildenbrand Created issue and added "Fixes" to cover page Changes since v7: Removed comments from patch 1 describing issues that we fixed or addressed Added Acked-by for patch 1 from Cornelia Huck [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting conformance.tex |6 + content.tex | 286 ++- 2 files changed, 286 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v8 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Reviewed-by: Cornelia Huck
Reviewed-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 81 ++-
2 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/conformance.tex b/conformance.tex
index 5038b36324ac..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 0569d633e3ba..2f6ddc64b9c0 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5040,6 +5045,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5099,11 +5108,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
identify the free_page_vq.
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated,
+ identify the reporting_vq.
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated, then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5510,7 +5523,8 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host that the guest is initializing
free pages with \field{poison_val}. When the feature is enabled, pages
[virtio-dev] Re: [PATCH v7 0/3] virtio-spec: Add documentation for recently added balloon features
On Mon, Aug 24, 2020 at 7:10 AM Alexander Duyck wrote: > > This patch set is meant to add documentation for balloon features that have > been recently added to the Linux kernel[1,2] and that were recently added > to QEMU[3]. > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 Since there was only one minor nit about v6 which is addressed in v7 I feel that this patch set is ready for inclusion. If there are no further comments I request a vote for inclusion. Thanks. - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v7 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Reviewed-by: Cornelia Huck
Reviewed-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 81 ++-
2 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/conformance.tex b/conformance.tex
index 5038b36324ac..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 0569d633e3ba..2f6ddc64b9c0 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5040,6 +5045,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5099,11 +5108,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
identify the free_page_vq.
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated,
+ identify the reporting_vq.
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated, then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5510,7 +5523,8 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host that the guest is initializing
free pages with \field{poison_val}. When the feature is enabled, pages
[virtio-dev] [PATCH v7 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Reviewed-by: Cornelia Huck
Reviewed-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 46 ++
2 files changed, 48 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index a14e26edfcb2..5038b36324ac 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 76dfce919b97..0569d633e3ba 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field
\field{free_page_hint_cmd_id}
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5033,6 +5036,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5054,12 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
\field{free_page_hint_cmd_id} is available if
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5087,6 +5097,8 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated,
identify the free_page_vq.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
+ the \field{poison_val} configuration field.
\item DRIVER_OK is set: device operation begins.
@@ -5494,6 +5506,40 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
endian of the guest rather than (necessarily when not using the legacy
interface) little-endian.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host that the guest is initializing
+free
[virtio-dev] [PATCH v7 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that were recently added to QEMU[3]. Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand Changes since v4: Further clean-ups for page hinting based on input from David Hildenbrand Changes since v5: Reverted patcher order changes from v3 placing hinting as first patch Addressed comments and added Reviewed-by from Cornelia Huck Addressed further items I found in free page hinting Added requirement that output buffers be 4 bytes in size Replaced reference to descriptor with buffer Replaced references to free_page_hint_vq with free_page_vq Changes since v6: Fixed line that was left over from a merge conflict Added Reviewed-by from David Hildenbrand Created issue and added "Fixes" to cover page [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting conformance.tex |6 + content.tex | 286 ++- 2 files changed, 286 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v7 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
In working on this the specification as pointed out certain issues with the
Linux driver and QEMU device implementation. The issues include:
1. The Linux driver does not re-initialize pages when it reuses them
before receiving the "DONE" command, as such this can lead to possible data
corruption.
2. The QEMU device is not returning the "DONE" command if a migration
fails. This results in the guest holding onto pages until forced out by the
shrinker.
There are also additional issues that have been found not related to the
specification.
There is currently discussion on if the feature should be removed so this
patch is a place-holder for if we decide to keep the feature and fix the
issues. Otherwise this patch can be dropped and we can work on a patch to
document the need to avoid the feature.
Reviewed-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 161 +--
2 files changed, 157 insertions(+), 6 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..a14e26edfcb2 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..76dfce919b97 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field
\field{free_page_hint_cmd_id}
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALL
[virtio-dev] Re: [virtio-comment] [PATCH RESEND v6 2/3] content: Document balloon feature page poison
On Fri, Aug 21, 2020 at 11:24 AM David Hildenbrand wrote:
>
> On 18.08.20 19:32, Alexander Duyck wrote:
> > From: Alexander Duyck
> > +\devicenormative{\paragraph}{Page Poison}{Device Types / Memory Balloon
> > Device / Device Operation / Page Poison}
> > +
> > +Normative statements in this section apply if the
> > +VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
> > +
> > +The device MAY use the content of \field{poison_val} as a hint to guest
> > +behavior.
> > +>>>>>>> patched
>
> ^ looks strange
Yeah, this must have been a merge conflict that didn't get sorted
correctly. I will fix and submit a v7 next week with your
"Reviewed-by" included.
> Apart from that
>
> Reviewed-by: David Hildenbrand
>
> Thanks!
Thanks for taking the time to review this. I will push v7 out on
Monday and ask for a vote for inclusion.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
Re: [virtio-dev] Re: [PATCH RESEND v6 0/3] virtio-spec: Add documentation for recently added balloon features
On Tue, Aug 18, 2020 at 10:41 AM David Hildenbrand wrote: > > On 18.08.20 19:32, Alexander Duyck wrote: > > I am resending this patch set with the hope of getting final reviews sorted > > out as I had no feedback on v6. If there are no further comments to be made > > I will create an issue and ask for inclusion of this patch set. > > > > You can open an issue right away and ask for inclusion in a week or so. > I'll try to have a look this week but I consider this good enough already :) Thanks. I was already thinking along the same lines. I will follow up with this next week if I don't hear anything. - Alex Fixes: https://github.com/oasis-tcs/virtio-spec/issues/84 - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH RESEND v6 0/3] virtio-spec: Add documentation for recently added balloon features
I am resending this patch set with the hope of getting final reviews sorted out as I had no feedback on v6. If there are no further comments to be made I will create an issue and ask for inclusion of this patch set. This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that were recently added to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand Changes since v4: Further clean-ups for page hinting based on input from David Hildenbrand Changes since v5: Reverted patcher order changes from v3 placing hinting as first patch Addressed comments and added Reviewed-by from Cornelia Huck Addressed further items I found in free page hinting Added requirement that output buffers be 4 bytes in size Replaced reference to descriptor with buffer Replaced references to free_page_hint_vq with free_page_vq [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting conformance.tex |6 + content.tex | 287 ++- 2 files changed, 287 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH RESEND v6 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
In working on this the specification as pointed out certain issues with the
Linux driver and QEMU device implementation. The issues include:
1. The Linux driver does not re-initialize pages when it reuses them
before receiving the "DONE" command, as such this can lead to possible data
corruption.
2. The QEMU device is not returning the "DONE" command if a migration
fails. This results in the guest holding onto pages until forced out by the
shrinker.
There are also additional issues that have been found not related to the
specification.
There is currently discussion on if the feature should be removed so this
patch is a place-holder for if we decide to keep the feature and fix the
issues. Otherwise this patch can be dropped and we can work on a patch to
document the need to avoid the feature.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 161 +--
2 files changed, 157 insertions(+), 6 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..a14e26edfcb2 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..76dfce919b97 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field
\field{free_page_hint_cmd_id}
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is neg
[virtio-dev] [PATCH RESEND v6 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Reviewed-by: Cornelia Huck
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 81 ++-
2 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/conformance.tex b/conformance.tex
index 5038b36324ac..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index cedd8e702c16..c9174b88924b 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5040,6 +5045,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5099,11 +5108,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
identify the free_page_vq.
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated,
+ identify the reporting_vq.
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated, then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5510,7 +5523,8 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host that the guest is initializing
free pages with \field{poison_val}. When the feature is enabled, pages will
-be immediately written
[virtio-dev] [PATCH RESEND v6 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Reviewed-by: Cornelia Huck
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 47 +++
2 files changed, 49 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index a14e26edfcb2..5038b36324ac 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 76dfce919b97..cedd8e702c16 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field
\field{free_page_hint_cmd_id}
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5033,6 +5036,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5054,12 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
\field{free_page_hint_cmd_id} is available if
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5087,6 +5097,8 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated,
identify the free_page_vq.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
+ the \field{poison_val} configuration field.
\item DRIVER_OK is set: device operation begins.
@@ -5494,6 +5506,41 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
endian of the guest rather than (necessarily when not using the legacy
interface) little-endian.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host that the guest is initializing
+free pages with \field{poison_val}. When
[virtio-dev] Re: [PATCH v6 0/3] virtio-spec: Add documentation for recently added balloon features
So it has been almost a month since I submitted this and there hasn't been any feedback. I'm wondering if this is ready for acceptance. If so what are the next steps to getting this accepted into the virtio spec? Thanks. - Alex On Mon, Jul 13, 2020 at 11:24 AM Alexander Duyck wrote: > > This patch set is meant to add documentation for balloon features that have > been recently added to the Linux kernel[1,2] and that were recently added > to QEMU[3]. > > Changes since RFC: > Incorporated suggestions from Cornelia Huck > Fixed a few additional spelling errors > > Changes since v1: > Incorporated additional suggestions from Cornelia Huck > Dropped documentation referring to free page reporting from page poison patch > > Changes since v2: > Rewrote multiple statements based on input from David Hildenbrand > Dropped use of balloon and deflate from page hinting description > Dropped use of free page reporting from page poison description > Cleaned up several spots that didn't match RFC2119 style comments > Added conformance links. > Various other clean-ups. > Updated balloon command IDs based on input from Cornelia Huck > > Changes since v3: > Reordered patches to place free page hinting at end of patch set > Moved contents out of patch to poison and free page reporting patches > Updated patch description to document some known issues with feature > Further clean-ups based on input from David Hildenbrand > > Changes since v4: > Further clean-ups for page hinting based on input from David Hildenbrand > > Changes since v5: > Reverted patcher order changes from v3 placing hinting as first patch > Addressed comments and added Reviewed-by from Cornelia Huck > Addressed further items I found in free page hinting > Added requirement that output buffers be 4 bytes in size > Replaced reference to descriptor with buffer > Replaced references to free_page_hint_vq with free_page_vq > > [1]: > https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ > [2]: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc > [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html > > --- > > Alexander Duyck (3): > content: Document balloon feature free page hints > content: Document balloon feature page poison > content: Document balloon feature free page reporting > > > conformance.tex |6 + > content.tex | 279 > ++- > 2 files changed, 279 insertions(+), 6 deletions(-) > > -- > - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v3 QEMU 3/3] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
From: Alexander Duyck
Recently a feature named Free Page Reporting was added to the virtio
balloon. In order to avoid any confusion we should drop the use of the word
'report' when referring to Free Page Hinting. So what this patch does is go
through and replace all instances of 'report' with 'hint" when we are
referring to free page hinting.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 76 ++--
include/hw/virtio/virtio-balloon.h | 20 +
2 files changed, 48 insertions(+), 48 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 6e2d1293402f..22cb5df717b1 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -526,22 +526,22 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED &&
-id == dev->free_page_report_cmd_id) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_REQUESTED &&
+id == dev->free_page_hint_cmd_id) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_START;
} else {
/*
* Stop the optimization only when it has started. This
* avoids a stale stop sign for the previous command.
*/
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
}
}
}
if (elem->in_num) {
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
qemu_guest_free_page_hint(elem->in_sg[0].iov_base,
elem->in_sg[0].iov_len);
}
@@ -567,11 +567,11 @@ static void virtio_ballloon_get_free_page_hints(void
*opaque)
qemu_mutex_unlock(>free_page_lock);
virtio_notify(vdev, vq);
/*
- * Start to poll the vq once the reporting started. Otherwise, continue
+ * Start to poll the vq once the hinting started. Otherwise, continue
* only when there are entries on the vq, which need to be given back.
*/
} while (continue_to_get_hints ||
- dev->free_page_report_status == FREE_PAGE_REPORT_S_START);
+ dev->free_page_hint_status == FREE_PAGE_HINT_S_START);
virtio_queue_set_notification(vq, 1);
}
@@ -594,14 +594,14 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
qemu_mutex_lock(>free_page_lock);
-if (s->free_page_report_cmd_id == UINT_MAX) {
-s->free_page_report_cmd_id =
- VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
+if (s->free_page_hint_cmd_id == UINT_MAX) {
+s->free_page_hint_cmd_id =
+ VIRTIO_BALLOON_FREE_PAGE_HINT_CMD_ID_MIN;
} else {
-s->free_page_report_cmd_id++;
+s->free_page_hint_cmd_id++;
}
-s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+s->free_page_hint_status = FREE_PAGE_HINT_S_REQUESTED;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
@@ -611,18 +611,18 @@ static void virtio_balloon_free_page_stop(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_STOP) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_STOP) {
/*
* The lock also guarantees us that the
* virtio_ballloon_get_free_page_hints exits after the
- * free_page_report_status is set to S_STOP.
+ * free_page_hint_status is set to S_STOP.
*/
qemu_mutex_lock(>free_page_lock);
/*
- * The guest hasn't done the reporting, so host sends a notification
- * to the guest to actively stop the reporting.
+ * The guest isn't done hinting, so send a notification
+ * to the guest to actively stop the hinting.
*/
-s->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+s->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
}
@@ -632,20 +632,20 @@ static void virtio_balloon_free_page_done(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_DONE) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_DONE) {
/* See virtio_balloon_free_page_stop() */
q
[virtio-dev] [PATCH v3 QEMU 2/3] virtio-balloon: Add locking to prevent possible race when starting hinting
From: Alexander Duyck
There is already locking in place when we are stopping free page hinting
but there is not similar protections in place when we start. I can only
assume this was overlooked as in most cases the page hinting should not be
occurring when we are starting the hinting, however there is still a chance
we could be processing hints by the time we get back around to restarting
the hinting so we are better off making sure to protect the state with the
mutex lock rather than just updating the value with no protections.
Based on feedback from Peter Maydell this issue had also been spotted by
Coverity: CID 1430269
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c |4
1 file changed, 4 insertions(+)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index ce70adcc6925..6e2d1293402f 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -592,6 +592,8 @@ static void virtio_balloon_free_page_start(VirtIOBalloon *s)
return;
}
+qemu_mutex_lock(>free_page_lock);
+
if (s->free_page_report_cmd_id == UINT_MAX) {
s->free_page_report_cmd_id =
VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
@@ -600,6 +602,8 @@ static void virtio_balloon_free_page_start(VirtIOBalloon *s)
}
s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+qemu_mutex_unlock(>free_page_lock);
+
virtio_notify_config(vdev);
}
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v3 QEMU 1/3] virtio-balloon: Prevent guest from starting a report when we didn't request one
From: Alexander Duyck
Based on code review it appears possible for the driver to force the device
out of a stopped state when hinting by repeating the last ID it was
provided.
Prevent this by only allowing a transition to the start state when we are
in the requested state. This way the driver is only allowed to send one
descriptor that will transition the device into the start state. All others
will leave it in the stop state once it has finished.
Fixes: c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c |3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index e670f1e59534..ce70adcc6925 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -526,7 +526,8 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (id == dev->free_page_report_cmd_id) {
+if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED &&
+id == dev->free_page_report_cmd_id) {
dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
} else {
/*
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v3 QEMU 0/3] virtio-balloon: Free page hinting clean-ups
This series contains a couple minor cleanups related to free page hinting. The first patch addresses what I believe is a possible issue in which the driver could potentially force the device out of the stop state and back into the running state if it were to replay an earlier virtqueue element containing the same ID it had submitted earlier. The second patch takes care of a possible race due to a mutex lock not being used when starting the hinting from the device-side. The final patch takes care of renaming various hinting objects that were using "reporting" in the name to try and clarify which objects are for free page reporting and which are for free page hinting. Changes from v1: Split first patch into two patches as each addresses a separate issue. Added acked-by for first patch. Changes from v2: Added Acked-by for patch 2 Added comment to patch 2 about it fixing Coverity issue Rebased on latest pull of QEMU --- Alexander Duyck (3): virtio-balloon: Prevent guest from starting a report when we didn't request one virtio-balloon: Add locking to prevent possible race when starting hinting virtio-balloon: Replace free page hinting references to 'report' with 'hint' hw/virtio/virtio-balloon.c | 79 +++- include/hw/virtio/virtio-balloon.h | 20 + 2 files changed, 52 insertions(+), 47 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v6 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
In working on this the specification as pointed out certain issues with the
Linux driver and QEMU device implementation. The issues include:
1. The Linux driver does not re-initialize pages when it reuses them
before receiving the "DONE" command, as such this can lead to possible data
corruption.
2. The QEMU device is not returning the "DONE" command if a migration
fails. This results in the guest holding onto pages until forced out by the
shrinker.
There are also additional issues that have been found not related to the
specification.
There is currently discussion on if the feature should be removed so this
patch is a place-holder for if we decide to keep the feature and fix the
issues. Otherwise this patch can be dropped and we can work on a patch to
document the need to avoid the feature.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 154 +--
2 files changed, 150 insertions(+), 6 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..a14e26edfcb2 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..352911abe6b9 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field
\field{free_page_hint_cmd_id}
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is neg
[virtio-dev] [PATCH v6 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that were recently added to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand Changes since v4: Further clean-ups for page hinting based on input from David Hildenbrand Changes since v5: Reverted patcher order changes from v3 placing hinting as first patch Addressed comments and added Reviewed-by from Cornelia Huck Addressed further items I found in free page hinting Added requirement that output buffers be 4 bytes in size Replaced reference to descriptor with buffer Replaced references to free_page_hint_vq with free_page_vq [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting conformance.tex |6 + content.tex | 279 ++- 2 files changed, 279 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v6 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Reviewed-by: Cornelia Huck
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 81 ++-
2 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/conformance.tex b/conformance.tex
index 5038b36324ac..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 3d0600dc0b4c..f1fdfbb85223 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5040,6 +5045,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5099,11 +5108,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
identify the free_page_vq.
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated,
+ identify the reporting_vq.
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated, then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5503,7 +5516,8 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host that the guest is initializing
free pages with \field{poison_val}. When the feature is enabled, pages will
-be immediately written
[virtio-dev] [PATCH v6 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Reviewed-by: Cornelia Huck
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 46 ++
2 files changed, 48 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index a14e26edfcb2..5038b36324ac 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 352911abe6b9..3d0600dc0b4c 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field
\field{free_page_hint_cmd_id}
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5033,6 +5036,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5054,12 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
\field{free_page_hint_cmd_id} is available if
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5087,6 +5097,8 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated,
identify the free_page_vq.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, update
+ the \field{poison_val} configuration field.
\item DRIVER_OK is set: device operation begins.
@@ -5487,6 +5499,40 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
\field{free_page_hint_cmd_id} when it no longer has need for the
previously hinted pages.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host that the guest is initializing
+free pages with \field{poison_val}. When
[virtio-dev] Re: [PATCH v5 1/3] content: Document balloon feature page poison
On Wed, Jul 8, 2020 at 4:03 AM Cornelia Huck wrote:
>
> On Mon, 06 Jul 2020 14:18:22 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Page poison provides a way for the guest to notify the host that it is
> > initializing or poisoning freed pages with some specific poison value. As a
> > result of this we can infer a couple traits about the guest:
> >
> > 1. Free pages will contain a specific pattern within the guest.
> > 2. Modifying free pages from this value may cause an error in the guest.
> > 3. Pages will be immediately written to by the driver when deflated.
> >
> > There are currently no existing features that make use of this data. In the
> > upcoming feature free page reporting we will need to make use of this to
> > identify if we can evict pages from the guest without causing data
> > corruption.
> >
> > Add documentation for the page poison feature describing the basic
> > functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > conformance.tex |2 ++
> > content.tex | 59
> > +++
> > 2 files changed, 57 insertions(+), 4 deletions(-)
>
> (...)
>
> > @@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> >
> > \subsection{Device configuration layout}\label{sec:Device Types / Memory
> > Balloon Device / Device configuration layout}
> > - Both fields of this configuration
> > - are always available.
> > + \field{num_pages} and \field{actual} are always available.
> > +
> > + \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
> > +negotiated.
> >
> > \begin{lstlisting}
> > struct virtio_balloon_config {
> > le32 num_pages;
> > le32 actual;
> > +le32 free_page_hint_cmd_id;
>
> This is slightly confusing, as free page hinting is only introduced in
> a later patch; maybe call this 'reserved' and rename in the patch
> introducing page hinting? But no real need to do so, I just stumbled
> over it.
What I can do is switch back to the original patch order so that the
first patch added is free page hinting since it sounds like we are
keeping the feature. Then it will take care of adding this field.
> > +le32 poison_val;
> > };
> > \end{lstlisting}
> >
> > @@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> >\begin{enumerate}
> >\item Identify the stats virtqueue.
> >\item Add one empty buffer to the stats virtqueue.
> > - \item DRIVER_OK is set: device operation begins.
> > - \item Notify the device about the stats virtqueue buffer.
> >\end{enumerate}
> > +
> > +\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
> > + driver updates the \field{poison_val} configuration field.
>
> s/the driver updates/update/ would be more in line with the other
> wording here; more of a nit, feel free to keep as-is.
Okay, I will update this.
> > +
> > +\item DRIVER_OK is set: device operation begins.
> > +
> > +\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
> > + notify the device about the stats virtqueue buffer.
> > \end{enumerate}
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5345,6 +5362,40 @@ \subsubsection{Memory Statistics
> > Tags}\label{sec:Device Types / Memory Balloon D
> >allocations in the guest.
> > \end{description}
> >
> > +\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device
> > / Device Operation / Page Poison}
> > +
> > +Page Poison provides a way to notify the host that the guest is
> > initializing
> > +free pages with \field{poison_val}. When the feature is enabled, pages will
> > +be immediately written to by the driver after deflating.
> > +
> > +If the guest is not initializing freed pages the driver should reject the
>
> s/pages/pages,/
Done.
> > +VIRTIO_BALLOON_F_PAGE_POISON feature.
> > +
> > +If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the driver
> > +will place the initialization value into the \field{poison_val}
> > +configuration field data.
> > +
> > +\drivernormative{\paragraph}{Page Poison}{Device Types / Memory Balloon
> > Device / Device Operation / Page Poison}
>
[virtio-dev] [PATCH v5 2/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 89 +--
2 files changed, 87 insertions(+), 4 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index 4ed9d62e8088..18a5a94a72aa 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 4a0ab90260ff..403651d1413b 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[4] reporting_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5022,6 +5025,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5033,6 +5038,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5088,10 +5097,16 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
driver updates the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
+ reporting_vq is identified.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5365,8 +5380,9 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device
Types / Memory Balloon D
\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
Page Poison provides a way to notify the host that the guest is initializing
-free pages with \field{poison_val}. When the feature is enabled, pages will
-be immediately written to by the driver
[virtio-dev] [PATCH v5 1/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 59 +++
2 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..4ed9d62e8088 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..4a0ab90260ff 100644
--- a/content.tex
+++ b/content.tex
@@ -5019,6 +5019,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5026,6 +5029,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
+ driver updates the \field{poison_val} configuration field.
+
+\item DRIVER_OK is set: device operation begins.
+
+\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
+ notify the device about the stats virtqueue buffer.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory
[virtio-dev] [PATCH v5 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that were recently added to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand Changes since v4: Further clean-ups for page hinting based on input from David Hildenbrand [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature page poison content: Document balloon feature free page reporting content: Document balloon feature free page hints conformance.tex |6 + content.tex | 280 ++- 2 files changed, 280 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v2 0/3] virtio-balloon: Free page hinting clean-ups
This series contains a couple minor cleanups related to free page hinting. The first patch addresses what I believe is a possible issue in which the driver could potentially force the device out of the stop state and back into the running state if it were to replay an earlier virtqueue element containing the same ID it had submitted earlier. The second patch takes care of a possible race due to a mutex lock not being used when starting the hinting from the device-side. The final patch takes care of renaming various hinting objects that were using "reporting" in the name to try and clarify which objects are for free page reporting and which are for free page hinting. Changes from v1: Split first patch into two patches as each addresses a separate issue. Added acked-by for first patch. --- Alexander Duyck (3): virtio-balloon: Prevent guest from starting a report when we didn't request one virtio-balloon: Add locking to prevent possible race when starting hinting virtio-balloon: Replace free page hinting references to 'report' with 'hint' hw/virtio/virtio-balloon.c | 77 +++- include/hw/virtio/virtio-balloon.h | 20 + 2 files changed, 51 insertions(+), 46 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v2 2/3] virtio-balloon: Add locking to prevent possible race when starting hinting
From: Alexander Duyck
There is already locking in place when we are stopping free page hinting
but there is not similar protections in place when we start. I can only
assume this was overlooked as in most cases the page hinting should not be
occurring when we are starting the hinting, however there is still a chance
we could be processing hints by the time we get back around to restarting
the hinting so we are better off making sure to protect the state with the
mutex lock rather than just updating the value with no protections.
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c |4
1 file changed, 4 insertions(+)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 0c0fd7114799..b3e96a822b4d 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -593,6 +593,8 @@ static void virtio_balloon_free_page_start(VirtIOBalloon *s)
return;
}
+qemu_mutex_lock(>free_page_lock);
+
if (s->free_page_report_cmd_id == UINT_MAX) {
s->free_page_report_cmd_id =
VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
@@ -601,6 +603,8 @@ static void virtio_balloon_free_page_start(VirtIOBalloon *s)
}
s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+qemu_mutex_unlock(>free_page_lock);
+
virtio_notify_config(vdev);
}
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
Re: [virtio-dev] Re: [virtio-comment] [PATCH v4 3/3] content: Document balloon feature free page hints
On Wed, Jun 24, 2020 at 11:26 AM David Hildenbrand wrote:
>
> On 27.05.20 06:07, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Free page hints allow the balloon driver to provide information on what
> > pages are not currently in use so that we can avoid the cost of copying
> > them in migration scenarios. Add a feature description for free page hints
> > describing basic functioning and requirements.
> >
> > In working on this the specification as pointed out certain issues with the
> > Linux driver and QEMU device implementation. The issues include:
> > 1. The Linux driver does not re-initialize pages when it reuses them
> > before receiving the "DONE" command, as such this can lead to possible data
> > corruption.
> > 2. The QEMU device is not returning the "DONE" command if a migration
> > fails. This results in the guest holding onto pages until forced out by the
> > shrinker.
> >
> > There are also additional issues that have been found not related to the
> > specification.
> >
> > There is currently discussion on if the feature should be removed so this
> > patch is a place-holder for if we decide to keep the feature and fix the
> > issues. Otherwise this patch can be dropped and we can work on a patch to
> > document the need to avoid the feature.
>
> Looks like the feature will stay, hope we can document the expected
> semantics reasonably well now and fix the remaining issues. After all we
> spend quite some time in reverse-engineering and fixing already ...
Agreed. Might as well document as much as we can so somebody else will
not have to go back through the same process again.
> [...]
> >
> > +\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation / Free Page Hinting}
> > +
> > +Free page hinting is designed to be used during migration to determine what
> > +pages within the guest are currently unused so that they can be skipped
> > over
> > +while migrating the guest. The device will indicate that it is ready to
> > start
> > +performing hinting by setting the \field{free_page_hint_cmd_id} to one of
> > the
> > +non-reserved values that can be used as a command ID. The following values
> > +are reserved:
>
> Maybe mention somewhere (resulting from a discussion with Michael) that
> the semantics of hinted pages are similar to MADV_FREE, except
> - it's asynchronous (and the cmd_id is used to synchronize)
> - when reading pages after hinted, the content is undefined (might be
> something besides the old content or zero).
>
> Might help to understand what the semantics are.
I'll work on writing something up. I am not sure if mentioning
MADV_FREE makes much sense though since the behavior still differs
quite a bit, but I can describe the behavior.
> > +
> > +\begin{description}
> > +\item[VIRTIO_BALLOON_CMD_ID_STOP (0)] Any command ID previously supplied by
> > + the device is invalid. The driver should stop hinting free pages until a
> > + new command ID is supplied, but should not release any hinted pages for
> > + use by the guest.
> > +
> > +\item[VIRTIO_BALLOON_CMD_ID_DONE (1)] Any command ID previously supplied by
> > + the device is invalid. The driver should stop hinting free pages, and
> > + should release all hinted pages for use by the guest.
> > +\end{description}
> > +
> > +A request for free page hinting proceeds as follows:
> > +
> > +\begin{enumerate}
> > +
> > +\item The driver examines the \field{free_page_hint_cmd_id} configuration
> > field.
> > + If it contains a non-reserved value then free page hinting will begin.
> > +
> > +\item To supply free page hints:
> > + \begin{enumerate}
> > + \item The driver constructs an output descriptor containing the new value
> > +from the \field{free_page_hint_cmd_id} configuration field and adds it
> > to
> > +the free_page_hint_vq.
> > + \item The driver maps a series of pages and adds them to the
> > +free_page_hint_vq as individual scatter-gather input descriptor
> > entries.
> > + \item When the driver is no longer able to fetch additional pages to add
> > +to the free_page_hint_vq, it will construct an output descriptor
> > +containing the command ID VIRTIO_BALLOON_CMD_ID_STOP.
> > + \end{enumerate}
> > +
> > +\item A round of hinting ends either when the driver is no longer able to
> > + supply more pages for hinting as described above, or when the device
> > + updates \field{free_page_hint_cmd_id} configuration field t
[virtio-dev] Re: [PATCH 1/2] virtio-balloon: Prevent guest from starting a report when we didn't request one
On Mon, Jun 22, 2020 at 1:10 AM David Hildenbrand wrote:
>
> On 19.06.20 23:53, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Based on code review it appears possible for the driver to force the device
> > out of a stopped state when hinting by repeating the last ID it was
> > provided.
>
> Indeed, thanks for noticing.
>
> >
> > Prevent this by only allowing a transition to the start state when we are
> > in the requested state. This way the driver is only allowed to send one
> > descriptor that will transition the device into the start state. All others
> > will leave it in the stop state once it has finished.
> >
> > In addition add the necessary locking to provent any potential races
>
> s/provent/prevent/
Thanks for catching that. I will fix the typo.
> > between the accesses of the cmd_id and the status.
> >
> > Fixes: c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")
> > Signed-off-by: Alexander Duyck
> > ---
> > hw/virtio/virtio-balloon.c | 11 +++
> > 1 file changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
> > index 10507b2a430a..7f3af266f674 100644
> > --- a/hw/virtio/virtio-balloon.c
> > +++ b/hw/virtio/virtio-balloon.c
> > @@ -527,7 +527,8 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
> > ret = false;
> > goto out;
> > }
> > -if (id == dev->free_page_report_cmd_id) {
> > +if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED &&
> > +id == dev->free_page_report_cmd_id) {
> > dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
> > } else {
> > /*
>
> But doesn't that mean that, after the first hint, all further ones will
> be discarded and we'll enter the STOP state in the else case? Or am I
> missing something?
>
> Shouldn't this be something like
>
> if (id == dev->free_page_report_cmd_id) {
> if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED) {
> dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
> }
> /* Stay in FREE_PAGE_REPORT_S_START as long as the cmd_id match .*/
> } else { ...
There should only be one element containing an outbuf at the start of
the report. Once that is processed we should not see the driver
sending additional outbufs unless it is sending the STOP command ID.
> > @@ -592,14 +593,16 @@ static void
> > virtio_balloon_free_page_start(VirtIOBalloon *s)
> > return;
> > }
> >
> > -if (s->free_page_report_cmd_id == UINT_MAX) {
> > +qemu_mutex_lock(>free_page_lock);
> > +
> > +if (s->free_page_report_cmd_id++ == UINT_MAX) {
> > s->free_page_report_cmd_id =
> > VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
> > -} else {
> > -s->free_page_report_cmd_id++;
> > }
>
> Somewhat unrelated cleanup.
Agreed. I can drop it if preferred. I just took care of it because I
was adding the lock above and below to prevent us from getting into
any wierd states where the command ID might be updated but the report
status was not.
> >
> > s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
> > +qemu_mutex_unlock(>free_page_lock);
> > +
> > virtio_notify_config(vdev);
> > }
> >
> >
>
>
> --
> Thanks,
>
> David / dhildenb
>
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH 2/2] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
From: Alexander Duyck
Recently a feature named Free Page Reporting was added to the virtio
balloon. In order to avoid any confusion we should drop the use of the word
'report' when referring to Free Page Hinting. So what this patch does is go
through and replace all instances of 'report' with 'hint" when we are
referring to free page hinting.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 72 ++--
include/hw/virtio/virtio-balloon.h | 20 +-
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 7f3af266f674..eea53078a73f 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -527,22 +527,22 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED &&
-id == dev->free_page_report_cmd_id) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_REQUESTED &&
+id == dev->free_page_hint_cmd_id) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_START;
} else {
/*
* Stop the optimization only when it has started. This
* avoids a stale stop sign for the previous command.
*/
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
}
}
}
if (elem->in_num) {
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
qemu_guest_free_page_hint(elem->in_sg[0].iov_base,
elem->in_sg[0].iov_len);
}
@@ -568,11 +568,11 @@ static void virtio_ballloon_get_free_page_hints(void
*opaque)
qemu_mutex_unlock(>free_page_lock);
virtio_notify(vdev, vq);
/*
- * Start to poll the vq once the reporting started. Otherwise, continue
+ * Start to poll the vq once the hinting started. Otherwise, continue
* only when there are entries on the vq, which need to be given back.
*/
} while (continue_to_get_hints ||
- dev->free_page_report_status == FREE_PAGE_REPORT_S_START);
+ dev->free_page_hint_status == FREE_PAGE_HINT_S_START);
virtio_queue_set_notification(vq, 1);
}
@@ -595,12 +595,12 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
qemu_mutex_lock(>free_page_lock);
-if (s->free_page_report_cmd_id++ == UINT_MAX) {
-s->free_page_report_cmd_id =
- VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
+if (s->free_page_hint_cmd_id++ == UINT_MAX) {
+s->free_page_hint_cmd_id =
+ VIRTIO_BALLOON_FREE_PAGE_HINT_CMD_ID_MIN;
}
-s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+s->free_page_hint_status = FREE_PAGE_HINT_S_REQUESTED;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
@@ -610,18 +610,18 @@ static void virtio_balloon_free_page_stop(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_STOP) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_STOP) {
/*
* The lock also guarantees us that the
* virtio_ballloon_get_free_page_hints exits after the
- * free_page_report_status is set to S_STOP.
+ * free_page_hint_status is set to S_STOP.
*/
qemu_mutex_lock(>free_page_lock);
/*
- * The guest hasn't done the reporting, so host sends a notification
- * to the guest to actively stop the reporting.
+ * The guest isn't done hinting, so send a notification
+ * to the guest to actively stop the hinting.
*/
-s->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+s->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
}
@@ -631,15 +631,15 @@ static void virtio_balloon_free_page_done(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
+s->free_page_hint_status = FREE_PAGE_HINT_S_DONE;
virtio_notify_config(vdev);
}
static int
-virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)
+virtio_
[virtio-dev] [PATCH 1/2] virtio-balloon: Prevent guest from starting a report when we didn't request one
From: Alexander Duyck
Based on code review it appears possible for the driver to force the device
out of a stopped state when hinting by repeating the last ID it was
provided.
Prevent this by only allowing a transition to the start state when we are
in the requested state. This way the driver is only allowed to send one
descriptor that will transition the device into the start state. All others
will leave it in the stop state once it has finished.
In addition add the necessary locking to provent any potential races
between the accesses of the cmd_id and the status.
Fixes: c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 11 +++
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 10507b2a430a..7f3af266f674 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -527,7 +527,8 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (id == dev->free_page_report_cmd_id) {
+if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED &&
+id == dev->free_page_report_cmd_id) {
dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
} else {
/*
@@ -592,14 +593,16 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
return;
}
-if (s->free_page_report_cmd_id == UINT_MAX) {
+qemu_mutex_lock(>free_page_lock);
+
+if (s->free_page_report_cmd_id++ == UINT_MAX) {
s->free_page_report_cmd_id =
VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
-} else {
-s->free_page_report_cmd_id++;
}
s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+qemu_mutex_unlock(>free_page_lock);
+
virtio_notify_config(vdev);
}
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH 0/2] virtio-balloon: Free page hinting clean-ups
This series contains a couple minor cleanups related to free page hinting. The first patch addresses what I believe is a possible issue in which the driver could potentially force the device out of the stop state and back into the running state if it were to replay an earlier virtqueue element containing the same ID it had submitted earlier. The second patch takes care of renaming various hinting objects that were using "reporting" in the name to try and clarify which objects are for free page reporting and which are for free page hinting. --- Alexander Duyck (2): virtio-balloon: Prevent guest from starting a report when we didn't request one virtio-balloon: Replace free page hinting references to 'report' with 'hint' hw/virtio/virtio-balloon.c | 77 +++- include/hw/virtio/virtio-balloon.h | 20 + 2 files changed, 50 insertions(+), 47 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH v25 QEMU 3/3] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
On Thu, Jun 18, 2020 at 10:10 AM David Hildenbrand wrote: > > >> > >> Ugh, ... > >> > >> @MST, you might have missed that in another discussion, what's your > >> general opinion about removing free page hinting in QEMU (and Linux)? We > >> keep finding issues in the QEMU implementation, including non-trivial > >> ones, and have to speculate about the actual semantics. I can see that > >> e.g., libvirt does not support it yet. > > > > Not maintaining two similar features sounds attractive. Agreed. Just to make sure we are all on the same page I am adding Wei Wang since he was the original author for page hinting. > I consider free page hinting (in QEMU) to be in an unmaintainable state > (and it looks like Alex and I are fixing a feature we don't actually > intend to use / not aware of users). In contrast to that, the free page > reporting functionality/implementation is a walk in the park. > > > > > I'm still trying to get my head around the list of issues. So far they > > all look kind of minor to me. Would you like to summarize them > > somewhere? > > Some things I still have in my mind > > > 1. If migration fails during RAM precopy, the guest will never receive a > DONE notification. Probably easy to fix. Agreed. It is just a matter of finding the right point to add a hook so that if we abort the migration we can report DONE. > 2. Unclear semantics. Alex tried to document what the actual semantics > of hinted pages are. Assume the following in the guest to a previously > hinted page > > /* page was hinted and is reused now */ > if (page[x] != Y) > page[x] == Y; > /* migration ends, we now run on the destination */ > BUG_ON(page[x] != Y); > /* BUG, because the content chan > > A guest can observe that. And that could be a random driver that just > allocated a page. > > (I *assume* in Linux we might catch that using kasan, but I am not 100% > sure, also, the actual semantics to document are unclear - e.g., for > other guests) > > As Alex mentioned, it is not even guaranteed in QEMU that we receive a > zero page on the destination, it could also be something else (e.g., > previously migrated values). So this is only an issue for pages that are pushed out of the balloon as a part of the shrinker process though. So fixing it would be pretty straightforward as we would just have to initialize or at least dirty pages that are leaked as a part of the shrinker. That may have an impact on performance though as it would result in us dirtying pages that are freed as a result of the shrinker being triggered. > 3. If I am not wrong, the iothread works in > virtio_ballloon_get_free_page_hints() on the virtqueue only with holding > the free_page_lock (no BQL). > > Assume we're migrating, the iothread is active, and the guest triggers a > device reset. > > virtio_balloon_device_reset() will trigger a > virtio_balloon_free_page_stop(s). That won't actually wait for the > iothread to stop, it will only temporarily lock free_page_lock and > update s->free_page_report_status. > > I think there can be a race between the device reset and the iothread. > Once virtio_balloon_free_page_stop() returned, > virtio_ballloon_get_free_page_hints() can still call > - virtio_queue_set_notification(vq, 0); > - virtio_queue_set_notification(vq, 1); > - virtio_notify(vdev, vq); > - virtqueue_pop() > > I doubt this is very nice. And our conversation had me start looking though reference to virtio_balloon_free_page_stop. It looks like we call it for when we unrealize the device or reset the device. It might make more sense for us to look at pushing the status to DONE and forcing the iothread to be flushed out. > There are other concerns I had regarding the iothread (e.g., while > reporting is active, virtio_ballloon_get_free_page_hints() is > essentially a busy loop, in contrast to documented - > continue_to_get_hints will always be true). > > > The appeal of hinting is that it's 0 overhead outside migration, > > and pains were taken to avoid keeping pages locked while > > hypervisor is busy. > > > > If we are to drop hinting completely we need to show that reporting > > can be comparable, and we'll probably want to add a mode for > > reporting that behaves somewhat similarly. > > Depends on the actual users. If we're dropping a feature that nobody is > actively using, I don't think we have to show anything. > > This feature obviously saw no proper review. I'm pretty sure it had some, as it went through several iterations as I recall. However I don't think the review of the virtio interface was very detailed as I think most of the attention was on the kernel interface. As far as trying to do this with page reporting it would be doable, but I would need to use something like the command interface so that I would have a way to tell the driver when to drop the reported bit from the pages and when to stop/resume hinting. However it still wouldn't resolve the issue of copy on write style pages where the page is only read and
[virtio-dev] Re: [PATCH v25 QEMU 3/3] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
On Thu, Jun 18, 2020 at 5:54 AM David Hildenbrand wrote:
>
> On 13.06.20 22:07, Alexander Duyck wrote:
> > On Tue, May 26, 2020 at 9:14 PM Alexander Duyck
> > wrote:
> >>
> >> From: Alexander Duyck
> >>
> >> In an upcoming patch a feature named Free Page Reporting is about to be
> >> added. In order to avoid any confusion we should drop the use of the word
> >> 'report' when referring to Free Page Hinting. So what this patch does is go
> >> through and replace all instances of 'report' with 'hint" when we are
> >> referring to free page hinting.
> >>
> >> Acked-by: David Hildenbrand
> >> Signed-off-by: Alexander Duyck
> >> ---
> >> hw/virtio/virtio-balloon.c | 78
> >> ++--
> >> include/hw/virtio/virtio-balloon.h | 20 +
> >> 2 files changed, 49 insertions(+), 49 deletions(-)
> >>
> >> diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
> >> index 3e2ac1104b5f..dc15409b0bb6 100644
> >> --- a/hw/virtio/virtio-balloon.c
> >> +++ b/hw/virtio/virtio-balloon.c
> >
> > ...
> >
> >> @@ -817,14 +817,14 @@ static int virtio_balloon_post_load_device(void
> >> *opaque, int version_id)
> >> return 0;
> >> }
> >>
> >> -static const VMStateDescription vmstate_virtio_balloon_free_page_report =
> >> {
> >> +static const VMStateDescription vmstate_virtio_balloon_free_page_hint = {
> >> .name = "virtio-balloon-device/free-page-report",
> >> .version_id = 1,
> >> .minimum_version_id = 1,
> >> .needed = virtio_balloon_free_page_support,
> >> .fields = (VMStateField[]) {
> >> -VMSTATE_UINT32(free_page_report_cmd_id, VirtIOBalloon),
> >> -VMSTATE_UINT32(free_page_report_status, VirtIOBalloon),
> >> +VMSTATE_UINT32(free_page_hint_cmd_id, VirtIOBalloon),
> >> +VMSTATE_UINT32(free_page_hint_status, VirtIOBalloon),
> >> VMSTATE_END_OF_LIST()
> >> }
> >> };
> >
> > So I noticed this patch wasn't in the list of patches pulled, but that
> > is probably for the best since I believe the change above might have
> > broken migration as VMSTATE_UINT32 does a stringify on the first
> > parameter.
>
> Indeed, it's the name of the vmstate field. But I don't think it is
> relevant for migration. It's and indicator if a field is valid and it's
> used in traces/error messages.
>
> See git grep "field->name"
>
> I don't think renaming this is problematic. Can you rebase and resent?
> Thanks!
Okay, I will.
> > Any advice on how to address it, or should I just give up on renaming
> > free_page_report_cmd_id and free_page_report_status?
> >
> > Looking at this I wonder why we even need to migrate these values? It
> > seems like if we are completing a migration the cmd_id should always
> > be "DONE" shouldn't it? It isn't as if we are going to migrate the
>
> The *status* should be DONE IIUC. The cmd_id might be relevant, no? It's
> always incremented until it wraps.
The thing is, the cmd_id visible to the driver if the status is DONE
is the cmd_id value for DONE. So as long as the driver acknowledges
the value we could essentially start over the cmd_id without any
negative effect. The driver would have to put down a new descriptor to
start a block of hinting in order to begin reporting again so there
shouldn't be any risk of us falsely hinting pages that were in a
previous epoch.
Ugh, although now looking at it I think we might have a bug in the
QEMU code in that the driver could in theory force its way past a
"STOP" by just replaying the last command_id descriptor and then keep
going. Should be a pretty easy fix though as we should only allow a
transition to S_START if the status is S_REQUESTED/
> > hinting from one host to another. We will have to start over which is
> > essentially the signal that the "DONE" value provides. Same thing for
> > the status. We shouldn't be able to migrate unless both of these are
> > already in the "DONE" state so if anything I wonder if we shouldn't
> > have that as the initial state for the device and just drop the
> > migration info.
>
> We'll have to glue that to a compat machine unfortunately, so we can
> just keep migrating it ... :(
Yeah, I kind of figured that would be the case. However if the name
change is not an issue then it should not be a problem.
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH v25 QEMU 0/3] virtio-balloon: add support for page poison and free page reporting
It's been almost 2 weeks since I submitted this. Just thought I would
follow up and see if there is any ETA on when this might be applied,
or if I missed the need to fix something and resubmit.
Thanks.
- Alex
On Tue, May 26, 2020 at 9:13 PM Alexander Duyck
wrote:
>
> This series provides an asynchronous means of reporting free guest pages
> to QEMU through virtio-balloon so that the memory associated with those
> pages can be dropped and reused by other processes and/or guests on the
> host. Using this it is possible to avoid unnecessary I/O to disk and
> greatly improve performance in the case of memory overcommit on the host.
>
> I originally submitted this patch series back on February 11th 2020[1],
> but at that time I was focused primarily on the kernel portion of this
> patch set. However as of April 7th those patches are now included in
> Linus's kernel tree[2] and so I am submitting the QEMU pieces for
> inclusion.
>
> [1]:
> https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/
> [2]:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc
>
> Changes from v17:
> Fixed typo in patch 1 title
> Addressed white-space issues reported via checkpatch
> Added braces {} for two if statements to match expected coding style
>
> Changes from v18:
> Updated patches 2 and 3 based on input from dhildenb
> Added comment to patch 2 describing what keeps us from reporting a bad page
> Added patch to address issue with ROM devices being directly writable
>
> Changes from v19:
> Added std-headers change to match changes pushed for linux kernel headers
> Added patch to remove "report" from page hinting code paths
> Updated comment to better explain why we disable hints w/ page poisoning
> Removed code that was modifying config size for poison vs hinting
> Dropped x-page-poison property
> Added code to bounds check the reported region vs the RAM block
> Dropped patch for ROM devices as that was already pulled in by Paolo
>
> Changes from v20:
> Rearranged patches to push Linux header sync patches to front
> Removed association between free page hinting and VIRTIO_BALLOON_F_PAGE_POISON
> Added code to enable VIRTIO_BALLOON_F_PAGE_POISON if page reporting is enabled
> Fixed possible resource leak if poison or qemu_balloon_is_inhibited return
> true
>
> Changes from v21:
> Added ack for patch 3
> Rewrote patch description for page poison reporting feature
> Made page-poison independent property and set to enabled by default
> Added logic to migrate poison_val
> Added several comments in code to better explain features
> Switched free-page-reporting property to disabled by default
>
> Changes from v22:
> Added ack for patches 4 & 5
> Added additional comment fixes in patch 3 to remove "reporting" reference
> Renamed rvq in patch 5 to reporting_vq to better match linux kernel
> Moved call adding reporting_vq to after free_page_vq
>
> Changes from v23:
> Rebased on latest QEMU
> Dropped patches 1 & 2 as Linux kernel headers were synced
> Added compat machine code for page-poison feature
>
> Changes from v24:
> Moved free page hinting rename to end of set as feature may be removed
> entirely
> Added code to cleanup reporting_vq
>
> ---
>
> Alexander Duyck (3):
> virtio-balloon: Implement support for page poison reporting feature
> virtio-balloon: Provide an interface for free page reporting
> virtio-balloon: Replace free page hinting references to 'report' with
> 'hint'
>
>
> hw/core/machine.c |4 +
> hw/virtio/virtio-balloon.c | 179
>
> include/hw/virtio/virtio-balloon.h | 23 ++---
> 3 files changed, 155 insertions(+), 51 deletions(-)
>
> --
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v4 1/3] content: Document balloon feature page poison
On Fri, May 29, 2020 at 1:13 AM David Hildenbrand wrote:
>
> On 27.05.20 06:06, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Page poison provides a way for the guest to notify the host that it is
> > initializing or poisoning freed pages with some specific poison value. As a
> > result of this we can infer a couple traits about the guest:
> >
> > 1. Free pages will contain a specific pattern within the guest.
> > 2. Modifying free pages from this value may cause an error in the guest.
> > 3. Pages will be immediately written to by the driver when deflated.
> >
> > There are currently no existing features that make use of this data. In the
> > upcoming feature free page reporting we will need to make use of this to
> > identify if we can evict pages from the guest without causing data
> > corruption.
> >
> > Add documentation for the page poison feature describing the basic
> > functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > conformance.tex |2 ++
> > content.tex | 59
> > +++
> > 2 files changed, 57 insertions(+), 4 deletions(-)
> >
> > diff --git a/conformance.tex b/conformance.tex
> > index b6fdec090383..4ed9d62e8088 100644
> > --- a/conformance.tex
> > +++ b/conformance.tex
> > @@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Feature
> > bits}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > +\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Page Poison}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Driver
> > Conformance}\label{sec:Conformance / Driver Conformance / SCSI Host Driver
> > Conformance}
> > @@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Feature
> > bits}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > +\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Page Poison}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Device
> > Conformance}\label{sec:Conformance / Device Conformance / SCSI Host Device
> > Conformance}
> > diff --git a/content.tex b/content.tex
> > index 91735e3eb018..4a0ab90260ff 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5019,6 +5019,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > memory statistics is present.
> > \item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
> > guest out of memory condition.
> > +\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the
> > driver
> > +will immediately write \field{poison_val} to pages after deflating
> > them.
> > +Configuration field \field{poison_val} is valid.
> >
>
> Here we have "that the driver will immediately" ...
>
> But we never document that in form of a normative statement (e.g., "The
> driver MUST initialize pages with \field{poison_val} after deflating").
I'm pretty sure we did document that. In the normative statement for
the driver below we have:
+The driver MUST initialize the deflated pages with \field{poison_val} when
+they are reused by the driver.
> Just wondering if that is intended (I imagine it will be different with
> free page reporting)?
I think we add some additional items with page reporting, but I think
those are in the page reporting section if I recall correctly.
> Apart from that looks good to me!
>
> --
> Thanks,
>
> David / dhildenb
>
Thanks for taking all the time to review this.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v25 QEMU 3/3] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
From: Alexander Duyck
In an upcoming patch a feature named Free Page Reporting is about to be
added. In order to avoid any confusion we should drop the use of the word
'report' when referring to Free Page Hinting. So what this patch does is go
through and replace all instances of 'report' with 'hint" when we are
referring to free page hinting.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 78 ++--
include/hw/virtio/virtio-balloon.h | 20 +
2 files changed, 49 insertions(+), 49 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 3e2ac1104b5f..dc15409b0bb6 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -527,21 +527,21 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (id == dev->free_page_report_cmd_id) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
+if (id == dev->free_page_hint_cmd_id) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_START;
} else {
/*
* Stop the optimization only when it has started. This
* avoids a stale stop sign for the previous command.
*/
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
}
}
}
if (elem->in_num) {
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
qemu_guest_free_page_hint(elem->in_sg[0].iov_base,
elem->in_sg[0].iov_len);
}
@@ -567,11 +567,11 @@ static void virtio_ballloon_get_free_page_hints(void
*opaque)
qemu_mutex_unlock(>free_page_lock);
virtio_notify(vdev, vq);
/*
- * Start to poll the vq once the reporting started. Otherwise, continue
+ * Start to poll the vq once the hinting started. Otherwise, continue
* only when there are entries on the vq, which need to be given back.
*/
} while (continue_to_get_hints ||
- dev->free_page_report_status == FREE_PAGE_REPORT_S_START);
+ dev->free_page_hint_status == FREE_PAGE_HINT_S_START);
virtio_queue_set_notification(vq, 1);
}
@@ -592,14 +592,14 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
return;
}
-if (s->free_page_report_cmd_id == UINT_MAX) {
-s->free_page_report_cmd_id =
- VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
+if (s->free_page_hint_cmd_id == UINT_MAX) {
+s->free_page_hint_cmd_id =
+ VIRTIO_BALLOON_FREE_PAGE_HINT_CMD_ID_MIN;
} else {
-s->free_page_report_cmd_id++;
+s->free_page_hint_cmd_id++;
}
-s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+s->free_page_hint_status = FREE_PAGE_HINT_S_REQUESTED;
virtio_notify_config(vdev);
}
@@ -607,18 +607,18 @@ static void virtio_balloon_free_page_stop(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_STOP) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_STOP) {
/*
* The lock also guarantees us that the
* virtio_ballloon_get_free_page_hints exits after the
- * free_page_report_status is set to S_STOP.
+ * free_page_hint_status is set to S_STOP.
*/
qemu_mutex_lock(>free_page_lock);
/*
- * The guest hasn't done the reporting, so host sends a notification
- * to the guest to actively stop the reporting.
+ * The guest isn't done hinting, so send a notification
+ * to the guest to actively stop the hinting.
*/
-s->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+s->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
}
@@ -628,15 +628,15 @@ static void virtio_balloon_free_page_done(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
+s->free_page_hint_status = FREE_PAGE_HINT_S_DONE;
virtio_notify_config(vdev);
}
static int
-virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)
+virtio_balloon_free_page_hint_notify(NotifierWithReturn *n, void *data)
{
VirtIOBalloon *dev = container_of(n, VirtIOBalloon,
- free_page_r
[virtio-dev] [PATCH v25 QEMU 2/3] virtio-balloon: Provide an interface for free page reporting
From: Alexander Duyck
Add support for free page reporting. The idea is to function very similar
to how the balloon works in that we basically end up madvising the page as
not being used. However we don't really need to bother with any deflate
type logic since the page will be faulted back into the guest when it is
read or written to.
This provides a new way of letting the guest proactively report free
pages to the hypervisor, so the hypervisor can reuse them. In contrast to
inflate/deflate that is triggered via the hypervisor explicitly.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 72
include/hw/virtio/virtio-balloon.h |2 +
2 files changed, 73 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 26f6a7ca2e35..3e2ac1104b5f 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -321,6 +321,67 @@ static void balloon_stats_set_poll_interval(Object *obj,
Visitor *v,
balloon_stats_change_timer(s, 0);
}
+static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
+{
+VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
+VirtQueueElement *elem;
+
+while ((elem = virtqueue_pop(vq, sizeof(VirtQueueElement {
+unsigned int i;
+
+/*
+ * When we discard the page it has the effect of removing the page
+ * from the hypervisor itself and causing it to be zeroed when it
+ * is returned to us. So we must not discard the page if it is
+ * accessible by another device or process, or if the guest is
+ * expecting it to retain a non-zero value.
+ */
+if (qemu_balloon_is_inhibited() || dev->poison_val) {
+goto skip_element;
+}
+
+for (i = 0; i < elem->in_num; i++) {
+void *addr = elem->in_sg[i].iov_base;
+size_t size = elem->in_sg[i].iov_len;
+ram_addr_t ram_offset;
+RAMBlock *rb;
+
+/*
+ * There is no need to check the memory section to see if
+ * it is ram/readonly/romd like there is for handle_output
+ * below. If the region is not meant to be written to then
+ * address_space_map will have allocated a bounce buffer
+ * and it will be freed in address_space_unmap and trigger
+ * and unassigned_mem_write before failing to copy over the
+ * buffer. If more than one bad descriptor is provided it
+ * will return NULL after the first bounce buffer and fail
+ * to map any resources.
+ */
+rb = qemu_ram_block_from_host(addr, false, _offset);
+if (!rb) {
+trace_virtio_balloon_bad_addr(elem->in_addr[i]);
+continue;
+}
+
+/*
+ * For now we will simply ignore unaligned memory regions, or
+ * regions that overrun the end of the RAMBlock.
+ */
+if (!QEMU_IS_ALIGNED(ram_offset | size, qemu_ram_pagesize(rb)) ||
+(ram_offset + size) > qemu_ram_get_used_length(rb)) {
+continue;
+}
+
+ram_block_discard_range(rb, ram_offset, size);
+}
+
+skip_element:
+virtqueue_push(vq, elem, 0);
+virtio_notify(vdev, vq);
+g_free(elem);
+}
+}
+
static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
{
VirtIOBalloon *s = VIRTIO_BALLOON(vdev);
@@ -841,6 +902,12 @@ static void virtio_balloon_device_realize(DeviceState
*dev, Error **errp)
virtio_error(vdev, "iothread is missing");
}
}
+
+if (virtio_has_feature(s->host_features, VIRTIO_BALLOON_F_REPORTING)) {
+s->reporting_vq = virtio_add_queue(vdev, 32,
+ virtio_balloon_handle_report);
+}
+
reset_stats(s);
}
@@ -863,6 +930,9 @@ static void virtio_balloon_device_unrealize(DeviceState
*dev)
if (s->free_page_vq) {
virtio_delete_queue(s->free_page_vq);
}
+if (s->reporting_vq) {
+virtio_delete_queue(s->reporting_vq);
+}
virtio_cleanup(vdev);
}
@@ -945,6 +1015,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_PAGE_POISON, true),
+DEFINE_PROP_BIT("free-page-reporting", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_REPORTING, false),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/inclu
[virtio-dev] [PATCH v25 QEMU 0/3] virtio-balloon: add support for page poison and free page reporting
This series provides an asynchronous means of reporting free guest pages
to QEMU through virtio-balloon so that the memory associated with those
pages can be dropped and reused by other processes and/or guests on the
host. Using this it is possible to avoid unnecessary I/O to disk and
greatly improve performance in the case of memory overcommit on the host.
I originally submitted this patch series back on February 11th 2020[1],
but at that time I was focused primarily on the kernel portion of this
patch set. However as of April 7th those patches are now included in
Linus's kernel tree[2] and so I am submitting the QEMU pieces for
inclusion.
[1]:
https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/
[2]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc
Changes from v17:
Fixed typo in patch 1 title
Addressed white-space issues reported via checkpatch
Added braces {} for two if statements to match expected coding style
Changes from v18:
Updated patches 2 and 3 based on input from dhildenb
Added comment to patch 2 describing what keeps us from reporting a bad page
Added patch to address issue with ROM devices being directly writable
Changes from v19:
Added std-headers change to match changes pushed for linux kernel headers
Added patch to remove "report" from page hinting code paths
Updated comment to better explain why we disable hints w/ page poisoning
Removed code that was modifying config size for poison vs hinting
Dropped x-page-poison property
Added code to bounds check the reported region vs the RAM block
Dropped patch for ROM devices as that was already pulled in by Paolo
Changes from v20:
Rearranged patches to push Linux header sync patches to front
Removed association between free page hinting and VIRTIO_BALLOON_F_PAGE_POISON
Added code to enable VIRTIO_BALLOON_F_PAGE_POISON if page reporting is enabled
Fixed possible resource leak if poison or qemu_balloon_is_inhibited return true
Changes from v21:
Added ack for patch 3
Rewrote patch description for page poison reporting feature
Made page-poison independent property and set to enabled by default
Added logic to migrate poison_val
Added several comments in code to better explain features
Switched free-page-reporting property to disabled by default
Changes from v22:
Added ack for patches 4 & 5
Added additional comment fixes in patch 3 to remove "reporting" reference
Renamed rvq in patch 5 to reporting_vq to better match linux kernel
Moved call adding reporting_vq to after free_page_vq
Changes from v23:
Rebased on latest QEMU
Dropped patches 1 & 2 as Linux kernel headers were synced
Added compat machine code for page-poison feature
Changes from v24:
Moved free page hinting rename to end of set as feature may be removed entirely
Added code to cleanup reporting_vq
---
Alexander Duyck (3):
virtio-balloon: Implement support for page poison reporting feature
virtio-balloon: Provide an interface for free page reporting
virtio-balloon: Replace free page hinting references to 'report' with
'hint'
hw/core/machine.c |4 +
hw/virtio/virtio-balloon.c | 179
include/hw/virtio/virtio-balloon.h | 23 ++---
3 files changed, 155 insertions(+), 51 deletions(-)
--
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v25 QEMU 1/3] virtio-balloon: Implement support for page poison reporting feature
From: Alexander Duyck
We need to make certain to advertise support for page poison reporting if
we want to actually get data on if the guest will be poisoning pages.
Add a value for reporting the poison value being used if page poisoning is
enabled in the guest. With this we can determine if we will need to skip
free page reporting when it is enabled in the future.
The value currently has no impact on existing balloon interfaces. In the
case of existing balloon interfaces the onus is on the guest driver to
reapply whatever poison is in place.
When we add free page reporting the poison value is used to determine if
we can perform in-place page reporting. The expectation is that a reported
page will already contain the value specified by the poison, and the
reporting of the page should not change that value.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/core/machine.c |4 +++-
hw/virtio/virtio-balloon.c | 29 +
include/hw/virtio/virtio-balloon.h |1 +
3 files changed, 33 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index bb3a7b18b193..9eca7d8c9bfe 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -28,7 +28,9 @@
#include "hw/mem/nvdimm.h"
#include "migration/vmstate.h"
-GlobalProperty hw_compat_5_0[] = {};
+GlobalProperty hw_compat_5_0[] = {
+{ "virtio-balloon-device", "page-poison", "false" },
+};
const size_t hw_compat_5_0_len = G_N_ELEMENTS(hw_compat_5_0);
GlobalProperty hw_compat_4_2[] = {
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 065cd450f10f..26f6a7ca2e35 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -634,6 +634,7 @@ static void virtio_balloon_get_config(VirtIODevice *vdev,
uint8_t *config_data)
config.num_pages = cpu_to_le32(dev->num_pages);
config.actual = cpu_to_le32(dev->actual);
+config.poison_val = cpu_to_le32(dev->poison_val);
if (dev->free_page_report_status == FREE_PAGE_REPORT_S_REQUESTED) {
config.free_page_report_cmd_id =
@@ -683,6 +684,14 @@ static ram_addr_t get_current_ram_size(void)
return size;
}
+static bool virtio_balloon_page_poison_support(void *opaque)
+{
+VirtIOBalloon *s = opaque;
+VirtIODevice *vdev = VIRTIO_DEVICE(s);
+
+return virtio_vdev_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
+}
+
static void virtio_balloon_set_config(VirtIODevice *vdev,
const uint8_t *config_data)
{
@@ -697,6 +706,10 @@ static void virtio_balloon_set_config(VirtIODevice *vdev,
qapi_event_send_balloon_change(vm_ram_size -
((ram_addr_t) dev->actual <<
VIRTIO_BALLOON_PFN_SHIFT));
}
+dev->poison_val = 0;
+if (virtio_balloon_page_poison_support(dev)) {
+dev->poison_val = le32_to_cpu(config.poison_val);
+}
trace_virtio_balloon_set_config(dev->actual, oldactual);
}
@@ -755,6 +768,17 @@ static const VMStateDescription
vmstate_virtio_balloon_free_page_report = {
}
};
+static const VMStateDescription vmstate_virtio_balloon_page_poison = {
+.name = "vitio-balloon-device/page-poison",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = virtio_balloon_page_poison_support,
+.fields = (VMStateField[]) {
+VMSTATE_UINT32(poison_val, VirtIOBalloon),
+VMSTATE_END_OF_LIST()
+}
+};
+
static const VMStateDescription vmstate_virtio_balloon_device = {
.name = "virtio-balloon-device",
.version_id = 1,
@@ -767,6 +791,7 @@ static const VMStateDescription
vmstate_virtio_balloon_device = {
},
.subsections = (const VMStateDescription * []) {
_virtio_balloon_free_page_report,
+_virtio_balloon_page_poison,
NULL
}
};
@@ -854,6 +879,8 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)
g_free(s->stats_vq_elem);
s->stats_vq_elem = NULL;
}
+
+s->poison_val = 0;
}
static void virtio_balloon_set_status(VirtIODevice *vdev, uint8_t status)
@@ -916,6 +943,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_DEFLATE_ON_OOM, false),
DEFINE_PROP_BIT("free-page-hint", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
+DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_PAGE_POISON, true),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/include/hw/virtio/virtio-balloon.h
index d1c968d2376e..7fe78e5c14d7 100644
--- a/include/hw/virtio/virtio-balloon.h
+
[virtio-dev] [PATCH v4 3/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
In working on this the specification as pointed out certain issues with the
Linux driver and QEMU device implementation. The issues include:
1. The Linux driver does not re-initialize pages when it reuses them
before receiving the "DONE" command, as such this can lead to possible data
corruption.
2. The QEMU device is not returning the "DONE" command if a migration
fails. This results in the guest holding onto pages until forced out by the
shrinker.
There are also additional issues that have been found not related to the
specification.
There is currently discussion on if the feature should be removed so this
patch is a place-holder for if we decide to keep the feature and fix the
issues. Otherwise this patch can be dropped and we can work on a patch to
document the need to avoid the feature.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 125 +++
2 files changed, 127 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index 18a5a94a72aa..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
diff --git a/content.tex b/content.tex
index 403651d1413b..a3a9c13516fd 100644
--- a/content.tex
+++ b/content.tex
@@ -5006,11 +5006,14 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[0] inflateq
\item[1] deflateq
\item[2] statsq
+\item[3] free_page_vq
\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+
reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
@@ -5022,6 +5025,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field
\field{free_page_hint_cmd_id}
+is valid.
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
@@ -5060,6 +5067,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
\field{num_pages} and \field{actual} are always available.
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
+
\field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
negotiated.
@@ -5094,6 +5105,9 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item Add one empty buffer to the stats virtqueue.
\end{enumerate}
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
+ free_page_vq is
[virtio-dev] [PATCH v4 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that we are currently working on adding to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck Changes since v3: Reordered patches to place free page hinting at end of patch set Moved contents out of patch to poison and free page reporting patches Updated patch description to document some known issues with feature Further clean-ups based on input from David Hildenbrand [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature page poison content: Document balloon feature free page reporting content: Document balloon feature free page hints conformance.tex |6 + content.tex | 269 ++- 2 files changed, 269 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v4 1/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host that it is
initializing or poisoning freed pages with some specific poison value. As a
result of this we can infer a couple traits about the guest:
1. Free pages will contain a specific pattern within the guest.
2. Modifying free pages from this value may cause an error in the guest.
3. Pages will be immediately written to by the driver when deflated.
There are currently no existing features that make use of this data. In the
upcoming feature free page reporting we will need to make use of this to
identify if we can evict pages from the guest without causing data
corruption.
Add documentation for the page poison feature describing the basic
functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 59 +++
2 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..4ed9d62e8088 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..4a0ab90260ff 100644
--- a/content.tex
+++ b/content.tex
@@ -5019,6 +5019,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
+will immediately write \field{poison_val} to pages after deflating them.
+Configuration field \field{poison_val} is valid.
\end{description}
@@ -5026,6 +5029,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it will
+not immediately write \field{poison_val} to deflated pages (e.g., to
+initialize them, or fill them with a poison value).
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
+ driver updates the \field{poison_val} configuration field.
+
+\item DRIVER_OK is set: device operation begins.
+
+\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
+ notify the device about the stats virtqueue buffer.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory
[virtio-dev] [PATCH v4 2/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 89 +--
2 files changed, 87 insertions(+), 4 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index 4ed9d62e8088..18a5a94a72aa 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 4a0ab90260ff..403651d1413b 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[4] reporting_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5022,6 +5025,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] A hint to the device, that the driver
will immediately write \field{poison_val} to pages after deflating them.
Configuration field \field{poison_val} is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5033,6 +5038,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
not immediately write \field{poison_val} to deflated pages (e.g., to
initialize them, or fill them with a poison value).
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5088,10 +5097,16 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
driver updates the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
+ reporting_vq is identified.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
+ begin reporting free pages to the device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5365,8 +5380,9 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device
Types / Memory Balloon D
\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
Page Poison provides a way to notify the host that the guest is initializing
-free pages with \field{poison_val}. When the feature is enabled, pages will
-be immediately written to by the driver
[virtio-dev] Re: [virtio-comment] [PATCH v3 2/3] content: Document balloon feature page poison
On Tue, May 26, 2020 at 8:38 AM Cornelia Huck wrote: > > On Tue, 26 May 2020 17:28:00 +0200 > David Hildenbrand wrote: > > > On 26.05.20 16:50, Alexander Duyck wrote: > > > On Tue, May 26, 2020 at 1:24 AM David Hildenbrand > > > wrote: > > > >> Still wondering what to do with free page hinting ... in the meantime > > >> I'll have a look at free page reporting :) > > > > > > The problem is it is already out there so I worry we wouldn't ever be > > > able to get rid of it. At most we could deprecate it, but we are still > > > stuck with it consuming bit resources and such. > > > > Yeah, that's not an issue, they will simply turn to dead bits with > > minimal documentation. I just don't see us fixing/supporting that > > feature, really. Let's see what @MST things when he has time to look > > into this. > > > > If free page hinting is broken enough that we don't want anybody to try > implementing it, we maybe could: > > - reserve the feature bit, > - say that the device SHOULD NOT offer it, > - say that the driver SHOULD NOT accept it? > > Would avoid conflicts, and tell implementors that they should not > bother. (We can then proceed to start deprecating the Linux/QEMU > implementations.) What I might do is just try to work around it for now. I might re-order my patches so that I can push the page poison and free page reporting bits with the free page hinting changes at the end of the set. That way if we decide to take a different route then we can always go back and change that later and it won't have an impact on the earlier changes. Thanks. - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH v2 resubmit] virtio-balloon: Disable free page reporting if page poison reporting is not enabled
Do I need to resubmit this patch? It has been over two weeks now since
it was originally submitted, and a week and a half since I last sent
out an email following up. I'm just wondering if there is some list I
missed as I am assuming the maintainers and lists I contacted here are
correct? It looks like we are at RC7 now so I am worried this won't
make it in before LInus releases 5.7.
Thanks.
- Alex
On Fri, May 15, 2020 at 10:02 AM Alexander Duyck
wrote:
>
> Just following up. It has been a week since I submitted this. I was
> hoping we could get it in for 5.7 since this affects free page
> reporting which will be introduced with that kernel release.
>
> Thanks.
>
> - Alex
>
> On Fri, May 8, 2020 at 10:40 AM Alexander Duyck
> wrote:
> >
> > From: Alexander Duyck
> >
> > We should disable free page reporting if page poisoning is enabled but we
> > cannot report it via the balloon interface. This way we can avoid the
> > possibility of corrupting guest memory. Normally the page poisoning feature
> > should always be present when free page reporting is enabled on the
> > hypervisor, however this allows us to correctly handle a case of the
> > virtio-balloon device being possibly misconfigured.
> >
> > Fixes: 5d757c8d518d ("virtio-balloon: add support for providing free page
> > reports to host")
> > Acked-by: David Hildenbrand
> > Signed-off-by: Alexander Duyck
> > ---
> >
> > Changes since v1:
> > Originally this patch also modified free page hinting, that has been
> > removed.
> > Updated patch title and description.
> > Added a comment explaining reasoning for disabling free page reporting.
> >
> > Resbumitting v2 w/ Ack from David Hildebrand.
> >
> > drivers/virtio/virtio_balloon.c |9 -
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/virtio/virtio_balloon.c
> > b/drivers/virtio/virtio_balloon.c
> > index 51086a5afdd4..1f157d2f4952 100644
> > --- a/drivers/virtio/virtio_balloon.c
> > +++ b/drivers/virtio/virtio_balloon.c
> > @@ -1107,11 +1107,18 @@ static int virtballoon_restore(struct virtio_device
> > *vdev)
> >
> > static int virtballoon_validate(struct virtio_device *vdev)
> > {
> > - /* Tell the host whether we care about poisoned pages. */
> > + /*
> > +* Inform the hypervisor that our pages are poisoned or
> > +* initialized. If we cannot do that then we should disable
> > +* page reporting as it could potentially change the contents
> > +* of our free pages.
> > +*/
> > if (!want_init_on_free() &&
> > (IS_ENABLED(CONFIG_PAGE_POISONING_NO_SANITY) ||
> > !page_poisoning_enabled()))
> > __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
> > + else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON))
> > + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING);
> >
> > __virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PLATFORM);
> > return 0;
> >
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v2 1/3] content: Document balloon feature free page hints
On Wed, May 20, 2020 at 1:24 AM David Hildenbrand wrote:
>
> On 19.05.20 23:00, Alexander Duyck wrote:
> > On Tue, May 19, 2020 at 9:09 AM David Hildenbrand wrote:
[...]
> >>>>
> >>>> Let's think this through, what about this scenario:
> >>>>
> >>>> The device sets \field{free_page_hint_cmd_id} = X
> >>>> The driver starts reporting free pages (and reports all pages it has)
> >>>> 1. Sends X to start the windows
> >>>> 2. Sends all page hints (\field{free_page_hint_cmd_id} stays X)
> >>>> 3. Sends VIRTIO_BALLOON_CMD_ID_STOP to end the window
> >>>> The driver sets \field{free_page_hint_cmd_id} = DONE or STOP
> >>>>
> >>>> The guest can reuse the pages any time (triggered by the shrinker),
> >>>> especially, during 2, before the hypervisor even processed a hint
> >>>> request. It can happen that the guest reuses a page before the
> >>>> hypervisor processes the request and before
> >>>> \field{free_page_hint_cmd_id} changes.
> >>>>
> >>>> In QEMU, the double-bitmap magic makes sure that this is guaranteed to
> >>>> work IIRC.
> >>>>
> >>>> In that case, the page has to be migrated in that windows, the
> >>>> hypervisor must not modify the content.
> >>>
> >>> If by "reuse" you mean write to or reinitialize then that is correct.
> >>> All that is really happening is that any pages that are hinted have
> >>> the potential to be left behind with the original VM and not migrated
> >>> to the new one. We get the notification that the migration happened
> >>> when CMD_ID_DONE is passed to us. At that point the hinting is
> >>> complete and the device has no use for additional data.
> >>>
> >>> Instead of CMD_ID_STOP it probably would have made more sense to call
> >>> it something like CMD_ID_PAUSE or CMD_ID_HOLD as that is what it is
> >>> really doing. It is just temporarily holding the hints off while the
> >>> hypervisor synchronizes the dirty bits from the host.
> >>
> >> I think if migration fails, it will be left set to STOP. Guess we should
> >> specify that possibility somehow as well.
> >
> > Actually I think that is a bug in the QEMU implementation. We cannot
> > let that happen otherwise the guest is never going to let the memory
> > go. If we have to abort the migration we should be calling
> > virtio_balloon_free_page_done(), probably in response to something
> > like PRECOPY_NOTIFY_CLEANUP?
> >
> > The problem is I am not that familiar with the migration process
> > itself within QEMU so I am not sure which labels mean what in the
> > notifier, but we should be signaling DONE if we are either going to
> > abort the migration or if we completed it and are now on the new
> > system.
> >
>
> We are starting to see way too many issues in QEMU code that are hard to
> tackle.
>
> 1. The issue with hinted pages not being 0 or the old content when
> re-accessed, due to the way they are skipped during migration. MST
> called that a memory corruption and that this would be broken. Very hard
> to fix, maybe impossible.
>
> 2. Some cases (migration failing while we are only sending memory) not
> setting the status to DONE. The handling is just a mess.
>
> 3. Take a look at that asynchronous iothread handling.
> virtio_ballloon_get_free_page_hints(). I think you can trick it into
> doing nasty things while you are e.g., resetting the device. Also,
> virtio_ballloon_get_free_page_hints() will just busy-loop while waiting
> for requests. Just ugly. Also that block_iothread, just hard to grasp.
>
> Besides all the other issues I keep finding.
>
> At this point I'd just love to rip out the QEMU implement and let
> whoever wants to really have it try again. The Linux side seems to be in
> a better state.
>
> This feature is in an unmaintainable state in QEMU.
>
> @MST, is there something that speaks against ripping out the QEMU and
> Linux parts, documenting in the spec that this feature should not be used?
Have we mad a decision on this? I plan to resubmit my QEMU patch set
as I had a bug in patch 3 that needs to be addressed. I'm just
wondering if I should rebase on David's patches that fix free page
hinting, or if I should just replace the patch with one that removes
free page hinting?
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v3 2/3] content: Document balloon feature page poison
On Tue, May 26, 2020 at 1:24 AM David Hildenbrand wrote:
>
> On 20.05.20 18:25, Alexander Duyck wrote:
> > On Wed, May 20, 2020 at 2:28 AM David Hildenbrand wrote:
> >>
> >> On 20.05.20 04:02, Alexander Duyck wrote:
> >>> From: Alexander Duyck
> >>>
> >>> Page poison provides a way for the guest to notify the host of the content
> >>> expected to be found in pages when they are added back to the guest after
> >>> being discarded. The feature currently doesn't apply to the existing
> >>> balloon features, however it will apply to an upcoming feature, free page
> >>> reporting. Add documentation for the page poison feature describing the
> >>> basic functionality and requirements.
> >>>
> >>
> >> I would rephrase this, starting what it does *without* free page
> >> reporting (which is not "provides a way for the guest to notify ..."),
> >> and then eventually how this feature will also be used in the future as
> >> well with free page reporting.
> >
> > Below is a rewrite on this description. I'm thinking that we can
> > probably call out the advantage to free page reporting in a different
> > way. Basically with the page poison feature we know a few things about
> > the behavior and I have called them out in the new patch description:
> >
> > Page poison provides a way for the guest to notify the host that it is
> > initializing or poisoning freed pages with some specific poison value. As a
> > result of this we can infer a couple traits about the guest:
> >
> > 1. Free pages will contain a specific pattern within the guest.
> > 2. Modifying free pages from this value may cause an error in the guest.
> > 3. Pages will be immediately written to by the driver when deflated.
> >
> > There are currently no existing features that make use of this data. In the
> > upcoming feature free page reporting we will need to make use of this to
> > identify if we can evict pages from the guest without causing data
> > corruption.
> >
> > Add documentation for the page poison feature describing the basic
> > functionality and requirements.
> >
[...]
> >>> +
> >>> +If the guest is not initializing or poisoning freed pages it should
> >>> reject
> >>
> >> Sometimes you use "write to pages after deflating", here you use "freed
> >> pages"
> >
> > So when I am referencing "freed pages" I am talking about all free
> > memory, while when I refer to "pages after deflating" I am talking
> > about pages coming out of the balloon.
> >
> > My thought is that there maybe be additional uses for "poison_val" be
> > to feed it into some future use other than just the balloon portion of
> > the deflation. Basically what this is telling us is that we could look
> > for a pattern of pages containing nothing but poison_val if we wanted
> > to do some sort of same page merging, or maybe define something to
> > optimize migration by defining a poison page similar to a zero page
> > that could be used to reduce migration overhead in the future.
> >
> >>> +the VIRTIO_BALLOON_F_PAGE_POISON feature.
> >>> +
> >>> +If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
> >>> +will place the expected poison value into the \field{poison_val}
> >>
> >> again, "expected" is misleading in the context of this patch only.
> >
> > I will rewrite this statement at follows:
> > If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the driver
> > will place the initialization and/or poison value into the
> > \field{poison_val}
> > configuration field data.
> >
> > I think I might strengthen things a bit as well. In the driver
> > normative section I think I might add the following:
> > The driver MUST initialize and/or poison the deflated pages with
> > \field{poison_val} when they are reused by the driver.
> >
>
> Maybe simplify that whole "initialize and/or poison " handling across
> this patch to "initialize with \field{poison_val}" - if the
> initialization is used for poisoning or initialization doesn't matter
> from spec POV.
>
> In general looks good to me, I'll have another look at the full result.
Okay, I will go through and try to flush out the "and/or poison" in
favor of just calling out the initialization.
> Still wondering what to do with free page hinting ... in the meantime
> I'll have a look at free page reporting :)
The problem is it is already out there so I worry we wouldn't ever be
able to get rid of it. At most we could deprecate it, but we are still
stuck with it consuming bit resources and such.
Anyway I'll try to get another iteration of these patches out later today.
Thanks.
- Alex
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v3 3/3] content: Document balloon feature free page reporting
On Tue, May 26, 2020 at 2:06 AM David Hildenbrand wrote:
>
> On 20.05.20 04:02, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Free page reporting is a feature that allows the guest to proactively
> > report unused pages to the host. By making use of this feature is is
> > possible to reduce the overall memory footprint of the guest in cases where
> > some significant portion of the memory is idle. Add documentation for the
> > free page reporting feature describing the functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > conformance.tex |2 +
> > content.tex | 82
> > ++-
> > 2 files changed, 83 insertions(+), 1 deletion(-)
> >
> > diff --git a/conformance.tex b/conformance.tex
> > index 5038b36324ac..5496a25e93ef 100644
> > --- a/conformance.tex
> > +++ b/conformance.tex
> > @@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Hinting}
> > \item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Page Poison}
> > +\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Reporting}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Driver
> > Conformance}\label{sec:Conformance / Driver Conformance / SCSI Host Driver
> > Conformance}
> > @@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
> > Conformance Targets}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Memory Statistics}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Hinting}
> > \item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Page Poison}
> > +\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
> > Operation / Free Page Reporting}
> > \end{itemize}
> >
> > \conformance{\subsection}{SCSI Host Device
> > Conformance}\label{sec:Conformance / Device Conformance / SCSI Host Device
> > Conformance}
> > diff --git a/content.tex b/content.tex
> > index 89e9948b7399..acdbcfc81538 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \item[1] deflateq
> > \item[2] statsq
> > \item[3] free_page_vq
> > +\item[4] reporting_vq
> > \end{description}
> >
> >statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> >
> >free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >
> > + reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
> > +
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > \item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
> > @@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > \item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
> > the driver is expecting balloon pages to contain a certain value when
> > returned. Configuration field poison_val is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
> > +page reporting. A virtqueue for reporting free guest memory is present.
> >
> > \end{description}
> >
> > @@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > expecting any specific value to be stored in the page.
> >
> > +If the driver is expecting the pages to retain some initialized value,
>
> "some" -> the value communicated via poison_val?
So I left this somewhat vague on purpose. What this is referring to is
that if the driver/guest is expecting any given pattern to be retained
by the driver and the VIRTIO_BALLOON_F_PAGE_POISON value is not set
then we must not accept the reporting feature. As such in this case
there is no value in poison_val as VIRTIO_BALLOON_F_PAGE_POISON is not
negotiated.
> > +it MUST NOT accept VIRT
[virtio-dev] [PATCH v3 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 82 ++-
2 files changed, 83 insertions(+), 1 deletion(-)
diff --git a/conformance.tex b/conformance.tex
index 5038b36324ac..5496a25e93ef 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -151,6 +151,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -335,6 +336,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Reporting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 89e9948b7399..acdbcfc81538 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
the driver is expecting balloon pages to contain a certain value when
returned. Configuration field poison_val is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
expecting any specific value to be stored in the page.
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5101,10 +5110,16 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
driver updates the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
+ reporting_vq is identified.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
+ begin reporting free pages to device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5490,7 +5505,8 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host that the guest is initializing
and/or poisoning free pages. When the feature is enabled pages that are
-deflated might be immediately written to by the guest.
+deflated might
[virtio-dev] [PATCH v3 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 +
content.tex | 140 +--
2 files changed, 136 insertions(+), 6 deletions(-)
diff --git a/conformance.tex b/conformance.tex
index b6fdec090383..a14e26edfcb2 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -149,6 +149,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -331,6 +332,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Feature bits}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 91735e3eb018..816b6c1b052e 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field
\field{free_page_hint_cmd_id}
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
+ free_page_vq is identified.
+
+\item DRIVER_OK is set: device operation begins.
+
+\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
+ notify the device about the stats virtqueue buffer.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5345,6 +5362,117 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device
Types / Memory Balloon D
allocations in the guest.
\end{description}
+\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Hinting}
+
+Free page hinting is designed to be used during migration to determine what
+pages within the guest are currently unused so that they can be skipped over
+while migrating the guest. The device
[virtio-dev] [PATCH v3 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host of the content
expected to be found in pages when they are added back to the guest after
being discarded. The feature currently doesn't apply to the existing
balloon features, however it will apply to an upcoming feature, free page
reporting. Add documentation for the page poison feature describing the
basic functionality and requirements.
Signed-off-by: Alexander Duyck
---
conformance.tex |2 ++
content.tex | 44
2 files changed, 46 insertions(+)
diff --git a/conformance.tex b/conformance.tex
index a14e26edfcb2..5038b36324ac 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -150,6 +150,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{drivernormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Driver Conformance}\label{sec:Conformance
/ Driver Conformance / SCSI Host Driver Conformance}
@@ -333,6 +334,7 @@ \section{Conformance Targets}\label{sec:Conformance /
Conformance Targets}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Memory Statistics}
\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Free Page Hinting}
+\item \ref{devicenormative:Device Types / Memory Balloon Device / Device
Operation / Page Poison}
\end{itemize}
\conformance{\subsection}{SCSI Host Device Conformance}\label{sec:Conformance
/ Device Conformance / SCSI Host Device Conformance}
diff --git a/content.tex b/content.tex
index 816b6c1b052e..89e9948b7399 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field
\field{free_page_hint_cmd_id}
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
+the driver is expecting balloon pages to contain a certain value when
+returned. Configuration field poison_val is valid.
\end{description}
@@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
+expecting any specific value to be stored in the page.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5055,11 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
+
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5088,6 +5098,9 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
free_page_vq is identified.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
+ driver updates the \field{poison_val} configuration field.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
@@ -5473,6 +5486,37 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
\field{free_page_hint_cmd_id} once it no longer has need for the
previously hinted pages.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host that the guest is initializing
+and/or poisoning free pages. When the feature is enabled pages that are
+deflated might be immediately written to by the guest.
+
+If the guest is not initializing or poisoning freed pages it should reject
+the VIRTIO_BALLOON_F_PAGE_POISON feature.
+
+If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
+will place the expected poison value into the \field{poison_val}
+configuration field data.
+
+\drivernormative{\paragraph
[virtio-dev] [PATCH v3 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that we are currently working on adding to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch Changes since v2: Rewrote multiple statements based on input from David Hildenbrand Dropped use of balloon and deflate from page hinting description Dropped use of free page reporting from page poison description Cleaned up several spots that didn't match RFC2119 style comments Added conformance links. Various other clean-ups. Updated balloon command IDs based on input from Cornelia Huck [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting conformance.tex |6 + content.tex | 264 ++- 2 files changed, 264 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v2 1/3] content: Document balloon feature free page hints
On Tue, May 19, 2020 at 9:09 AM David Hildenbrand wrote:
>
> >> I proposed that the driver MUST reinitialize the pages when reusing
> >> (which is what Linux does), so then this is true. Reuse implies
> >> initializing, implies modification. It's somewhat simpler than what you
> >> propose, leaving the case open where the driver would reuse pages by
> >> only reading them (I don't really see a use case for that ...). But I
> >> don't care as long as it's consistent and correct :)
> >
> > Linux does not reinitialize the pages when it frees them. That only
>
> Whoever uses the pages has to initialize. Again, I don't think we should
> make difference between the guest and the driver. From spec POV, they
> are one piece. Everything else is implementation detail.
Right, but the problem is "use". In the case of balloon it was pages
being pulled out of the balloon. In the case of free pages nobody is
really using them. They are "free" already. Part of the issue here is
that unlike the balloon or page reporting we don't really have a good
definition for where they are. Getting back to the wording I have been
using for free page hinting I am looking at something like:
The driver MUST reinitialize the contents of any previously hinted page
released before receiving the command ID VIRTIO_BALLOON_CMD_ID_DONE.
I might reference that as well as the earlier comment about treating
the hinted pages as uninitialized memory.
> > happens if poison or init_on_free are enabled which are rare cases.
> > When it does reinitialize the pages then I agree that the device
> > cannot modify the contents.
>
> What about a user who relies on the content of uninitialized pages?
> Like, read it, if it has the value, don't set it to the value. Unlikely
> but possible, no? We could have data corruption.
>
> We should document that in some way, because this is what could happen
> with the *current* QEMU implementation
Agreed. This is a problem with the current QEMU/Linux driver
implementation. What worries me is that I wonder if this might not be
more possible then we realize. For example I wonder if something like
KSM could read the page and try merging it with others just for the
value to eventually change.
So I was documenting the driver side mostly as-is for the
specification. What we probably do need to do is update both the
driver and the specification to address this since if we are pulling
the page out before we get "DONE" we probably should reinitialize it
so that the state if fixed going forward and it cannot change.
> >
> > The current implementation is assuming QEMU live-migration with the
> > Linux guest as the only use case. As such I want to make sure we
> > correctly capture all of the behaviors that are expected based on
> > those assumptions, but I want to avoid inserting behaviors we would
> > like to see occur but aren't really a part of this.
>
> Exactly that's why I bring this ^ up.
>
> >
> >>>
> >>> The driver can end up releasing the pages back to the buddy allocator
> >>> and if they are not poisoned/init_on_free then they will go there and
> >>> can still potentially change until such time as the guest writes to
> >>> the page modifying it or the balloon driver switches the cmd ID to
> >>> VIRTIO_BALLOON_CMD_ID_DONE. That was one of the reasons for trying to
> >>> frame it the way I did. So what I can do is reword the two statements
> >>> as follows:
> >>>
> >>> If the content of a previously hinted page has not been modified by the
> >>> guest since the device issued the \field{free_page_hint_cmd_id}
> >>> associated
> >>> with the hint, the device MAY modify the contents of the page.
> >>>
> >>> The device MUST NOT modify the content of a previously hinted page
> >>> after
> >>> \field{free_page_hint_cmd_id} is set to VIRTIO_BALLOON_CMD_ID_DONE.
> >> Is it really only "DONE" that closes the current window? I think a
> >> "STOP" from the device will also close the window. DONE is only set at
> >> the very last iteration during memory migration.
> >
> > So the CMD_ID_DONE is issued when the migration has occurred. The
> > migration is what is actually modifying the memory.
> >
> >> (virtio_balloon_free_page_report_notify() in QEMU)
> >>
> >> I consider one window == one iteration == one value of
> >> \field{free_page_hint_cmd_id} until either DONE or STOP
> >
> > CMD_ID_STOP will close the current window for providing hints, but the
> > migration hasn't happened yet. We are still accumulating the hints. We
> > don't receive CMD_ID_DONE from the device until the migration has
> > occurred. It is the migration that will alter the content of the pages
> > by leaving them behind on the previous VM.
>
> I'll have to think about again if your statements reflect the reality
> today. I'll have to dive once again into QEMU code :( Complicated stuff.
So I am operating on the assumption that the memory isn't going to
change until the migration occurs. If you take a look at
[virtio-dev] Re: [virtio-comment] [PATCH v2 3/3] content: Document balloon feature free page reporting
On Tue, May 19, 2020 at 3:27 AM David Hildenbrand wrote:
>
> On 15.05.20 19:33, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Free page reporting is a feature that allows the guest to proactively
> > report unused pages to the host. By making use of this feature is is
> > possible to reduce the overall memory footprint of the guest in cases where
> > some significant portion of the memory is idle. Add documentation for the
> > free page reporting feature describing the functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 84
> > ++-
> > 1 file changed, 83 insertions(+), 1 deletion(-)
> >
> > diff --git a/content.tex b/content.tex
> > index 3d30fd5bb6fa..3cb38105f794 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \item[1] deflateq
> > \item[2] statsq
> > \item[3] free_page_vq
> > +\item[4] reporting_vq
> > \end{description}
> >
> >statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> >
> >free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >
> > + reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
> > +
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > \item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
> > @@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > \item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
> > the driver is expecting balloon pages to contain a certain value when
> > returned. Configuration field poison_val is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
> > +page reporting. A virtqueue for reporting free guest memory is present.
> >
> > \end{description}
> >
> > @@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > expecting any specific value to be stored in the page.
> >
> > +If the driver is expecting the pages to retain some initialized value,
> > +it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
> > +negotiates VIRTIO_BALLOON_F_PAGE_POISON.
> > +
>
> Is "accept" really the right word here? Below you use "negotiate", which
> makes more sense.
Okay, I'll change that.
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
> > @@ -5101,10 +5110,16 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> > \item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
> >driver updates the \field{poison_val} configuration field.
> >
> > +\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
> > + reporting_vq is identified.
> > +
> > \item DRIVER_OK is set: device operation begins.
> >
> > \item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
> >notify the device about the stats virtqueue buffer.
> > +
> > +\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
> > + begin reporting free pages to device.
> > \end{enumerate}
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5478,7 +5493,9 @@ \subsubsection{Page Poison}\label{sec:Device Types /
> > Memory Balloon Device / Dev
> >
> > Page Poison provides a way to notify the host of the contents that are
> > currently in the balloon pages, and those that are expected to be in the
> > -pages when they are pulled from the balloon.
> > +pages when they are pulled from the balloon. It is used for in-place
> > +reporting of pages without needing to pull them from the memory allocator
> > +of the guest.
>
> Let's see how that looks like after you modify patch #2.
What I currently have is:
Page Poison provides a way to notify the host that the guest is initializing
and/or poisoning free pages. When the feature is enabled pages that are
d
[virtio-dev] Re: [virtio-comment] [PATCH v2 1/3] content: Document balloon feature free page hints
On Mon, May 18, 2020 at 11:54 PM Cornelia Huck wrote:
>
> On Mon, 18 May 2020 13:15:50 -0700
> Alexander Duyck wrote:
>
> > On Mon, May 18, 2020 at 6:18 AM David Hildenbrand wrote:
> > >
> > > On 15.05.20 19:33, Alexander Duyck wrote:
>
> > > > +\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory
> > > > Balloon Device / Device Operation / Free Page Hinting}
> > > > +
> > > > +Free page hinting is designed to be used during migration to determine
> > > > what
> > > > +pages within the guest are currently unused so that they can be
> > > > skipped over
> > > > +while migrating the guest. The device will indicate that it is ready
> > > > to start
> > > > +performing hinting by setting the \field{free_page_hint_cmd_id} to one
> > > > of the
> > > > +non-reserved values that can be used as a command ID. The following
> > > > values
> > > > +are reserved:
> > > > +
> > > > +\begin{description}
> > > > +\item[VIRTIO_BALLOON_CMD_ID_STOP (0)] Any command ID previously
> > > > supplied by
> > > > + the device is invalid. The driver should halt all hinting until a new
> > > > + command ID is supplied.
> > >
> > > Maybe "The driver should stop hinting free pages, but should not reuse
> > > all previously hinted pages."
> >
> > The "reuse all previously hinted pages" seems rather unclear to me. I
> > would like to make clear that in this case the "use" is the guest
> > making use of the memory, not the driver doing something like
> > recycling hints. So in the spots where you reference "the driver
> > reusing pages" I think I might prefer to go with something along the
> > lines of "releasing pages for use by the guest". The problem is that
> > when you have a balloon we were referencing using pages from the
> > balloon. Since we cannot reference the balloon I figure I will go with
> > language where we "supply" and "release" hinted pages. That way we
> > acknowledge that the driver is holding onto pages and not freeing them
> > for use by the guest.
> >
> > I'll probably go with something like:
> > The driver should stop hinting free pages, but
> > should not release any hinted pages for use by the guest.
>
> "The driver should stop hinting free pages until a new command ID is
> supplied, but should not release..."
>
> would sound good to me.
Thanks, I will make that tweak.
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] [PATCH v2 1/3] content: Document balloon feature free page hints
On Tue, May 19, 2020 at 2:40 AM David Hildenbrand wrote:
>
> [...]
> >>> +\begin{description}
> >>> +\item[VIRTIO_BALLOON_CMD_ID_STOP (0)] Any command ID previously supplied
> >>> by
> >>> + the device is invalid. The driver should halt all hinting until a new
> >>> + command ID is supplied.
> >>
> >> Maybe "The driver should stop hinting free pages, but should not reuse
> >> all previously hinted pages."
> >
> > The "reuse all previously hinted pages" seems rather unclear to me. I
> > would like to make clear that in this case the "use" is the guest
> > making use of the memory, not the driver doing something like
> > recycling hints. So in the spots where you reference "the driver
>
> IMHO, In term of use/reuse, I think it does not matter. From spec POV,
> whatever happens in the guest in respect to hinting is under driver
> control. The driver just has to find a way that the memory won't be reused.
My only complaint about the use/reuse term is that when it was
previously employed it was to describe things in relation to a
balloon. So you would "use/reuse a page from the balloon" which
implied removing it. Without the balloon terminology it becomes much
more vague as "use" could mean many different things when the page
might be left on the guest already.
> > reusing pages" I think I might prefer to go with something along the
> > lines of "releasing pages for use by the guest". The problem is that
> > when you have a balloon we were referencing using pages from the
> > balloon. Since we cannot reference the balloon I figure I will go with
> > language where we "supply" and "release" hinted pages. That way we
> > acknowledge that the driver is holding onto pages and not freeing them
> > for use by the guest.
> >
> > I'll probably go with something like:
> > The driver should stop hinting free pages, but
> > should not release any hinted pages for use by the guest.
> >
>
> I'd say "release any hinted pages" is an implementation detail in the
> guest to make sure the pages won't be reused. But I don't have a strong
> opinion here as long as it helps to describe what has to be done :)
Well the wording here is "should not" which implies that not releasing
the pages is optional but recommended. The problem with releasing the
pages is that without something like the page reporting code I did for
the Linux kernel the guest would end up having to hint all the pages
again to get to where it was before.
>
> >>> +
> >>> +The driver SHOULD return pages for use once \field{free_page_hint_cmd_id}
> >>> +reports a value of VIRTIO_BALLOON_CMD_ID_DONE.
> >>
> >> "return pages" -> "start to reuse all previously hinted pages".
> >
> > The driver SHOULD release all hinted pages for use by the guest once
> > \field{free_page_hint_cmd_id} reports a value of VIRTIO_BALLOON_CMD_ID_DONE.
> >
> >> Also,
> >>
> >> "The driver MUST reinitialize hinted pages before reusing them."
> >
> > That isn't quite correct though. It is only necessary to initialize
> > the pages if the guest depends on them being initialized.
> >
> > Maybe something like:
> > The driver MUST treat the content of all hinted pages as uninitialized
> > memory.
> >
>
> Makes sense.
>
> >>> +
> >>> +\devicenormative{\paragraph}{Free Page Hinting}{Device Types / Memory
> >>> Balloon Device / Device Operation / Free Page Hinting}
> >>> +
> >>> +Normative statements in this section apply if the
> >>> +VIRTIO_BALLOON_F_FREE_PAGE_HINT feature has been negotiated.
> >>> +
> >>> +The device MUST set \field{free_page_hint_cmd_id} to
> >>> +VIRTIO_BALLOON_CMD_ID_STOP any time that the dirty pages for the given
> >>> +guest are being recorded.
> >>> +
> >>> +The device MUST NOT reuse a command ID until it has received an output
> >>> +descriptor containing VIRTIO_BALLOON_CMD_ID_STOP from the driver.
> >>> +
> >>> +The device MUST ignore pages that are provided with a command ID that
> >>> does
> >>> +not match the current value in \field{free_page_hint_cmd_id}.
> >>> +
> >>> +The device MAY modify the contents of the page in the balloon if the page
> >>> +has not been modified by the guest since the
> >>> \field{free_page_hint_cmd_id}
> >>> +associated with the hint was issued by the device.
> >>
> >> "page in the balloon" -> "previously hinted pages"
> >>
> >> But it's not that easy in respect to the guest reusing the pages.
> >>
> >> "previously hinted pages and not reused pages" ?
> >>
> >> Also, something like
> >>
> >> "The device MUST NOT modify the contents of previously hinted pages in
> >> case they are reused by the devices, even if they are reused by the
> >> driver before the hinting request is processed."
> >
> > That is not quite true.
>
> I proposed that the driver MUST reinitialize the pages when reusing
> (which is what Linux does), so then this is true. Reuse implies
> initializing, implies modification. It's somewhat simpler than what you
> propose, leaving the case open where the driver would reuse pages by
> only reading them (I don't
[virtio-dev] Re: [virtio-comment] [PATCH v2 1/3] content: Document balloon feature free page hints
On Mon, May 18, 2020 at 6:18 AM David Hildenbrand wrote:
>
> On 15.05.20 19:33, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Free page hints allow the balloon driver to provide information on what
> > pages are not currently in use so that we can avoid the cost of copying
> > them in migration scenarios. Add a feature description for free page hints
> > describing basic functioning and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 128
> > ---
> > 1 file changed, 122 insertions(+), 6 deletions(-)
> >
> > diff --git a/content.tex b/content.tex
> > index 91735e3eb018..ec0abf177526 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \begin{description}
> > \item[0] inflateq
> > \item[1] deflateq
> > -\item[2] statsq.
> > +\item[2] statsq
> > +\item[3] free_page_vq
> > \end{description}
> >
> > - Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
> > + statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> > +
> > + free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > @@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > memory statistics is present.
> > \item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
> > guest out of memory condition.
> > +\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
> > +page hinting. A virtqueue for providing hints as to what memory is
> > +currently free is present. Configuration field free_page_hint_cmd_id
> > +is valid.
>
> \field{free_page_hint_cmd_id} ?
>
> >
> > \end{description}
> >
> > @@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> >
> > \subsection{Device configuration layout}\label{sec:Device Types / Memory
> > Balloon Device / Device configuration layout}
> > - Both fields of this configuration
> > - are always available.
> > + \field{num_pages} and \field{actual} are always available.
> > +
> > + \field{free_page_hint_cmd_id} is available if
> > +VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
> > +the driver.
> >
> > \begin{lstlisting}
> > struct virtio_balloon_config {
> > le32 num_pages;
> > le32 actual;
> > +le32 free_page_hint_cmd_id;
> > };
> > \end{lstlisting}
> >
> > @@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> >\begin{enumerate}
> >\item Identify the stats virtqueue.
> >\item Add one empty buffer to the stats virtqueue.
> > - \item DRIVER_OK is set: device operation begins.
> > - \item Notify the device about the stats virtqueue buffer.
> >\end{enumerate}
> > +
> > +\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
> > + free_page_vq is identified.
> > +
> > +\item DRIVER_OK is set: device operation begins.
> > +
> > +\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
> > + notify the device about the stats virtqueue buffer.
> > \end{enumerate}
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5345,6 +5362,105 @@ \subsubsection{Memory Statistics
> > Tags}\label{sec:Device Types / Memory Balloon D
> >allocations in the guest.
> > \end{description}
> >
> > +\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation / Free Page Hinting}
> > +
> > +Free page hinting is designed to be used during migration to determine what
> > +pages within the guest are currently unused so that they can be skipped
> > over
> > +while migrating the guest. The device will indicate that it is ready to
> > start
> > +performing hinting by setting the \field{free_page_hint_cmd_id} to one of
> > the
> > +non-reserved values that can be used as a command ID. The following values
> > +are reserved:
> > +
&g
[virtio-dev] Re: [PATCH v24 QEMU 3/3] virtio-balloon: Provide an interface for free page reporting
On Fri, May 8, 2020 at 2:30 PM Alexander Duyck
wrote:
>
> From: Alexander Duyck
>
> Add support for free page reporting. The idea is to function very similar
> to how the balloon works in that we basically end up madvising the page as
> not being used. However we don't really need to bother with any deflate
> type logic since the page will be faulted back into the guest when it is
> read or written to.
>
> This provides a new way of letting the guest proactively report free
> pages to the hypervisor, so the hypervisor can reuse them. In contrast to
> inflate/deflate that is triggered via the hypervisor explicitly.
>
> Acked-by: David Hildenbrand
> Signed-off-by: Alexander Duyck
I just realized that the patch below added the code to add the
reporting_vq but I never cleaned it up. I will submit a v25 in the
next couple of days that contains a fix for that.
> ---
> hw/virtio/virtio-balloon.c | 69
>
> include/hw/virtio/virtio-balloon.h |2 +
> 2 files changed, 70 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
> index 1666132a24c1..53abba290274 100644
> --- a/hw/virtio/virtio-balloon.c
> +++ b/hw/virtio/virtio-balloon.c
> @@ -321,6 +321,67 @@ static void balloon_stats_set_poll_interval(Object *obj,
> Visitor *v,
> balloon_stats_change_timer(s, 0);
> }
>
> +static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
> +{
> +VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
> +VirtQueueElement *elem;
> +
> +while ((elem = virtqueue_pop(vq, sizeof(VirtQueueElement {
> +unsigned int i;
> +
> +/*
> + * When we discard the page it has the effect of removing the page
> + * from the hypervisor itself and causing it to be zeroed when it
> + * is returned to us. So we must not discard the page if it is
> + * accessible by another device or process, or if the guest is
> + * expecting it to retain a non-zero value.
> + */
> +if (qemu_balloon_is_inhibited() || dev->poison_val) {
> +goto skip_element;
> +}
> +
> +for (i = 0; i < elem->in_num; i++) {
> +void *addr = elem->in_sg[i].iov_base;
> +size_t size = elem->in_sg[i].iov_len;
> +ram_addr_t ram_offset;
> +RAMBlock *rb;
> +
> +/*
> + * There is no need to check the memory section to see if
> + * it is ram/readonly/romd like there is for handle_output
> + * below. If the region is not meant to be written to then
> + * address_space_map will have allocated a bounce buffer
> + * and it will be freed in address_space_unmap and trigger
> + * and unassigned_mem_write before failing to copy over the
> + * buffer. If more than one bad descriptor is provided it
> + * will return NULL after the first bounce buffer and fail
> + * to map any resources.
> + */
> +rb = qemu_ram_block_from_host(addr, false, _offset);
> +if (!rb) {
> +trace_virtio_balloon_bad_addr(elem->in_addr[i]);
> +continue;
> +}
> +
> +/*
> + * For now we will simply ignore unaligned memory regions, or
> + * regions that overrun the end of the RAMBlock.
> + */
> +if (!QEMU_IS_ALIGNED(ram_offset | size, qemu_ram_pagesize(rb)) ||
> +(ram_offset + size) > qemu_ram_get_used_length(rb)) {
> +continue;
> +}
> +
> +ram_block_discard_range(rb, ram_offset, size);
> +}
> +
> +skip_element:
> +virtqueue_push(vq, elem, 0);
> +virtio_notify(vdev, vq);
> +g_free(elem);
> +}
> +}
> +
> static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
> {
> VirtIOBalloon *s = VIRTIO_BALLOON(vdev);
> @@ -841,6 +902,12 @@ static void virtio_balloon_device_realize(DeviceState
> *dev, Error **errp)
> virtio_error(vdev, "iothread is missing");
> }
> }
> +
> +if (virtio_has_feature(s->host_features, VIRTIO_BALLOON_F_REPORTING)) {
> +s->reporting_vq = virtio_add_queue(vdev, 32,
> + virtio_balloon_handle_report);
> +}
> +
> reset_stats(s);
> }
>
> @@ -945,6 +1012,8 @@ static Property virtio_balloon_properties[] = {
> VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
> DEFINE_PROP_BIT("page-poison&quo
[virtio-dev] Re: [virtio-comment] [PATCH 2/3] content: Document balloon feature page poison
On Fri, May 15, 2020 at 10:17 AM David Hildenbrand wrote:
>
> On 08.05.20 19:16, Alexander Duyck wrote:
> > From: Alexander Duyck
> >
> > Page poison provides a way for the guest to notify the host of the content
> > expected to be found in pages when they are added back to the guest after
> > being discarded. The feature currently doesn't apply to the existing
> > balloon features, however it will apply to an upcoming feature, free page
> > reporting. Add documentation for the page poison feature describing the
> > basic functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 45 +
> > 1 file changed, 45 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index 7d91604178fd..e154948a9a1a 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > page hinting. A virtqueue for providing hints as to what memory is
> > currently free is present. Configuration field free_page_hint_cmd_id
> > is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
> > +is expecting reported pages to contain a certain value when returned.
> > +Configuration field poison_val is valid.
> >
> > \end{description}
> >
> > @@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
> > feature if offered by the device.
> >
> > +The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > +expecting any specific value to be stored in the page.
> > +
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
> > @@ -5055,11 +5061,15 @@ \subsection{Device configuration
> > layout}\label{sec:Device Types / Memory Balloon
> > VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
> > the driver.
> >
> > + \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
> > +negotiated.
> > +
> > \begin{lstlisting}
> > struct virtio_balloon_config {
> > le32 num_pages;
> > le32 actual;
> > le32 free_page_hint_cmd_id;
> > +le32 poison_val;
> > };
> > \end{lstlisting}
> >
> > @@ -5088,6 +5098,9 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> > \item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
> >free_page_vq is identified.
> >
> > +\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
> > + the driver MUST update the poison_val configuration field.
> > +
> > \item DRIVER_OK is set: device operation begins.
> >
> > \item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated then
> > @@ -5461,6 +5474,38 @@ \subsubsection{Free Page Hinting}\label{sec:Device
> > Types / Memory Balloon Device
> > The device MAY NOT modify the contents of the balloon after
> > \field{free_page_hint_cmd_id} is set to VIRTIO_BALLOON_CMD_ID_DONE.
> >
> > +\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device
> > / Device Operation / Page Poison}
> > +
> > +Page Poison provides a way to notify the host of the contents that are
> > +currently in the balloon pages, and those that are expected to be in the
> > +pages when they are pulled from the balloon. It is used for in-place
>
> "when they are pulled from the balloon". That's not correct. This only
> applies to free page reporting (-> patch #3).
Okay, I will pull that bit.
> Without free page reporting, poisoning only tells the hypervisor that
> pages pages that are getting deflated might immediately be written by
> the hypervisor again.
>
> Or am I missing something?
No that is a valid point. In the case of the current balloon
implementation they are likely to be poisoned/initialized again by the
guest when they are deflated.
> > +reporting of pages without needing to pull them from the memory allocator
> > +of the guest.
> > +
> > +If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
> > +will place the expected poison value in \field{poison_val} configuration
> > +d
[virtio-dev] [PATCH v2 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 84 ++-
1 file changed, 83 insertions(+), 1 deletion(-)
diff --git a/content.tex b/content.tex
index 3d30fd5bb6fa..3cb38105f794 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
the driver is expecting balloon pages to contain a certain value when
returned. Configuration field poison_val is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] The device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
expecting any specific value to be stored in the page.
+If the driver is expecting the pages to retain some initialized value,
+it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
+negotiates VIRTIO_BALLOON_F_PAGE_POISON.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5101,10 +5110,16 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
driver updates the \field{poison_val} configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
+ reporting_vq is identified.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
notify the device about the stats virtqueue buffer.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
+ begin reporting free pages to device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5478,7 +5493,9 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
Page Poison provides a way to notify the host of the contents that are
currently in the balloon pages, and those that are expected to be in the
-pages when they are pulled from the balloon.
+pages when they are pulled from the balloon. It is used for in-place
+reporting of pages without needing to pull them from the memory allocator
+of the guest.
If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
will place the expected poison value into the \field{poison_val}
@@ -5504,6 +5521,71 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
page hinting, the device MAY ignore the content of \field{poison_val}
for those operations.
+\subsubsection{Free Page Reporting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Reporting}
+
+Free Page Reporting provides a mechanism similar to balloon inflation,
+however it does not provide a deflation queue. The expectation is that the
+device will have a means by which it can detect the guest page access and
+fault in such pages with some initial value, likely a zero page.
+
+The driver will respond to to memory conditions and begin reporting free
+pages when some number of pages are available.
+
+\begin{enumerate}
+
+\item The driver determines it has enough pages available to begin
+ reporting pages.
+
+\item The driver gathers pages into a scatter-gather list and adds them to
+ the reporting_vq.
+
+\item The device acknowledges the reporting request by using the
+ reporting_vq descriptor.
+
+\item Once the device has acknowledged the report, the pages can be
+ returned to the location from which they were pulled.
+
+\item The driver can then continue
[virtio-dev] [PATCH v2 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host of the content
expected to be found in pages when they are added back to the guest after
being discarded. The feature currently doesn't apply to the existing
balloon features, however it will apply to an upcoming feature, free page
reporting. Add documentation for the page poison feature describing the
basic functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 43 +++
1 file changed, 43 insertions(+)
diff --git a/content.tex b/content.tex
index ec0abf177526..3d30fd5bb6fa 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field free_page_hint_cmd_id
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] The device has to be notified if
+the driver is expecting balloon pages to contain a certain value when
+returned. Configuration field poison_val is valid.
\end{description}
@@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
+expecting any specific value to be stored in the page.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5055,11 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
+
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5088,6 +5098,9 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
free_page_vq is identified.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated, the
+ driver updates the \field{poison_val} configuration field.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
@@ -5461,6 +5474,36 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
The device MAY NOT modify the contents of the balloon after
\field{free_page_hint_cmd_id} is set to VIRTIO_BALLOON_CMD_ID_DONE.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host of the contents that are
+currently in the balloon pages, and those that are expected to be in the
+pages when they are pulled from the balloon.
+
+If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
+will place the expected poison value into the \field{poison_val}
+configuration field data.
+
+\drivernormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+The driver MUST populate the \field{poison_val} configuration data if it is
+expecting the page to contain some fixed value when free.
+
+The driver MAY opt to reject the feature if it will take care of
+re-initializing pages when first accessing them.
+
+\devicenormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+As page poisoning is optional for normal balloon operations and free
+page hinting, the device MAY ignore the content of \field{poison_val}
+for those operations.
+
\section{SCSI Host Device}\label{sec:Device Types / SCSI Host Device}
The virtio SCSI host device groups together one or more virtual
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v2 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 128 ---
1 file changed, 122 insertions(+), 6 deletions(-)
diff --git a/content.tex b/content.tex
index 91735e3eb018..ec0abf177526 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] The device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field free_page_hint_cmd_id
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated, the
+ free_page_vq is identified.
+
+\item DRIVER_OK is set: device operation begins.
+
+\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated, then
+ notify the device about the stats virtqueue buffer.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5345,6 +5362,105 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device
Types / Memory Balloon D
allocations in the guest.
\end{description}
+\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Hinting}
+
+Free page hinting is designed to be used during migration to determine what
+pages within the guest are currently unused so that they can be skipped over
+while migrating the guest. The device will indicate that it is ready to start
+performing hinting by setting the \field{free_page_hint_cmd_id} to one of the
+non-reserved values that can be used as a command ID. The following values
+are reserved:
+
+\begin{description}
+\item[VIRTIO_BALLOON_CMD_ID_STOP (0)] Any command ID previously supplied by
+ the device is invalid. The driver should halt all hinting until a new
+ command ID is supplied.
+
+\item[VIRTIO_BALLOON_CMD_ID_DONE (1)] Any command ID previously supplied by
+ the device is invalid. The driver should halt all hinting and the hinting
+ balloon can now be deflated returning all pages to the guest.
+\end{description}
+
+A request for free page hinting proceeds as follows:
+
+\begin{enumerate}
+
+\item The driver examines the \field{free_page_hint_cmd_id} configuration
field.
+ If it contains a non-reserved value then inflation of the balloon will begin.
+
+\item To supply memory to the hinting balloon:
+ \begin{enumerate}
+ \item The driver constructs an output descriptor containing the new value
+from the \field{free_page_hint_cmd_id} configuration field and adds it to
+the free_page_hint_vq.
+ \item The driver maps a series of pages and adds them to the
+free_page_hint_vq as individual scatter-gather input descriptor entries.
+ \item When the driver is no longer able to fetch additional pages to add
+to the free_page_hint_vq, it will construct an output
[virtio-dev] [PATCH v2 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that we are currently working on adding to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors Changes since v1: Incorporated additional suggestions from Cornelia Huck Dropped documentation referring to free page reporting from page poison patch [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting content.tex | 253 ++- 1 file changed, 247 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH 0/3] virtio-spec: Add documentation for recently added balloon features
On Mon, May 11, 2020 at 5:44 AM David Hildenbrand wrote: > > On 11.05.20 14:38, Cornelia Huck wrote: > > On Fri, 08 May 2020 10:16:14 -0700 > > Alexander Duyck wrote: > > > >> This patch set is meant to add documentation for balloon features that have > >> been recently added to the Linux kernel[1,2] and that we are currently > >> working on adding to QEMU[3]. > >> > >> Changes since RFC: > >> Incorporated suggestions from Cornelia Huck > >> Fixed a few additional spelling errors > >> > >> [1]: > >> https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ > >> [2]: > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc > >> [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html > >> > >> --- > >> > >> Alexander Duyck (3): > >> content: Document balloon feature free page hints > >> content: Document balloon feature page poison > >> content: Document balloon feature free page reporting > >> > >> > >> content.tex | 248 > >> ++- > >> 1 file changed, 242 insertions(+), 6 deletions(-) > >> > >> -- > >> > > > > I think this has moved a lot into the right direction; but the patches > > would really benefit from review by someone more familiar with the > > balloon than me. > > On my list, will have a look this week. Any ETA on when you might be able to get to that review? I'm just considering if I should submit v2 with your tweak and the suggestions from Cornelia or if I should wait for your feedback. Thanks. - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH v2 resubmit] virtio-balloon: Disable free page reporting if page poison reporting is not enabled
Just following up. It has been a week since I submitted this. I was
hoping we could get it in for 5.7 since this affects free page
reporting which will be introduced with that kernel release.
Thanks.
- Alex
On Fri, May 8, 2020 at 10:40 AM Alexander Duyck
wrote:
>
> From: Alexander Duyck
>
> We should disable free page reporting if page poisoning is enabled but we
> cannot report it via the balloon interface. This way we can avoid the
> possibility of corrupting guest memory. Normally the page poisoning feature
> should always be present when free page reporting is enabled on the
> hypervisor, however this allows us to correctly handle a case of the
> virtio-balloon device being possibly misconfigured.
>
> Fixes: 5d757c8d518d ("virtio-balloon: add support for providing free page
> reports to host")
> Acked-by: David Hildenbrand
> Signed-off-by: Alexander Duyck
> ---
>
> Changes since v1:
> Originally this patch also modified free page hinting, that has been removed.
> Updated patch title and description.
> Added a comment explaining reasoning for disabling free page reporting.
>
> Resbumitting v2 w/ Ack from David Hildebrand.
>
> drivers/virtio/virtio_balloon.c |9 -
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
> index 51086a5afdd4..1f157d2f4952 100644
> --- a/drivers/virtio/virtio_balloon.c
> +++ b/drivers/virtio/virtio_balloon.c
> @@ -1107,11 +1107,18 @@ static int virtballoon_restore(struct virtio_device
> *vdev)
>
> static int virtballoon_validate(struct virtio_device *vdev)
> {
> - /* Tell the host whether we care about poisoned pages. */
> + /*
> +* Inform the hypervisor that our pages are poisoned or
> +* initialized. If we cannot do that then we should disable
> +* page reporting as it could potentially change the contents
> +* of our free pages.
> +*/
> if (!want_init_on_free() &&
> (IS_ENABLED(CONFIG_PAGE_POISONING_NO_SANITY) ||
> !page_poisoning_enabled()))
> __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
> + else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON))
> + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING);
>
> __virtio_clear_bit(vdev, VIRTIO_F_IOMMU_PLATFORM);
> return 0;
>
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH 0/3] virtio-spec: Add documentation for recently added balloon features
On Mon, May 11, 2020 at 5:44 AM David Hildenbrand wrote: > > On 11.05.20 14:38, Cornelia Huck wrote: > > On Fri, 08 May 2020 10:16:14 -0700 > > Alexander Duyck wrote: > > > >> This patch set is meant to add documentation for balloon features that have > >> been recently added to the Linux kernel[1,2] and that we are currently > >> working on adding to QEMU[3]. > >> > >> Changes since RFC: > >> Incorporated suggestions from Cornelia Huck > >> Fixed a few additional spelling errors > >> > >> [1]: > >> https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ > >> [2]: > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc > >> [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html > >> > >> --- > >> > >> Alexander Duyck (3): > >> content: Document balloon feature free page hints > >> content: Document balloon feature page poison > >> content: Document balloon feature free page reporting > >> > >> > >> content.tex | 248 > >> ++- > >> 1 file changed, 242 insertions(+), 6 deletions(-) > >> > >> -- > >> > > > > I think this has moved a lot into the right direction; but the patches > > would really benefit from review by someone more familiar with the > > balloon than me. > > On my list, will have a look this week. > > Minor nit I spotted: Patch #2 should not document things (e.g., how > poisoning interacts with reported pages), before the free reporting > feature is actually introduced in patch #3. I'll tweak thinks a bit to work around that. The issue is that the page poison feature was introduced before page reporting, however it wasn't really put into use until page reporting was added. > BTW: Thanks Alex for tackling this! No problem. Thanks to you and Cornelia for taking the time to review this. - Alex - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [PATCH 3/3] content: Document balloon feature free page reporting
I have incorporated your feedback.
Thanks.
- Alex
On Mon, May 11, 2020 at 5:36 AM Cornelia Huck wrote:
>
> On Fri, 08 May 2020 10:16:36 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Free page reporting is a feature that allows the guest to proactively
> > report unused pages to the host. By making use of this feature is is
> > possible to reduce the overall memory footprint of the guest in cases where
> > some significant portion of the memory is idle. Add documentation for the
> > free page reporting feature describing the functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 75
> > +++
> > 1 file changed, 75 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index e154948a9a1a..97a2c01d498f 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \item[1] deflateq
> > \item[2] statsq
> > \item[3] free_page_vq
> > +\item[4] reporting_vq
> > \end{description}
> >
> >statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
> >
> >free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
> >
> > + reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
> > +
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > \item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
> > @@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > \item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
> > is expecting reported pages to contain a certain value when returned.
> > Configuration field poison_val is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] Device has support for free
>
> s/Device/The device/
>
> > +page reporting. A virtqueue for reporting free guest memory is present.
> >
> > \end{description}
> >
> > @@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > expecting any specific value to be stored in the page.
> >
> > +The driver MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING if it is
> > expecting
> > +the pages to retain some initialized value and it has not negotiated
> > +VIRTIO_BALLOON_F_PAGE_POISON as a feature.
>
> "If the driver is expecting the pages to retain some initialized value,
> it MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING unless it also
> negotiates VIRTIO_BALLOON_F_PAGE_POISON." ?
>
> > +
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
>
> (...)
>
> > +\drivernormative{\paragraph}{Free Page Reporting}{Device Types / Memory
> > Balloon Device / Device Operation / Free Page Reporting}
> > +
> > +Normative statements in this section apply if the
> > +VIRTIO_BALLOON_F_PAGE_REPORTING feature has been negotiated.
> > +
> > +If the driver is expecting the free page to contain some initial value it
> > +MUST NOT negotiate this feature without negotiating the
> > +VIRTIO_BALLOON_F_PAGE_POISON feature as well and supply this value via
> > +\field{poison_val}.
>
> Maybe use my suggestion from above (and add the "and supply...")?
>
> > +
> > +The driver MUST NOT use the reported pages until the device has
> > +acknowledged the reporting request.
> > +
> > +The driver MAY report free pages any time after DRIVER_OK is set.
> > +
> > +It is RECOMMENDED that the driver avoid unnecessary reads or writes to the
> > +page contents as this could reduce the performance for free page reporting.
> > +
> > +\devicenormative{\paragraph}{Free Page Reporting}{Device Types / Memory
> > Balloon Device / Device Operation / Free Page Reporting}
> > +
> > +Normative statements in this section apply if the
> > +VIRTIO_BALLOON_F_PAGE_REPORTING feature has been negotiated.
> > +
> > +The device MAY modify the contents of a page in the report after detecting
> > +its physical number in an report request and before acknowledgin
[virtio-dev] Re: [PATCH 2/3] content: Document balloon feature page poison
Thanks. I will incorporate the feedback below for v2.
On Mon, May 11, 2020 at 5:25 AM Cornelia Huck wrote:
>
> On Fri, 08 May 2020 10:16:29 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Page poison provides a way for the guest to notify the host of the content
> > expected to be found in pages when they are added back to the guest after
> > being discarded. The feature currently doesn't apply to the existing
> > balloon features, however it will apply to an upcoming feature, free page
> > reporting. Add documentation for the page poison feature describing the
> > basic functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 45 +
> > 1 file changed, 45 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index 7d91604178fd..e154948a9a1a 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > page hinting. A virtqueue for providing hints as to what memory is
> > currently free is present. Configuration field free_page_hint_cmd_id
> > is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
>
> Maybe "The device"/"the driver"? In any case, with a definite article :)
>
> > +is expecting reported pages to contain a certain value when returned.
> > +Configuration field poison_val is valid.
>
> >
> > \end{description}
> >
> > @@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
> > feature if offered by the device.
> >
> > +The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > +expecting any specific value to be stored in the page.
> > +
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
> > @@ -5055,11 +5061,15 @@ \subsection{Device configuration
> > layout}\label{sec:Device Types / Memory Balloon
> > VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
> > the driver.
> >
> > + \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
> > +negotiated.
> > +
> > \begin{lstlisting}
> > struct virtio_balloon_config {
> > le32 num_pages;
> > le32 actual;
> > le32 free_page_hint_cmd_id;
> > +le32 poison_val;
> > };
> > \end{lstlisting}
> >
> > @@ -5088,6 +5098,9 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> > \item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
> >free_page_vq is identified.
> >
> > +\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
> > + the driver MUST update the poison_val configuration field.
>
> "...negotiated, the driver updates the \field{poison_val} configuration
> field."
>
> > +
> > \item DRIVER_OK is set: device operation begins.
> >
> > \item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated then
> > @@ -5461,6 +5474,38 @@ \subsubsection{Free Page Hinting}\label{sec:Device
> > Types / Memory Balloon Device
> > The device MAY NOT modify the contents of the balloon after
> > \field{free_page_hint_cmd_id} is set to VIRTIO_BALLOON_CMD_ID_DONE.
> >
> > +\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device
> > / Device Operation / Page Poison}
> > +
> > +Page Poison provides a way to notify the host of the contents that are
> > +currently in the balloon pages, and those that are expected to be in the
> > +pages when they are pulled from the balloon. It is used for in-place
> > +reporting of pages without needing to pull them from the memory allocator
> > +of the guest.
> > +
> > +If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
> > +will place the expected poison value in \field{poison_val} configuration
> > +data.
>
> "...into the \field{poison_val} configuration data field."
>
> > +
> > +\drivernormative{\paragraph}{Page Poison}{Device Types / Memory Balloon
> > Device / Device Operation / Page Poison}
> >
[virtio-dev] Re: [PATCH 1/3] content: Document balloon feature free page hints
On Mon, May 11, 2020 at 5:10 AM Cornelia Huck wrote:
>
> On Fri, 08 May 2020 10:16:22 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Free page hints allow the balloon driver to provide information on what
> > pages are not currently in use so that we can avoid the cost of copying
> > them in migration scenarios. Add a feature description for free page hints
> > describing basic functioning and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 128
> > ---
> > 1 file changed, 122 insertions(+), 6 deletions(-)
> >
>
I've taken care of the spelling/grammar items pointed out. Will be
fixed for next version.
> > +hinting request.
> > +
> > +The driver SHOULD return pages for use once \field{free_page_hint_cmd_id}
> > +reports a value of VIRTIO_BALLOON_CMD_ID_DONE.
> > +
> > +\devicenormative{\paragraph}{Free Page Hinting}{Device Types / Memory
> > Balloon Device / Device Operation / Free Page Hinting}
> > +
> > +Normative statements in this section apply if the
> > +VIRTIO_BALLOON_F_FREE_PAGE_HINT feature has been negotiated.
> > +
> > +The device MUST set \field{free_page_hint_cmd_id} to
> > +VIRTIO_BALLOON_CMD_ID_STOP any time that the dirty pages for the given
> > +guest are being recorded.
> > +
> > +The device MUST NOT reuse a command ID until it has received an output
>
> What does 'reuse' mean in this context? I would expect that it is
> forbidden to do the following:
> - device sets id in config space
> - driver queues descriptor with id
> - device changes id to anything other than the current id or the
> reserved ids <== bad
>
> Maybe "The device MUST NOT change a command ID actively in use by the
> driver until..." ?
Actually what I was referring to was something more like:
- device sets the ID in config space
- device sets the ID value to CMD_ID_STOP.
- device set the ID value to again.
In order to be able to do that the driver has to reply indicating it
has stopped reporting hints before we can go back and reuse . The
expectation is that the device will use when it resumes.
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v24 QEMU 3/3] virtio-balloon: Provide an interface for free page reporting
From: Alexander Duyck
Add support for free page reporting. The idea is to function very similar
to how the balloon works in that we basically end up madvising the page as
not being used. However we don't really need to bother with any deflate
type logic since the page will be faulted back into the guest when it is
read or written to.
This provides a new way of letting the guest proactively report free
pages to the hypervisor, so the hypervisor can reuse them. In contrast to
inflate/deflate that is triggered via the hypervisor explicitly.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 69
include/hw/virtio/virtio-balloon.h |2 +
2 files changed, 70 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 1666132a24c1..53abba290274 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -321,6 +321,67 @@ static void balloon_stats_set_poll_interval(Object *obj,
Visitor *v,
balloon_stats_change_timer(s, 0);
}
+static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
+{
+VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
+VirtQueueElement *elem;
+
+while ((elem = virtqueue_pop(vq, sizeof(VirtQueueElement {
+unsigned int i;
+
+/*
+ * When we discard the page it has the effect of removing the page
+ * from the hypervisor itself and causing it to be zeroed when it
+ * is returned to us. So we must not discard the page if it is
+ * accessible by another device or process, or if the guest is
+ * expecting it to retain a non-zero value.
+ */
+if (qemu_balloon_is_inhibited() || dev->poison_val) {
+goto skip_element;
+}
+
+for (i = 0; i < elem->in_num; i++) {
+void *addr = elem->in_sg[i].iov_base;
+size_t size = elem->in_sg[i].iov_len;
+ram_addr_t ram_offset;
+RAMBlock *rb;
+
+/*
+ * There is no need to check the memory section to see if
+ * it is ram/readonly/romd like there is for handle_output
+ * below. If the region is not meant to be written to then
+ * address_space_map will have allocated a bounce buffer
+ * and it will be freed in address_space_unmap and trigger
+ * and unassigned_mem_write before failing to copy over the
+ * buffer. If more than one bad descriptor is provided it
+ * will return NULL after the first bounce buffer and fail
+ * to map any resources.
+ */
+rb = qemu_ram_block_from_host(addr, false, _offset);
+if (!rb) {
+trace_virtio_balloon_bad_addr(elem->in_addr[i]);
+continue;
+}
+
+/*
+ * For now we will simply ignore unaligned memory regions, or
+ * regions that overrun the end of the RAMBlock.
+ */
+if (!QEMU_IS_ALIGNED(ram_offset | size, qemu_ram_pagesize(rb)) ||
+(ram_offset + size) > qemu_ram_get_used_length(rb)) {
+continue;
+}
+
+ram_block_discard_range(rb, ram_offset, size);
+}
+
+skip_element:
+virtqueue_push(vq, elem, 0);
+virtio_notify(vdev, vq);
+g_free(elem);
+}
+}
+
static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
{
VirtIOBalloon *s = VIRTIO_BALLOON(vdev);
@@ -841,6 +902,12 @@ static void virtio_balloon_device_realize(DeviceState
*dev, Error **errp)
virtio_error(vdev, "iothread is missing");
}
}
+
+if (virtio_has_feature(s->host_features, VIRTIO_BALLOON_F_REPORTING)) {
+s->reporting_vq = virtio_add_queue(vdev, 32,
+ virtio_balloon_handle_report);
+}
+
reset_stats(s);
}
@@ -945,6 +1012,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_PAGE_POISON, true),
+DEFINE_PROP_BIT("free-page-reporting", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_REPORTING, false),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/include/hw/virtio/virtio-balloon.h
index 3ca2a78e1aca..28fd2b396087 100644
--- a/include/hw/virtio/virtio-balloon.h
+++ b/include/hw/virtio/virtio-balloon.h
@@ -42,7 +42,7 @@ enum virtio_balloon_free_page_hint_status {
typedef struct VirtIOBalloon {
VirtIODevice parent_obj;
-VirtQ
[virtio-dev] [PATCH v24 QEMU 2/3] virtio-balloon: Implement support for page poison reporting feature
From: Alexander Duyck
We need to make certain to advertise support for page poison reporting if
we want to actually get data on if the guest will be poisoning pages.
Add a value for reporting the poison value being used if page poisoning is
enabled in the guest. With this we can determine if we will need to skip
free page reporting when it is enabled in the future.
The value currently has no impact on existing balloon interfaces. In the
case of existing balloon interfaces the onus is on the guest driver to
reapply whatever poison is in place.
When we add free page reporting the poison value is used to determine if
we can perform in-place page reporting. The expectation is that a reported
page will already contain the value specified by the poison, and the
reporting of the page should not change that value.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/core/machine.c |4 +++-
hw/virtio/virtio-balloon.c | 29 +
include/hw/virtio/virtio-balloon.h |1 +
3 files changed, 33 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 7a50dd518f4c..da26d58634dc 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -28,7 +28,9 @@
#include "hw/mem/nvdimm.h"
#include "migration/vmstate.h"
-GlobalProperty hw_compat_5_0[] = {};
+GlobalProperty hw_compat_5_0[] = {
+{ "virtio-balloon-device", "page-poison", "false" },
+};
const size_t hw_compat_5_0_len = G_N_ELEMENTS(hw_compat_5_0);
GlobalProperty hw_compat_4_2[] = {
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index a209706b4d8d..1666132a24c1 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -634,6 +634,7 @@ static void virtio_balloon_get_config(VirtIODevice *vdev,
uint8_t *config_data)
config.num_pages = cpu_to_le32(dev->num_pages);
config.actual = cpu_to_le32(dev->actual);
+config.poison_val = cpu_to_le32(dev->poison_val);
if (dev->free_page_hint_status == FREE_PAGE_HINT_S_REQUESTED) {
config.free_page_hint_cmd_id =
@@ -683,6 +684,14 @@ static ram_addr_t get_current_ram_size(void)
return size;
}
+static bool virtio_balloon_page_poison_support(void *opaque)
+{
+VirtIOBalloon *s = opaque;
+VirtIODevice *vdev = VIRTIO_DEVICE(s);
+
+return virtio_vdev_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
+}
+
static void virtio_balloon_set_config(VirtIODevice *vdev,
const uint8_t *config_data)
{
@@ -697,6 +706,10 @@ static void virtio_balloon_set_config(VirtIODevice *vdev,
qapi_event_send_balloon_change(vm_ram_size -
((ram_addr_t) dev->actual <<
VIRTIO_BALLOON_PFN_SHIFT));
}
+dev->poison_val = 0;
+if (virtio_balloon_page_poison_support(dev)) {
+dev->poison_val = le32_to_cpu(config.poison_val);
+}
trace_virtio_balloon_set_config(dev->actual, oldactual);
}
@@ -755,6 +768,17 @@ static const VMStateDescription
vmstate_virtio_balloon_free_page_hint = {
}
};
+static const VMStateDescription vmstate_virtio_balloon_page_poison = {
+.name = "vitio-balloon-device/page-poison",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = virtio_balloon_page_poison_support,
+.fields = (VMStateField[]) {
+VMSTATE_UINT32(poison_val, VirtIOBalloon),
+VMSTATE_END_OF_LIST()
+}
+};
+
static const VMStateDescription vmstate_virtio_balloon_device = {
.name = "virtio-balloon-device",
.version_id = 1,
@@ -767,6 +791,7 @@ static const VMStateDescription
vmstate_virtio_balloon_device = {
},
.subsections = (const VMStateDescription * []) {
_virtio_balloon_free_page_hint,
+_virtio_balloon_page_poison,
NULL
}
};
@@ -854,6 +879,8 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)
g_free(s->stats_vq_elem);
s->stats_vq_elem = NULL;
}
+
+s->poison_val = 0;
}
static void virtio_balloon_set_status(VirtIODevice *vdev, uint8_t status)
@@ -916,6 +943,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_DEFLATE_ON_OOM, false),
DEFINE_PROP_BIT("free-page-hint", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
+DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_PAGE_POISON, true),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/include/hw/virtio/virtio-balloon.h
index 108cff97e71a..3ca2a78e1aca 100644
--- a/include/hw/virtio/virtio-balloon.h
+
[virtio-dev] [PATCH v24 QEMU 0/3] virtio-balloon: add support for page poison reporting and free page reporting
This series provides an asynchronous means of reporting free guest pages
to QEMU through virtio-balloon so that the memory associated with those
pages can be dropped and reused by other processes and/or guests on the
host. Using this it is possible to avoid unnecessary I/O to disk and
greatly improve performance in the case of memory overcommit on the host.
I originally submitted this patch series back on February 11th 2020[1],
but at that time I was focused primarily on the kernel portion of this
patch set. However as of April 7th those patches are now included in
Linus's kernel tree[2] and so I am submitting the QEMU pieces for
inclusion.
[1]:
https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/
[2]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc
Changes from v17:
Fixed typo in patch 1 title
Addressed white-space issues reported via checkpatch
Added braces {} for two if statements to match expected coding style
Changes from v18:
Updated patches 2 and 3 based on input from dhildenb
Added comment to patch 2 describing what keeps us from reporting a bad page
Added patch to address issue with ROM devices being directly writable
Changes from v19:
Added std-headers change to match changes pushed for linux kernel headers
Added patch to remove "report" from page hinting code paths
Updated comment to better explain why we disable hints w/ page poisoning
Removed code that was modifying config size for poison vs hinting
Dropped x-page-poison property
Added code to bounds check the reported region vs the RAM block
Dropped patch for ROM devices as that was already pulled in by Paolo
Changes from v20:
Rearranged patches to push Linux header sync patches to front
Removed association between free page hinting and VIRTIO_BALLOON_F_PAGE_POISON
Added code to enable VIRTIO_BALLOON_F_PAGE_POISON if page reporting is enabled
Fixed possible resource leak if poison or qemu_balloon_is_inhibited return true
Changes from v21:
Added ack for patch 3
Rewrote patch description for page poison reporting feature
Made page-poison independent property and set to enabled by default
Added logic to migrate poison_val
Added several comments in code to better explain features
Switched free-page-reporting property to disabled by default
Changes from v22:
Added ack for patches 4 & 5
Added additional comment fixes in patch 3 to remove "reporting" reference
Renamed rvq in patch 5 to reporting_vq to better match linux kernel
Moved call adding reporting_vq to after free_page_vq
Changes from v23:
Rebased on latest QEMU
Dropped patches 1 & 2 as Linux kernel headers were synced
Added compat machine bits for page-poison feature
---
Alexander Duyck (3):
virtio-balloon: Replace free page hinting references to 'report' with
'hint'
virtio-balloon: Implement support for page poison reporting feature
virtio-balloon: Provide an interface for free page reporting
hw/core/machine.c |4 +
hw/virtio/virtio-balloon.c | 176
include/hw/virtio/virtio-balloon.h | 23 ++---
3 files changed, 152 insertions(+), 51 deletions(-)
--
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v24 QEMU 1/3] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
From: Alexander Duyck
In an upcoming patch a feature named Free Page Reporting is about to be
added. In order to avoid any confusion we should drop the use of the word
'report' when referring to Free Page Hinting. So what this patch does is go
through and replace all instances of 'report' with 'hint" when we are
referring to free page hinting.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 78 ++--
include/hw/virtio/virtio-balloon.h | 20 +
2 files changed, 49 insertions(+), 49 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index a4729f7fc930..a209706b4d8d 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -466,21 +466,21 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (id == dev->free_page_report_cmd_id) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
+if (id == dev->free_page_hint_cmd_id) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_START;
} else {
/*
* Stop the optimization only when it has started. This
* avoids a stale stop sign for the previous command.
*/
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
}
}
}
if (elem->in_num) {
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
qemu_guest_free_page_hint(elem->in_sg[0].iov_base,
elem->in_sg[0].iov_len);
}
@@ -506,11 +506,11 @@ static void virtio_ballloon_get_free_page_hints(void
*opaque)
qemu_mutex_unlock(>free_page_lock);
virtio_notify(vdev, vq);
/*
- * Start to poll the vq once the reporting started. Otherwise, continue
+ * Start to poll the vq once the hinting started. Otherwise, continue
* only when there are entries on the vq, which need to be given back.
*/
} while (continue_to_get_hints ||
- dev->free_page_report_status == FREE_PAGE_REPORT_S_START);
+ dev->free_page_hint_status == FREE_PAGE_HINT_S_START);
virtio_queue_set_notification(vq, 1);
}
@@ -531,14 +531,14 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
return;
}
-if (s->free_page_report_cmd_id == UINT_MAX) {
-s->free_page_report_cmd_id =
- VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
+if (s->free_page_hint_cmd_id == UINT_MAX) {
+s->free_page_hint_cmd_id =
+ VIRTIO_BALLOON_FREE_PAGE_HINT_CMD_ID_MIN;
} else {
-s->free_page_report_cmd_id++;
+s->free_page_hint_cmd_id++;
}
-s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+s->free_page_hint_status = FREE_PAGE_HINT_S_REQUESTED;
virtio_notify_config(vdev);
}
@@ -546,18 +546,18 @@ static void virtio_balloon_free_page_stop(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_STOP) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_STOP) {
/*
* The lock also guarantees us that the
* virtio_ballloon_get_free_page_hints exits after the
- * free_page_report_status is set to S_STOP.
+ * free_page_hint_status is set to S_STOP.
*/
qemu_mutex_lock(>free_page_lock);
/*
- * The guest hasn't done the reporting, so host sends a notification
- * to the guest to actively stop the reporting.
+ * The guest isn't done hinting, so send a notification
+ * to the guest to actively stop the hinting.
*/
-s->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+s->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
}
@@ -567,15 +567,15 @@ static void virtio_balloon_free_page_done(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
+s->free_page_hint_status = FREE_PAGE_HINT_S_DONE;
virtio_notify_config(vdev);
}
static int
-virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)
+virtio_balloon_free_page_hint_notify(NotifierWithReturn *n, void *data)
{
VirtIOBalloon *dev = container_of(n, VirtIOBalloon,
- free_page_r
[virtio-dev] Re: [PATCH v23 QEMU 0/5] virtio-balloon: add support for page poison reporting and free page reporting
I just wanted to follow up since it has been a little over a week
since I submitted this and I haven't heard anything back. It looks
like the linux-headers patches can be dropped since the headers appear
to have been synced. I was wondering if I should resubmit with just
the 3 patches that are adding the functionality, or if this patch-set
is good as-is?
Thanks.
- Alex
On Mon, Apr 27, 2020 at 5:53 PM Alexander Duyck
wrote:
>
> This series provides an asynchronous means of reporting free guest pages
> to QEMU through virtio-balloon so that the memory associated with those
> pages can be dropped and reused by other processes and/or guests on the
> host. Using this it is possible to avoid unnecessary I/O to disk and
> greatly improve performance in the case of memory overcommit on the host.
>
> I originally submitted this patch series back on February 11th 2020[1],
> but at that time I was focused primarily on the kernel portion of this
> patch set. However as of April 7th those patches are now included in
> Linus's kernel tree[2] and so I am submitting the QEMU pieces for
> inclusion.
>
> [1]:
> https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/
> [2]:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc
>
> Changes from v17:
> Fixed typo in patch 1 title
> Addressed white-space issues reported via checkpatch
> Added braces {} for two if statements to match expected coding style
>
> Changes from v18:
> Updated patches 2 and 3 based on input from dhildenb
> Added comment to patch 2 describing what keeps us from reporting a bad page
> Added patch to address issue with ROM devices being directly writable
>
> Changes from v19:
> Added std-headers change to match changes pushed for linux kernel headers
> Added patch to remove "report" from page hinting code paths
> Updated comment to better explain why we disable hints w/ page poisoning
> Removed code that was modifying config size for poison vs hinting
> Dropped x-page-poison property
> Added code to bounds check the reported region vs the RAM block
> Dropped patch for ROM devices as that was already pulled in by Paolo
>
> Changes from v20:
> Rearranged patches to push Linux header sync patches to front
> Removed association between free page hinting and VIRTIO_BALLOON_F_PAGE_POISON
> Added code to enable VIRTIO_BALLOON_F_PAGE_POISON if page reporting is enabled
> Fixed possible resource leak if poison or qemu_balloon_is_inhibited return
> true
>
> Changes from v21:
> Added ack for patch 3
> Rewrote patch description for page poison reporting feature
> Made page-poison independent property and set to enabled by default
> Added logic to migrate poison_val
> Added several comments in code to better explain features
> Switched free-page-reporting property to disabled by default
>
> Changes from v22:
> Added ack for patches 4 & 5
> Added additional comment fixes in patch 3 to remove "reporting" reference
> Renamed rvq in patch 5 to reporting_vq to improve readability
> Moved call adding reporting_vq to after free_page_vq to fix VQ ordering
>
> ---
>
> Alexander Duyck (5):
> linux-headers: Update to allow renaming of free_page_report_cmd_id
> linux-headers: update to contain virito-balloon free page reporting
> virtio-balloon: Replace free page hinting references to 'report' with
> 'hint'
> virtio-balloon: Implement support for page poison reporting feature
> virtio-balloon: Provide an interface for free page reporting
>
>
> hw/virtio/virtio-balloon.c | 176
> ++-
> include/hw/virtio/virtio-balloon.h | 23 ++-
> include/standard-headers/linux/virtio_balloon.h | 12 +-
> 3 files changed, 159 insertions(+), 52 deletions(-)
>
> --
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH 1/3] content: Document balloon feature free page hints
From: Alexander Duyck
Free page hints allow the balloon driver to provide information on what
pages are not currently in use so that we can avoid the cost of copying
them in migration scenarios. Add a feature description for free page hints
describing basic functioning and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 128 ---
1 file changed, 122 insertions(+), 6 deletions(-)
diff --git a/content.tex b/content.tex
index 91735e3eb018..7d91604178fd 100644
--- a/content.tex
+++ b/content.tex
@@ -5005,10 +5005,13 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\begin{description}
\item[0] inflateq
\item[1] deflateq
-\item[2] statsq.
+\item[2] statsq
+\item[3] free_page_vq
\end{description}
- Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
+ statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
+
+ free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
@@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
memory statistics is present.
\item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
guest out of memory condition.
+\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] Device has support for free
+page hinting. A virtqueue for providing hints as to what memory is
+currently free is present. Configuration field free_page_hint_cmd_id
+is valid.
\end{description}
@@ -5042,13 +5049,17 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
\subsection{Device configuration layout}\label{sec:Device Types / Memory
Balloon Device / Device configuration layout}
- Both fields of this configuration
- are always available.
+ \field{num_pages} and \field{actual} are always available.
+
+ \field{free_page_hint_cmd_id} is available if
+VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
+the driver.
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
+le32 free_page_hint_cmd_id;
};
\end{lstlisting}
@@ -5072,9 +5083,15 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\begin{enumerate}
\item Identify the stats virtqueue.
\item Add one empty buffer to the stats virtqueue.
- \item DRIVER_OK is set: device operation begins.
- \item Notify the device about the stats virtqueue buffer.
\end{enumerate}
+
+\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
+ free_page_vq is identified.
+
+\item DRIVER_OK is set: device operation begins.
+
+\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated then
+ notify the device about the stats virtqueue buffer.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5345,6 +5362,105 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device
Types / Memory Balloon D
allocations in the guest.
\end{description}
+\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Hinting}
+
+Free page hinting is designed to be used during migration to determine what
+pages within the guest are currently unused so that they can be skipped over
+while migrating the guest. The device will indicate that it is ready to start
+performing hinting by setting the \field{free_page_hint_cmd_id} to one of the
+non-reserved values that can be used as a command ID. The following values
+are reserved:
+
+\begin{description}
+\item[VIRTIO_BALLOON_CMD_ID_STOP (0)] Any command ID previously supplied by
+ the device is invalid. The driver should halt all hinting until a new
+ command ID is supplied.
+
+\item[VIRTIO_BALLOON_CMD_ID_DONE (1)] Any command ID previously supplied by
+ the device is invalid. The driver should halt all hinting and the hinting
+ balloon can now be deflated returning all pages to the guest.
+\end{description}
+
+A request for free page hinting proceeds as follows:
+
+\begin{enumerate}
+
+\item The driver examines \field{free_page_hint_cmd_id} configuration field.
+ If it contains a non-reserved value then inflation of the balloon will begin.
+
+\item To supply memory to the hinting balloon:
+ \begin{enumerate}
+ \item The driver constructs an output descriptor containing the new value
+from \field{free_page_hint_cmd_id} configuration field and adds it to
+the free_page_hint_vq.
+ \item The driver maps a series of pages and adds them to the
+free_page_hint_vq as individual scatter-gather input descriptor entries.
+ \item When the driver is no longer able to fetch additional pages to add
+to the free_page_hint_vq it will construct an output descriptor
[virtio-dev] [PATCH 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 75 +++
1 file changed, 75 insertions(+)
diff --git a/content.tex b/content.tex
index e154948a9a1a..97a2c01d498f 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq
\item[3] free_page_vq
+\item[4] reporting_vq
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ is set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT is set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING is set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
is expecting reported pages to contain a certain value when returned.
Configuration field poison_val is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] Device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
expecting any specific value to be stored in the page.
+The driver MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING if it is expecting
+the pages to retain some initialized value and it has not negotiated
+VIRTIO_BALLOON_F_PAGE_POISON as a feature.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5101,10 +5110,16 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
the driver MUST update the poison_val configuration field.
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated the
+ reporting_vq is identified.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated then
notify the device about the stats virtqueue buffer.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated then
+ begin reporting free pages to device.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5506,6 +5521,66 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
free page hinting as this feature did not exist prior to those features being
added.
+\subsubsection{Free Page Reporting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Reporting}
+
+Free Page Reporting provides a mechanism similar to balloon inflation,
+however it does not provide a deflation queue. The expectation is that the
+device will have a means by which it can detect the guest page access and
+fault in such pages with some initial value, likely a zero page.
+
+The driver will respond to to memory conditions and begin reporting free
+pages when some number of pages are available.
+
+\begin{enumerate}
+
+\item The driver determines it has enough pages available to begin
+ reporting pages.
+
+\item The driver gathers pages into a scatter-gather list and adds them to
+ the reporting_vq.
+
+\item The device acknowledges the reporting request by using the
+ reporting_vq descriptor.
+
+\item Once the device has acknowledged the report, the pages can be
+ returned to the location from which they were pulled.
+
+\item The driver can then continue to gather and report pages until it
+ has determined it has reported a sufficient quantity of pages.
+
+\end{enumerate}
+
+\drivernormative{\paragraph}{Free Page Reporting}{Device Types / Memory
Balloon Device / Device Operation / Free Page Reporting}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_PAGE_REPORTING feature has been negotiated.
+
+If the driver is expecting the free page to contain some initial value it
+MUST NOT negotiate this feature without negotiating the
+VIRTIO_BALLOON_F_PAGE_POISON feature as well and supply this value via
+\field{poison_val}.
+
+The driver MUST NOT use
[virtio-dev] [PATCH 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host of the content
expected to be found in pages when they are added back to the guest after
being discarded. The feature currently doesn't apply to the existing
balloon features, however it will apply to an upcoming feature, free page
reporting. Add documentation for the page poison feature describing the
basic functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 45 +
1 file changed, 45 insertions(+)
diff --git a/content.tex b/content.tex
index 7d91604178fd..e154948a9a1a 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field free_page_hint_cmd_id
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
+is expecting reported pages to contain a certain value when returned.
+Configuration field poison_val is valid.
\end{description}
@@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
+expecting any specific value to be stored in the page.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5055,11 +5061,15 @@ \subsection{Device configuration
layout}\label{sec:Device Types / Memory Balloon
VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
the driver.
+ \field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has been
+negotiated.
+
\begin{lstlisting}
struct virtio_balloon_config {
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5088,6 +5098,9 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
free_page_vq is identified.
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
+ the driver MUST update the poison_val configuration field.
+
\item DRIVER_OK is set: device operation begins.
\item If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated then
@@ -5461,6 +5474,38 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
The device MAY NOT modify the contents of the balloon after
\field{free_page_hint_cmd_id} is set to VIRTIO_BALLOON_CMD_ID_DONE.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host of the contents that are
+currently in the balloon pages, and those that are expected to be in the
+pages when they are pulled from the balloon. It is used for in-place
+reporting of pages without needing to pull them from the memory allocator
+of the guest.
+
+If VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated, the guest
+will place the expected poison value in \field{poison_val} configuration
+data.
+
+\drivernormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+The driver MUST populate the \field{poison_val} configuration data if it is
+expecting the page to contain some fixed value when free.
+
+The driver MAY opt to disable the feature if it will take care of
+re-initializing pages when first accessing them.
+
+\devicenormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+The device MAY ignore the \field{poison_val} for normal balloon operations and
+free page hinting as this feature did not exist prior to those features being
+added.
+
\section{SCSI Host Device}\label{sec:Device Types / SCSI Host Device}
The virtio SCSI host device groups together one or more virtual
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that we are currently working on adding to QEMU[3]. Changes since RFC: Incorporated suggestions from Cornelia Huck Fixed a few additional spelling errors [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting content.tex | 248 ++- 1 file changed, 242 insertions(+), 6 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] Re: [virtio-comment] Re: [PATCH RFC 3/3] content: Document balloon feature free page reporting
On Thu, May 7, 2020 at 8:59 AM Cornelia Huck wrote:
>
> On Wed, 29 Apr 2020 11:57:57 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Free page reporting is a feature that allows the guest to proactively
> > report unused pages to the host. By making use of this feature is is
> > possible to reduce the overall memory footprint of the guest in cases where
> > some significant portion of the memory is idle. Add documentation for the
> > free page reporting feature describing the functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 74
> > +++
> > 1 file changed, 74 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index c98b8ea9526a..52955c8ff007 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \item[1] deflateq
> > \item[2] statsq.
> > \item[3] free_page_vq.
> > +\item[4] reporting_vq.
>
> Also lose the trailing period here.
>
> > \end{description}
> >
> >statsq only exists if VIRTIO_BALLOON_F_STATS_VQ set.
> >
> >free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT set.
> >
> > + reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING set.
>
> s/set/is set/
>
> > +
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > \item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
> > @@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > \item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
> > is expecting reported pages to contain a certain value when returned.
> > Configuration field poison_val is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] Device has support for free
> > +page reporting. A virtqueue for reporting free guest memory is present.
> >
> > \end{description}
> >
> > @@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > expecting any specific value to be stored in the page.
> >
> > +The driver MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING if it is
> > expecting
> > +the pages to retain some initialized value and it has not negotiated
> > +VIRTIO_BALLOON_F_PAGE_POISON as a feature.
> > +
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
> > @@ -5097,6 +5106,13 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> >
> > \item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
> >the driver MUST update the poison_val configuration field.
> > +
> > +\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated:
> > + \begin{enumerate}
> > + \item Identify the reporting virtqueue.
> > + \item DRIVER_OK is set: device operation begins.
> > + \item Begin reporting free pages to device.
> > + \end{enumerate}
> > \end{enumerate}
>
> See my previous comments regarding reordering this.
I'll work on restructuring all this. With things being split up I can
drop some of the enumerated lists.
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5497,6 +5513,64 @@ \subsubsection{Page Poison}\label{sec:Device Types /
> > Memory Balloon Device / Dev
> > free page hinting as this feature did not exist prior to those features
> > being
> > added.
> >
> > +\subsubsection{Free Page Reporting}\label{sec:Device Types / Memory
> > Balloon Device / Device Operation / Free Page Reporting}
> > +
> > +Free Page Reporting provides a mechanism similar to balloon inflation,
> > +however it does not provide a deflation queue. The expectation is that the
> > +device will have a means by which it can detect the guest page access and
> > +fault in such pages with some initial value, likely a zero page.
> > +
> > +The driver will respond to to memory conditions and begin reporting free
> > +pages w
[virtio-dev] Re: [PATCH RFC 2/3] content: Document balloon feature page poison
On Thu, May 7, 2020 at 8:48 AM Cornelia Huck wrote:
>
> On Wed, 29 Apr 2020 11:57:51 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Page poison provides a way for the guest to notify the host of the content
> > expected to be found in pages when they are added back to the guest after
> > being discarded. The feature currently doesn't apply to the existing
> > balloon features, however it will apply to an upcoming feature, free page
> > reporting. Add documentation for the page poison feature describing the
> > basic functionality and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 45 +
> > 1 file changed, 45 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index 796901e83a71..c98b8ea9526a 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > page hinting. A virtqueue for providing hints as to what memory is
> > currently free is present. Configuration field free_page_hint_cmd_id
> > is valid.
> > +\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
> > +is expecting reported pages to contain a certain value when returned.
> > +Configuration field poison_val is valid.
> >
> > \end{description}
> >
> > @@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
> > feature if offered by the device.
> >
> > +The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
> > +expecting any specific value to be stored in the page.
> > +
> > \devicenormative{\subsubsection}{Feature bits}{Device Types / Memory
> > Balloon Device / Feature bits}
> > If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
> > bit, and if the driver did not accept this feature bit, the
> > @@ -5058,6 +5064,7 @@ \subsection{Device configuration
> > layout}\label{sec:Device Types / Memory Balloon
> > le32 num_pages;
> > le32 actual;
> > le32 free_page_hint_cmd_id;
> > +le32 poison_val;
>
> Add
>
> "\field{poison_val} is available if VIRTIO_BALLOON_F_PAGE_POISON has
> been negotiated."
>
> above.
>
> > };
> > \end{lstlisting}
> >
> > @@ -5087,6 +5094,9 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> >
> > \item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
> >free_page_vq is identified.
> > +
> > +\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
> > + the driver MUST update the poison_val configuration field.
>
> When does the driver set the field? Before or after DRIVER_OK?
>
> Can it change it later?
It should be before DRIVER_OK, and it should not change. The expected
use case is that this provides information that should be static about
behaviors expected of the balloon driver so if the value is changed we
would likely need to unload and reload the driver, or at least deflate
the balloon and reset reporting.
> > \end{enumerate}
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5452,6 +5462,41 @@ \subsubsection{Free Page Hinting}\label{sec:Device
> > Types / Memory Balloon Device
> > the guest or \field{free_page_hint_cmd_id} is set to
> > VIRTIO_BALLOON_CMD_ID_DONE.
> >
> > +\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device
> > / Device Operation / Page Poison}
> > +
> > +Page Poison provides a way to notify the host of the contents that are
> > +currently in the balloon pages, and those that are expected to be in the
> > +pages when they are pulled from the balloon. It is used for in-place
> > +reporting of pages without needing to pull them from the memory allocator
> > +of the guest.
> > +
> > +\begin{enumerate}
> > +
> > +\item If VIRTIO_BALLOON_F_PAGE_POISON feature is negotiated, the guest will
>
> s/is/has been/
>
> s/the guest/the driver/
>
> > + place the expected poison value in \field{poison_val} configuration data.
> > +
> > +\end{enumerate}
>
> Do you really need a list with one item?
Probably not, what I was trying to do is just define this as a
separate part from the explanation of what it is in the para
[virtio-dev] Re: [PATCH RFC 1/3] content: Document balloon feature free page hints
Thanks for the review. For anything below where I didn't comment I am
just going with your suggestion.
On Thu, May 7, 2020 at 8:35 AM Cornelia Huck wrote:
>
> On Wed, 29 Apr 2020 11:57:45 -0700
> Alexander Duyck wrote:
>
> > From: Alexander Duyck
> >
> > Free page hints allow the balloon driver to provide information on what
> > pages are not currently in use so that we can avoid the cost of copying
> > them in migration scenarios. Add a feature description for free page hints
> > describing basic functioning and requirements.
> >
> > Signed-off-by: Alexander Duyck
> > ---
> > content.tex | 113
> > +--
> > 1 file changed, 110 insertions(+), 3 deletions(-)
> >
> > diff --git a/content.tex b/content.tex
> > index b91a132df146..796901e83a71 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -5006,9 +5006,12 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Memory Balloon Device / Virtque
> > \item[0] inflateq
> > \item[1] deflateq
> > \item[2] statsq.
> > +\item[3] free_page_vq.
>
> I'd lose the trailing period (and probably also remove it from statsq;
> it just looks weird.)
>
> > \end{description}
> >
> > - Virtqueue 2 only exists if VIRTIO_BALLOON_F_STATS_VQ set.
> > + statsq only exists if VIRTIO_BALLOON_F_STATS_VQ set.
>
> As you are touching this line anyway, s/set/is set/
>
> > +
> > + free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT set.
>
> s/set/is set/
>
> >
> > \subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
> > Feature bits}
> > \begin{description}
> > @@ -5019,6 +5022,10 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > memory statistics is present.
> > \item[VIRTIO_BALLOON_F_DEFLATE_ON_OOM (2) ] Deflate balloon on
> > guest out of memory condition.
> > +\item[ VIRTIO_BALLOON_F_FREE_PAGE_HINT(3) ] Device has support for free
> > +page hinting. A virtqueue for providing hints as to what memory is
> > +currently free is present. Configuration field free_page_hint_cmd_id
> > +is valid.
> >
> > \end{description}
> >
> > @@ -5042,13 +5049,15 @@ \subsection{Feature bits}\label{sec:Device Types /
> > Memory Balloon Device / Featu
> > VIRTIO_BALLOON_F_MUST_TELL_HOST is not negotiated.
> >
> > \subsection{Device configuration layout}\label{sec:Device Types / Memory
> > Balloon Device / Device configuration layout}
> > - Both fields of this configuration
> > - are always available.
> > + The first two fields of this configuration are always present. The
> > + availability of the others all depend on various feature bits as
> > + indicated above.
>
> Maybe make this more explicit?
>
> "
> \field{num_pages} and \field{actual} are always available.
>
> \field{free_page_hint_cmd_id} is available if
> VIRTIO_BALLOON_F_FREE_PAGE_HINT has been negotiated and is read-only by
> the driver.
> "
>
> >
> > \begin{lstlisting}
> > struct virtio_balloon_config {
> > le32 num_pages;
> > le32 actual;
> > +le32 free_page_hint_cmd_id;
> > };
> > \end{lstlisting}
> >
> > @@ -5075,6 +5084,9 @@ \subsection{Device Initialization}\label{sec:Device
> > Types / Memory Balloon Devic
> >\item DRIVER_OK is set: device operation begins.
> >\item Notify the device about the stats virtqueue buffer.
> >\end{enumerate}
> > +
> > +\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
> > + free_page_vq is identified.
>
> This startup sequence now seems wrong (identifying a vq after
> DRIVER_OK). Should probably be:
>
> - id inflate/deflate vqs
> - if STATS_VQ:
> - id stats vq
> - put buffer on stats vq
> - if FREE_PAGE_HINT:
> - id free page vq
> - DRIVER_OK is set
> - if STATS_VQ:
> - notify about buffer
>
> > \end{enumerate}
> >
> > \subsection{Device Operation}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation}
> > @@ -5345,6 +5357,101 @@ \subsubsection{Memory Statistics
> > Tags}\label{sec:Device Types / Memory Balloon D
> >allocations in the guest.
> > \end{description}
> >
> > +\subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon
> > Device / Device Operation / Free Page Hinting}
> > +
> > +Free page hinting is used during migration to determine what pages within
>
> s/used
[virtio-dev] [PATCH RFC 0/3] virtio-spec: Add documentation for recently added balloon features
This patch set is meant to add documentation for balloon features that have been recently added to the Linux kernel[1,2] and that we are currently working on adding to QEMU[3]. I am submitting them as an RFC as it has been a while since I have written any requirements or specification documents, so my wording on things will likely need some updates. In addition I have not before submitted virtio-spec updates. As such I would appreciate information on if this is the correct process for updating the document, or if I need to take some other/additional steps. [1]: https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/ [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc [3]: https://lists.oasis-open.org/archives/virtio-dev/202004/msg00180.html --- Alexander Duyck (3): content: Document balloon feature free page hints content: Document balloon feature page poison content: Document balloon feature free page reporting content.tex | 232 ++- 1 file changed, 229 insertions(+), 3 deletions(-) -- - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH RFC 3/3] content: Document balloon feature free page reporting
From: Alexander Duyck
Free page reporting is a feature that allows the guest to proactively
report unused pages to the host. By making use of this feature is is
possible to reduce the overall memory footprint of the guest in cases where
some significant portion of the memory is idle. Add documentation for the
free page reporting feature describing the functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 74 +++
1 file changed, 74 insertions(+)
diff --git a/content.tex b/content.tex
index c98b8ea9526a..52955c8ff007 100644
--- a/content.tex
+++ b/content.tex
@@ -5007,12 +5007,15 @@ \subsection{Virtqueues}\label{sec:Device Types / Memory
Balloon Device / Virtque
\item[1] deflateq
\item[2] statsq.
\item[3] free_page_vq.
+\item[4] reporting_vq.
\end{description}
statsq only exists if VIRTIO_BALLOON_F_STATS_VQ set.
free_page_vq only exists if VIRTIO_BALLOON_F_FREE_PAGE_HINT set.
+ reporting_vq only exists if VIRTIO_BALLOON_F_PAGE_REPORTING set.
+
\subsection{Feature bits}\label{sec:Device Types / Memory Balloon Device /
Feature bits}
\begin{description}
\item[VIRTIO_BALLOON_F_MUST_TELL_HOST (0)] Host has to be told before
@@ -5029,6 +5032,8 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
is expecting reported pages to contain a certain value when returned.
Configuration field poison_val is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_REPORTING(5) ] Device has support for free
+page reporting. A virtqueue for reporting free guest memory is present.
\end{description}
@@ -5039,6 +5044,10 @@ \subsection{Feature bits}\label{sec:Device Types /
Memory Balloon Device / Featu
The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
expecting any specific value to be stored in the page.
+The driver MUST NOT accept VIRTIO_BALLOON_F_PAGE_REPORTING if it is expecting
+the pages to retain some initialized value and it has not negotiated
+VIRTIO_BALLOON_F_PAGE_POISON as a feature.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5097,6 +5106,13 @@ \subsection{Device Initialization}\label{sec:Device
Types / Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
the driver MUST update the poison_val configuration field.
+
+\item If the VIRTIO_BALLOON_F_PAGE_REPORTING feature bit is negotiated:
+ \begin{enumerate}
+ \item Identify the reporting virtqueue.
+ \item DRIVER_OK is set: device operation begins.
+ \item Begin reporting free pages to device.
+ \end{enumerate}
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5497,6 +5513,64 @@ \subsubsection{Page Poison}\label{sec:Device Types /
Memory Balloon Device / Dev
free page hinting as this feature did not exist prior to those features being
added.
+\subsubsection{Free Page Reporting}\label{sec:Device Types / Memory Balloon
Device / Device Operation / Free Page Reporting}
+
+Free Page Reporting provides a mechanism similar to balloon inflation,
+however it does not provide a deflation queue. The expectation is that the
+device will have a means by which it can detect the guest page access and
+fault in such pages with some initial value, likely a zero page.
+
+The driver will respond to to memory conditions and begin reporting free
+pages when some number of pages are available.
+
+\begin{enumerate}
+
+\item The driver determines it has enough pages available to begin
+ reporting pages.
+
+\item The driver gathers pages into a scatter-gather list and adds them to
+ the reporting_vq.
+
+\item The device acknowledges the reporting request.
+
+\item Once the device has acknowledged the report, the pages can be
+ returned to the location from which they were pulled.
+
+\item The driver can then continue to gather and report pages until it
+ has determined it has reported a suffcient quantity of pages.
+
+\end{enumerate}
+
+\drivernormative{\paragraph}{Free Page Reporting}{Device Types / Memory
Balloon Device / Device Operation / Free Page Reporting}
+
+Normative statements in this section apply if and only if the
+VIRTIO_BALLOON_F_PAGE_REPORTING feature has been negotiated.
+
+If the guest is expecting the free page to contain some initial value it
+MUST make use of the VIRTIO_BALLOON_F_PAGE_POISON feature to notify the
+device of this expectation via \field{poison_val}.
+
+The driver MUST NOT use the reported pages until the device has
+acknowledged the reporting request.
+
+The driver MAY report free pages any time after DRIVER_OK is set.
+
+It is RECOMMENDED that the driver avoid unecessary reads or writes
[virtio-dev] [PATCH RFC 2/3] content: Document balloon feature page poison
From: Alexander Duyck
Page poison provides a way for the guest to notify the host of the content
expected to be found in pages when they are added back to the guest after
being discarded. The feature currently doesn't apply to the existing
balloon features, however it will apply to an upcoming feature, free page
reporting. Add documentation for the page poison feature describing the
basic functionality and requirements.
Signed-off-by: Alexander Duyck
---
content.tex | 45 +
1 file changed, 45 insertions(+)
diff --git a/content.tex b/content.tex
index 796901e83a71..c98b8ea9526a 100644
--- a/content.tex
+++ b/content.tex
@@ -5026,6 +5026,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
page hinting. A virtqueue for providing hints as to what memory is
currently free is present. Configuration field free_page_hint_cmd_id
is valid.
+\item[ VIRTIO_BALLOON_F_PAGE_POISON(4) ] Host has to be notified if guest
+is expecting reported pages to contain a certain value when returned.
+Configuration field poison_val is valid.
\end{description}
@@ -5033,6 +5036,9 @@ \subsection{Feature bits}\label{sec:Device Types / Memory
Balloon Device / Featu
The driver SHOULD accept the VIRTIO_BALLOON_F_MUST_TELL_HOST
feature if offered by the device.
+The driver SHOULD clear the VIRTIO_BALLOON_F_PAGE_POISON flag if it is not
+expecting any specific value to be stored in the page.
+
\devicenormative{\subsubsection}{Feature bits}{Device Types / Memory Balloon
Device / Feature bits}
If the device offers the VIRTIO_BALLOON_F_MUST_TELL_HOST feature
bit, and if the driver did not accept this feature bit, the
@@ -5058,6 +5064,7 @@ \subsection{Device configuration layout}\label{sec:Device
Types / Memory Balloon
le32 num_pages;
le32 actual;
le32 free_page_hint_cmd_id;
+le32 poison_val;
};
\end{lstlisting}
@@ -5087,6 +5094,9 @@ \subsection{Device Initialization}\label{sec:Device Types
/ Memory Balloon Devic
\item If the VIRTIO_BALLOON_F_FREE_PAGE_HINT feature bit is negotiated the
free_page_vq is identified.
+
+\item If the VIRTIO_BALLOON_F_PAGE_POISON feature bit is negotiated then
+ the driver MUST update the poison_val configuration field.
\end{enumerate}
\subsection{Device Operation}\label{sec:Device Types / Memory Balloon Device /
Device Operation}
@@ -5452,6 +5462,41 @@ \subsubsection{Free Page Hinting}\label{sec:Device Types
/ Memory Balloon Device
the guest or \field{free_page_hint_cmd_id} is set to
VIRTIO_BALLOON_CMD_ID_DONE.
+\subsubsection{Page Poison}\label{sec:Device Types / Memory Balloon Device /
Device Operation / Page Poison}
+
+Page Poison provides a way to notify the host of the contents that are
+currently in the balloon pages, and those that are expected to be in the
+pages when they are pulled from the balloon. It is used for in-place
+reporting of pages without needing to pull them from the memory allocator
+of the guest.
+
+\begin{enumerate}
+
+\item If VIRTIO_BALLOON_F_PAGE_POISON feature is negotiated, the guest will
+ place the expected poison value in \field{poison_val} configuration data.
+
+\end{enumerate}
+
+\drivernormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if and only if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+The driver MUST populate the \field{poison_val} configuration data if it is
+expecting the page to contain some fixed value when free.
+
+The driver MAY opt to disable the feature if it will take care of
+re-initializing pages when first accessing them.
+
+\devicenormative{\paragraph}{Page Poison}{Device Types / Memory Balloon Device
/ Device Operation / Page Poison}
+
+Normative statements in this section apply if and only if the
+VIRTIO_BALLOON_F_PAGE_POISON feature has been negotiated.
+
+The device MAY ignore the \field{poison_val} for normal balloon operations and
+free page hinting as this feature did not exist prior to those features being
+added.
+
\section{SCSI Host Device}\label{sec:Device Types / SCSI Host Device}
The virtio SCSI host device groups together one or more virtual
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v23 QEMU 4/5] virtio-balloon: Implement support for page poison reporting feature
From: Alexander Duyck
We need to make certain to advertise support for page poison reporting if
we want to actually get data on if the guest will be poisoning pages.
Add a value for reporting the poison value being used if page poisoning is
enabled in the guest. With this we can determine if we will need to skip
free page reporting when it is enabled in the future.
The value currently has no impact on existing balloon interfaces. In the
case of existing balloon interfaces the onus is on the guest driver to
reapply whatever poison is in place.
When we add free page reporting the poison value is used to determine if
we can perform in-place page reporting. The expectation is that a reported
page will already contain the value specified by the poison, and the
reporting of the page should not change that value.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 29 +
include/hw/virtio/virtio-balloon.h |1 +
2 files changed, 30 insertions(+)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index a209706b4d8d..1666132a24c1 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -634,6 +634,7 @@ static void virtio_balloon_get_config(VirtIODevice *vdev,
uint8_t *config_data)
config.num_pages = cpu_to_le32(dev->num_pages);
config.actual = cpu_to_le32(dev->actual);
+config.poison_val = cpu_to_le32(dev->poison_val);
if (dev->free_page_hint_status == FREE_PAGE_HINT_S_REQUESTED) {
config.free_page_hint_cmd_id =
@@ -683,6 +684,14 @@ static ram_addr_t get_current_ram_size(void)
return size;
}
+static bool virtio_balloon_page_poison_support(void *opaque)
+{
+VirtIOBalloon *s = opaque;
+VirtIODevice *vdev = VIRTIO_DEVICE(s);
+
+return virtio_vdev_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON);
+}
+
static void virtio_balloon_set_config(VirtIODevice *vdev,
const uint8_t *config_data)
{
@@ -697,6 +706,10 @@ static void virtio_balloon_set_config(VirtIODevice *vdev,
qapi_event_send_balloon_change(vm_ram_size -
((ram_addr_t) dev->actual <<
VIRTIO_BALLOON_PFN_SHIFT));
}
+dev->poison_val = 0;
+if (virtio_balloon_page_poison_support(dev)) {
+dev->poison_val = le32_to_cpu(config.poison_val);
+}
trace_virtio_balloon_set_config(dev->actual, oldactual);
}
@@ -755,6 +768,17 @@ static const VMStateDescription
vmstate_virtio_balloon_free_page_hint = {
}
};
+static const VMStateDescription vmstate_virtio_balloon_page_poison = {
+.name = "vitio-balloon-device/page-poison",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = virtio_balloon_page_poison_support,
+.fields = (VMStateField[]) {
+VMSTATE_UINT32(poison_val, VirtIOBalloon),
+VMSTATE_END_OF_LIST()
+}
+};
+
static const VMStateDescription vmstate_virtio_balloon_device = {
.name = "virtio-balloon-device",
.version_id = 1,
@@ -767,6 +791,7 @@ static const VMStateDescription
vmstate_virtio_balloon_device = {
},
.subsections = (const VMStateDescription * []) {
_virtio_balloon_free_page_hint,
+_virtio_balloon_page_poison,
NULL
}
};
@@ -854,6 +879,8 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)
g_free(s->stats_vq_elem);
s->stats_vq_elem = NULL;
}
+
+s->poison_val = 0;
}
static void virtio_balloon_set_status(VirtIODevice *vdev, uint8_t status)
@@ -916,6 +943,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_DEFLATE_ON_OOM, false),
DEFINE_PROP_BIT("free-page-hint", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
+DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_PAGE_POISON, true),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/include/hw/virtio/virtio-balloon.h
index 108cff97e71a..3ca2a78e1aca 100644
--- a/include/hw/virtio/virtio-balloon.h
+++ b/include/hw/virtio/virtio-balloon.h
@@ -70,6 +70,7 @@ typedef struct VirtIOBalloon {
uint32_t host_features;
bool qemu_4_0_config_size;
+uint32_t poison_val;
} VirtIOBalloon;
#endif
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v23 QEMU 5/5] virtio-balloon: Provide an interface for free page reporting
From: Alexander Duyck
Add support for free page reporting. The idea is to function very similar
to how the balloon works in that we basically end up madvising the page as
not being used. However we don't really need to bother with any deflate
type logic since the page will be faulted back into the guest when it is
read or written to.
This provides a new way of letting the guest proactively report free
pages to the hypervisor, so the hypervisor can reuse them. In contrast to
inflate/deflate that is triggered via the hypervisor explicitly.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 69
include/hw/virtio/virtio-balloon.h |2 +
2 files changed, 70 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index 1666132a24c1..53abba290274 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -321,6 +321,67 @@ static void balloon_stats_set_poll_interval(Object *obj,
Visitor *v,
balloon_stats_change_timer(s, 0);
}
+static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
+{
+VirtIOBalloon *dev = VIRTIO_BALLOON(vdev);
+VirtQueueElement *elem;
+
+while ((elem = virtqueue_pop(vq, sizeof(VirtQueueElement {
+unsigned int i;
+
+/*
+ * When we discard the page it has the effect of removing the page
+ * from the hypervisor itself and causing it to be zeroed when it
+ * is returned to us. So we must not discard the page if it is
+ * accessible by another device or process, or if the guest is
+ * expecting it to retain a non-zero value.
+ */
+if (qemu_balloon_is_inhibited() || dev->poison_val) {
+goto skip_element;
+}
+
+for (i = 0; i < elem->in_num; i++) {
+void *addr = elem->in_sg[i].iov_base;
+size_t size = elem->in_sg[i].iov_len;
+ram_addr_t ram_offset;
+RAMBlock *rb;
+
+/*
+ * There is no need to check the memory section to see if
+ * it is ram/readonly/romd like there is for handle_output
+ * below. If the region is not meant to be written to then
+ * address_space_map will have allocated a bounce buffer
+ * and it will be freed in address_space_unmap and trigger
+ * and unassigned_mem_write before failing to copy over the
+ * buffer. If more than one bad descriptor is provided it
+ * will return NULL after the first bounce buffer and fail
+ * to map any resources.
+ */
+rb = qemu_ram_block_from_host(addr, false, _offset);
+if (!rb) {
+trace_virtio_balloon_bad_addr(elem->in_addr[i]);
+continue;
+}
+
+/*
+ * For now we will simply ignore unaligned memory regions, or
+ * regions that overrun the end of the RAMBlock.
+ */
+if (!QEMU_IS_ALIGNED(ram_offset | size, qemu_ram_pagesize(rb)) ||
+(ram_offset + size) > qemu_ram_get_used_length(rb)) {
+continue;
+}
+
+ram_block_discard_range(rb, ram_offset, size);
+}
+
+skip_element:
+virtqueue_push(vq, elem, 0);
+virtio_notify(vdev, vq);
+g_free(elem);
+}
+}
+
static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
{
VirtIOBalloon *s = VIRTIO_BALLOON(vdev);
@@ -841,6 +902,12 @@ static void virtio_balloon_device_realize(DeviceState
*dev, Error **errp)
virtio_error(vdev, "iothread is missing");
}
}
+
+if (virtio_has_feature(s->host_features, VIRTIO_BALLOON_F_REPORTING)) {
+s->reporting_vq = virtio_add_queue(vdev, 32,
+ virtio_balloon_handle_report);
+}
+
reset_stats(s);
}
@@ -945,6 +1012,8 @@ static Property virtio_balloon_properties[] = {
VIRTIO_BALLOON_F_FREE_PAGE_HINT, false),
DEFINE_PROP_BIT("page-poison", VirtIOBalloon, host_features,
VIRTIO_BALLOON_F_PAGE_POISON, true),
+DEFINE_PROP_BIT("free-page-reporting", VirtIOBalloon, host_features,
+VIRTIO_BALLOON_F_REPORTING, false),
/* QEMU 4.0 accidentally changed the config size even when free-page-hint
* is disabled, resulting in QEMU 3.1 migration incompatibility. This
* property retains this quirk for QEMU 4.1 machine types.
diff --git a/include/hw/virtio/virtio-balloon.h
b/include/hw/virtio/virtio-balloon.h
index 3ca2a78e1aca..28fd2b396087 100644
--- a/include/hw/virtio/virtio-balloon.h
+++ b/include/hw/virtio/virtio-balloon.h
@@ -42,7 +42,7 @@ enum virtio_balloon_free_page_hint_status {
typedef struct VirtIOBalloon {
VirtIODevice parent_obj;
-VirtQ
[virtio-dev] [PATCH v23 QEMU 2/5] linux-headers: update to contain virito-balloon free page reporting
From: Alexander Duyck Sync the latest upstream changes for free page reporting. To be replaced by a full linux header sync. Signed-off-by: Alexander Duyck --- include/standard-headers/linux/virtio_balloon.h |1 + 1 file changed, 1 insertion(+) diff --git a/include/standard-headers/linux/virtio_balloon.h b/include/standard-headers/linux/virtio_balloon.h index af0a6b59dab2..af3b7a1fa263 100644 --- a/include/standard-headers/linux/virtio_balloon.h +++ b/include/standard-headers/linux/virtio_balloon.h @@ -36,6 +36,7 @@ #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM2 /* Deflate balloon on OOM */ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ +#define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v23 QEMU 0/5] virtio-balloon: add support for page poison reporting and free page reporting
This series provides an asynchronous means of reporting free guest pages
to QEMU through virtio-balloon so that the memory associated with those
pages can be dropped and reused by other processes and/or guests on the
host. Using this it is possible to avoid unnecessary I/O to disk and
greatly improve performance in the case of memory overcommit on the host.
I originally submitted this patch series back on February 11th 2020[1],
but at that time I was focused primarily on the kernel portion of this
patch set. However as of April 7th those patches are now included in
Linus's kernel tree[2] and so I am submitting the QEMU pieces for
inclusion.
[1]:
https://lore.kernel.org/lkml/20200211224416.29318.44077.stgit@localhost.localdomain/
[2]:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b0c504f154718904ae49349147e3b7e6ae91ffdc
Changes from v17:
Fixed typo in patch 1 title
Addressed white-space issues reported via checkpatch
Added braces {} for two if statements to match expected coding style
Changes from v18:
Updated patches 2 and 3 based on input from dhildenb
Added comment to patch 2 describing what keeps us from reporting a bad page
Added patch to address issue with ROM devices being directly writable
Changes from v19:
Added std-headers change to match changes pushed for linux kernel headers
Added patch to remove "report" from page hinting code paths
Updated comment to better explain why we disable hints w/ page poisoning
Removed code that was modifying config size for poison vs hinting
Dropped x-page-poison property
Added code to bounds check the reported region vs the RAM block
Dropped patch for ROM devices as that was already pulled in by Paolo
Changes from v20:
Rearranged patches to push Linux header sync patches to front
Removed association between free page hinting and VIRTIO_BALLOON_F_PAGE_POISON
Added code to enable VIRTIO_BALLOON_F_PAGE_POISON if page reporting is enabled
Fixed possible resource leak if poison or qemu_balloon_is_inhibited return true
Changes from v21:
Added ack for patch 3
Rewrote patch description for page poison reporting feature
Made page-poison independent property and set to enabled by default
Added logic to migrate poison_val
Added several comments in code to better explain features
Switched free-page-reporting property to disabled by default
Changes from v22:
Added ack for patches 4 & 5
Added additional comment fixes in patch 3 to remove "reporting" reference
Renamed rvq in patch 5 to reporting_vq to improve readability
Moved call adding reporting_vq to after free_page_vq to fix VQ ordering
---
Alexander Duyck (5):
linux-headers: Update to allow renaming of free_page_report_cmd_id
linux-headers: update to contain virito-balloon free page reporting
virtio-balloon: Replace free page hinting references to 'report' with
'hint'
virtio-balloon: Implement support for page poison reporting feature
virtio-balloon: Provide an interface for free page reporting
hw/virtio/virtio-balloon.c | 176 ++-
include/hw/virtio/virtio-balloon.h | 23 ++-
include/standard-headers/linux/virtio_balloon.h | 12 +-
3 files changed, 159 insertions(+), 52 deletions(-)
--
-
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
[virtio-dev] [PATCH v23 QEMU 3/5] virtio-balloon: Replace free page hinting references to 'report' with 'hint'
From: Alexander Duyck
In an upcoming patch a feature named Free Page Reporting is about to be
added. In order to avoid any confusion we should drop the use of the word
'report' when referring to Free Page Hinting. So what this patch does is go
through and replace all instances of 'report' with 'hint" when we are
referring to free page hinting.
Acked-by: David Hildenbrand
Signed-off-by: Alexander Duyck
---
hw/virtio/virtio-balloon.c | 78 ++--
include/hw/virtio/virtio-balloon.h | 20 +
2 files changed, 49 insertions(+), 49 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index a4729f7fc930..a209706b4d8d 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -466,21 +466,21 @@ static bool get_free_page_hints(VirtIOBalloon *dev)
ret = false;
goto out;
}
-if (id == dev->free_page_report_cmd_id) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_START;
+if (id == dev->free_page_hint_cmd_id) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_START;
} else {
/*
* Stop the optimization only when it has started. This
* avoids a stale stop sign for the previous command.
*/
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
-dev->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
+dev->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
}
}
}
if (elem->in_num) {
-if (dev->free_page_report_status == FREE_PAGE_REPORT_S_START) {
+if (dev->free_page_hint_status == FREE_PAGE_HINT_S_START) {
qemu_guest_free_page_hint(elem->in_sg[0].iov_base,
elem->in_sg[0].iov_len);
}
@@ -506,11 +506,11 @@ static void virtio_ballloon_get_free_page_hints(void
*opaque)
qemu_mutex_unlock(>free_page_lock);
virtio_notify(vdev, vq);
/*
- * Start to poll the vq once the reporting started. Otherwise, continue
+ * Start to poll the vq once the hinting started. Otherwise, continue
* only when there are entries on the vq, which need to be given back.
*/
} while (continue_to_get_hints ||
- dev->free_page_report_status == FREE_PAGE_REPORT_S_START);
+ dev->free_page_hint_status == FREE_PAGE_HINT_S_START);
virtio_queue_set_notification(vq, 1);
}
@@ -531,14 +531,14 @@ static void virtio_balloon_free_page_start(VirtIOBalloon
*s)
return;
}
-if (s->free_page_report_cmd_id == UINT_MAX) {
-s->free_page_report_cmd_id =
- VIRTIO_BALLOON_FREE_PAGE_REPORT_CMD_ID_MIN;
+if (s->free_page_hint_cmd_id == UINT_MAX) {
+s->free_page_hint_cmd_id =
+ VIRTIO_BALLOON_FREE_PAGE_HINT_CMD_ID_MIN;
} else {
-s->free_page_report_cmd_id++;
+s->free_page_hint_cmd_id++;
}
-s->free_page_report_status = FREE_PAGE_REPORT_S_REQUESTED;
+s->free_page_hint_status = FREE_PAGE_HINT_S_REQUESTED;
virtio_notify_config(vdev);
}
@@ -546,18 +546,18 @@ static void virtio_balloon_free_page_stop(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-if (s->free_page_report_status != FREE_PAGE_REPORT_S_STOP) {
+if (s->free_page_hint_status != FREE_PAGE_HINT_S_STOP) {
/*
* The lock also guarantees us that the
* virtio_ballloon_get_free_page_hints exits after the
- * free_page_report_status is set to S_STOP.
+ * free_page_hint_status is set to S_STOP.
*/
qemu_mutex_lock(>free_page_lock);
/*
- * The guest hasn't done the reporting, so host sends a notification
- * to the guest to actively stop the reporting.
+ * The guest isn't done hinting, so send a notification
+ * to the guest to actively stop the hinting.
*/
-s->free_page_report_status = FREE_PAGE_REPORT_S_STOP;
+s->free_page_hint_status = FREE_PAGE_HINT_S_STOP;
qemu_mutex_unlock(>free_page_lock);
virtio_notify_config(vdev);
}
@@ -567,15 +567,15 @@ static void virtio_balloon_free_page_done(VirtIOBalloon
*s)
{
VirtIODevice *vdev = VIRTIO_DEVICE(s);
-s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
+s->free_page_hint_status = FREE_PAGE_HINT_S_DONE;
virtio_notify_config(vdev);
}
static int
-virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)
+virtio_balloon_free_page_hint_notify(NotifierWithReturn *n, void *data)
{
VirtIOBalloon *dev = container_of(n, VirtIOBalloon,
- free_page_r
