Re: [PATCH 6/6] doc/kvm: Add halt polling documentation

[Suraj Jitindar Singh]

On Fri, 2016-10-14 at 09:16 +0800, Wanpeng Li wrote:
> 2016-10-14 8:53 GMT+08:00 Suraj Jitindar Singh  com>:
> > 
> > There is currently no documentation about the halt polling
> > capabilities
> > of the kvm module. Add some documentation describing the mechanism
> > as well
> > as the module parameters to all better understanding of how halt
> > polling
> > should be used and the effect of tuning the module parameters.
> How about replace "halt-polling" by "Adaptive halt-polling"? Btw,
Yeah that's slightly more descriptive I guess
> thanks for your docs.
> 
> Regards,
> Wanpeng Li
> 
> > 
> > 
> > Signed-off-by: Suraj Jitindar Singh 
> > ---
> >  Documentation/virtual/kvm/00-INDEX |   2 +
> >  Documentation/virtual/kvm/halt-polling.txt | 127
> > +
> >  2 files changed, 129 insertions(+)
> >  create mode 100644 Documentation/virtual/kvm/halt-polling.txt
> > 
> > diff --git a/Documentation/virtual/kvm/00-INDEX
> > b/Documentation/virtual/kvm/00-INDEX
> > index fee9f2b..69fe1a8 100644
> > --- a/Documentation/virtual/kvm/00-INDEX
> > +++ b/Documentation/virtual/kvm/00-INDEX
> > @@ -6,6 +6,8 @@ cpuid.txt
> > - KVM-specific cpuid leaves (x86).
> >  devices/
> > - KVM_CAP_DEVICE_CTRL userspace API.
> > +halt-polling.txt
> > +   - notes on halt-polling
> >  hypercalls.txt
> > - KVM hypercalls.
> >  locking.txt
> > diff --git a/Documentation/virtual/kvm/halt-polling.txt
> > b/Documentation/virtual/kvm/halt-polling.txt
> > new file mode 100644
> > index 000..4a84183
> > --- /dev/null
> > +++ b/Documentation/virtual/kvm/halt-polling.txt
> > @@ -0,0 +1,127 @@
> > +The KVM halt polling system
> > +===
> > +
> > +The KVM halt polling system provides a feature within KVM whereby
> > the latency
> > +of a guest can, under some circumstances, be reduced by polling in
> > the host
> > +for some time period after the guest has elected to no longer run
> > by cedeing.
> > +That is, when a guest vcpu has ceded, or in the case of powerpc
> > when all of the
> > +vcpus of a single vcore have ceded, the host kernel polls for
> > wakeup conditions
> > +before giving up the cpu to the scheduler in order to let
> > something else run.
> > +
> > +Polling provides a latency advantage in cases where the guest can
> > be run again
> > +very quickly by at least saving us a trip through the scheduler,
> > normally on
> > +the order of a few micro-seconds, although performance benefits
> > are workload
> > +dependant. In the event that no wakeup source arrives during the
> > polling
> > +interval or some other task on the runqueue is runnable the
> > scheduler is
> > +invoked. Thus halt polling is especially useful on workloads with
> > very short
> > +wakeup periods where the time spent halt polling is minimised and
> > the time
> > +savings of not invoking the scheduler are distinguishable.
> > +
> > +The generic halt polling code is implemented in:
> > +
> > +   virt/kvm/kvm_main.c: kvm_vcpu_block()
> > +
> > +The powerpc kvm-hv specific case is implemented in:
> > +
> > +   arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked()
> > +
> > +Halt Polling Interval
> > +=
> > +
> > +The maximum time for which to poll before invoking the scheduler,
> > referred to
> > +as the halt polling interval, is increased and decreased based on
> > the perceived
> > +effectiveness of the polling in an attempt to limit pointless
> > polling.
> > +This value is stored in either the vcpu struct:
> > +
> > +   kvm_vcpu->halt_poll_ns
> > +
> > +or in the case of powerpc kvm-hv, in the vcore struct:
> > +
> > +   kvmppc_vcore->halt_poll_ns
> > +
> > +Thus this is a per vcpu (or vcore) value.
> > +
> > +During polling if a wakeup source is received within the halt
> > polling interval,
> > +the interval is left unchanged. In the event that a wakeup source
> > isn't
> > +received during the polling interval (and thus schedule is
> > invoked) there are
> > +two options, either the polling interval and total block time[0]
> > were less than
> > +the global max polling interval (see module params below), or the
> > total block
> > +time was greater than the global max polling interval.
> > +
> > +In the event that both the polling interval and total block time
> > were less than
> > +the global max polling interval then the polling interval can be
> > increased in
> > +the hope that next time during the longer polling interval the
> > wake up source
> > +will be received while the host is polling and the latency
> > benefits will be
> > +received. The polling interval is grown in the function
> > grow_halt_poll_ns() and
> > +is multiplied by the module parameter halt_poll_ns_grow.
> > +
> > +In the event that the total block time was greater than the global
> > max polling
> > +interval then the host will never poll for long enough (limited by
> > the global
> > +max) to wakeup during 

Re: [PATCH 6/6] doc/kvm: Add halt polling documentation

[Wanpeng Li]

2016-10-14 8:53 GMT+08:00 Suraj Jitindar Singh :
> There is currently no documentation about the halt polling capabilities
> of the kvm module. Add some documentation describing the mechanism as well
> as the module parameters to all better understanding of how halt polling
> should be used and the effect of tuning the module parameters.

How about replace "halt-polling" by "Adaptive halt-polling"? Btw,
thanks for your docs.

Regards,
Wanpeng Li

>
> Signed-off-by: Suraj Jitindar Singh 
> ---
>  Documentation/virtual/kvm/00-INDEX |   2 +
>  Documentation/virtual/kvm/halt-polling.txt | 127 
> +
>  2 files changed, 129 insertions(+)
>  create mode 100644 Documentation/virtual/kvm/halt-polling.txt
>
> diff --git a/Documentation/virtual/kvm/00-INDEX 
> b/Documentation/virtual/kvm/00-INDEX
> index fee9f2b..69fe1a8 100644
> --- a/Documentation/virtual/kvm/00-INDEX
> +++ b/Documentation/virtual/kvm/00-INDEX
> @@ -6,6 +6,8 @@ cpuid.txt
> - KVM-specific cpuid leaves (x86).
>  devices/
> - KVM_CAP_DEVICE_CTRL userspace API.
> +halt-polling.txt
> +   - notes on halt-polling
>  hypercalls.txt
> - KVM hypercalls.
>  locking.txt
> diff --git a/Documentation/virtual/kvm/halt-polling.txt 
> b/Documentation/virtual/kvm/halt-polling.txt
> new file mode 100644
> index 000..4a84183
> --- /dev/null
> +++ b/Documentation/virtual/kvm/halt-polling.txt
> @@ -0,0 +1,127 @@
> +The KVM halt polling system
> +===
> +
> +The KVM halt polling system provides a feature within KVM whereby the latency
> +of a guest can, under some circumstances, be reduced by polling in the host
> +for some time period after the guest has elected to no longer run by cedeing.
> +That is, when a guest vcpu has ceded, or in the case of powerpc when all of 
> the
> +vcpus of a single vcore have ceded, the host kernel polls for wakeup 
> conditions
> +before giving up the cpu to the scheduler in order to let something else run.
> +
> +Polling provides a latency advantage in cases where the guest can be run 
> again
> +very quickly by at least saving us a trip through the scheduler, normally on
> +the order of a few micro-seconds, although performance benefits are workload
> +dependant. In the event that no wakeup source arrives during the polling
> +interval or some other task on the runqueue is runnable the scheduler is
> +invoked. Thus halt polling is especially useful on workloads with very short
> +wakeup periods where the time spent halt polling is minimised and the time
> +savings of not invoking the scheduler are distinguishable.
> +
> +The generic halt polling code is implemented in:
> +
> +   virt/kvm/kvm_main.c: kvm_vcpu_block()
> +
> +The powerpc kvm-hv specific case is implemented in:
> +
> +   arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked()
> +
> +Halt Polling Interval
> +=
> +
> +The maximum time for which to poll before invoking the scheduler, referred to
> +as the halt polling interval, is increased and decreased based on the 
> perceived
> +effectiveness of the polling in an attempt to limit pointless polling.
> +This value is stored in either the vcpu struct:
> +
> +   kvm_vcpu->halt_poll_ns
> +
> +or in the case of powerpc kvm-hv, in the vcore struct:
> +
> +   kvmppc_vcore->halt_poll_ns
> +
> +Thus this is a per vcpu (or vcore) value.
> +
> +During polling if a wakeup source is received within the halt polling 
> interval,
> +the interval is left unchanged. In the event that a wakeup source isn't
> +received during the polling interval (and thus schedule is invoked) there are
> +two options, either the polling interval and total block time[0] were less 
> than
> +the global max polling interval (see module params below), or the total block
> +time was greater than the global max polling interval.
> +
> +In the event that both the polling interval and total block time were less 
> than
> +the global max polling interval then the polling interval can be increased in
> +the hope that next time during the longer polling interval the wake up source
> +will be received while the host is polling and the latency benefits will be
> +received. The polling interval is grown in the function grow_halt_poll_ns() 
> and
> +is multiplied by the module parameter halt_poll_ns_grow.
> +
> +In the event that the total block time was greater than the global max 
> polling
> +interval then the host will never poll for long enough (limited by the global
> +max) to wakeup during the polling interval so it may as well be shrunk in 
> order
> +to avoid pointless polling. The polling interval is shrunk in the function
> +shrink_halt_poll_ns() and is divided by the module parameter
> +halt_poll_ns_shrink, or set to 0 iff halt_poll_ns_shrink == 0.
> +
> +It is worth noting that this adjustment process attempts to hone in on some
> +steady state polling interval but will only really do a 

[PATCH 6/6] doc/kvm: Add halt polling documentation

[Suraj Jitindar Singh]

There is currently no documentation about the halt polling capabilities
of the kvm module. Add some documentation describing the mechanism as well
as the module parameters to all better understanding of how halt polling
should be used and the effect of tuning the module parameters.

Signed-off-by: Suraj Jitindar Singh 
---
 Documentation/virtual/kvm/00-INDEX |   2 +
 Documentation/virtual/kvm/halt-polling.txt | 127 +
 2 files changed, 129 insertions(+)
 create mode 100644 Documentation/virtual/kvm/halt-polling.txt

diff --git a/Documentation/virtual/kvm/00-INDEX 
b/Documentation/virtual/kvm/00-INDEX
index fee9f2b..69fe1a8 100644
--- a/Documentation/virtual/kvm/00-INDEX
+++ b/Documentation/virtual/kvm/00-INDEX
@@ -6,6 +6,8 @@ cpuid.txt
- KVM-specific cpuid leaves (x86).
 devices/
- KVM_CAP_DEVICE_CTRL userspace API.
+halt-polling.txt
+   - notes on halt-polling
 hypercalls.txt
- KVM hypercalls.
 locking.txt
diff --git a/Documentation/virtual/kvm/halt-polling.txt 
b/Documentation/virtual/kvm/halt-polling.txt
new file mode 100644
index 000..4a84183
--- /dev/null
+++ b/Documentation/virtual/kvm/halt-polling.txt
@@ -0,0 +1,127 @@
+The KVM halt polling system
+===
+
+The KVM halt polling system provides a feature within KVM whereby the latency
+of a guest can, under some circumstances, be reduced by polling in the host
+for some time period after the guest has elected to no longer run by cedeing.
+That is, when a guest vcpu has ceded, or in the case of powerpc when all of the
+vcpus of a single vcore have ceded, the host kernel polls for wakeup conditions
+before giving up the cpu to the scheduler in order to let something else run.
+
+Polling provides a latency advantage in cases where the guest can be run again
+very quickly by at least saving us a trip through the scheduler, normally on
+the order of a few micro-seconds, although performance benefits are workload
+dependant. In the event that no wakeup source arrives during the polling
+interval or some other task on the runqueue is runnable the scheduler is
+invoked. Thus halt polling is especially useful on workloads with very short
+wakeup periods where the time spent halt polling is minimised and the time
+savings of not invoking the scheduler are distinguishable.
+
+The generic halt polling code is implemented in:
+
+   virt/kvm/kvm_main.c: kvm_vcpu_block()
+
+The powerpc kvm-hv specific case is implemented in:
+
+   arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked()
+
+Halt Polling Interval
+=
+
+The maximum time for which to poll before invoking the scheduler, referred to
+as the halt polling interval, is increased and decreased based on the perceived
+effectiveness of the polling in an attempt to limit pointless polling.
+This value is stored in either the vcpu struct:
+
+   kvm_vcpu->halt_poll_ns
+
+or in the case of powerpc kvm-hv, in the vcore struct:
+
+   kvmppc_vcore->halt_poll_ns
+
+Thus this is a per vcpu (or vcore) value.
+
+During polling if a wakeup source is received within the halt polling interval,
+the interval is left unchanged. In the event that a wakeup source isn't
+received during the polling interval (and thus schedule is invoked) there are
+two options, either the polling interval and total block time[0] were less than
+the global max polling interval (see module params below), or the total block
+time was greater than the global max polling interval.
+
+In the event that both the polling interval and total block time were less than
+the global max polling interval then the polling interval can be increased in
+the hope that next time during the longer polling interval the wake up source
+will be received while the host is polling and the latency benefits will be
+received. The polling interval is grown in the function grow_halt_poll_ns() and
+is multiplied by the module parameter halt_poll_ns_grow.
+
+In the event that the total block time was greater than the global max polling
+interval then the host will never poll for long enough (limited by the global
+max) to wakeup during the polling interval so it may as well be shrunk in order
+to avoid pointless polling. The polling interval is shrunk in the function
+shrink_halt_poll_ns() and is divided by the module parameter
+halt_poll_ns_shrink, or set to 0 iff halt_poll_ns_shrink == 0.
+
+It is worth noting that this adjustment process attempts to hone in on some
+steady state polling interval but will only really do a good job for wakeups
+which come at an approximately constant rate, otherwise there will be constant
+adjustment of the polling interval.
+
+[0] total block time: the time between when the halt polling function is
+ invoked and a wakeup source received (irrespective of
+ whether the scheduler is invoked within that function).
+
+Module Parameters
+=
+
+The