Re: [PATCH 6/6] doc/kvm: Add halt polling documentation
On Fri, 2016-10-14 at 09:16 +0800, Wanpeng Li wrote: > 2016-10-14 8:53 GMT+08:00 Suraj Jitindar Singhcom>: > > > > There is currently no documentation about the halt polling > > capabilities > > of the kvm module. Add some documentation describing the mechanism > > as well > > as the module parameters to all better understanding of how halt > > polling > > should be used and the effect of tuning the module parameters. > How about replace "halt-polling" by "Adaptive halt-polling"? Btw, Yeah that's slightly more descriptive I guess > thanks for your docs. > > Regards, > Wanpeng Li > > > > > > > Signed-off-by: Suraj Jitindar Singh > > --- > > Documentation/virtual/kvm/00-INDEX | 2 + > > Documentation/virtual/kvm/halt-polling.txt | 127 > > + > > 2 files changed, 129 insertions(+) > > create mode 100644 Documentation/virtual/kvm/halt-polling.txt > > > > diff --git a/Documentation/virtual/kvm/00-INDEX > > b/Documentation/virtual/kvm/00-INDEX > > index fee9f2b..69fe1a8 100644 > > --- a/Documentation/virtual/kvm/00-INDEX > > +++ b/Documentation/virtual/kvm/00-INDEX > > @@ -6,6 +6,8 @@ cpuid.txt > > - KVM-specific cpuid leaves (x86). > > devices/ > > - KVM_CAP_DEVICE_CTRL userspace API. > > +halt-polling.txt > > + - notes on halt-polling > > hypercalls.txt > > - KVM hypercalls. > > locking.txt > > diff --git a/Documentation/virtual/kvm/halt-polling.txt > > b/Documentation/virtual/kvm/halt-polling.txt > > new file mode 100644 > > index 000..4a84183 > > --- /dev/null > > +++ b/Documentation/virtual/kvm/halt-polling.txt > > @@ -0,0 +1,127 @@ > > +The KVM halt polling system > > +=== > > + > > +The KVM halt polling system provides a feature within KVM whereby > > the latency > > +of a guest can, under some circumstances, be reduced by polling in > > the host > > +for some time period after the guest has elected to no longer run > > by cedeing. > > +That is, when a guest vcpu has ceded, or in the case of powerpc > > when all of the > > +vcpus of a single vcore have ceded, the host kernel polls for > > wakeup conditions > > +before giving up the cpu to the scheduler in order to let > > something else run. > > + > > +Polling provides a latency advantage in cases where the guest can > > be run again > > +very quickly by at least saving us a trip through the scheduler, > > normally on > > +the order of a few micro-seconds, although performance benefits > > are workload > > +dependant. In the event that no wakeup source arrives during the > > polling > > +interval or some other task on the runqueue is runnable the > > scheduler is > > +invoked. Thus halt polling is especially useful on workloads with > > very short > > +wakeup periods where the time spent halt polling is minimised and > > the time > > +savings of not invoking the scheduler are distinguishable. > > + > > +The generic halt polling code is implemented in: > > + > > + virt/kvm/kvm_main.c: kvm_vcpu_block() > > + > > +The powerpc kvm-hv specific case is implemented in: > > + > > + arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked() > > + > > +Halt Polling Interval > > += > > + > > +The maximum time for which to poll before invoking the scheduler, > > referred to > > +as the halt polling interval, is increased and decreased based on > > the perceived > > +effectiveness of the polling in an attempt to limit pointless > > polling. > > +This value is stored in either the vcpu struct: > > + > > + kvm_vcpu->halt_poll_ns > > + > > +or in the case of powerpc kvm-hv, in the vcore struct: > > + > > + kvmppc_vcore->halt_poll_ns > > + > > +Thus this is a per vcpu (or vcore) value. > > + > > +During polling if a wakeup source is received within the halt > > polling interval, > > +the interval is left unchanged. In the event that a wakeup source > > isn't > > +received during the polling interval (and thus schedule is > > invoked) there are > > +two options, either the polling interval and total block time[0] > > were less than > > +the global max polling interval (see module params below), or the > > total block > > +time was greater than the global max polling interval. > > + > > +In the event that both the polling interval and total block time > > were less than > > +the global max polling interval then the polling interval can be > > increased in > > +the hope that next time during the longer polling interval the > > wake up source > > +will be received while the host is polling and the latency > > benefits will be > > +received. The polling interval is grown in the function > > grow_halt_poll_ns() and > > +is multiplied by the module parameter halt_poll_ns_grow. > > + > > +In the event that the total block time was greater than the global > > max polling > > +interval then the host will never poll for long enough (limited by > > the global > > +max) to wakeup during
Re: [PATCH 6/6] doc/kvm: Add halt polling documentation
2016-10-14 8:53 GMT+08:00 Suraj Jitindar Singh: > There is currently no documentation about the halt polling capabilities > of the kvm module. Add some documentation describing the mechanism as well > as the module parameters to all better understanding of how halt polling > should be used and the effect of tuning the module parameters. How about replace "halt-polling" by "Adaptive halt-polling"? Btw, thanks for your docs. Regards, Wanpeng Li > > Signed-off-by: Suraj Jitindar Singh > --- > Documentation/virtual/kvm/00-INDEX | 2 + > Documentation/virtual/kvm/halt-polling.txt | 127 > + > 2 files changed, 129 insertions(+) > create mode 100644 Documentation/virtual/kvm/halt-polling.txt > > diff --git a/Documentation/virtual/kvm/00-INDEX > b/Documentation/virtual/kvm/00-INDEX > index fee9f2b..69fe1a8 100644 > --- a/Documentation/virtual/kvm/00-INDEX > +++ b/Documentation/virtual/kvm/00-INDEX > @@ -6,6 +6,8 @@ cpuid.txt > - KVM-specific cpuid leaves (x86). > devices/ > - KVM_CAP_DEVICE_CTRL userspace API. > +halt-polling.txt > + - notes on halt-polling > hypercalls.txt > - KVM hypercalls. > locking.txt > diff --git a/Documentation/virtual/kvm/halt-polling.txt > b/Documentation/virtual/kvm/halt-polling.txt > new file mode 100644 > index 000..4a84183 > --- /dev/null > +++ b/Documentation/virtual/kvm/halt-polling.txt > @@ -0,0 +1,127 @@ > +The KVM halt polling system > +=== > + > +The KVM halt polling system provides a feature within KVM whereby the latency > +of a guest can, under some circumstances, be reduced by polling in the host > +for some time period after the guest has elected to no longer run by cedeing. > +That is, when a guest vcpu has ceded, or in the case of powerpc when all of > the > +vcpus of a single vcore have ceded, the host kernel polls for wakeup > conditions > +before giving up the cpu to the scheduler in order to let something else run. > + > +Polling provides a latency advantage in cases where the guest can be run > again > +very quickly by at least saving us a trip through the scheduler, normally on > +the order of a few micro-seconds, although performance benefits are workload > +dependant. In the event that no wakeup source arrives during the polling > +interval or some other task on the runqueue is runnable the scheduler is > +invoked. Thus halt polling is especially useful on workloads with very short > +wakeup periods where the time spent halt polling is minimised and the time > +savings of not invoking the scheduler are distinguishable. > + > +The generic halt polling code is implemented in: > + > + virt/kvm/kvm_main.c: kvm_vcpu_block() > + > +The powerpc kvm-hv specific case is implemented in: > + > + arch/powerpc/kvm/book3s_hv.c: kvmppc_vcore_blocked() > + > +Halt Polling Interval > += > + > +The maximum time for which to poll before invoking the scheduler, referred to > +as the halt polling interval, is increased and decreased based on the > perceived > +effectiveness of the polling in an attempt to limit pointless polling. > +This value is stored in either the vcpu struct: > + > + kvm_vcpu->halt_poll_ns > + > +or in the case of powerpc kvm-hv, in the vcore struct: > + > + kvmppc_vcore->halt_poll_ns > + > +Thus this is a per vcpu (or vcore) value. > + > +During polling if a wakeup source is received within the halt polling > interval, > +the interval is left unchanged. In the event that a wakeup source isn't > +received during the polling interval (and thus schedule is invoked) there are > +two options, either the polling interval and total block time[0] were less > than > +the global max polling interval (see module params below), or the total block > +time was greater than the global max polling interval. > + > +In the event that both the polling interval and total block time were less > than > +the global max polling interval then the polling interval can be increased in > +the hope that next time during the longer polling interval the wake up source > +will be received while the host is polling and the latency benefits will be > +received. The polling interval is grown in the function grow_halt_poll_ns() > and > +is multiplied by the module parameter halt_poll_ns_grow. > + > +In the event that the total block time was greater than the global max > polling > +interval then the host will never poll for long enough (limited by the global > +max) to wakeup during the polling interval so it may as well be shrunk in > order > +to avoid pointless polling. The polling interval is shrunk in the function > +shrink_halt_poll_ns() and is divided by the module parameter > +halt_poll_ns_shrink, or set to 0 iff halt_poll_ns_shrink == 0. > + > +It is worth noting that this adjustment process attempts to hone in on some > +steady state polling interval but will only really do a