Signed-off-by: Dario Faggioli <>
Cc: Meng Xu <>
Cc: George Dunlap <>
Cc: Wei Liu <>
Cc: Lars Kurth <>
Cc: Andrew Cooper <>
Cc: Ian Jackson <>
Cc: Jan Beulich <>
Cc: Konrad Rzeszutek Wilk <>
Cc: Stefano Stabellini <>
 docs/features/rtds.pandoc |  125 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 125 insertions(+)
 create mode 100644 docs/features/rtds.pandoc

diff --git a/docs/features/rtds.pandoc b/docs/features/rtds.pandoc
new file mode 100644
index 0000000..76aa165
--- /dev/null
+++ b/docs/features/rtds.pandoc
@@ -0,0 +1,125 @@
+% Revision 1
+# Basics
+---------------- ----------------------------------------------------
+         Status: e.g. **Experimental**
+Architecture(s): e.g. x86, arm
+      Component: e.g. Hypervisor
+---------------- ----------------------------------------------------
+# Overview
+RTDS is one of the virtual CPU (vCPU) scheduler available in the Xen
+hypervisor. The job of an hypervisor's virtual CPU scheduler is to
+decide, among all the various vCPUs of the various virtual machines,
+which ones should execute on the host's physical CPUs (pCPUs), at any
+given point in time.
+RTDS is a real--time scheduler, so its purpose is enabling
+**deterministic** scheduling of the virtual machine's vCPUs. It has
+been originally developed in the context of the RT-Xen project.
+# User details
+RTDS is not in use by default. In order to use it as the Xen scheduler
+the following parameter should be passed to the hypervisor at boot:
+    `sched=rtds`
+Once the system is live, for creating a cpupool with RTDS as its
+scheduler, either compile a cpupool configuration file, as described
+in `docs/man/xlcpupool.cfg.pod.5` (and as exemplified in
+`tools/examples/cpupool`), or use just `xl` directly:
+    xl cpupool-create name=\"pool-rt\" sched=\"rtds\" cpus=[4,5,6,8]
+For checking or changing a VM's scheduling parameters from xl, do
+as follows:
+    * `xl sched-rtds -d vm-rt`
+    * `xl sched-rtds -d vm-rt -t 10000 -b 25000`
+It is possible, for a multiple vCPUs VM, to change the parameters of
+each vCPU individually:
+    * `xl sched-rtds -d vm-rt -v 0 -p 20000 -b 10000 -v 1 -p 45000 -b 12000`
+# Technical details
+The implementation entirely lives in the hypervisor. Xen has a
+pluggable, hook based, architecture for schedulers. RTDS code
+is all inside one file: `xen/common/sched_rt.c`
+In libxl, the availability of the RTDS scheduler is advertised by
+the presence of the LIBXL\_HAVE\_SCHED\_RTDS symbol. The ability of
+specifying different scheduling parameters for each vcpu has been
+introduced later, and is available if the following symbols are defined:
+# Limitations
+RTDS is a special purpose scheduling. This is by design, and not at
+all a limitation, but it is certainly something to keep in mind when
+thinking about using it. The purpose of the scheduler is enabling
+deterministic and statically analyzable behavior (as per the
+real-time academic literature), according to the scheduling parameters
+assigned to each vCPU.
+Using RTDS a the Xen scheduler, and/or for general purpose workloads
+is definitely possible, but the vCPU scheduling parameters (of both
+Domain0 and of the various VMs) would probably require tweaking, with
+respect to their default values.
+# Testing
+Any change done in RTDS must be tested by doing the following:
+* create a cpupool with RTDS as its scheduler,
+* create a few virtual machines a move them in and out of the pool,
+* create a few virtual machines, directly inside the pool, and verify
+  that they boot and can run some basic workload (e.g., login into them
+  and run simple commands),
+* shutdown/reboot the virtual machines,
+The fact that the system boots fine when passing `sched=rtds` to Xen
+should also be verified.
+Finally, to check that the scheduler is working properly (although only
+at a macroscopic level), the following should be done:
+* create a VM with 1 vCPU and put it in the RTDS cpupool,
+* set the scheduling parameters such as it has a 50% reservation, with
+  `xl sched-rtds -d vm -t 100000 -b 50000`,
+* run a CPU-burning process inside the VM (e.g., `yes`),
+* check with `xentop` (in Domain0) that the VM is getting no more than
+  50% pCPU time.
+# Areas for improvement
+* Work-conserving mode to be added;
+* performance assessment, especially focusing on what level of real-time
+  behavior the scheduler enables.
+# Known issues
+* OSSTest reports occasional failures on ARM.
+# References
+* "RT-Xen: Real-Time Virtualization" [XPDS14 
+* "Scheduling in Xen" [XPDS15 
+* [RT-Xen Project](
+* [RTDS-Based-Scheduler](
+* [Xen Project 
+# History
+Date       Revision Version  Notes
+---------- -------- -------- -------------------------------------------
+2016-10-13 1        Xen 4.8  Document written
+---------- -------- -------- -------------------------------------------

Xen-devel mailing list

Reply via email to