Hi all,

For now in there is no agreement on the remote guest console solution, so I decide to do some investigation continue the discussion.

Our goal
VM serial console remote access in CLI mode. That means the client runs without X environment.

There are several proposals.

1. Sandboxed sshd
VDSM runs a new host sshd instance in virtual machine/sandbox and redirects the virtio console to it.
2. Third-party sshd
VDSM runs third-party sshd library/implementation and redirects virtio console to it.
3. Spice
Extend spice to support console and implement a client to be run without GUI environment
4. oVirt shell -> Engine -> libvirt
The user connects to Engine via oVirt CLI, then issues a "serial-console" command, then Engine locates the host and connect to the guest console. Currently there is a workaround, it invokes "virsh -c qemu+tls://host/qemu console vmid" from Engine side.
5. VDSM console streaming API
VDSM exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC binding. Then implement the related client in vdsClient and Engine

Detailed discussion

1. Sandboxes
Solution 1 and 2 allow users connect to console using their favorite ssh client. The login name is vmid, the password is set by setVmTicket() call of VDSM. The connection will be lost during migration. This is similar to VNC in oVirt.

I take a look at several sandbox technologies, including libvirt-sandbox, lxc and selinux. a) libvirt-sandbox boots a VM using host kernel and initramfs, then passthru the host file system to the VM in read only mode. We can also add extra binding to the guest file system. It's very easy to use. To run shell in a VM, one can just issues

virt-sandbox -c qemu:///session  /bin/sh

Then the VM will be ready in several seconds.
However it will trigger some selinux violations. Currently there is no official support for selinux policy configuration from this project. In the project page this is put in the todo list.

b) lxc utilize Linux container to run a process in sandbox. It needs to be configured properly. I find in the package lxc-templates there is an example configuration file for running sshd in lxc.

c) sandbox command in the package policycoreutils-python makes use of selinux to run a process in sandbox, but there is no official or example policy files for sshd.

In a word, for sandbox technologies, we have to configure the policies/file system binding/network carefully and test the compatibility with popular sshd implementations (openssh-server). When those sshd upgrade, the policy must be upgraded by us at the same time. Since the policies are not maintained by who implements sshd, this is a burden for us.

Work to do
  Write and maintain the policies.
  Find ways for auth callback and redirecting data to openssh-server.

  Re-use existing pieces and technologies (host sshd, sandbox).
  User friendly, they can use existing ssh clients.
Connection is lost in migration, this is not a big problem because 1) VNC connection share the same problem, 2) the user can reconnect manually. It's not easy to maintain the sandbox policies/file system binding/network for compatibility with sshd.

2. Third-party sshd implementations
Almost the same as solution 1 but with better flexibility. VDSM can import a third-party sshd library and let that library deal with auth and transport. VDSM just have to implement the data forwarding. Many people consider this is insecure but I think the ticket solution for VNC is even not as secure as this. Currently most of us only trust openssh-server and think the quality of third-party sshd is low. I searched for a while and found twisted.conch from the popular twisted project. I'm not familiar with twisted.conch, but I still put it in this mail to collect opinions from potential twisted.conch experts.

In a word, I prefer sandbox technologies to third-party sshd implementations unless there is a implementation as good as openssh-server.

Work to do
  Integrate twisted.conch into VDSM

Very flexible. If library provide auth callback to VDSM, then VDSM can just compares the login password to the VM ticket without knowing SSH detials.
Third party implementations are not as secure and carefully maintained as sshd in the host (probably openssh-server).

3. Extend Spice to support console
Is it possible to implement a spice client can be run in pure text mode without GUI environment? If we extend the protocol to support console stream but the client must be run in GUI, it will be less useful.

  No new VMs and server process, easy for maintenance.
  Must wait for Spice developers to commit the support.
Need special client program in CLI, the user may prefer existing client program like ssh. It not a big problem because this feature can be put in to oVirt shell.

4. oVirt shell -> Engine -> libvirtd
This is the current workaround described in


The design is good but I do not like Engine talking to libvirtd directly, thus comes the VDSM console streaming API below.

Work to do
  Provide console streaming API from Engine to be invoked in oVirt shell.
  Implement the "serial-console" command in oVirt shell.

Support migration. Engine can reconnect to the guest automatically after migration while keeping the connection from oVirt shell. Fit well in the current oVirt architecture: no new server process introduced, no new VM introduced, easy to maintain and manage.
  Engine talking to libvirtd directly breaks the encapsulation of VDSM.
Users only can get the console stream from Engine, they can not directly connect to the host as VNC and the above two sshd solutions do.

5. VDSM console streaming API
Implement new APIs in VDSM to forward the raw data from console. It exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC binding. Then Engine can get the console data stream via API instead of directly connecting to libvirtd. Other things will be the same as solution 4.

Work to do
  Implement getConsoleReadStream() and getConsoleWriteStream() in VDSM.
  Provide console streaming API from Engine to be invoked in oVirt shell.
  Implement the "serial-console" command in oVirt shell.
Optional: Implement a client program in vdsClient to consume the stream API.

  Same as solution 4
We can not allow ordinary user directly connect to VDSM and invoke the stream API, because there is no ACL in VDSM, once a client cert is setup for the ordinary user, he can call all the APIs in VDSM and get total control. So the ordinary user can only get the stream from Engine, and we leave Engine to do the ACL.

I like solution 4 best.

Thanks and best regards!

Zhou Zheng Sheng / 周征晟
E-mail: zhshz...@linux.vnet.ibm.com
Telephone: 86-10-82454397

vdsm-devel mailing list

Reply via email to