The best solution would of course be 3 (Or something similar that keeps the 
terminal state inside the VM memory so that migration works).
Tunelling screen can do that but it requires having screen (or something 
similar) installed on the guest which is hard to do.

But I think the more practical solution is 2 as it has semantics similar to VNC.
Running a real ssh (ie. 1) is problematic because we have less control over the 
daemon and there are more vectors the user can try and use to break out of the 
Further more, setting up sandboxes is a bit problematic ATM.

I don't really understand 5. What does those methods return the virtio dev path?

----- Original Message -----
> From: "Zhou Zheng Sheng" <>
> To: "VDSM Project Development" <>
> Sent: Tuesday, November 27, 2012 4:22:20 AM
> Subject: Re: [vdsm] [RFC]about the implement of text-based console
> Hi all,
> For now in there is no agreement on the remote guest console
> solution,
> so I decide to do some investigation continue the discussion.
> Our goal
>    VM serial console remote access in CLI mode. That means the client
> runs without X environment.
Do you mean like running qemu with -curses?
> There are several proposals.
> 1. Sandboxed sshd
>    VDSM runs a new host sshd instance in virtual machine/sandbox and
> redirects the virtio console to it.
> 2. Third-party sshd
>    VDSM runs third-party sshd library/implementation and redirects
> virtio console to it.
> 3. Spice
>    Extend spice to support console and implement a client to be run
> without GUI environment
> 4. oVirt shell -> Engine -> libvirt
>    The user connects to Engine via oVirt CLI, then issues a
> "serial-console" command, then Engine locates the host and connect to
> the guest console. Currently there is a workaround, it invokes "virsh
> -c
> qemu+tls://host/qemu console vmid" from Engine side.
> 5. VDSM console streaming API
>    VDSM exposes getConsoleReadStream() and getConsoleWriteStream()
>    via
> XMLRPC binding. Then implement the related client in vdsClient and
> Engine
> Detailed discussion
> 1. Sandboxes
> Solution 1 and 2 allow users connect to console using their favorite
> ssh
> client. The login name is vmid, the password is set by setVmTicket()
> call of VDSM. The connection will be lost during migration. This is
> similar to VNC in oVirt.
> I take a look at several sandbox technologies, including
> libvirt-sandbox, lxc and selinux.
> a) libvirt-sandbox boots a VM using host kernel and initramfs, then
> passthru the host file system to the VM in read only mode. We can
> also
> add extra binding to the guest file system. It's very easy to use. To
> run shell in a VM, one can just issues
> virt-sandbox -c qemu:///session  /bin/sh
> Then the VM will be ready in several seconds.
> However it will trigger some selinux violations. Currently there is
> no
> official support for selinux policy configuration from this project.
> In
> the project page this is put in the todo list.
> b) lxc utilize Linux container to run a process in sandbox. It needs
> to
> be configured properly. I find in the package lxc-templates there is
> an
> example configuration file for running sshd in lxc.
> c) sandbox command in the package policycoreutils-python makes use of
> selinux to run a process in sandbox, but there is no official or
> example
> policy files for sshd.
> In a word, for sandbox technologies, we have to configure the
> policies/file system binding/network carefully and test the
> compatibility with popular sshd implementations (openssh-server).
> When
> those sshd upgrade, the policy must be upgraded by us at the same
> time.
> Since the policies are not maintained by who implements sshd, this is
> a
> burden for us.
> Work to do
>    Write and maintain the policies.
>    Find ways for auth callback and redirecting data to
>    openssh-server.
> pros
>    Re-use existing pieces and technologies (host sshd, sandbox).
>    User friendly, they can use existing ssh clients.
> cons
>    Connection is lost in migration, this is not a big problem because
>    1)
> VNC connection share the same problem, 2) the user can reconnect
> manually.
>    It's not easy to maintain the sandbox policies/file system
> binding/network for compatibility with sshd.
> 2. Third-party sshd implementations
> Almost the same as solution 1 but with better flexibility. VDSM can
> import a third-party sshd library and let that library deal with auth
> and transport. VDSM just have to implement the data forwarding. Many
> people consider this is insecure but I think the ticket solution for
> is even not as secure as this. Currently most of us only trust
> openssh-server and think the quality of third-party sshd is low. I
> searched for a while and found twisted.conch from the popular twisted
> project. I'm not familiar with twisted.conch, but I still put it in
> this
> mail to collect opinions from potential twisted.conch experts.
> In a word, I prefer sandbox technologies to third-party sshd
> implementations unless there is a implementation as good as
> openssh-server.
> Work to do
>    Integrate twisted.conch into VDSM
> pros
>    Very flexible. If library provide auth callback to VDSM, then VDSM
> can just compares the login password to the VM ticket without knowing
> SSH detials.
> cons
>    Third party implementations are not as secure and carefully
> maintained as sshd in the host (probably openssh-server).
> 3. Extend Spice to support console
> Is it possible to implement a spice client can be run in pure text
> mode
> without GUI environment? If we extend the protocol to support console
> stream but the client must be run in GUI, it will be less useful.
> pros
>    No new VMs and server process, easy for maintenance.
> cons
>    Must wait for Spice developers to commit the support.
>    Need special client program in CLI, the user may prefer existing
> client program like ssh. It not a big problem because this feature
> can
> be put in to oVirt shell.
> 4. oVirt shell -> Engine -> libvirtd
> This is the current workaround described in
> The design is good but I do not like Engine talking to libvirtd
> directly, thus comes the VDSM console streaming API below.
> Work to do
>    Provide console streaming API from Engine to be invoked in oVirt
>    shell.
>    Implement the "serial-console" command in oVirt shell.
> pros
>    Support migration. Engine can reconnect to the guest automatically
> after migration while keeping the connection from oVirt shell.
>    Fit well in the current oVirt architecture: no new server process
> introduced, no new VM introduced, easy to maintain and manage.
> cons
>    Engine talking to libvirtd directly breaks the encapsulation of
>    VDSM.
>    Users only can get the console stream from Engine, they can not
> directly connect to the host as VNC and the above two sshd solutions
> do.
> 5. VDSM console streaming API
> Implement new APIs in VDSM to forward the raw data from console. It
> exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC
> binding. Then Engine can get the console data stream via API instead
> of
> directly connecting to libvirtd. Other things will be the same as
> solution 4.
> Work to do
>    Implement getConsoleReadStream() and getConsoleWriteStream() in
>    VDSM.
>    Provide console streaming API from Engine to be invoked in oVirt
>    shell.
>    Implement the "serial-console" command in oVirt shell.
>    Optional: Implement a client program in vdsClient to consume the
> stream API.
> pros
>    Same as solution 4
> cons
>    We can not allow ordinary user directly connect to VDSM and invoke
> the stream API, because there is no ACL in VDSM, once a client cert
> is
> setup for the ordinary user, he can call all the APIs in VDSM and get
> total control. So the ordinary user can only get the stream from
> Engine,
> and we leave Engine to do the ACL.
> I like solution 4 best.
> --
> Thanks and best regards!
> Zhou Zheng Sheng / 周征晟
> E-mail:
> Telephone: 86-10-82454397
> _______________________________________________
> vdsm-devel mailing list
vdsm-devel mailing list

Reply via email to