On Thu, 2014-07-24 at 11:06 +0100, Daniel P. Berrange wrote: > On Wed, Jul 23, 2014 at 10:32:44PM -0700, Nicholas A. Bellinger wrote: > > *) vhost-scsi doesn't support migration > > > > Since it's initial merge in QEMU v1.5, vhost-scsi has a migration blocker > > set. This is primarily due to requiring some external orchestration in > > order to setup the necessary vhost-scsi endpoints on the migration > > destination to match what's running on the migration source. > > > > Here are a couple of points that Stefan detailed some time ago about what's > > involved for properly supporting live migration with vhost-scsi: > > > > (1) vhost-scsi needs to tell QEMU when it dirties memory pages, either by > > DMAing to guest memory buffers or by modifying the virtio vring (which also > > lives in guest memory). This should be straightforward since the > > infrastructure is already present in vhost (it's called the "log") and used > > by drivers/vhost/net.c. > > > > (2) The harder part is seamless target handover to the destination host. > > vhost-scsi needs to serialize any SCSI target state from the source machine > > and load it on the destination machine. We could be in the middle of > > emulating a SCSI command. > > > > An obvious solution is to only support active-passive or active-active HA > > setups where tcm already knows how to fail over. This typically requires > > shared storage and maybe some communication for the clustering mechanism. > > There are more sophisticated approaches, so this straightforward one is just > > an example. > > > > That said, we do intended to support live migration for vhost-scsi using > > iSCSI/iSER/FC shared storage. > > > > *) vhost-scsi doesn't support qcow2 > > > > Given all other cinder drivers do not use QEMU qcow2 to access storage > > blocks, with the exception of the Netapp and Gluster driver, this argument > > is not particularly relevant here. > > > > However, this doesn't mean that vhost-scsi (and target-core itself) cannot > > support qcow2 images. There is currently an effort to add a userspace > > backend driver for the upstream target (tcm_core_user [3]), that will allow > > for supporting various disk formats in userspace. > > > > The important part for vhost-scsi is that regardless of what type of target > > backend driver is put behind the fabric LUNs (raw block devices using > > IBLOCK, qcow2 images using target_core_user, etc) the changes required in > > Nova and libvirt to support vhost-scsi remain the same. They do not change > > based on the backend driver. > > > > *) vhost-scsi is not intended for production > > > > vhost-scsi has been included the upstream kernel since the v3.6 release, and > > included in QEMU since v1.5. vhost-scsi runs unmodified out of the box on a > > number of popular distributions including Fedora, Ubuntu, and OpenSuse. It > > also works as a QEMU boot device with Seabios, and even with the Windows > > virtio-scsi mini-port driver. > > > > There is at least one vendor who has already posted libvirt patches to > > support vhost-scsi, so vhost-scsi is already being pushed beyond a debugging > > and development tool. > > > > For instance, here are a few specific use cases where vhost-scsi is > > currently the only option for virtio-scsi guests: > > > > - Low (sub 100 usec) latencies for AIO reads/writes with small iodepth > > workloads > > - 1M+ small block IOPs workloads at low CPU utilization with large > > iopdeth workloads. > > - End-to-end data integrity using T10 protection information (DIF) > > IIUC, there is also missing support for block jobs like drive-mirror > which is needed by Nova. >
This limitation can be considered an acceptable trade-off in initial support by some users, given the already considerable performance + efficiency gains that vhost logic provides to KVM/virtio guests. Others would like to utilize virtio-scsi to access features like SPC-3 Persistent Reservations, ALUA Explicit/Implicit Multipath, and EXTENDED_COPY offload provided by the LIO target subsystem. Note these three SCSI features are enabled (by default) on all LIO target fabric LUNs, starting with v3.12 kernel code. > From a functionality POV migration & drive-mirror support are the two > core roadblocks to including vhost-scsi in Nova (as well as libvirt > support for it of course). Realistically it doesn't sound like these > are likely to be solved soon enough to give us confidence in taking > this for the Juno release cycle. > The spec is for initial support of vhost-scsi controller endpoints during Juno, and as mentioned earlier by Vish, should be considered a experimental feature given caveats you've highlighted above. We also understand that code will ultimately have to pass Nova + libvirt upstream review in order to be merged, and that the approval of any vhost-scsi spec now is not a guarantee the feature will actually make it into any official Juno release. Thanks, --nab _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
