On Fri, Jun 24, 2016 at 2:19 AM, Daniel P. Berrange <[email protected]> wrote:
> On Thu, Jun 23, 2016 at 09:09:44AM -0700, Walter A. Boring IV wrote: > > > > volumes connected to QEMU instances eventually become directly connected? > > > > > Our long term goal is that 100% of all network storage will be > connected > Oh, didn't know this at all. Is this something Nova has been working on for a while? I'd love to hear more about the reasoning, the plan etc. It would also be really neat to have an opportunity to participate. > > > to directly by QEMU. We already have the ability to partially do this > with > > > iSCSI, but it is lacking support for multipath. As & when that gap is > > > addressed though, we'll stop using the host OS for any iSCSI stuff. > Any chance anybody has any insight on how to make this work? I tried configuring this last week and it appears to be broken in a few places. > > > > > So if you're requiring access to host iSCSI volumes, it'll work in the > > > short-medium term, but in the medium-long term we're not going to use > > > that so plan accordingly. > > > > What is the benefit of this largely monolithic approach? It seems that > > moving everything into QEMU is diametrically opposed to the unix model > > itself and > > is just a re-implementation of what already exists in the linux world > > outside of QEMU. > > There are many benefits to having it inside QEMU. First it gives us > improved isolation between VMs, because we can control the network > I/O directly against the VM using cgroup resource controls. It gives > us improved security, particularly in combination with LUKS encryption > since the unencrypted block device is not directly visible / accessible > to any other process. It gives us improved reliability / managability > since we avoid having to spawn the iscsi client tools which have poor > error reporting and have been frequent sources of instability in our > True, the iscsi tools aren't the greatest. > infrastructure (eg see how we have to blindly re-run the same command > many times over because it randomly times out). It will give us improved > I/O performance because of a shorter I/O path to get requests from QEMU > out to the network. > I'd love to hear more on the design and how it all comes together. Particularly the performance info. Like I said, I tried to set it up against master but seems I'm either missing something in the config or it's broken. > > NB, this is not just about iSCSI, the same is all true for RBD where > we've also stopped using in-kernel RBD client and do it all in QEMU. > > > Does QEMU support hardware initiators? iSER? > No, this is only for case where you're doing pure software based > iSCSI client connections. If we're relying on local hardware that's > a different story. > I'm confused, so what's the iser driver referenced in the patch commit message: https://review.openstack.org/#/c/135854/ So there's a different story for that? > > > > > We regularly fix issues with iSCSI attaches in the release cycles of > > OpenStack, > > because it's all done in python using existing linux packages. How often > > This is a great example of the benefit that in-QEMU client gives us. The > Linux iSCSI client tools have proved very unreliable in use by OpenStack. > This is a reflection of the very architectural approach. We have individual > resources needed by distinct VMs, but we're having to manage them as a host > wide resource and that's creating us unneccessary complexity and having a > poor effect on our reliability overall. > > > are QEMU > > releases done and upgraded on customer deployments vs. python packages > > (os-brick)? > > We're removing the entire layer of instability by removing the need to > deal with any command line tools, and thus greatly simplifying our > setup on compute nodes. No matter what we might do in os-brick it'll > never give us a simple or reliable system - we're just papering over > the flaws by doing stuff like blindly re-trying iscsi commands upon > failure. > This all sounds like it could be a good direction to go in. I'd love to see more info on the plan, how it works, and how to test it out a bit. Didn't find a spec, any links, reviews or config info available? Wish I would've caught this on ML or IRC or wherever, would've loved to have participated a bit. > Regards, > Daniel > -- > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ > :| > |: http://libvirt.org -o- http://virt-manager.org > :| > |: http://autobuild.org -o- http://search.cpan.org/~danberr/ > :| > |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc > :| > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
