Re: [openstack-dev] [Nova] Interface in PEER2PEER live migration
On Thu, 2016-08-25 at 11:41 -0400, Daniel P. Berrange wrote: > > I think where the confusion is coming is that libvirt will work in > two different ways with P2P migration. If the TUNNELLED flag is set, > then the migration data will go over the Libvirtd <-> libvirtd > connection, which is influenced by the live_migration_inbound_addr > parameter. Right. > If the TUNNELED flag is not set the data is QEMU <-> QEMU directly, > and that needs the extra URI set. Exactly. > What we need todo is fix the Nova code so that when the TUNNELLED > flag > is *not* set, we also provide the extra URI, using the hostname/ip > addr > listed in live_migration_inbound_addr, falling back to the compute > hostname > if live_migration_inbound_addr is not set. This will work, but I have the feeling that this will give to live_migration_inbound_addr two meanings, that depends on the kind of live migration configured. I will adapt the patch to follow this path! -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Interface in PEER2PEER live migration
On Thu, Aug 25, 2016 at 12:01:40PM +0200, Alberto Planas Dominguez wrote: > On Wed, 2016-08-24 at 11:18 -0400, Daniel P. Berrange wrote: > > On Wed, Aug 24, 2016 at 05:07:50PM +0200, Alberto Planas Dominguez > > wrote: > > Daniel, thanks for the fast reply!! > > > > Unfortunately was closed as invalid, and the solution provided is > > > completely unrelated. The solution suggested is based on > > > `live_migration_inbound_addr`, that is related with the libvirtd > > > URI, > > > not the qmeu one. I tested several times and yes, this solution is > > > not > > > related with the problem. > > > > The "live_migration_inbound_addr" configuration parameters was > > intended > > to affect both libvirt & QEMU traffic. If that is not working > > correctly, > > then we should be fixing that, nto adding yet another parameter. > > The code in libvirt is very clear: if uri_in is NULL will ask to the > hostname to the other side. I checked the code in 1.2.18: > > https://github.com/libvirt/libvirt/blob/v1.2.18-maint/src/qemu/qemu_mig > ration.c#L3601 > > https://github.com/libvirt/libvirt/blob/v1.2.18-maint/src/qemu/qemu_mig > ration.c#L3615 > > The same logic is in master: > > https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_migration. > c#L4013 > > But we can go back to 0.9.12: > > https://github.com/libvirt/libvirt/blob/v0.9.12-maint/src/qemu/qemu_mig > ration.c#L1472 > > Nova set migration_uri parameter to None, that this means that uri_in > is NULL. > > How can I affect the the QEMU part? The code path AAIU is: if we do not > set miguri (migrateToURI2) or migrate_uri (migrateToURI3), is a > uri_in=NULL. > > I am not familiar with libvirt code, please, help me to find how I can > affect this uri_in parameter to have a value different from the > hostname of the other node, without setting the correct value in > migrateToURI[23] in the Nova side. I think where the confusion is coming is that libvirt will work in two different ways with P2P migration. If the TUNNELLED flag is set, then the migration data will go over the Libvirtd <-> libvirtd connection, which is influenced by the live_migration_inbound_addr parameter. If the TUNNELED flag is not set the data is QEMU <-> QEMU directly, and that needs the extra URI set. What we need todo is fix the Nova code so that when the TUNNELLED flag is *not* set, we also provide the extra URI, using the hostname/ip addr listed in live_migration_inbound_addr, falling back to the compute hostname if live_migration_inbound_addr is not set. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Interface in PEER2PEER live migration
On Wed, 2016-08-24 at 11:18 -0400, Daniel P. Berrange wrote: > On Wed, Aug 24, 2016 at 05:07:50PM +0200, Alberto Planas Dominguez > wrote: Daniel, thanks for the fast reply!! > > Unfortunately was closed as invalid, and the solution provided is > > completely unrelated. The solution suggested is based on > > `live_migration_inbound_addr`, that is related with the libvirtd > > URI, > > not the qmeu one. I tested several times and yes, this solution is > > not > > related with the problem. > > The "live_migration_inbound_addr" configuration parameters was > intended > to affect both libvirt & QEMU traffic. If that is not working > correctly, > then we should be fixing that, nto adding yet another parameter. The code in libvirt is very clear: if uri_in is NULL will ask to the hostname to the other side. I checked the code in 1.2.18: https://github.com/libvirt/libvirt/blob/v1.2.18-maint/src/qemu/qemu_mig ration.c#L3601 https://github.com/libvirt/libvirt/blob/v1.2.18-maint/src/qemu/qemu_mig ration.c#L3615 The same logic is in master: https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_migration. c#L4013 But we can go back to 0.9.12: https://github.com/libvirt/libvirt/blob/v0.9.12-maint/src/qemu/qemu_mig ration.c#L1472 Nova set migration_uri parameter to None, that this means that uri_in is NULL. How can I affect the the QEMU part? The code path AAIU is: if we do not set miguri (migrateToURI2) or migrate_uri (migrateToURI3), is a uri_in=NULL. I am not familiar with libvirt code, please, help me to find how I can affect this uri_in parameter to have a value different from the hostname of the other node, without setting the correct value in migrateToURI[23] in the Nova side. > > this patch will create a second uri: > > > > migrate_uri=tcp://fast.%s/ > > While you can do that hack, the fact that is works is simply luck - > it > certainly was not designed with this kind of usage in mind. We would > in fact like to remove the live_migration_uri config parameter > entirely > and having the libvirt driver automatically use the correct URI. This is a very interesting suggestion! I can see that in /etc/libvirt.qemu.conf there is a parameter for 'migration_address', that is the -incoming side of the migration. What I am not able to see is how I can change the hostname (to select the correct interface) from the source of the migration. I see the default value in `migration_host`, but my tests didn't work to set the uri_in properly -- SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Interface in PEER2PEER live migration
On Wed, Aug 24, 2016 at 05:07:50PM +0200, Alberto Planas Dominguez wrote: > Unfortunately was closed as invalid, and the solution provided is > completely unrelated. The solution suggested is based on > `live_migration_inbound_addr`, that is related with the libvirtd URI, > not the qmeu one. I tested several times and yes, this solution is not > related with the problem. The "live_migration_inbound_addr" configuration parameters was intended to affect both libvirt & QEMU traffic. If that is not working correctly, then we should be fixing that, nto adding yet another parameter. > > I worked in a patch for mater here: > > https://review.openstack.org/#/c/356558/ > > This patch worked as expected. This create a second URI, based on the > hostname given in live_migration_uri parameter, to build a second one > that will be used by qemu/kvm for the second connection. > > So, for example if: > > live_migration_uri=qemu+tcp://fast.%s/system > > this patch will create a second uri: > > migrate_uri=tcp://fast.%s/ While you can do that hack, the fact that is works is simply luck - it certainly was not designed with this kind of usage in mind. We would in fact like to remove the live_migration_uri config parameter entirely and having the libvirt driver automatically use the correct URI. As such adding further URI config parmaeters is not a direction we want to go in. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
