Re: [Openstack] multi-nic blueprint - data migration
Ryu, yes, but it's not quite that simple since flat networks don't belong to projects. Also note that the part of multi-nic that sits above the virt layer is not yet in trunk. -tr3buchet On Tue, Mar 29, 2011 at 4:45 AM, Ishimoto, Ryu r...@midokura.jp wrote: ahhh yes, I misread the blueprint, it makes a lot of sense now. Thanks. Does this mean that when an instance launches, all the MAC addresses/Networks that belong in the same project get assigned to that instance? I couldn't find this in the code so I just wanted to verify. Ryu On Tue, Mar 29, 2011 at 1:13 AM, Trey Morris trey.mor...@rackspace.comwrote: Ryu, the new mac_address table is going to associate a mac_address with an instance and a network. When the VIFs are created for the instance, they are given the mac_address from the table and attached to the network from the table. Does that help? -trey On Mon, Mar 28, 2011 at 10:54 AM, Ishimoto, Ryu r...@midokura.jp wrote: Thanks, that cleared it up for me. Staying on the topic of multi-nics, I have been trying to understand the direction in which Nova is going in regards to networking, and reading up on the multi-nics blueprint, it certainly seems like it's going in the right direction, but I do have one question: What was the reasoning behind associating networks, and not NICs, to an instance? I understand that each instance NIC would belong to a distinct network, but it just seems more intuitive to imagine that an instance has NICs, and these NICs are connected to networks(or even more intuitively, connected to virtual ports that belong to various networks - Assigning a port to a NIC is like allocating an IP address). One problem I see from my suggestion above is that there is no association between NICs and networks, which means there is no way to select a network to grab an IP address from for each NIC at the time of VM launch. I might be missing something completely here, but why not just let the user manage all this through the management API before the VM launch? Let the user create NICs, Networks(and Ports with IP addresses), and map the NICs to ports. Then pass this list of 'connected' NICs as a parameter to launch a VM. This parameter is optional, and if omitted, it should be treated as launching a single NIC instance, with a new NIC created and associated with the instance on the fly. This prevents it from breaking the way it works now. If the concept of NICs for instance makes sense to everyone, I would love to help out and look further into what work needs to be done to extend the current multi-nic model into this one. I hope this made sense. I apologize for the length. Thanks, Ryu On Tue, Mar 29, 2011 at 12:47 AM, Trey Morris trey.mor...@rackspace.com wrote: I've written a migration to handle moving the data in the current instances table mac_address column into the new mac_address table before the column is removed. I agree with Jay, data should never be discarded when migrating forward. I don't think there has been a case yet where data is migrated in nova. This could be the first. -trey On Mon, Mar 28, 2011 at 8:17 AM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Mar 28, 2011 at 2:31 AM, Ishimoto, Ryu r...@midokura.jp wrote: Hi All, I was looking at the multi-nic blueprint(https://blueprints.launchpad.net/nova/+spec/nova-multi-nic), and in particular: 1) removing mac_address column from the instances table and creating a mac_addresses table. This is for storing which instances own which mac addresses as well as which network each mac is for. What happens(or should happen) to the MAC addresses that are already associated with instances? Will they be migrated to the new mac_addresses table? Or will they be discarded completely? Data should never be discarded in situations like this where a column is moved to another table's schema (or to be records in another table). I was curious to know how Nova usually handles data migration issues like this. No idea whether/if Nova's data migrations have previously needed to preserve data in this way. Glance does, however, and you can use the following Python changescript to get an idea how to perform this exact type of change: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_add_disk_format.py Note that SQLite has issues if you try to add and drop columns in the same changescript and you also have an unrelated column that is indexed (see: http://code.google.com/p/sqlalchemy-migrate/issues/detail?id=117). If this is the case, you will need to write SQL-based changescripts specfically for SQLite. You can see examples of how this is accomplished in Glance for the same DB version here: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_sqlite_upgrade.sql
Re: [Openstack] multi-nic blueprint - data migration
ahhh yes, I misread the blueprint, it makes a lot of sense now. Thanks. Does this mean that when an instance launches, all the MAC addresses/Networks that belong in the same project get assigned to that instance? I couldn't find this in the code so I just wanted to verify. Ryu On Tue, Mar 29, 2011 at 1:13 AM, Trey Morris trey.mor...@rackspace.comwrote: Ryu, the new mac_address table is going to associate a mac_address with an instance and a network. When the VIFs are created for the instance, they are given the mac_address from the table and attached to the network from the table. Does that help? -trey On Mon, Mar 28, 2011 at 10:54 AM, Ishimoto, Ryu r...@midokura.jp wrote: Thanks, that cleared it up for me. Staying on the topic of multi-nics, I have been trying to understand the direction in which Nova is going in regards to networking, and reading up on the multi-nics blueprint, it certainly seems like it's going in the right direction, but I do have one question: What was the reasoning behind associating networks, and not NICs, to an instance? I understand that each instance NIC would belong to a distinct network, but it just seems more intuitive to imagine that an instance has NICs, and these NICs are connected to networks(or even more intuitively, connected to virtual ports that belong to various networks - Assigning a port to a NIC is like allocating an IP address). One problem I see from my suggestion above is that there is no association between NICs and networks, which means there is no way to select a network to grab an IP address from for each NIC at the time of VM launch. I might be missing something completely here, but why not just let the user manage all this through the management API before the VM launch? Let the user create NICs, Networks(and Ports with IP addresses), and map the NICs to ports. Then pass this list of 'connected' NICs as a parameter to launch a VM. This parameter is optional, and if omitted, it should be treated as launching a single NIC instance, with a new NIC created and associated with the instance on the fly. This prevents it from breaking the way it works now. If the concept of NICs for instance makes sense to everyone, I would love to help out and look further into what work needs to be done to extend the current multi-nic model into this one. I hope this made sense. I apologize for the length. Thanks, Ryu On Tue, Mar 29, 2011 at 12:47 AM, Trey Morris trey.mor...@rackspace.comwrote: I've written a migration to handle moving the data in the current instances table mac_address column into the new mac_address table before the column is removed. I agree with Jay, data should never be discarded when migrating forward. I don't think there has been a case yet where data is migrated in nova. This could be the first. -trey On Mon, Mar 28, 2011 at 8:17 AM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Mar 28, 2011 at 2:31 AM, Ishimoto, Ryu r...@midokura.jp wrote: Hi All, I was looking at the multi-nic blueprint(https://blueprints.launchpad.net/nova/+spec/nova-multi-nic), and in particular: 1) removing mac_address column from the instances table and creating a mac_addresses table. This is for storing which instances own which mac addresses as well as which network each mac is for. What happens(or should happen) to the MAC addresses that are already associated with instances? Will they be migrated to the new mac_addresses table? Or will they be discarded completely? Data should never be discarded in situations like this where a column is moved to another table's schema (or to be records in another table). I was curious to know how Nova usually handles data migration issues like this. No idea whether/if Nova's data migrations have previously needed to preserve data in this way. Glance does, however, and you can use the following Python changescript to get an idea how to perform this exact type of change: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_add_disk_format.py Note that SQLite has issues if you try to add and drop columns in the same changescript and you also have an unrelated column that is indexed (see: http://code.google.com/p/sqlalchemy-migrate/issues/detail?id=117). If this is the case, you will need to write SQL-based changescripts specfically for SQLite. You can see examples of how this is accomplished in Glance for the same DB version here: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_sqlite_upgrade.sql http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_sqlite_downgrade.sql Cheers, jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net
Re: [Openstack] multi-nic blueprint - data migration
Also, curious, who assigns mac_addresses? or how the table is built initially? Ram On Tue, Mar 29, 2011 at 2:45 AM, Ishimoto, Ryu r...@midokura.jp wrote: ahhh yes, I misread the blueprint, it makes a lot of sense now. Thanks. Does this mean that when an instance launches, all the MAC addresses/Networks that belong in the same project get assigned to that instance? I couldn't find this in the code so I just wanted to verify. Ryu On Tue, Mar 29, 2011 at 1:13 AM, Trey Morris trey.mor...@rackspace.com wrote: Ryu, the new mac_address table is going to associate a mac_address with an instance and a network. When the VIFs are created for the instance, they are given the mac_address from the table and attached to the network from the table. Does that help? -trey On Mon, Mar 28, 2011 at 10:54 AM, Ishimoto, Ryu r...@midokura.jp wrote: Thanks, that cleared it up for me. Staying on the topic of multi-nics, I have been trying to understand the direction in which Nova is going in regards to networking, and reading up on the multi-nics blueprint, it certainly seems like it's going in the right direction, but I do have one question: What was the reasoning behind associating networks, and not NICs, to an instance? I understand that each instance NIC would belong to a distinct network, but it just seems more intuitive to imagine that an instance has NICs, and these NICs are connected to networks(or even more intuitively, connected to virtual ports that belong to various networks - Assigning a port to a NIC is like allocating an IP address). One problem I see from my suggestion above is that there is no association between NICs and networks, which means there is no way to select a network to grab an IP address from for each NIC at the time of VM launch. I might be missing something completely here, but why not just let the user manage all this through the management API before the VM launch? Let the user create NICs, Networks(and Ports with IP addresses), and map the NICs to ports. Then pass this list of 'connected' NICs as a parameter to launch a VM. This parameter is optional, and if omitted, it should be treated as launching a single NIC instance, with a new NIC created and associated with the instance on the fly. This prevents it from breaking the way it works now. If the concept of NICs for instance makes sense to everyone, I would love to help out and look further into what work needs to be done to extend the current multi-nic model into this one. I hope this made sense. I apologize for the length. Thanks, Ryu On Tue, Mar 29, 2011 at 12:47 AM, Trey Morris trey.mor...@rackspace.com wrote: I've written a migration to handle moving the data in the current instances table mac_address column into the new mac_address table before the column is removed. I agree with Jay, data should never be discarded when migrating forward. I don't think there has been a case yet where data is migrated in nova. This could be the first. -trey On Mon, Mar 28, 2011 at 8:17 AM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Mar 28, 2011 at 2:31 AM, Ishimoto, Ryu r...@midokura.jp wrote: Hi All, I was looking at the multi-nic blueprint(https://blueprints.launchpad.net/nova/+spec/nova-multi-nic), and in particular: 1) removing mac_address column from the instances table and creating a mac_addresses table. This is for storing which instances own which mac addresses as well as which network each mac is for. What happens(or should happen) to the MAC addresses that are already associated with instances? Will they be migrated to the new mac_addresses table? Or will they be discarded completely? Data should never be discarded in situations like this where a column is moved to another table's schema (or to be records in another table). I was curious to know how Nova usually handles data migration issues like this. No idea whether/if Nova's data migrations have previously needed to preserve data in this way. Glance does, however, and you can use the following Python changescript to get an idea how to perform this exact type of change: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_add_disk_format.py Note that SQLite has issues if you try to add and drop columns in the same changescript and you also have an unrelated column that is indexed (see: http://code.google.com/p/sqlalchemy-migrate/issues/detail?id=117). If this is the case, you will need to write SQL-based changescripts specfically for SQLite. You can see examples of how this is accomplished in Glance for the same DB version here: http://bazaar.launchpad.net/~glance-core/glance/cactus-trunk/view/head:/glance/registry/db/migrate_repo/versions/003_sqlite_upgrade.sql
[Openstack] multi-nic blueprint - data migration
Hi All, I was looking at the multi-nic blueprint( https://blueprints.launchpad.net/nova/+spec/nova-multi-nic), and in particular: 1) removing mac_address column from the instances table and creating a mac_addresses table. This is for storing which instances own which mac addresses as well as which network each mac is for. What happens(or should happen) to the MAC addresses that are already associated with instances? Will they be migrated to the new mac_addresses table? Or will they be discarded completely? I was curious to know how Nova usually handles data migration issues like this. Thanks, Ryu ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
