Re: [Openstack] Basic networking/configuration woes
2012/2/24 Justin Santa Barbara jus...@fathomdb.com: I have contributed a patch (which has merged) which should allow you to stop editing the SQL: https://review.openstack.org/#change,3816 With that, you should be able to pass the full range, with an additional argument specifying the subset that nova controls: e.g.-fixed_cidr=10.200.0.0/16 When I boot my VM, I think it gets a real address from my DHCP server (because the VM can reach the DHCP server), but not the address nova assigned it! I believe the nova iptables rules mean that the machine can't then do TCP/IP, but even if I am wrong/could overcome that, I don't think cloud-init could then configure the correct address. The instance firewall should be configured to only allow DHCP responses from the IP it believes to be the correct DHCP server. Perhaps it has the wrong idea? -- Soren Hansen | http://linux2go.dk/ Senior Software Engineer | http://www.cisco.com/ Ubuntu Developer | http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
Justin Santa Barbara wrote Yes, I do have a non-nova DHCP server. However, even if I didn't, and even if iptables allowed talking to 169.254 with the magic link-local, cloud-init still couldn't configure the IP address... :-( If Linux assigned a 169.254 address to an Ethernet port then iptables should be set up for 169.254. But you can't use the link local network for unicast messages unless you have a link local address. I don't believe Linux will assign an 169.254 address by default if another IP address has been assigned manually or by DHCP. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
The instance firewall should be configured to only allow DHCP responses from the IP it believes to be the correct DHCP server. Perhaps it has the wrong idea? Ah yes - it probably does because of my unusual network subrange config. It seems I'm not missing something, so I've proposed a init script for contrib that can apply the configuration from a config_drive. It solved my problem, and it avoids all of the DHCP/cloud-init complexity without needing direct injection. Still very much a version 1 though: https://review.openstack.org/#change,4498 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Basic networking/configuration woes
I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be. I'm hoping someone can point me to the right way, or we can figure out what needs to change. My wishlist: * I want my instances to be on my network e.g. 10.0.0.0/16 * As Nova can't pull IPs from my DHCP server, I'm willing to allocate it a sub-range, e.g. 10.200.0.0/16 First decision: Choosing a networking mode: * I don't want / need VLANs * If I use FlatDHCPManager, I can't do the subrange stuff - it seems that this mode assumes it controls the entire address range. * So it's FlatManager. It works, but now I don't have DHCP, so I just have to inject info into the instance. Next decision: How to inject info (at least the IP address): * Supposedly the 'right way' is to use cloud-init. It looks like I'd still need DHCP before I can reach 169.254..., and I don't have that. It looks like cloud-init can't do network configuration even if nova passed the information in. And I'd be locked into cloud-init images - no Windows, no Debian etc. * The next best way is config_drive. It looks like I'd have to bundle my own image. Maybe I could use cloud-init, maybe with an OVF formatted config_drive, but even then I couldn't configure networking (?) * So now I'm back to file injection. That just works. So now I'm using FlatManager and file injection; and yet I feel this is the dodgy back alley of OpenStack, and I should be in the well-lit nice area. I worry that things like file injection and FlatManager are less favored and may be deprecated in future. But every time I try to do things right I just waste a lot of time and make no progress. Yet I feel I didn't really have a choice here. How are other people making this work? What is the right way? Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
I'd assume FlatDHCPManager works much like FlatManager, but maybe I'm wrong. I use FlatManager and I always end up having to modify the fixed_ips table manually after running nova-manage because I think I'm trying to do something similar as you. I have a /23... and I want to give nova a /25 out of it. Though I'm giving nova a /25, it's still really a /23. I use nova-manage to add my /23 and then I edit the fixed_ips table and mark a lot of addresses as 'reserved'... or just remove them altogether. (When I try to specify the /25 to nova-manage, it doesn't go so well) As far as 169.254... you can reach that without any address assigned. Your NIC should receive a link local address when there's no other IP assigned which is in the 169.254.* range. Not sure if that helped much :) - Chris On Feb 23, 2012, at 3:12 PM, Justin Santa Barbara wrote: I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be. I'm hoping someone can point me to the right way, or we can figure out what needs to change. My wishlist: * I want my instances to be on my network e.g. 10.0.0.0/16 * As Nova can't pull IPs from my DHCP server, I'm willing to allocate it a sub-range, e.g. 10.200.0.0/16 First decision: Choosing a networking mode: * I don't want / need VLANs * If I use FlatDHCPManager, I can't do the subrange stuff - it seems that this mode assumes it controls the entire address range. * So it's FlatManager. It works, but now I don't have DHCP, so I just have to inject info into the instance. Next decision: How to inject info (at least the IP address): * Supposedly the 'right way' is to use cloud-init. It looks like I'd still need DHCP before I can reach 169.254..., and I don't have that. It looks like cloud-init can't do network configuration even if nova passed the information in. And I'd be locked into cloud-init images - no Windows, no Debian etc. * The next best way is config_drive. It looks like I'd have to bundle my own image. Maybe I could use cloud-init, maybe with an OVF formatted config_drive, but even then I couldn't configure networking (?) * So now I'm back to file injection. That just works. So now I'm using FlatManager and file injection; and yet I feel this is the dodgy back alley of OpenStack, and I should be in the well-lit nice area. I worry that things like file injection and FlatManager are less favored and may be deprecated in future. But every time I try to do things right I just waste a lot of time and make no progress. Yet I feel I didn't really have a choice here. How are other people making this work? What is the right way? Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
Thanks for chipping in. I have contributed a patch (which has merged) which should allow you to stop editing the SQL: https://review.openstack.org/#change,3816 With that, you should be able to pass the full range, with an additional argument specifying the subset that nova controls: e.g.-fixed_cidr=10.200.0.0/16 When I boot my VM, I think it gets a real address from my DHCP server (because the VM can reach the DHCP server), but not the address nova assigned it! I believe the nova iptables rules mean that the machine can't then do TCP/IP, but even if I am wrong/could overcome that, I don't think cloud-init could then configure the correct address. Justin On Thu, Feb 23, 2012 at 3:38 PM, Chris Behrens cbehr...@codestud.com wrote: I'd assume FlatDHCPManager works much like FlatManager, but maybe I'm wrong. I use FlatManager and I always end up having to modify the fixed_ips table manually after running nova-manage because I think I'm trying to do something similar as you. I have a /23... and I want to give nova a /25 out of it. Though I'm giving nova a /25, it's still really a /23. I use nova-manage to add my /23 and then I edit the fixed_ips table and mark a lot of addresses as 'reserved'... or just remove them altogether. (When I try to specify the /25 to nova-manage, it doesn't go so well) As far as 169.254... you can reach that without any address assigned. Your NIC should receive a link local address when there's no other IP assigned which is in the 169.254.* range. Not sure if that helped much :) - Chris On Feb 23, 2012, at 3:12 PM, Justin Santa Barbara wrote: I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be. I'm hoping someone can point me to the right way, or we can figure out what needs to change. My wishlist: * I want my instances to be on my network e.g. 10.0.0.0/16 * As Nova can't pull IPs from my DHCP server, I'm willing to allocate it a sub-range, e.g. 10.200.0.0/16 First decision: Choosing a networking mode: * I don't want / need VLANs * If I use FlatDHCPManager, I can't do the subrange stuff - it seems that this mode assumes it controls the entire address range. * So it's FlatManager. It works, but now I don't have DHCP, so I just have to inject info into the instance. Next decision: How to inject info (at least the IP address): * Supposedly the 'right way' is to use cloud-init. It looks like I'd still need DHCP before I can reach 169.254..., and I don't have that. It looks like cloud-init can't do network configuration even if nova passed the information in. And I'd be locked into cloud-init images - no Windows, no Debian etc. * The next best way is config_drive. It looks like I'd have to bundle my own image. Maybe I could use cloud-init, maybe with an OVF formatted config_drive, but even then I couldn't configure networking (?) * So now I'm back to file injection. That just works. So now I'm using FlatManager and file injection; and yet I feel this is the dodgy back alley of OpenStack, and I should be in the well-lit nice area. I worry that things like file injection and FlatManager are less favored and may be deprecated in future. But every time I try to do things right I just waste a lot of time and make no progress. Yet I feel I didn't really have a choice here. How are other people making this work? What is the right way? Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
On Feb 23, 2012, at 3:55 PM, Justin Santa Barbara wrote: Thanks for chipping in. I have contributed a patch (which has merged) which should allow you to stop editing the SQL: https://review.openstack.org/#change,3816 With that, you should be able to pass the full range, with an additional argument specifying the subset that nova controls: e.g.-fixed_cidr=10.200.0.0/16 Oh cool.. that'll save me some pain. :) When I boot my VM, I think it gets a real address from my DHCP server (because the VM can reach the DHCP server), but not the address nova assigned it! I believe the nova iptables rules mean that the machine can't then do TCP/IP, but even if I am wrong/could overcome that, I don't think cloud-init could then configure the correct address. If you're going to go the cloud-init route... you wouldn't need DHCP, right? There should be iptables rules to allow you to talk to the metadata service over 169.254.* (And linux should give you a default link-local address that allows you to talk to the MD service magically) Do you have a non-nova DHCP server running as well? - Chris Justin On Thu, Feb 23, 2012 at 3:38 PM, Chris Behrens cbehr...@codestud.com wrote: I'd assume FlatDHCPManager works much like FlatManager, but maybe I'm wrong. I use FlatManager and I always end up having to modify the fixed_ips table manually after running nova-manage because I think I'm trying to do something similar as you. I have a /23... and I want to give nova a /25 out of it. Though I'm giving nova a /25, it's still really a /23. I use nova-manage to add my /23 and then I edit the fixed_ips table and mark a lot of addresses as 'reserved'... or just remove them altogether. (When I try to specify the /25 to nova-manage, it doesn't go so well) As far as 169.254... you can reach that without any address assigned. Your NIC should receive a link local address when there's no other IP assigned which is in the 169.254.* range. Not sure if that helped much :) - Chris On Feb 23, 2012, at 3:12 PM, Justin Santa Barbara wrote: I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be. I'm hoping someone can point me to the right way, or we can figure out what needs to change. My wishlist: * I want my instances to be on my network e.g. 10.0.0.0/16 * As Nova can't pull IPs from my DHCP server, I'm willing to allocate it a sub-range, e.g. 10.200.0.0/16 First decision: Choosing a networking mode: * I don't want / need VLANs * If I use FlatDHCPManager, I can't do the subrange stuff - it seems that this mode assumes it controls the entire address range. * So it's FlatManager. It works, but now I don't have DHCP, so I just have to inject info into the instance. Next decision: How to inject info (at least the IP address): * Supposedly the 'right way' is to use cloud-init. It looks like I'd still need DHCP before I can reach 169.254..., and I don't have that. It looks like cloud-init can't do network configuration even if nova passed the information in. And I'd be locked into cloud-init images - no Windows, no Debian etc. * The next best way is config_drive. It looks like I'd have to bundle my own image. Maybe I could use cloud-init, maybe with an OVF formatted config_drive, but even then I couldn't configure networking (?) * So now I'm back to file injection. That just works. So now I'm using FlatManager and file injection; and yet I feel this is the dodgy back alley of OpenStack, and I should be in the well-lit nice area. I worry that things like file injection and FlatManager are less favored and may be deprecated in future. But every time I try to do things right I just waste a lot of time and make no progress. Yet I feel I didn't really have a choice here. How are other people making this work? What is the right way? Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
If you're going to go the cloud-init route... you wouldn't need DHCP, right? There should be iptables rules to allow you to talk to the metadata service over 169.254.* (And linux should give you a default link-local address that allows you to talk to the MD service magically) Do you have a non-nova DHCP server running as well? Yes, I do have a non-nova DHCP server. However, even if I didn't, and even if iptables allowed talking to 169.254 with the magic link-local, cloud-init still couldn't configure the IP address... :-( ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Basic networking/configuration woes
On Feb 23, 2012, at 5:42 PM, Justin Santa Barbara wrote: If you're going to go the cloud-init route... you wouldn't need DHCP, right? There should be iptables rules to allow you to talk to the metadata service over 169.254.* (And linux should give you a default link-local address that allows you to talk to the MD service magically) Do you have a non-nova DHCP server running as well? Yes, I do have a non-nova DHCP server. However, even if I didn't, and even if iptables allowed talking to 169.254 with the magic link-local, cloud-init still couldn't configure the IP address... :-( It is definitely possible to make this work with flatdhcp in multihost mode. Can you configure your home router to route for the whole range but not give out leases for macs it doesn't know? Then you can use --dnsmasq_config_file=/path/to/config in that config file you can use: dhcp_opiton=3,ip of router to force vms to use your router as their gateway. You may have to setup a forward on the router to get metadata to work, but I think this gets you 90% of the way there. (it may work to just run nova-api on each compute and leave metadata_host on the host_ip, but i'm not sure if it will go out to the gateway by default and skip the iptables rule on localhost) I know this basic setup was working as far back as diablo because it is the setup that freecloud is using. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
