Re: [Openstack] PREROUTING 169.254.169.254 rule shoud not on Compute node.......
Hi, this is an issue in our Multi-Node arquitecture of the Stackops Openstack Nova Distro. We have not verified 100% yet if it fix the issue, but I think Hugo has found the reason and the solution, so my sincere kudos, Hugo. We will release a maintenance release once we verify that the rules works fine in our lab. Regards -- Diego Parrilla CEO www.stackops.com | diego.parri...@stackops.com | +34 649 94 43 29 | skype:diegoparrilla ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law. On Wed, May 11, 2011 at 4:08 AM, 郭耀謙 tonyt...@gmail.com wrote: Hello , guys There's a problem while separate instance's network and nova-management network. EX. Nova management network : 192.168.1.0/24 eth0 Instance network : 10.0.0.0/12 eth1 bridge to br100 During cloud-setup : Instance try to retrieve metadata from 169.254.169.254. Instances(10.0.0.0/12) request 169.254.169.254 PREROUTING from gateway(nova-network). But If PREROUTING rule is already been set on nova-Compute node, instance request will be redirected on VM host instead of nova-network host. So If your topology is like A diadram from StackOps , Plz Check iptables rule on Compute nodes. -A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.2:8773 And del this rule , your instance will get metadata correctly ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PREROUTING 169.254.169.254 rule shoud not on Compute node.......
We're using quagga for route exchange (via ospf) between the network servers, and the api server. The api server peers with each network server. This is about as simple a quagga setup as you can have. Basically, when nova-network starts up a new project network, quagga begins to advertise that network into ospf. The api server sees that advertisement and can use that to reach the node (for metadata requests). Incidentally, you might want to lock down your POSTROUTING rule to ensure that vms can't hit your management network. IIRC, we were able to use a similar rule, except with a specific host and port as opposed to a blanket rule for the whole network. -nld 2011/5/10 郭耀謙 tonyt...@gmail.com: #iptables -t nat -A nova-network-POSTROUTING -s 10.0.0.0/12 -d 192.168.1.0/24 -j ACCEPT That's what I did on nova-network host . btw , I always isolate nova-network. I'm interesting about quagga in your environment. 2011/5/11 Narayan Desai narayan.de...@gmail.com For what it's worth, we're running in a configuration similar to the one in the attached diagram using VlanManager. When we moved the nova-network service off of the machine with nova-api, we needed to add an additional prerouting rule on the network server that prevented the traffic from being sent out via NAT (which caused the source address to be lost, resulting in a metadata resolution error). Once the packets arrive at the api service with the correct source address, they need a route back, via the nova-network server in order to get the response packets onto the correct vlan. With a single nova-network server, a static route will do. With multiple nova-network instances on different systems, things get a little more complicated. We ended up setting up route distribution via quagga between the nova-api server, and the nova-network servers. This ensures that nova-api knows which nova-network instance to use to reach any particular project network. -nld On Tue, May 10, 2011 at 9:08 PM, 郭耀謙 tonyt...@gmail.com wrote: Hello , guys There's a problem while separate instance's network and nova-management network. EX. Nova management network : 192.168.1.0/24 eth0 Instance network : 10.0.0.0/12 eth1 bridge to br100 During cloud-setup : Instance try to retrieve metadata from 169.254.169.254. Instances(10.0.0.0/12) request 169.254.169.254 PREROUTING from gateway(nova-network). But If PREROUTING rule is already been set on nova-Compute node, instance request will be redirected on VM host instead of nova-network host. So If your topology is like A diadram from StackOps , Plz Check iptables rule on Compute nodes. -A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.2:8773 And del this rule , your instance will get metadata correctly ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PREROUTING 169.254.169.254 rule shoud not on Compute node.......
For what it's worth, we're running in a configuration similar to the one in the attached diagram using VlanManager. When we moved the nova-network service off of the machine with nova-api, we needed to add an additional prerouting rule on the network server that prevented the traffic from being sent out via NAT (which caused the source address to be lost, resulting in a metadata resolution error). Once the packets arrive at the api service with the correct source address, they need a route back, via the nova-network server in order to get the response packets onto the correct vlan. With a single nova-network server, a static route will do. With multiple nova-network instances on different systems, things get a little more complicated. We ended up setting up route distribution via quagga between the nova-api server, and the nova-network servers. This ensures that nova-api knows which nova-network instance to use to reach any particular project network. -nld On Tue, May 10, 2011 at 9:08 PM, 郭耀謙 tonyt...@gmail.com wrote: Hello , guys There's a problem while separate instance's network and nova-management network. EX. Nova management network : 192.168.1.0/24 eth0 Instance network : 10.0.0.0/12 eth1 bridge to br100 During cloud-setup : Instance try to retrieve metadata from 169.254.169.254. Instances(10.0.0.0/12) request 169.254.169.254 PREROUTING from gateway(nova-network). But If PREROUTING rule is already been set on nova-Compute node, instance request will be redirected on VM host instead of nova-network host. So If your topology is like A diadram from StackOps , Plz Check iptables rule on Compute nodes. -A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.2:8773 And del this rule , your instance will get metadata correctly ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PREROUTING 169.254.169.254 rule shoud not on Compute node.......
*#iptables -t nat -A nova-network-POSTROUTING -s 10.0.0.0/12 -d 192.168.1.0/24 -j ACCEPT* * * ***That's what I did on nova-network host .* * * *btw , I always isolate nova-network.* *I'm interesting about **quagga in your environment.* 2011/5/11 Narayan Desai narayan.de...@gmail.com For what it's worth, we're running in a configuration similar to the one in the attached diagram using VlanManager. When we moved the nova-network service off of the machine with nova-api, we needed to add an additional prerouting rule on the network server that prevented the traffic from being sent out via NAT (which caused the source address to be lost, resulting in a metadata resolution error). Once the packets arrive at the api service with the correct source address, they need a route back, via the nova-network server in order to get the response packets onto the correct vlan. With a single nova-network server, a static route will do. With multiple nova-network instances on different systems, things get a little more complicated. We ended up setting up route distribution via quagga between the nova-api server, and the nova-network servers. This ensures that nova-api knows which nova-network instance to use to reach any particular project network. -nld On Tue, May 10, 2011 at 9:08 PM, 郭耀謙 tonyt...@gmail.com wrote: Hello , guys There's a problem while separate instance's network and nova-management network. EX. Nova management network : 192.168.1.0/24 eth0 Instance network : 10.0.0.0/12 eth1 bridge to br100 During cloud-setup : Instance try to retrieve metadata from 169.254.169.254. Instances(10.0.0.0/12) request 169.254.169.254 PREROUTING from gateway(nova-network). But If PREROUTING rule is already been set on nova-Compute node, instance request will be redirected on VM host instead of nova-network host. So If your topology is like A diadram from StackOps , Plz Check iptables rule on Compute nodes. -A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.2:8773 And del this rule , your instance will get metadata correctly ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
