Re: [Openstack] metadata api with Quantum and provider networks
Hi, Thanks for you patience Dan answering all the questions, I have solved all the issues and got the thing working. I could possibly add some content to the Quantum docs about my experience, who should I contant in this regard? Regards, --janis ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] metadata api with Quantum and provider networks
On Wed, Oct 10, 2012 at 8:57 AM, Jānis Ģeņģeris janis.genge...@gmail.com wrote: Hi, Thanks for you patience Dan answering all the questions, I have solved all the issues and got the thing working. I could possibly add some content to the Quantum docs about my experience, who should I contant in this regard? woohoo, I love it when people volunteer to improve the docs :) Anne and Diane are the primary docs team leaders for quantum docs. If you have only small changes, its probably easy to just create a bug in https://bugs.launchpad.net/openstack-manuals, tag it with 'quantum', and then write new text you would suggest, where it should be inserted, and then someone from the docs team can implement the suggested change. If you want to make more complex changes, you could use git to checkout the doc source, and submit changes for review yourself. See: http://wiki.openstack.org/Documentation/HowTo Dan Regards, --janis -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] metadata api with Quantum and provider networks
Hi,
I have managed to create the Quantum router so that all the VMs created on
provider network are going through it. I had to simply use
router-interface-add.
But I have now the problem with accessing metadata service. As L3 agent
adds DNAT rule in router, the packets are rewritten and arrives at metadata
host, but when coming back it doesn't go through the router but directly to
the source host resulting in TCP handshake failure. Could it be because all
the ports in br-int which are somehow related to this provider network are
withing the same VLAN(have the same tag in ovs-vsctl output) thus being in
the same broadcast domain?
So the above could be solved with SNAT rule to router address, but then
metadata service sees routers address instead of VMs and returns HTTP 404
error.
Is it possible to fix this with additional flow in openvswitch somehow? The
vswitch man pages say that it is possible to match against IPs.
What other solution is here to fix this?
Thanks,
--janis
On Mon, Oct 8, 2012 at 11:04 PM, Jānis Ģeņģeris janis.genge...@gmail.comwrote:
Here is the output, with few details.
# quantum router-show router_vlan1501
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| external_gateway_info | |
| id| 3c7383c7-7759-4db6-ba5b-19e754280cb8 |
| name | router_vlan1501 |
| status| ACTIVE |
| tenant_id | 7246a7e9d61f42b8a644bc1551a2a396 |
+---+--+
# quantum net-show vlan1501
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| id| c2161824-a439-40e5-8809-5599f80df2fe |
| name | vlan1501 |
| provider:network_type | vlan |
| provider:physical_network | default |
| provider:segmentation_id | 1501 |
| router:external | False|
| shared| False|
| status| ACTIVE |
| subnets | af6eac1e-dfec-49a0-bfc2-3fbf9a7063b3 |
| tenant_id | 7246a7e9d61f42b8a644bc1551a2a396 |
+---+--+
# quantum router-gateway-set 3c7383c7-7759-4db6-ba5b-19e754280cb8
c2161824-a439-40e5-8809-5599f80df2fe
Bad router request: Network c2161824-a439-40e5-8809-5599f80df2fe is not a
valid external network
I assume this happens because 'router:external' if False, when switching
it to True the above command succeeds. But then if I want to switch back
later (with no ports attached, no routers, even no subnets), I get this:
quantum net-update vlan1501 --router:external False
External network e011d68b-6abd-43a4-b6c3-9dbadf5344ee cannot be updated to
be made non-external, since it has existing gateway ports
Anyway, I have set 'router:external' to True for 'vlan1501' network, set
the gateway for my newly created quantum router(router_vlan1501) with
'router-gateway-set' as a result quantum have created router ns with single
interface from my configured provider network 'vlan1501' with default
gateway set to the one of the 'vlan1501' nets attached subnet.
But now if I boot the VM with '--nic=vlan1501_id' option, I get VM with
default gateway set to networks('vlan1501') gateway(not the freshly created
router gateway, am I missing something in configs?), which is the same as
the created routers gateway. Another thing if I switch to newly created
'router_vlan1501' namespace I can't actually ping the external gateway,
that is there as a default gw for the 'vlan1501' net.
So my thinking is I need to change the default gw in VM to the virtual
router?
On Mon, Oct 8, 2012 at 9:32 PM, Dan Wendlandt d...@nicira.com wrote:
On Mon, Oct 8, 2012 at 12:27 PM, Jānis Ģeņģeris
janis.genge...@gmail.com wrote:
On Mon, Oct 8, 2012 at 6:24 PM, Dan Wendlandt d...@nicira.com wrote:
On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris
janis.genge...@gmail.com
wrote:
Hello,
When using provider networks in Quantum, where should the metadata
service
rule mapping (e.g. 169.254.169.254:80 -
metadata_server:metadata_port)
must
be set?
For example, for floating IPs l3-agent handles this, but for provider
networks router is not used. I tried to
Re: [Openstack] metadata api with Quantum and provider networks
On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris janis.genge...@gmail.com wrote: Hello, When using provider networks in Quantum, where should the metadata service rule mapping (e.g. 169.254.169.254:80 - metadata_server:metadata_port) must be set? For example, for floating IPs l3-agent handles this, but for provider networks router is not used. I tried to set custom iptables rule for this, but have a hard time understanding where to set it, as there is openvswitch and namespaces. I'm using provider network configuration with VLANs. You actually could use the Quantum L3 router as your gateway even if VMs are on a provider network, but I suspect your question is actually more along the lines of: if I want my gateway to be a physical router not managed by Quantum, how do I does the DNAT rule for metadata get applied? In this case, you need to apply the DNAT rule manually to the physical router, which I believe is the same as if you were using flat networking with Nova with a physical router. There may also be a more complex solution achievable via quantum in which the provider creates a quantum router with an interface on the provider network, VMs are each given a host route to route traffic destined for 169.254.169.254/32 to this quantum router IP, rather than the physical default gateway, and this quantum router performs the DNAT. However, its probably much easier to just apply this rule to your physical router. Dan Regards, --janis ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] metadata api with Quantum and provider networks
On Mon, Oct 8, 2012 at 6:24 PM, Dan Wendlandt d...@nicira.com wrote: On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris janis.genge...@gmail.com wrote: Hello, When using provider networks in Quantum, where should the metadata service rule mapping (e.g. 169.254.169.254:80 - metadata_server:metadata_port) must be set? For example, for floating IPs l3-agent handles this, but for provider networks router is not used. I tried to set custom iptables rule for this, but have a hard time understanding where to set it, as there is openvswitch and namespaces. I'm using provider network configuration with VLANs. You actually could use the Quantum L3 router as your gateway even if VMs are on a provider network, but I suspect your question is actually more along the lines of: if I want my gateway to be a physical router not managed by Quantum, how do I does the DNAT rule for metadata get applied? In this case, you need to apply the DNAT rule manually to the physical router, which I believe is the same as if you were using flat networking with Nova with a physical router. Adding the rule in physical router is not a good idea, because then the configuration of the OpenStack crosses the actual software/server border into network equipment, than can add to complexity later. I tried to add provider network to quantum router, and the quantum CLI was rejecting it. AFAIK router-interface-add is for internal networks, and router-gateway-set is also failing. Which CLI command to use for adding provider network to existing quantum router? There may also be a more complex solution achievable via quantum in which the provider creates a quantum router with an interface on the provider network, VMs are each given a host route to route traffic destined for 169.254.169.254/32 to this quantum router IP, rather than the physical default gateway, and this quantum router performs the DNAT. However, its probably much easier to just apply this rule to your physical router. No, this is no good. Dan Regards, --janis ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ Regards, --janis ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] metadata api with Quantum and provider networks
Here is the output, with few details.
# quantum router-show router_vlan1501
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| external_gateway_info | |
| id| 3c7383c7-7759-4db6-ba5b-19e754280cb8 |
| name | router_vlan1501 |
| status| ACTIVE |
| tenant_id | 7246a7e9d61f42b8a644bc1551a2a396 |
+---+--+
# quantum net-show vlan1501
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| id| c2161824-a439-40e5-8809-5599f80df2fe |
| name | vlan1501 |
| provider:network_type | vlan |
| provider:physical_network | default |
| provider:segmentation_id | 1501 |
| router:external | False|
| shared| False|
| status| ACTIVE |
| subnets | af6eac1e-dfec-49a0-bfc2-3fbf9a7063b3 |
| tenant_id | 7246a7e9d61f42b8a644bc1551a2a396 |
+---+--+
# quantum router-gateway-set 3c7383c7-7759-4db6-ba5b-19e754280cb8
c2161824-a439-40e5-8809-5599f80df2fe
Bad router request: Network c2161824-a439-40e5-8809-5599f80df2fe is not a
valid external network
I assume this happens because 'router:external' if False, when switching it
to True the above command succeeds. But then if I want to switch back later
(with no ports attached, no routers, even no subnets), I get this:
quantum net-update vlan1501 --router:external False
External network e011d68b-6abd-43a4-b6c3-9dbadf5344ee cannot be updated to
be made non-external, since it has existing gateway ports
Anyway, I have set 'router:external' to True for 'vlan1501' network, set
the gateway for my newly created quantum router(router_vlan1501) with
'router-gateway-set' as a result quantum have created router ns with single
interface from my configured provider network 'vlan1501' with default
gateway set to the one of the 'vlan1501' nets attached subnet.
But now if I boot the VM with '--nic=vlan1501_id' option, I get VM with
default gateway set to networks('vlan1501') gateway(not the freshly created
router gateway, am I missing something in configs?), which is the same as
the created routers gateway. Another thing if I switch to newly created
'router_vlan1501' namespace I can't actually ping the external gateway,
that is there as a default gw for the 'vlan1501' net.
So my thinking is I need to change the default gw in VM to the virtual
router?
On Mon, Oct 8, 2012 at 9:32 PM, Dan Wendlandt d...@nicira.com wrote:
On Mon, Oct 8, 2012 at 12:27 PM, Jānis Ģeņģeris
janis.genge...@gmail.com wrote:
On Mon, Oct 8, 2012 at 6:24 PM, Dan Wendlandt d...@nicira.com wrote:
On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris
janis.genge...@gmail.com
wrote:
Hello,
When using provider networks in Quantum, where should the metadata
service
rule mapping (e.g. 169.254.169.254:80 -
metadata_server:metadata_port)
must
be set?
For example, for floating IPs l3-agent handles this, but for provider
networks router is not used. I tried to set custom iptables rule for
this,
but have a hard time understanding where to set it, as there is
openvswitch
and namespaces.
I'm using provider network configuration with VLANs.
You actually could use the Quantum L3 router as your gateway even if
VMs are on a provider network, but I suspect your question is actually
more along the lines of: if I want my gateway to be a physical router
not managed by Quantum, how do I does the DNAT rule for metadata get
applied? In this case, you need to apply the DNAT rule manually to
the physical router, which I believe is the same as if you were using
flat networking with Nova with a physical router.
Adding the rule in physical router is not a good idea, because then the
configuration of the OpenStack crosses the actual software/server border
into network equipment, than can add to complexity later.
Yes, its hard to have it both ways... if you want everything done
automatically via software, I'd suggest using the quantum router as
the gateway, not an external physical router.
I tried to add provider network to quantum
