Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
On 8/4/14, 5:39 PM, mar...@redhat.com mandr...@redhat.com wrote: On 03/08/14 13:07, Gary Kotton wrote: Hi, Happy you asked about this. This is an idea that we have: Below is a suggestion on how we can improve the metadata service. This can be done by leveraging the a Load balancers supports X-Forwarded-For.The following link has two diagrams. The first is the existing support (may be a little rusty here, so please feel free to correct) and the second is the proposal. https://urldefense.proofpoint.com/v1/url?u=https://docs.google.com/drawin gs/d/19JCirhj2NVVFZ0Vbnsxhyxrm1jjzEAS3ZAMzfBRk=oIvRg1%2BdGAgOoM1BIlLLqw% 3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=pDHtkey4 U%2FmCkIGoEa0vUaWK4o93GK5Ep2QhTvy2gAw%3D%0As=98f30ec826fdd475b3ecca4a22b a6d3664652d7281a1a95b88eba9de570cc678 C-0E/edit?usp=sharing Metadata proxy support: the proxy will receive the HTTP request. It will then perform a query to the Neutron service (1) to retrieve the tenant id and the instance id from the neutron service. A proxy request will be sent to Nova for the metadata details (2). Proposed support: 1. There will be a load balancer vip 254.169.254.169 (this can be reached either via the L3 agent of the DG on the DHCP. 2. The LB will have a server farm of all of the Nova API's (this makes the soon highly available) 1. Replace the destination IP and port with the Nova metadata IP and port 2. Replace the source IP with the interface IP 3. Insert the header X-Fowarded-For (this will have the original source IP of the VM) 1. When the Nova metadata service receives the request, according to a configuration variable (https://urldefense.proofpoint.com/v1/url?u=https://github.com/openstack/ nova/blob/master/nova/api/metadata/handler.pyk=oIvRg1%2BdGAgOoM1BIlLLqw% 3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=pDHtkey4 U%2FmCkIGoEa0vUaWK4o93GK5Ep2QhTvy2gAw%3D%0As=5f11211dd96938ae5a319c12eb6 e01f9fa585d76aa5ee6a36a96561644fbb625 #L134), will interface with the neutron service to get the instance_id and the tenant id. This will be done by using a new extension. With the details provided by Neutron Nova will provide the correct metadata for the instance 2. A new extension will be added to Neutron that will enable a port lookup. The port lookup will have two input values and will return the port which has the instance id and the tenant id. 1. LB source IP this is the LB source IP that interfaces with the Nova API. When we create the edge router for the virtual network we will have a mapping of the edge LB ip - network id. This will enable us to get the virtual network for the port 2. Fixed port IP this with the virtual network will enable us to get the specific port. Hopefully in the coming days a spec will be posted that will provide more details thanks for that info Gary, the diagram in particular forced me to go read a bit about the metadata agent (i was mostly just proxying for the original question). I obviously miss a lot of the details (will be easier once the spec is out) but it seems like you're proposing an addition (port-lookup) that will change the way the metadata agent is called; in fact does it make the neutron metadata proxy obsolete? I will keep a look out for the spec, At the moment there is already a port lookup. This is done by the metadata proxy. The proposed solution will have less hops and fewer elements that can fail. Hopefully we Can get the spec posted in the near future. Sadly this will not be approved for Juno. thanks, marios Thanks Gary On 8/1/14, 6:11 PM, mar...@redhat.com mandr...@redhat.com wrote: Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://urldefense.proofpoint.com/v1/url?u=https://launchpad.net/neutron /% 2Bmilestone/juno-3k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF 6h goMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=TZXQIMHmAX22lC0YOyItiXOrAA%2FegHqY5 cN I73%2B0jJ8%3D%0As=b81f4d5919b317628f56d0313143cee8fca6e47f639a59784eb19 da 3d88681da [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3 -h igh-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
On 02/08/14 02:22, Assaf Muller wrote: Hey Marios, comments inline. - Original Message - Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? With L3 HA turned on you can run the metadata agent on all network nodes. The active instance of each router will have the proxy up in its namespace and it will forward it to the agent as expected. perfect thanks Assaf as well as for the follow up clarifications on irc, I didn't realise this was the case, all the best, marios thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://launchpad.net/neutron/+milestone/juno-3 [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3-high-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
On 03/08/14 13:07, Gary Kotton wrote: Hi, Happy you asked about this. This is an idea that we have: Below is a suggestion on how we can improve the metadata service. This can be done by leveraging the a Load balancers supports X-Forwarded-For.The following link has two diagrams. The first is the existing support (may be a little rusty here, so please feel free to correct) and the second is the proposal. https://docs.google.com/drawings/d/19JCirhj2NVVFZ0Vbnsxhyxrm1jjzEAS3ZAMzfBR C-0E/edit?usp=sharing Metadata proxy support: the proxy will receive the HTTP request. It will then perform a query to the Neutron service (1) to retrieve the tenant id and the instance id from the neutron service. A proxy request will be sent to Nova for the metadata details (2). Proposed support: 1. There will be a load balancer vip 254.169.254.169 (this can be reached either via the L3 agent of the DG on the DHCP. 2. The LB will have a server farm of all of the Nova API's (this makes the soon highly available) 1. Replace the destination IP and port with the Nova metadata IP and port 2. Replace the source IP with the interface IP 3. Insert the header X-Fowarded-For (this will have the original source IP of the VM) 1. When the Nova metadata service receives the request, according to a configuration variable (https://github.com/openstack/nova/blob/master/nova/api/metadata/handler.py #L134), will interface with the neutron service to get the instance_id and the tenant id. This will be done by using a new extension. With the details provided by Neutron Nova will provide the correct metadata for the instance 2. A new extension will be added to Neutron that will enable a port lookup. The port lookup will have two input values and will return the port which has the instance id and the tenant id. 1. LB source IP this is the LB source IP that interfaces with the Nova API. When we create the edge router for the virtual network we will have a mapping of the edge LB ip - network id. This will enable us to get the virtual network for the port 2. Fixed port IP this with the virtual network will enable us to get the specific port. Hopefully in the coming days a spec will be posted that will provide more details thanks for that info Gary, the diagram in particular forced me to go read a bit about the metadata agent (i was mostly just proxying for the original question). I obviously miss a lot of the details (will be easier once the spec is out) but it seems like you're proposing an addition (port-lookup) that will change the way the metadata agent is called; in fact does it make the neutron metadata proxy obsolete? I will keep a look out for the spec, thanks, marios Thanks Gary On 8/1/14, 6:11 PM, mar...@redhat.com mandr...@redhat.com wrote: Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://urldefense.proofpoint.com/v1/url?u=https://launchpad.net/neutron/% 2Bmilestone/juno-3k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6h goMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=TZXQIMHmAX22lC0YOyItiXOrAA%2FegHqY5cN I73%2B0jJ8%3D%0As=b81f4d5919b317628f56d0313143cee8fca6e47f639a59784eb19da 3d88681da [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3-h igh-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
Hi, Happy you asked about this. This is an idea that we have: Below is a suggestion on how we can improve the metadata service. This can be done by leveraging the a Load balancers supports X-Forwarded-For.The following link has two diagrams. The first is the existing support (may be a little rusty here, so please feel free to correct) and the second is the proposal. https://docs.google.com/drawings/d/19JCirhj2NVVFZ0Vbnsxhyxrm1jjzEAS3ZAMzfBR C-0E/edit?usp=sharing Metadata proxy support: the proxy will receive the HTTP request. It will then perform a query to the Neutron service (1) to retrieve the tenant id and the instance id from the neutron service. A proxy request will be sent to Nova for the metadata details (2). Proposed support: 1. There will be a load balancer vip 254.169.254.169 (this can be reached either via the L3 agent of the DG on the DHCP. 2. The LB will have a server farm of all of the Nova API's (this makes the soon highly available) 1. Replace the destination IP and port with the Nova metadata IP and port 2. Replace the source IP with the interface IP 3. Insert the header X-Fowarded-For (this will have the original source IP of the VM) 1. When the Nova metadata service receives the request, according to a configuration variable (https://github.com/openstack/nova/blob/master/nova/api/metadata/handler.py #L134), will interface with the neutron service to get the instance_id and the tenant id. This will be done by using a new extension. With the details provided by Neutron Nova will provide the correct metadata for the instance 2. A new extension will be added to Neutron that will enable a port lookup. The port lookup will have two input values and will return the port which has the instance id and the tenant id. 1. LB source IP this is the LB source IP that interfaces with the Nova API. When we create the edge router for the virtual network we will have a mapping of the edge LB ip - network id. This will enable us to get the virtual network for the port 2. Fixed port IP this with the virtual network will enable us to get the specific port. Hopefully in the coming days a spec will be posted that will provide more details Thanks Gary On 8/1/14, 6:11 PM, mar...@redhat.com mandr...@redhat.com wrote: Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://urldefense.proofpoint.com/v1/url?u=https://launchpad.net/neutron/% 2Bmilestone/juno-3k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6h goMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=TZXQIMHmAX22lC0YOyItiXOrAA%2FegHqY5cN I73%2B0jJ8%3D%0As=b81f4d5919b317628f56d0313143cee8fca6e47f639a59784eb19da 3d88681da [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3-h igh-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
Hi, The link below is broken. Please see - https://docs.google.com/drawings/d/19JCirhj2NVVFZ0Vbnsxhyxrm1jjzEAS3ZAMzfBR C-0E/edit In short this will give a highly available service without the need for the metadata proxy. It will also have one less hop = better performance. Thanks Gary On 8/3/14, 1:07 PM, Gary Kotton gkot...@vmware.com wrote: Hi, Happy you asked about this. This is an idea that we have: Below is a suggestion on how we can improve the metadata service. This can be done by leveraging the a Load balancers supports X-Forwarded-For.The following link has two diagrams. The first is the existing support (may be a little rusty here, so please feel free to correct) and the second is the proposal. https://docs.google.com/drawings/d/19JCirhj2NVVFZ0Vbnsxhyxrm1jjzEAS3ZAMzfB R C-0E/edit?usp=sharing Metadata proxy support: the proxy will receive the HTTP request. It will then perform a query to the Neutron service (1) to retrieve the tenant id and the instance id from the neutron service. A proxy request will be sent to Nova for the metadata details (2). Proposed support: 1. There will be a load balancer vip 254.169.254.169 (this can be reached either via the L3 agent of the DG on the DHCP. 2. The LB will have a server farm of all of the Nova API's (this makes the soon highly available) 1. Replace the destination IP and port with the Nova metadata IP and port 2. Replace the source IP with the interface IP 3. Insert the header X-Fowarded-For (this will have the original source IP of the VM) 1. When the Nova metadata service receives the request, according to a configuration variable (https://github.com/openstack/nova/blob/master/nova/api/metadata/handler.p y #L134), will interface with the neutron service to get the instance_id and the tenant id. This will be done by using a new extension. With the details provided by Neutron Nova will provide the correct metadata for the instance 2. A new extension will be added to Neutron that will enable a port lookup. The port lookup will have two input values and will return the port which has the instance id and the tenant id. 1. LB source IP this is the LB source IP that interfaces with the Nova API. When we create the edge router for the virtual network we will have a mapping of the edge LB ip - network id. This will enable us to get the virtual network for the port 2. Fixed port IP this with the virtual network will enable us to get the specific port. Hopefully in the coming days a spec will be posted that will provide more details Thanks Gary On 8/1/14, 6:11 PM, mar...@redhat.com mandr...@redhat.com wrote: Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://urldefense.proofpoint.com/v1/url?u=https://launchpad.net/neutron/ % 2Bmilestone/juno-3k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6 h goMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=TZXQIMHmAX22lC0YOyItiXOrAA%2FegHqY5c N I73%2B0jJ8%3D%0As=b81f4d5919b317628f56d0313143cee8fca6e47f639a59784eb19d a 3d88681da [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3- h igh-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Status of A/A HA for neutron-metadata-agent?
Hey Marios, comments inline. - Original Message - Hi all, I have been asked by a colleague about the status of A/A HA for neutron-* processes. From the 'HA guide' [1], l3-agent and metadata-agent are the only neutron components that can't be deployed in A/A HA (corosync/pacemaker for a/p is documented as available 'out of the box' for both). The l3-agent work is approved for J3 [4] but I am unaware of any work on the metadata-agent and can't see any mention in [2][3]. Is this someone has looked at, or is planning to (though ultimately K would be the earliest right?)? With L3 HA turned on you can run the metadata agent on all network nodes. The active instance of each router will have the proxy up in its namespace and it will forward it to the agent as expected. thanks! marios [1] http://docs.openstack.org/high-availability-guide/content/index.html [2] https://wiki.openstack.org/wiki/NeutronJunoProjectPlan [3] https://launchpad.net/neutron/+milestone/juno-3 [4] http://git.openstack.org/cgit/openstack/neutron-specs/tree/specs/juno/l3-high-availability.rst ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
