Public bug reported: Problem ----------
Debugging common networking/neutron problems (1. cannot ping VM, 2. cannot ping FIP), tends to be manual, and requires root access to look into the state of the agents or the datapath on different hosts. Neutron needs to provide a "diagnostics" extension api which can be used for debugging networking problems. Each agent/driver exposes its own state in a structured (json) format via the diagnostics extension. The structured content can be parsed by automation to pin-point problems or at least help get to the next step of the debugging much faster than done manually. In addition, there should be diagnostics/operational support to ping a neutron port from the dhcp or l3 agents. Nova "diagnostics" serves as an example here. https://wiki.openstack.org/wiki/Nova_VM_Diagnostics Fix ---- A "diagnostics" extension is added to neutron Each agent and corresponding drivers supports a get_diagnostics() API invoked from neutron-server upon the following GET APIs (limited by policy to admin-only). The outputs are structured so, they can be processed by other tools. GET: /agent/:id/diagnostics example output from neutron-ovs agent: OVS bridges, ports and flows GET: /agent/:id/diagnostics/network/:id example output from dhcp-agent (dnsmasq driver): contents of host,lease files GET: /agent/:id/diagnostics/port/:id example output from dhcp-agent: dhcp transactions for that port (from dnsmasq logs) example output from ovs-agent: stats on qvo,qbr,tap interfaces GET: /agent/:id/diagnostics/port/:id/security-groups example output from l2-agent (iptables-firewalldriver): iptables rules programmed (ingress/egress/spoofing) for that port GET: /agent/:id/diagnostics/port/:id/ping This is an "operational" command - ping the port from the agent (dhcp/l3) network/router namespace Neutron Command-line Client supports the following new commands ---------------------------------------------------------------- neutron l2-diagnostics --network-id <> --port-id <> agent neutron dhcp-diagnostics --network-id <> --port-id <> --ping agent neutron l3-diagnostics --network-id <> --port-id <> --ping agent Sample Diagnostics Extension Code ------------------------------------------ See Attached Code Diff ** Affects: neutron Importance: Undecided Status: New ** Tags: rfe ** Patch added: "sample code diff for diagnostics extension" https://bugs.launchpad.net/bugs/1519537/+attachment/4524935/+files/diagnostics-extension-code-diff.txt ** Description changed: - Problem ---------- - Debugging common networking/neutron problems (1. cannot ping VM, 2. cannot ping FIP), + Debugging common networking/neutron problems (1. cannot ping VM, 2. cannot ping FIP), tends to be manual, and requires root-shell access to look into the state of the agents or the datapath on different hosts. Neutron needs to provide a "diagnostics" extension api which can be used for debugging networking problems. Each agent/driver exposes its own state in a structured (json) format via the diagnostics extension. The structured - content can be parsed by automation to pin-point problems or at least help get to - the next step of the debugging much faster than done manually. + content can be parsed by automation to pin-point problems or at least help get to the next step of the debugging much faster than done manually. In addition, there should be diagnostics/operational support to ping a neutron port from the dhcp or l3 agents. Nova "diagnostics" serves as an example here. https://wiki.openstack.org/wiki/Nova_VM_Diagnostics Fix ---- A "diagnostics" extension is added to neutron Each agent and corresponding drivers supports a get_diagnostics() API invoked from neutron-server upon - the following GET APIs (limited by policy to admin-only). The outputs are structured so, they can be - processed by other tools. + the following GET APIs (limited by policy to admin-only). The outputs are structured so, they can be processed by other tools. GET: /agent/:id/diagnostics - example output from neutron-ovs agent: OVS bridges, ports and flows + example output from neutron-ovs agent: OVS bridges, ports and flows GET: /agent/:id/diagnostics/network/:id - example output from dhcp-agent (dnsmasq driver): contents of host,lease files + example output from dhcp-agent (dnsmasq driver): contents of host,lease files GET: /agent/:id/diagnostics/port/:id - example output from dhcp-agent: dhcp transactions for that port (from dnsmasq logs) - + example output from dhcp-agent: dhcp transactions for that port (from dnsmasq logs) + example output from ovs-agent: stats on qvo,qbr,tap interfaces + GET: /agent/:id/diagnostics/port/:id/security-groups - example output from l2-agent (iptables-firewalldriver): iptables rules programmed (ingress/egress/spoofing) for that port + example output from l2-agent (iptables-firewalldriver): iptables rules programmed (ingress/egress/spoofing) for that port GET: /agent/:id/diagnostics/port/:id/ping - This is an "operational" command - ping the port from the agent (dhcp/l3) network/router namespace + This is an "operational" command - ping the port from the agent (dhcp/l3) network/router namespace Sample Diagnostics Extension Code ------------------------------------------ See Attached Code Diff ** Description changed: Problem ---------- Debugging common networking/neutron problems (1. cannot ping VM, 2. cannot ping FIP), tends to be manual, and requires root-shell access to look into the state of the agents or the datapath on different hosts. Neutron needs to provide a "diagnostics" extension api which can be used for debugging networking problems. Each agent/driver exposes its own state in a structured (json) format via the diagnostics extension. The structured content can be parsed by automation to pin-point problems or at least help get to the next step of the debugging much faster than done manually. In addition, there should be diagnostics/operational support to ping a neutron port from the dhcp or l3 agents. Nova "diagnostics" serves as an example here. https://wiki.openstack.org/wiki/Nova_VM_Diagnostics Fix ---- A "diagnostics" extension is added to neutron Each agent and corresponding drivers supports a get_diagnostics() API invoked from neutron-server upon the following GET APIs (limited by policy to admin-only). The outputs are structured so, they can be processed by other tools. GET: /agent/:id/diagnostics example output from neutron-ovs agent: OVS bridges, ports and flows GET: /agent/:id/diagnostics/network/:id example output from dhcp-agent (dnsmasq driver): contents of host,lease files GET: /agent/:id/diagnostics/port/:id example output from dhcp-agent: dhcp transactions for that port (from dnsmasq logs) - example output from ovs-agent: stats on qvo,qbr,tap interfaces + example output from ovs-agent: stats on qvo,qbr,tap interfaces GET: /agent/:id/diagnostics/port/:id/security-groups example output from l2-agent (iptables-firewalldriver): iptables rules programmed (ingress/egress/spoofing) for that port GET: /agent/:id/diagnostics/port/:id/ping This is an "operational" command - ping the port from the agent (dhcp/l3) network/router namespace + + Neutron Command-line Client supports the following new commands + ---------------------------------------------------------------- + + neutron l2-diagnostics --network-id <> --port-id <> agent + + neutron dhcp-diagnostics --network-id <> --port-id <> --ping agent + + neutron l3-diagnostics --network-id <> --port-id <> --ping agent + + Sample Diagnostics Extension Code ------------------------------------------ See Attached Code Diff -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1519537 Title: [RFE] - Diagnostics Extension for Neutron Status in neutron: New Bug description: Problem ---------- Debugging common networking/neutron problems (1. cannot ping VM, 2. cannot ping FIP), tends to be manual, and requires root access to look into the state of the agents or the datapath on different hosts. Neutron needs to provide a "diagnostics" extension api which can be used for debugging networking problems. Each agent/driver exposes its own state in a structured (json) format via the diagnostics extension. The structured content can be parsed by automation to pin-point problems or at least help get to the next step of the debugging much faster than done manually. In addition, there should be diagnostics/operational support to ping a neutron port from the dhcp or l3 agents. Nova "diagnostics" serves as an example here. https://wiki.openstack.org/wiki/Nova_VM_Diagnostics Fix ---- A "diagnostics" extension is added to neutron Each agent and corresponding drivers supports a get_diagnostics() API invoked from neutron-server upon the following GET APIs (limited by policy to admin-only). The outputs are structured so, they can be processed by other tools. GET: /agent/:id/diagnostics example output from neutron-ovs agent: OVS bridges, ports and flows GET: /agent/:id/diagnostics/network/:id example output from dhcp-agent (dnsmasq driver): contents of host,lease files GET: /agent/:id/diagnostics/port/:id example output from dhcp-agent: dhcp transactions for that port (from dnsmasq logs) example output from ovs-agent: stats on qvo,qbr,tap interfaces GET: /agent/:id/diagnostics/port/:id/security-groups example output from l2-agent (iptables-firewalldriver): iptables rules programmed (ingress/egress/spoofing) for that port GET: /agent/:id/diagnostics/port/:id/ping This is an "operational" command - ping the port from the agent (dhcp/l3) network/router namespace Neutron Command-line Client supports the following new commands ---------------------------------------------------------------- neutron l2-diagnostics --network-id <> --port-id <> agent neutron dhcp-diagnostics --network-id <> --port-id <> --ping agent neutron l3-diagnostics --network-id <> --port-id <> --ping agent Sample Diagnostics Extension Code ------------------------------------------ See Attached Code Diff To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1519537/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
