Re: [openstack-dev] [neutron] Neutron debugging tool
Thanks for the great suggestions and pointers, everyone! easyOVS seems to cover many use cases I had in my mind. I'll give it a try and see if/how I can extend it. I do agree with Salvatore about putting reference to all these tools in OpenStack docs. I filed a docs bug [1] suggesting that we need to add (sub-)section to Chapter 12. Network Troubleshooting in OpenStack Operations Guide [2]. This way, tools will be quickly exposed to the docs surface and one does not need to spend hours to discover them from scattered places. Again, thanks for all input! Nodir [1] https://bugs.launchpad.net/openstack-manuals/+bug/1499114 [2] http://docs.openstack.org/openstack-ops/content/network_troubleshooting.html On 22 September 2015 at 05:27, Carol Bouchard (caboucha) wrote: > There was a presentation on DON at the Vancouver summit. Here is the link: > > > > https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/don-diagnosing-ovs-in-neutron > > > > From: Salvatore Orlando [mailto:salv.orla...@gmail.com] > Sent: Tuesday, September 22, 2015 3:51 AM > To: OpenStack Development Mailing List (not for usage questions) > > > Subject: Re: [openstack-dev] [neutron] Neutron debugging tool > > > > Thanks Ganesh! > > > > I did not know about this tool. > > I also quite like the network visualization bits, though I wonder how > practical that would be when one debugs very large deployments. > > > > I think it won't be a bad idea to list these tools in the networking guide > or in neutron's devref, or both. > > > > Salvatore > > > > On 22 September 2015 at 04:25, Ganesh Narayanan (ganeshna) > wrote: > > Another project for diagnosing OVS in Neutron: > > > > https://github.com/CiscoSystems/don > > > > Thanks, > > Ganesh > > > > From: Salvatore Orlando > Reply-To: OpenStack Development Mailing List > > Date: Monday, 21 September 2015 2:55 pm > To: OpenStack Development Mailing List > Subject: Re: [openstack-dev] [neutron] Neutron debugging tool > > > > It sounds like indeed that easyOVS covers what you're aiming too. > > However, from what I gather there is still plenty to do in easy OVS, so > perhaps rather than starting a new toolset from scratch you might build on > the existing one. > > > > Personally I'd welcome its adoption into the Neutron stadium as debugging > control plane/data plane issues in the neutron reference impl is becoming > difficult also for expert users and developers. > > I'd just suggest renaming it because calling it "OVS" is just plain wrong. > The neutron reference implementation and OVS are two distinct things. > > > > As concern neutron-debug, this is a tool that was developed in the early > stages of the project to verify connectivity using "probes" in namespaces. > These probes are simply tap interfaces associated with neutron ports. The > neutron-debug tool is still used in some devstack exercises. Nevertheless, > I'd rather keep building something like easyOVS and then deprecated > neutron-debug rather than develop it. > > > > Salvatore > > > > > > On 21 September 2015 at 02:40, Li Ma wrote: > > AFAIK, there is a project available in the github that does the same thing. > https://github.com/yeasy/easyOVS > > I used it before. > > > On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov > wrote: >> Hello, >> >> I am planning to develop a tool for network debugging. Initially, it >> will handle DVR case, which can also be extended to other too. Based >> on my OpenStack deployment/operations experience, I am planning to >> handle common pitfalls/misconfigurations, such as: >> 1) check external gateway validity >> 2) check if appropriate qrouter/qdhcp/fip namespaces are created in >> compute/network hosts >> 3) execute probing commands inside namespaces, to verify reachability >> 4) etc. >> >> I came across neutron-debug [1], which mostly focuses on namespace >> debugging. Its coverage is limited to OpenStack, while I am planning >> to cover compute/network nodes as well. In my experience, I had to ssh >> to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases >> above). The tool I am considering will handle these, given the host >> credentials. >> >> I'd like get community's feedback on utility of such debugging tool. >> Do people use neutron-debug on their OpenStack environment? Does the >> tool I am planning to develop with complete diagnosis coverage sound >> useful? Anyone is interested to j
Re: [openstack-dev] [neutron] Neutron debugging tool
There was a presentation on DON at the Vancouver summit. Here is the link: https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/don-diagnosing-ovs-in-neutron From: Salvatore Orlando [mailto:salv.orla...@gmail.com] Sent: Tuesday, September 22, 2015 3:51 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] Neutron debugging tool Thanks Ganesh! I did not know about this tool. I also quite like the network visualization bits, though I wonder how practical that would be when one debugs very large deployments. I think it won't be a bad idea to list these tools in the networking guide or in neutron's devref, or both. Salvatore On 22 September 2015 at 04:25, Ganesh Narayanan (ganeshna) mailto:ganes...@cisco.com>> wrote: Another project for diagnosing OVS in Neutron: https://github.com/CiscoSystems/don Thanks, Ganesh From: Salvatore Orlando mailto:salv.orla...@gmail.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Monday, 21 September 2015 2:55 pm To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [neutron] Neutron debugging tool It sounds like indeed that easyOVS covers what you're aiming too. However, from what I gather there is still plenty to do in easy OVS, so perhaps rather than starting a new toolset from scratch you might build on the existing one. Personally I'd welcome its adoption into the Neutron stadium as debugging control plane/data plane issues in the neutron reference impl is becoming difficult also for expert users and developers. I'd just suggest renaming it because calling it "OVS" is just plain wrong. The neutron reference implementation and OVS are two distinct things. As concern neutron-debug, this is a tool that was developed in the early stages of the project to verify connectivity using "probes" in namespaces. These probes are simply tap interfaces associated with neutron ports. The neutron-debug tool is still used in some devstack exercises. Nevertheless, I'd rather keep building something like easyOVS and then deprecated neutron-debug rather than develop it. Salvatore On 21 September 2015 at 02:40, Li Ma mailto:skywalker.n...@gmail.com>> wrote: AFAIK, there is a project available in the github that does the same thing. https://github.com/yeasy/easyOVS I used it before. On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov mailto:nodir.qodi...@gmail.com>> wrote: > Hello, > > I am planning to develop a tool for network debugging. Initially, it > will handle DVR case, which can also be extended to other too. Based > on my OpenStack deployment/operations experience, I am planning to > handle common pitfalls/misconfigurations, such as: > 1) check external gateway validity > 2) check if appropriate qrouter/qdhcp/fip namespaces are created in > compute/network hosts > 3) execute probing commands inside namespaces, to verify reachability > 4) etc. > > I came across neutron-debug [1], which mostly focuses on namespace > debugging. Its coverage is limited to OpenStack, while I am planning > to cover compute/network nodes as well. In my experience, I had to ssh > to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases > above). The tool I am considering will handle these, given the host > credentials. > > I'd like get community's feedback on utility of such debugging tool. > Do people use neutron-debug on their OpenStack environment? Does the > tool I am planning to develop with complete diagnosis coverage sound > useful? Anyone is interested to join the development? All feedback are > welcome. > > Thanks, > > - Nodir > > [1] > http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Li Ma (Nick) Email: skywalker.n...@gmail.com<mailto:skywalker.n...@gmail.com> __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usa
Re: [openstack-dev] [neutron] Neutron debugging tool
Thanks Ganesh! I did not know about this tool. I also quite like the network visualization bits, though I wonder how practical that would be when one debugs very large deployments. I think it won't be a bad idea to list these tools in the networking guide or in neutron's devref, or both. Salvatore On 22 September 2015 at 04:25, Ganesh Narayanan (ganeshna) < ganes...@cisco.com> wrote: > Another project for diagnosing OVS in Neutron: > > https://github.com/CiscoSystems/don > > Thanks, > Ganesh > > From: Salvatore Orlando > Reply-To: OpenStack Development Mailing List < > openstack-dev@lists.openstack.org> > Date: Monday, 21 September 2015 2:55 pm > To: OpenStack Development Mailing List > Subject: Re: [openstack-dev] [neutron] Neutron debugging tool > > It sounds like indeed that easyOVS covers what you're aiming too. > However, from what I gather there is still plenty to do in easy OVS, so > perhaps rather than starting a new toolset from scratch you might build on > the existing one. > > Personally I'd welcome its adoption into the Neutron stadium as debugging > control plane/data plane issues in the neutron reference impl is becoming > difficult also for expert users and developers. > I'd just suggest renaming it because calling it "OVS" is just plain wrong. > The neutron reference implementation and OVS are two distinct things. > > As concern neutron-debug, this is a tool that was developed in the early > stages of the project to verify connectivity using "probes" in namespaces. > These probes are simply tap interfaces associated with neutron ports. The > neutron-debug tool is still used in some devstack exercises. Nevertheless, > I'd rather keep building something like easyOVS and then deprecated > neutron-debug rather than develop it. > > Salvatore > > > On 21 September 2015 at 02:40, Li Ma wrote: > >> AFAIK, there is a project available in the github that does the same >> thing. >> https://github.com/yeasy/easyOVS >> >> I used it before. >> >> On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov >> wrote: >> > Hello, >> > >> > I am planning to develop a tool for network debugging. Initially, it >> > will handle DVR case, which can also be extended to other too. Based >> > on my OpenStack deployment/operations experience, I am planning to >> > handle common pitfalls/misconfigurations, such as: >> > 1) check external gateway validity >> > 2) check if appropriate qrouter/qdhcp/fip namespaces are created in >> > compute/network hosts >> > 3) execute probing commands inside namespaces, to verify reachability >> > 4) etc. >> > >> > I came across neutron-debug [1], which mostly focuses on namespace >> > debugging. Its coverage is limited to OpenStack, while I am planning >> > to cover compute/network nodes as well. In my experience, I had to ssh >> > to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases >> > above). The tool I am considering will handle these, given the host >> > credentials. >> > >> > I'd like get community's feedback on utility of such debugging tool. >> > Do people use neutron-debug on their OpenStack environment? Does the >> > tool I am planning to develop with complete diagnosis coverage sound >> > useful? Anyone is interested to join the development? All feedback are >> > welcome. >> > >> > Thanks, >> > >> > - Nodir >> > >> > [1] >> http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html >> > >> > >> __ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> -- >> >> Li Ma (Nick) >> Email: skywalker.n...@gmail.com >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Neutron debugging tool
Another project for diagnosing OVS in Neutron: https://github.com/CiscoSystems/don Thanks, Ganesh From: Salvatore Orlando mailto:salv.orla...@gmail.com>> Reply-To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Date: Monday, 21 September 2015 2:55 pm To: OpenStack Development Mailing List mailto:openstack-dev@lists.openstack.org>> Subject: Re: [openstack-dev] [neutron] Neutron debugging tool It sounds like indeed that easyOVS covers what you're aiming too. However, from what I gather there is still plenty to do in easy OVS, so perhaps rather than starting a new toolset from scratch you might build on the existing one. Personally I'd welcome its adoption into the Neutron stadium as debugging control plane/data plane issues in the neutron reference impl is becoming difficult also for expert users and developers. I'd just suggest renaming it because calling it "OVS" is just plain wrong. The neutron reference implementation and OVS are two distinct things. As concern neutron-debug, this is a tool that was developed in the early stages of the project to verify connectivity using "probes" in namespaces. These probes are simply tap interfaces associated with neutron ports. The neutron-debug tool is still used in some devstack exercises. Nevertheless, I'd rather keep building something like easyOVS and then deprecated neutron-debug rather than develop it. Salvatore On 21 September 2015 at 02:40, Li Ma mailto:skywalker.n...@gmail.com>> wrote: AFAIK, there is a project available in the github that does the same thing. https://github.com/yeasy/easyOVS I used it before. On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov mailto:nodir.qodi...@gmail.com>> wrote: > Hello, > > I am planning to develop a tool for network debugging. Initially, it > will handle DVR case, which can also be extended to other too. Based > on my OpenStack deployment/operations experience, I am planning to > handle common pitfalls/misconfigurations, such as: > 1) check external gateway validity > 2) check if appropriate qrouter/qdhcp/fip namespaces are created in > compute/network hosts > 3) execute probing commands inside namespaces, to verify reachability > 4) etc. > > I came across neutron-debug [1], which mostly focuses on namespace > debugging. Its coverage is limited to OpenStack, while I am planning > to cover compute/network nodes as well. In my experience, I had to ssh > to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases > above). The tool I am considering will handle these, given the host > credentials. > > I'd like get community's feedback on utility of such debugging tool. > Do people use neutron-debug on their OpenStack environment? Does the > tool I am planning to develop with complete diagnosis coverage sound > useful? Anyone is interested to join the development? All feedback are > welcome. > > Thanks, > > - Nodir > > [1] > http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Li Ma (Nick) Email: skywalker.n...@gmail.com<mailto:skywalker.n...@gmail.com> __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Neutron debugging tool
It sounds like indeed that easyOVS covers what you're aiming too. However, from what I gather there is still plenty to do in easy OVS, so perhaps rather than starting a new toolset from scratch you might build on the existing one. Personally I'd welcome its adoption into the Neutron stadium as debugging control plane/data plane issues in the neutron reference impl is becoming difficult also for expert users and developers. I'd just suggest renaming it because calling it "OVS" is just plain wrong. The neutron reference implementation and OVS are two distinct things. As concern neutron-debug, this is a tool that was developed in the early stages of the project to verify connectivity using "probes" in namespaces. These probes are simply tap interfaces associated with neutron ports. The neutron-debug tool is still used in some devstack exercises. Nevertheless, I'd rather keep building something like easyOVS and then deprecated neutron-debug rather than develop it. Salvatore On 21 September 2015 at 02:40, Li Ma wrote: > AFAIK, there is a project available in the github that does the same thing. > https://github.com/yeasy/easyOVS > > I used it before. > > On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov > wrote: > > Hello, > > > > I am planning to develop a tool for network debugging. Initially, it > > will handle DVR case, which can also be extended to other too. Based > > on my OpenStack deployment/operations experience, I am planning to > > handle common pitfalls/misconfigurations, such as: > > 1) check external gateway validity > > 2) check if appropriate qrouter/qdhcp/fip namespaces are created in > > compute/network hosts > > 3) execute probing commands inside namespaces, to verify reachability > > 4) etc. > > > > I came across neutron-debug [1], which mostly focuses on namespace > > debugging. Its coverage is limited to OpenStack, while I am planning > > to cover compute/network nodes as well. In my experience, I had to ssh > > to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases > > above). The tool I am considering will handle these, given the host > > credentials. > > > > I'd like get community's feedback on utility of such debugging tool. > > Do people use neutron-debug on their OpenStack environment? Does the > > tool I am planning to develop with complete diagnosis coverage sound > > useful? Anyone is interested to join the development? All feedback are > > welcome. > > > > Thanks, > > > > - Nodir > > > > [1] > http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html > > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > > Li Ma (Nick) > Email: skywalker.n...@gmail.com > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Neutron debugging tool
AFAIK, there is a project available in the github that does the same thing. https://github.com/yeasy/easyOVS I used it before. On Mon, Sep 21, 2015 at 12:17 AM, Nodir Kodirov wrote: > Hello, > > I am planning to develop a tool for network debugging. Initially, it > will handle DVR case, which can also be extended to other too. Based > on my OpenStack deployment/operations experience, I am planning to > handle common pitfalls/misconfigurations, such as: > 1) check external gateway validity > 2) check if appropriate qrouter/qdhcp/fip namespaces are created in > compute/network hosts > 3) execute probing commands inside namespaces, to verify reachability > 4) etc. > > I came across neutron-debug [1], which mostly focuses on namespace > debugging. Its coverage is limited to OpenStack, while I am planning > to cover compute/network nodes as well. In my experience, I had to ssh > to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases > above). The tool I am considering will handle these, given the host > credentials. > > I'd like get community's feedback on utility of such debugging tool. > Do people use neutron-debug on their OpenStack environment? Does the > tool I am planning to develop with complete diagnosis coverage sound > useful? Anyone is interested to join the development? All feedback are > welcome. > > Thanks, > > - Nodir > > [1] > http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Li Ma (Nick) Email: skywalker.n...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] Neutron debugging tool
Hello, I am planning to develop a tool for network debugging. Initially, it will handle DVR case, which can also be extended to other too. Based on my OpenStack deployment/operations experience, I am planning to handle common pitfalls/misconfigurations, such as: 1) check external gateway validity 2) check if appropriate qrouter/qdhcp/fip namespaces are created in compute/network hosts 3) execute probing commands inside namespaces, to verify reachability 4) etc. I came across neutron-debug [1], which mostly focuses on namespace debugging. Its coverage is limited to OpenStack, while I am planning to cover compute/network nodes as well. In my experience, I had to ssh to the host(s) to accurately diagnose the failure (e.g., 1, 2 cases above). The tool I am considering will handle these, given the host credentials. I'd like get community's feedback on utility of such debugging tool. Do people use neutron-debug on their OpenStack environment? Does the tool I am planning to develop with complete diagnosis coverage sound useful? Anyone is interested to join the development? All feedback are welcome. Thanks, - Nodir [1] http://docs.openstack.org/cli-reference/content/neutron-debug_commands.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev