I saw that FAQ prior to posting. I probably should have mentioned that.
Unfortunately the ovs-appctl does not work and complains of not finding
/usr/var/run/openvswitch/ovs-vswitchd.pid. I tried both coping the pidfile and
symlinking it to where ovs-appctl is looking, but it still failed with the same
error message.
Running strace when the pid file is there shows that it does finde the file,
but it performs a fcntl( ..., F_GETLK, {type=F_UNLCK,...} ...) which to me
makes me believe it might be seeing if the file/process is locked and erroring
out because it finds it can't unlock it.
The error message ovs-appctl displays is the same for both cases though:
_cannot read pidfile "/usr/var/run/openvswitch/ovs-vswitchd.pid"_.
I just now out of curiousity tried adding the --pidfile to the ovs-dpdk init
script (where it invokes vswitchd daemon, and found that now ovs-appctl works!
I'll follow that faq again now armed with a working ovs-appctl. I wonder if a
bug should be opened against this to add the --pidfile to the ovs-dpdk-init
script.
Thanks for taking your time to help me out! I'll post back with anything else
I find that might be an defect.
Gabe
> -----Original Message-----
> From: Ben Pfaff [mailto:[email protected]]
> Sent: Friday, August 28, 2015 9:08 AM
> To: Gabe Black
> Cc: Mooney, Sean K; [email protected]
> Subject: Re: [ovs-discuss] millions of packets going to a flow?
>
> On Fri, Aug 28, 2015 at 03:01:52PM +0000, Gabe Black wrote:
> > However, all the commands I show in this inquiry are ovs-xyz commands.
> > My questions are around how to debug/troubleshoot where the packets
> > are going. I am new to all of this, but to me this did seem like the
> > most relevant list to post that question.
>
> Maybe you want this FAQ.
>
> ### Q: I have a sophisticated network setup involving Open vSwitch, VMs or
> multiple hosts, and other components. The behavior isn't what I
> expect. Help!
>
> A: To debug network behavior problems, trace the path of a packet,
> hop-by-hop, from its origin in one host to a remote host. If
> that's correct, then trace the path of the response packet back to
> the origin.
>
> Usually a simple ICMP echo request and reply ("ping") packet is
> good enough. Start by initiating an ongoing "ping" from the origin
> host to a remote host. If you are tracking down a connectivity
> problem, the "ping" will not display any successful output, but
> packets are still being sent. (In this case the packets being sent
> are likely ARP rather than ICMP.)
>
> Tools available for tracing include the following:
>
> - "tcpdump" and "wireshark" for observing hops across network
> devices, such as Open vSwitch internal devices and physical
> wires.
>
> - "ovs-appctl dpif/dump-flows <br>" in Open vSwitch 1.10 and
> later or "ovs-dpctl dump-flows <br>" in earlier versions.
> These tools allow one to observe the actions being taken on
> packets in ongoing flows.
>
> See ovs-vswitchd(8) for "ovs-appctl dpif/dump-flows"
> documentation, ovs-dpctl(8) for "ovs-dpctl dump-flows"
> documentation, and "Why are there so many different ways to
> dump flows?" above for some background.
>
> - "ovs-appctl ofproto/trace" to observe the logic behind how
> ovs-vswitchd treats packets. See ovs-vswitchd(8) for
> documentation. You can out more details about a given flow
> that "ovs-dpctl dump-flows" displays, by cutting and pasting
> a flow from the output into an "ovs-appctl ofproto/trace"
> command.
>
> - SPAN, RSPAN, and ERSPAN features of physical switches, to
> observe what goes on at these physical hops.
>
> Starting at the origin of a given packet, observe the packet at
> each hop in turn. For example, in one plausible scenario, you
> might:
>
> 1. "tcpdump" the "eth" interface through which an ARP egresses
> a VM, from inside the VM.
>
> 2. "tcpdump" the "vif" or "tap" interface through which the ARP
> ingresses the host machine.
>
> 3. Use "ovs-dpctl dump-flows" to spot the ARP flow and observe
> the host interface through which the ARP egresses the
> physical machine. You may need to use "ovs-dpctl show" to
> interpret the port numbers. If the output seems surprising,
> you can use "ovs-appctl ofproto/trace" to observe details of
> how ovs-vswitchd determined the actions in the "ovs-dpctl
> dump-flows" output.
>
> 4. "tcpdump" the "eth" interface through which the ARP egresses
> the physical machine.
>
> 5. "tcpdump" the "eth" interface through which the ARP
> ingresses the physical machine, at the remote host that
> receives the ARP.
>
> 6. Use "ovs-dpctl dump-flows" to spot the ARP flow on the
> remote host that receives the ARP and observe the VM "vif"
> or "tap" interface to which the flow is directed. Again,
> "ovs-dpctl show" and "ovs-appctl ofproto/trace" might help.
>
> 7. "tcpdump" the "vif" or "tap" interface to which the ARP is
> directed.
>
> 8. "tcpdump" the "eth" interface through which the ARP
> ingresses a VM, from inside the VM.
>
> It is likely that during one of these steps you will figure out the
> problem. If not, then follow the ARP reply back to the origin, in
> reverse.
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
