Hi Gurusharan
Thanks for the extremely helpful tip
I did as you suggested, and ran a strace on the kvm pid:
# strace -e trace=select -p 3740
Process 3740 attached - interrupt to quit
select(44, [7 11 14 15 17 19 20 40 41 42 43], [], [], {0, 584192}) = 0
(Timeout)
select(44, [7 11 14 15 17 19 20 40 41 42 43], [], [], {0, 999000}) = 1
(in [19], left {0, 665094})
select(44, [7 11 14 15 17 19 20 40 41 42 43], [], [], {0, 665000}) = 2
(in [7 17], left {0, 664997})
...
The FD of the vnet0 interface is not in the readfds fd_set (it should
include 39) This indicates why the network device is not pulling any
traffic.
Also, even though the network interface is not registering any drops or
errors, the tc queue attached to it does show drops (not sure the
reasons for that)
# tc -s qdisc show dev vnet0
qdisc pfifo_fast 0: root refcnt 2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1
1 1 1 1 1
Sent 359863 bytes 1983 pkt (dropped 2804865, overlimits 0 requeues 2)
rate 0bit 0pps backlog 82148b 500p requeues 2
Commits like this:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=68e5ec64009812dbaa03ed9cfded9344986f5304
possibly fix the problem at the qemu side.
So even though the fault is not in openvswitch, in this process, I have
learned a lot more about how ovs and flows work - thanks :)
Regards
Chris
On 2015/03/18 05:27 PM, Gurucharan Shetty wrote:
Fwiw, I had seen a similar issue a couple of years ago. It turned out
to be a system bug. I debugged (with help from Ben) like this:
ps -ef | grep kvm : Choose the process id of the kvm
ls -ltr /proc/$pid/fd
This will show you the fd of /dev/net/tun
So the KVM process will be listening on that fd for all the packets
that comes from host and is headed to VM.
"strace -o logfile -p $Pid" for a few seconds.
If you open the logfile, you will see a select system call on that fd
and then read system calls on that fd.
In my case, there was nothing being read. I don't remember very well
what exactly was the problem. The fix was to use a different network
device .
On Wed, Mar 18, 2015 at 8:14 AM, Chris Picton <[email protected]> wrote:
On 03/18/15 16:52, Ben Pfaff wrote:
On Wed, Mar 18, 2015 at 12:44:08PM +0200, Chris Picton wrote:
On 03/18/15 08:47, Chris Picton wrote:
The relevant flow is:
recirc_id(0),skb_priority(0),in_port(2),eth(src=00:15:17:91:05:4c,dst=52:54:00:14:54:c6),eth_type(0x8100),vlan(vid=3,pcp=0),encap(eth_type(0x0806)),
packets:2, bytes:120, used:0.014s, actions:pop_vlan,7
I can't find any definitive documentation for an action element
consisting
purely of a number - in this case "7".
I have just re-read the man page, and found the following:
"The target may be a decimal port number designating the physical
port on which to output the packet."
so the packet should, according to the above, be forwarded to port 7,
which
is vnet0.
However, the packet is not arriving on vnet0. Any ideas where I can look
to
find out why not?
I'd start by checking the kernel log (with "dmesg").
Hi Ben
I had already done system level debugging to try find out where issues may
lie. There are no kernel logs indicating dropped packets or transmit
failures.
A diff between the contents of /sys/class/net/vnet0 (which is the port that
gets no traffic) and /sys/class/net/veth-carbon1-3/ (which is the port that
gets traffic) shows very few differences at the kernel level (rx/tx
counts/address/etc differ, but type/features/carrier/flags/etc are the same)
What I know so far is that while openvswitch should be forwarding packets
onto vnet0 (which is what the flow indicates), vnet0 does not receive those
packets (as indicated with tcpdump). Also the tx_packets counter for vnet0
does not increment (as it should when packets get sent to the interface and
transmitted on)
Is this a kernel/openvswitch bug? I can probably get the interface working
by migrating the VM to a different host and migrating it back, but I would
rather leave it in a non working state to try and pinpoint the issue.
Regards
Chris
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
