On Sun, Apr 9, 2017 at 3:23 PM, Mickey Spiegel <mickeys....@gmail.com>
wrote:
>
>
> On Thu, Apr 6, 2017 at 7:34 AM, Guoshuai Li <l...@dtdream.com> wrote:
>
>>
>> revese my topology:
>>
>> +---------+--------+
>> | VM 172.16.1.7 |
>> +---------+--------+
>> |
>> +---------+--------+
>> | Logical Switch |
>> +---------+--------+
>> |172.16.1.254
>> 10.157.142.3 +-----------+--------+
>> +----------------+ Logical Router 1 +
>> | +--------------------+
>> +---------+--------+
>> | Logical Switch |
>> +------------------+
>> | +--------------------+
>> +----------------+ Logical Router 2 |
>> 10.157.142.1 +--------------------+
>>
>>
>> Hi All, I am having a problem for ovn and need help, thanks.
>>>
>>>
>>> I created two logical routes and connected the two LogicalRoutes through
>>> a external LogicalSwitch (connected to the external network) .
>>>
>>> And then LogicalRoute-1 connected to the VM through the internal
>>> LogicalSwitch .
>>>
>>> my topology:
>>>
>>> 10.157.142.3 172.16.1.254
>>> +--------------------+ +---------+--------+
>>> +---------+--------+
>>> +----------------+ Logical Router 1 +------------------|
>>> Logical Switch +-------------------+ VM 172.16.1.7 |
>>> | +--------------------+ +------------------+
>>> +------------------+
>>> +---------+--------+
>>> | Logical Switch |
>>> +------------------+
>>> | +--------------------+
>>> +----------------+ Logical Router 2 |
>>> +--------------------+
>>> 10.157.142.1
>>>
>>> I tested the master and Branch2.7, it Can not be transferred from VM
>>> (172.16.1.7) to LogicaRouter-2 's port (10.157.142.
>>>
>> Sorry, The destination address is 10.157.142.1, And The SNAT/unSNAT
>> address is 10.157.142.3.
>>
>>> ) via ping.
>>> My logical router is a distributed gateway, and the two logical router
>>> ports that connect external LogicalSwitch are on the same chassis.
>>> If the two logical router ports are not on the same chassis ping is also
>>> OK, And ping from VM (172.16.1.7) to external network is also OK.
>>>
>>> I looked at the openflow tables on gateway chassis, I suspected unsnat
>>> handling error in Router1 input for icmp replay.
>>> I think it is necessary to replace the destination address 10.157.142.3
>>> with 172.16.1.7 in Table 19 and route 172.16.1.7 in Table 21, but now the
>>> route match is 10.157.142.0/24.
>>>
>>> cookie=0x92bd0055, duration=68.468s, table=16, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=50,reg14=0x4,metadata=0x7,dl_dst=fa:16:3e:58:1c:8a
>>> actions=resubmit(,17)
>>> cookie=0x45765344, duration=68.467s, table=17, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=0,metadata=0x7 actions=resubmit(,18)
>>> cookie=0xaeaaed29, duration=68.479s, table=18, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=0,metadata=0x7 actions=resubmit(,19)
>>> cookie=0xce785d51, duration=68.479s, table=19, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=100,ip,reg14=0x4,metadata=0x7,nw_dst=10.157.142.3
>>> actions=ct(table=20,zone=NXM_NX_REG12[0..15],nat)
>>> cookie=0xbd994421, duration=68.481s, table=20, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=0,metadata=0x7 actions=resubmit(,21)
>>> cookie=0xaea3a6ae, duration=68.479s, table=21, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=49,ip,metadata=0x7,nw_dst=10.157.142.0/24
>>> actions=dec_ttl(),move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..12
>>> 7],load:0xa9d8e03->NXM_NX_XXREG0[64..95],mod_dl_src:fa:16:3e
>>> :58:1c:8a,load:0x4->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0]
>>> ,resubmit(,22)
>>> cookie=0xce6e8d4e, duration=68.482s, table=22, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=0,ip,metadata=0x7 actions=push:NXM_NX_REG0[],pus
>>> h:NXM_NX_XXREG0[96..127],pop:NXM_NX_REG0[],mod_dl_dst:00:00:
>>> 00:00:00:00,resubmit(,66),pop:NXM_NX_REG0[],resubmit(,23)
>>> cookie=0xce89c4ed, duration=68.481s, table=23, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=150,reg15=0x4,metadata=0x7,dl_dst=00:00:00:00:00:00
>>> actions=load:0x5->NXM_NX_REG15[],resubmit(,24)
>>> cookie=0xb2d84350, duration=68.469s, table=24, n_packets=1, n_bytes=98,
>>> idle_age=36, priority=100,ip,metadata=0x7,dl_dst=00:00:00:00:00:00
>>>
>>> I do not know why and need help, thanks.
>>>
>>
> I was able to reproduce this. I agree with your analysis. Looking at
> ovs-ofctl dump-flows, the packet counts indicate that the packet is subject
> to ct(...,nat), but the routing table match is as if NAT never occurred.
>
> I tried with gateway routers and it worked. There are some differences in
> ovs-dpctl dump-flows.
>
> For the case of gateway routers:
>
> vagrant@compute2:~$ sudo ovs-dpctl dump-flows
>
> recirc_id(0x14),tunnel(tun_id=0x6,src=192.168.33.31,dst=192.
> 168.33.32,geneve({}{}),flags(-df+csum+key)),in_port(4),eth(
> src=00:00:00:00:00:00/01:00:00:00:00:00,dst=00:00:02:02:
> 03:04),eth_type(0x0800),ipv4(src=172.16.1.3,dst=172.16.1.
> 10,proto=1,ttl=62,frag=no),icmp(type=8,code=0), packets:3, bytes:294,
> used:1.981s, actions:userspace(pid=2658598031,slow_path(action))
>
> recirc_id(0x16),tunnel(tun_id=0x6,src=192.168.33.31,dst=192.
> 168.33.32,geneve({class=0x102,type=0x80,len=4,0x10002}),
> flags(-df+csum+key)),in_port(4),eth(src=00:00:02:02:03:04,
> dst=00:00:02:01:02:03),eth_type(0x0800),ipv4(src=172.16.
> 1.10,dst=192.168.1.2,tos=0/0x3,ttl=254,frag=no), packets:3, bytes:294,
> used:1.981s, actions:set(tunnel(tun_id=0x3,dst=192.168.33.31,ttl=64,tp_
> src=24284,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,
> 0x10002}),flags(df|csum|key))),set(eth(src=00:00:01:01:02:
> 03,dst=f0:00:00:01:02:03)),set(ipv4(src=172.16.1.10,dst=
> 192.168.1.2,tos=0/0x3,ttl=252)),4
>
> recirc_id(0),tunnel(tun_id=0x6,src=192.168.33.31,dst=192.
> 168.33.32,geneve({class=0x102,type=0x80,len=4,0x10002/
> 0x7fffffff}),flags(-df+csum+key)),in_port(4),eth(src=00:
> 00:00:00:00:00/01:00:00:00:00:00,dst=00:00:04:01:02:04),eth_
> type(0x0800),ipv4(src=192.168.1.2/255.255.255.254,dst=172.
> 16.1.10,proto=1,ttl=63,frag=no), packets:3, bytes:294, used:1.981s,
> actions:ct(zone=1,nat),recirc(0x13)
>
> recirc_id(0x15),tunnel(tun_id=0x6,src=192.168.33.31,dst=192.
> 168.33.32,geneve({}{}),flags(-df+csum+key)),in_port(4),eth(
> src=00:00:02:01:02:03,dst=00:00:02:02:03:04),eth_type(
> 0x0800),ipv4(src=172.16.1.10,dst=172.16.1.3,proto=1,ttl=255,frag=no),
> packets:3, bytes:294, used:1.981s, actions:set(eth(src=00:00:02:
> 02:03:04,dst=00:00:02:01:02:03)),set(ipv4(src=172.16.1.10,
> dst=172.16.1.3,ttl=254)),ct(zone=2,nat),ct(commit,zone=1,
> nat(dst=192.168.1.2)),recirc(0x16)
>
> recirc_id(0x13),tunnel(tun_id=0x6,src=192.168.33.31,dst=192.
> 168.33.32,geneve({}{}),flags(-df+csum+key)),in_port(4),eth(
> src=00:00:04:01:02:03,dst=00:00:04:01:02:04),eth_type(
> 0x0800),ipv4(src=192.168.1.2,dst=172.16.1.10,ttl=63,frag=no), packets:3,
> bytes:294, used:1.981s, actions:set(eth(src=00:00:02:
> 01:02:03,dst=00:00:02:02:03:04)),set(ipv4(src=192.168.1.2,
> dst=172.16.1.10,ttl=62)),ct(commit,zone=2,nat(src=172.16.
> 1.3)),recirc(0x14)
>
> Note that recirc_id(0x15) goes to ct() actions after the ICMP response is
> generated in the slow path, which is required for ICMP type change.
>
>
> With distributed routers and distributed gateway ports:
>
> vagrant@compute2:~$ sudo ovs-dpctl dump-flows
>
> recirc_id(0),tunnel(tun_id=0x1,src=192.168.33.31,dst=192.
> 168.33.32,geneve({class=0x102,type=0x80,len=4,0x10004/
> 0x7fffffff}),flags(-df+csum+key)),in_port(4),eth_type(
> 0x0800),ipv4(src=192.168.1.3,frag=no), packets:3, bytes:294, used:2.388s,
> actions:ct(commit,zone=3,nat(src=172.16.1.1)),recirc(0x3)
>
> recirc_id(0x3),tunnel(tun_id=0x1,src=192.168.33.31,dst=192.
> 168.33.32,geneve({}{}),flags(-df+csum+key)),in_port(4),eth(
> src=00:00:02:01:02:03,dst=00:00:02:01:02:04),eth_type(
> 0x0800),ipv4(src=172.16.1.1,dst=172.16.1.10,proto=1,ttl=
> 63,frag=no),icmp(type=8,code=0), packets:3, bytes:294, used:2.389s,
> actions:userspace(pid=2248102802,slow_path(action))
>
> recirc_id(0x4),tunnel(tun_id=0x1,src=192.168.33.31,dst=192.
> 168.33.32,geneve({}{}),flags(-df+csum+key)),in_port(4),eth(
> src=00:00:02:01:02:04,dst=00:00:02:01:02:03),eth_type(
> 0x0800),ipv4(dst=172.16.1.1,ttl=254,frag=no), packets:3, bytes:294,
> used:2.389s, actions:userspace(pid=2248102802,slow_path(controller))
>
> The recirc_id(0x4) entry ends up with the slow_path(controller) action
> resulting from table 24. I do not yet know why the earlier ct() actions
> from table 19 were skipped.
>
The problems have to do with hitting NAT actions when already in the slow
path due to the change of ICMP type.
When pinging from either a gateway router or a distributed router with a
distributed gateway port, to the IP address of another gateway router, the
ping works. On the second gateway router, the hit in table 17 to generate
the ICMP echo reply forces slow path due to the action (change of ICMP
type). On gateway routers, all IP packets are subject to recirc in the DNAT
pipeline stage (table 20), even though the ICMP echo reply would never hit
a DNAT entry. The recirc forced by this action ends the slow path
processing. By the time the packet gets back to the first router (gateway
router or distributed router), the packet is no longer in the slow path and
NAT succeeds as it should.
When pinging from either a gateway router or a distributed router with a
distributed gateway port, to the IP address of another distributed router
with a distributed gateway port, the ping fails. There is no recirc between
the hit on the second gateway router in table 17 to generate the ICMP echo
reply (which forces slow path due to the ICMP type change), and the reply
packet's hit of the UNSNAT rule in table 19 of the first router (gateway
router or distributed router). The latter does not find a match, and the
packet ends up hitting the table 24 slow path controller entry that applies
when ARP could not resolve a MAC address.
Discussing this with Guru on IRC earlier today, the suggestion was to
create a system test in tests/system-traffic.at (without OVN) that
demonstrates the problem. From one namespace trying to ping a virtual IP,
there should be flows to apply SNAT, then apply clone and ct_clear
(emulating the behavior of OVN patch ports), then generate the ICMP echo
reply just like OVN does, back through clone and ct_clear, then UNSNAT,
then back to the source. Once we can demonstrate the problem in this way,
then we can ask for further help to debug this.
Mickey
Mickey
>
>
>
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
