Re: [vpp-dev] SIGSEGV when enabling nat64 on 22.02+

[Filip Varga via lists.fd.io]

Hi Yuxiang,

Thank you for the report. I am looking into replicating the issue. Please 
create a ticket on the public jira – you can track the progress there.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Yuxiang Zhu
Sent: Monday, June 13, 2022 9:43 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] SIGSEGV when enabling nat64 on 22.02+

Hi,

I am looking to set up nat64 for testing. However I got SIGSEGV when issuing 
set interface nat64 out . Is nat64 broken or was I doing something 
wrong?

I was running v22.06-rc1 official debian build on a Debian bullseye virtual 
machine with a very basic config. I also tried v22.06-rc1 and master and got 
the same error.



uname -a

Linux vpp-proto 5.10.0-15-amd64 #1 SMP Debian 5.10.120-1 (2022-06-09) x86_64 
GNU/Linux





vppctl show version verbose cmdline:

Version:  v22.06-rc1~5-gea4bcec98~b8

Compiled by:  root

Compile host: bc9b24a17e05

Compile date: 2022-06-10T21:37:09

Compile location: /w/workspace/vpp-merge-2206-debian11-x86_64

Compiler: Clang/LLVM 11.0.1

Current PID:  10735

Command line arguments:

  /usr/bin/vpp

  unix

{

nodaemon

log

/var/log/vpp/vpp.log

full-coredump

cli-listen

/run/vpp/cli.sock

gid

vpp

exec

/etc/vpp/bootstrap.vpp

}

  api-trace

{

on

}

  api-segment

{

gid

vpp

}

  socksvr

{

default

}

  memory

{

main-heap-size

512M

main-heap-page-size

default-hugepage

}

  cpu

{

main-core

1

workers

2

}

  dpdk

{

dev

:00:13.0

dev

:00:14.0

}
default-hugepage

}

  cpu

{

main-core

1

workers

2

}

  dpdk

{

dev

:00:13.0

dev

:00:14.0

}



/etc/vpp/bootstrap.vpp
set interface state GigabitEthernet0/13/0 up
set interface ip address GigabitEthernet0/13/0 2001:db8::1/64
set interface state GigabitEthernet0/14/0 up
set interface ip address GigabitEthernet0/14/0 
100.100.18.253/24

Both NICs are virtio nics connected to a linux bridge on the host:


# dpdk-devbind.py -s



Network devices using DPDK-compatible driver



:00:13.0 'Virtio network device 1000' drv=uio_pci_generic 
unused=virtio_pci,vfio-pci

:00:14.0 'Virtio network device 1000' drv=uio_pci_generic 
unused=virtio_pci,vfio-pci




Steps to produce:


# vppctl show int addr

GigabitEthernet0/13/0 (up):

  L3 2001:db8::1/64

GigabitEthernet0/14/0 (up):

  L3 100.100.18.253/24

local0 (dn):
# vppctl set interface nat64 in GigabitEthernet0/13/0

# vppctl set interface nat64 out GigabitEthernet0/14/0

After issuing `set interface nat64 out ` command, vpp will crash in 
a few minutes with the error message like:

Jun 13 15:08:21 vpp-proto vnet[10728]: received signal SIGSEGV, PC 
0x7fe97ba6e0f8, faulting address 0x848

Jun 13 15:08:21 vpp-proto vnet[10728]: #0  0x7fe99ebb153e 0x7fe99ebb153e

Jun 13 15:08:21 vpp-proto vnet[10728]: #1  0x7fe99eac8140 0x7fe99eac8140

Jun 13 15:08:21 vpp-proto vnet[10728]: #2  0x7fe97ba6e0f8 0x7fe97ba6e0f8

Jun 13 15:08:21 vpp-proto vnet[10728]: #3  0x7fe97ba57157 
nat64_out2in_node_fn_hsw + 0x447

Jun 13 15:08:21 vpp-proto vnet[10728]: #4  0x7fe99eb41451 vlib_worker_loop 
+ 0x1b31

Jun 13 15:08:21 vpp-proto vnet[10728]: #5  0x7fe99eb93f1a 
vlib_worker_thread_fn + 0xaa

Jun 13 15:08:21 vpp-proto vnet[10728]: #6  0x7fe99eb8e291 
vlib_worker_thread_bootstrap_fn + 0x51

Jun 13 15:08:21 vpp-proto vnet[10728]: #7  0x7fe99eabcea7 start_thread + 
0xd7

Jun 13 15:08:21 vpp-proto vnet[10728]: #8  0x7fe99e80fdef clone + 0x3f

Jun 13 15:08:21 vpp-proto systemd[1]: vpp.service: Main process exited, 
code=killed, status=6/ABRT


Best Regards,
Yuxiang Zhu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21534): https://lists.fd.io/g/vpp-dev/message/21534
Mute This Topic: https://lists.fd.io/mt/91722179/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Is it possible to have NAT44 and DET44 simultaneously?

[Filip Varga via lists.fd.io]

Hi Marcelo,

It is possible to run at once all of the nat plugins.

There are few issues I can see.

Using forwarding enabled configuration with nat44-ed should let all of the 
in2out traffic except static mapping get ignored / not translated. This should 
let you run both in2out nat44-ed and in2out det44 nodes on the same VPP 
interface.
Though running out2in nat44-ed and out2in det44 nodes on the same interface 
could cause trouble because of dropping. Even though nat44-ed forwarding 
enabled will let packet pass instead of dropping them – if there are no 
sessions / static mappings this might not be enough.

Please try your configuration if you haven’t yet and let me know. If you run in 
any issues please include packet traces.


Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Marcelo Caggiani 
Luizelli via lists.fd.io
Sent: Wednesday, April 27, 2022 7:15 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Is it possible to have NAT44 and DET44 simultaneously?
Importance: High

Hi all,

Is there any way to configure NAT44(with twice-nat rules) and be simultaneously 
 deterministic? I have a setup where I need deterministic port assignment (as 
DET44 does),  and twice-nat rules to access inside local hosts.

Any thoughts?

Regards, Marcelo.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21298): https://lists.fd.io/g/vpp-dev/message/21298
Mute This Topic: https://lists.fd.io/mt/90735805/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] about in/out arguments in "set interface nat44 .."command

[Filip Varga via lists.fd.io]

Hi,

I will do some testing and let you know.

For now:

There is one solution you can try at this moment:

  1.  configure both interfaces as inside (management traffic and business 
traffic)
  2.  configure G0 as outside interface
  3.  enable forwarding
  4.  add G0 as nat interface address
  5.  add static mappings for management IP addresses

The problem here is that when an interface is configured as outside all 
incomming traffic to G0 (reply to outgoing traffic from business traffic) get’s 
dropped because of default DENY policy. This is not yet configurable but it 
will be in a near future.

Best regards,
Filip

From: vpp-dev@lists.fd.io  On Behalf Of 
haiyan...@ilinkall.cn
Sent: Monday, March 14, 2022 10:34 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
vpp-dev 
Subject: Re: [vpp-dev] about in/out arguments in "set interface nat44 .."command
Importance: High

Hi Filip,

There are two kinds of traffic in my test.
one is managing traffic, which will comes into vpp only from a tap interfaces, 
for example tap's ip address is 10.10.1.1/24. we need these traffic to do snat 
and out from G0
one is business traffic, which are gererated interlanfrom vpp, these traffic 
will not do snat and out from G0
Both the two kinds of traffic share the same outside interface G0 which could 
access the internet.

I have tried:
1. seperate managing traffic into different vrf, ...
2. add static mapping for local 10.10.1.x/24 external ,  ...

but both did not work.

can vpp achieve this goal?  any suggestions will be appreciated.



haiyan...@ilinkall.cn

From: Filip Varga -X (fivarga - PANTHEON TECH SRO at 
Cisco)
Date: 2022-03-12 02:27
To: haiyan...@ilinkall.cn; 
vpp-dev
Subject: RE: [vpp-dev] about in/out arguments in "set interface nat44 .."command
Hi Haiyan,

Can you please be more specific. For example what other traffic except from 
interface A and B will be passing (are there other interfaces involved) ? Using 
nat44-ed forwarding is a bit tricky. It let’s pass traffic without translation 
if a preexisting session isn’t found so this would mean only static mapping 
get’s translated. Forwarding feature is used for specific purposes when in 
mixed environments you have A configured as inside and also outside and B 
configured as inside and outside also.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of 
haiyan...@ilinkall.cn
Sent: Thursday, March 10, 2022 12:54 AM
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: [vpp-dev] about in/out arguments in "set interface nat44 .."command

Dear all

if only want traffic from interface A to interface B done with nat44, others 
will be directly out from interface B, is this possible?

I did the following command in vpp, but did not work
nat44 forwarding enable
nat44 add address 
set interface nat44 in A out B

Thanks


haiyan...@ilinkall.cn

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21030): https://lists.fd.io/g/vpp-dev/message/21030
Mute This Topic: https://lists.fd.io/mt/89682456/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] about in/out arguments in "set interface nat44 .."command

[Filip Varga via lists.fd.io]

Hi Haiyan,

Can you please be more specific. For example what other traffic except from 
interface A and B will be passing (are there other interfaces involved) ? Using 
nat44-ed forwarding is a bit tricky. It let's pass traffic without translation 
if a preexisting session isn't found so this would mean only static mapping 
get's translated. Forwarding feature is used for specific purposes when in 
mixed environments you have A configured as inside and also outside and B 
configured as inside and outside also.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of 
haiyan...@ilinkall.cn
Sent: Thursday, March 10, 2022 12:54 AM
To: vpp-dev 
Subject: [vpp-dev] about in/out arguments in "set interface nat44 .."command

Dear all

if only want traffic from interface A to interface B done with nat44, others 
will be directly out from interface B, is this possible?

I did the following command in vpp, but did not work
nat44 forwarding enable
nat44 add address 
set interface nat44 in A out B

Thanks


haiyan...@ilinkall.cn

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21007): https://lists.fd.io/g/vpp-dev/message/21007
Mute This Topic: https://lists.fd.io/mt/89682456/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] route between two vrfs does not work

[Filip Varga via lists.fd.io]

Hi Haiyan,

VRF for nat44-ed and nat44-ei works as follows:

Scenario 2 VRF’s. VRF1 can reach the internet, VRF2 can’t.

1)
Enable nat44-ed plugin.

2)
VRF1:
Configure public facing interface to be used as outside interface for nat44-ed 
plugin.

VRF2:
Configure one or all interface (that you want to be able to communicate with 
public IP addresses) as inside interface[s] for nat44-ed plugin.

3)
Configure nat44-ed address range for VRF2.

tenant-vrf parameter is used to tell nat for which source VRF the address 
should be used for translation. So in this scenario we need it to be VRF2.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of 
haiyan...@ilinkall.cn
Sent: Tuesday, March 8, 2022 1:06 AM
To: vpp-dev 
Subject: [vpp-dev] route between two vrfs does not work


Dear all

my test uses two vrfs in vpp：
vrf A: interface(G0) with pulic ip address(172.16.0.73/24) exists，and i  also 
did "nat44 add address xxx tenant-vrf A"/"set interface nat44 out G0 
output-feature"/"nat44 forwarding enable"
vrf B: there is no public ip address, so need to access the internet through 
vfr A interface G0, so i did "ip route add 172.16.0.47/32 table B via 0.0.0.0 
next-hop-table A",but that does not work.
do I missing something or any suggestion ?
detail configurations shows below：
vpp# show version
vpp v20.01-natt~82-g061bec7 built by root on localhost.localdomain at 2022年 03月 
07日 星期一 16:04:34 CST
vpp#
vpp#
vpp# show nat
natnat44  nat64  nat66
vpp# show interface addr
G0 (up):
  L3 172.16.0.73/24 ip4 table-id 1 fib-idx 1
G1 (up):
  L2 bridge bd-id 1 idx 1 shg 0
local0 (dn):
loop21 (up):
  L2 bridge bd-id 1 idx 1 shg 0 bvi
  L3 192.168.1.1/24
tap10 (up):
  L3 10.10.1.1/24 ip4 table-id 1 fib-idx 1
tap20 (up):
  L2 bridge bd-id 1 idx 1 shg 0
vpp#
vpp# show ip fib
ipv4-VRF:0, fib_index:0, flow hash:[src dst sport dport proto ] 
locks:[src:plugin-hi:2, src:adjacency:1, src:default-route:1, ]
0.0.0.0/0
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:1 buckets:1 uRPF:0 to:[101:8484]]
[0] [@0]: dpo-drop ip4
0.0.0.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:2 buckets:1 uRPF:1 to:[0:0]]
[0] [@0]: dpo-drop ip4
172.16.0.47/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:33 buckets:1 uRPF:38 
to:[2220:186480]]
[0] [@13]: dst-address,unicast lookup in ipv4-VRF:1
192.168.1.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:27 buckets:1 uRPF:32 to:[0:0]]
[0] [@0]: dpo-drop ip4
192.168.1.0/24
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:26 buckets:1 uRPF:31 to:[10:960]]
[0] [@4]: ipv4-glean: loop21: mtu:9000 dead00150806
192.168.1.1/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:29 buckets:1 uRPF:36 to:[10:900]]
[0] [@2]: dpo-receive: 192.168.1.1 on loop21
192.168.1.200/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:30 buckets:1 uRPF:35 to:[5:480] 
via:[22:1848]]
[0] [@5]: ipv4 via 192.168.1.200 loop21: mtu:9000 
8a48fe5830d9dead00150800
192.168.1.255/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:28 buckets:1 uRPF:34 to:[0:0]]
[0] [@0]: dpo-drop ip4
224.0.0.0/4
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:4 buckets:1 uRPF:3 to:[0:0]]
[0] [@0]: dpo-drop ip4
240.0.0.0/4
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:3 buckets:1 uRPF:2 to:[0:0]]
[0] [@0]: dpo-drop ip4
255.255.255.255/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:5 buckets:1 uRPF:4 to:[0:0]]
[0] [@0]: dpo-drop ip4
ipv4-VRF:1, fib_index:1, flow hash:[src dst sport dport proto ] 
locks:[src:CLI:3, src:plugin-low:1, src:adjacency:8, 
src:recursive-resolution:1, ]
0.0.0.0/0
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:9 buckets:1 uRPF:20 to:[10:2415]]
[0] [@5]: ipv4 via 172.16.0.1 G0: mtu:9000 8446fe747dd4a0369f75ba8a0800
0.0.0.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:8 to:[0:0]]
[0] [@0]: dpo-drop ip4
10.10.1.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:20 buckets:1 uRPF:22 to:[0:0]]
[0] [@0]: dpo-drop ip4
10.10.1.0/24
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:19 buckets:1 uRPF:21 to:[0:0]]
[0] [@4]: ipv4-glean: tap10: mtu:9000 02fedd9ceb9b0806
10.10.1.1/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:22 buckets:1 uRPF:26 to:[0:0]]
[0] [@2]: dpo-receive: 10.10.1.1 on tap10
10.10.1.100/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:25 buckets:1 uRPF:29 
to:[1666:315090]]
[0] [@5]: ipv4 via 10.10.1.100 tap10: mtu:9000 963b645c5f7402fedd9ceb9b0800
10.10.1.255/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:21 buckets:1 uRPF:24 to:[0:0]]
[0] [@0]: dpo-drop ip4
172.16.0.0/32
  unicast-ip4-chain
  [@0]: dpo-load-balance: [proto:ip4 index:15 buckets:1 uRPF:14 to:[0:0]]
[0] [@0]: dpo-drop ip4

Re: [vpp-dev] linux-cp + nat44 possible?

[Filip Varga via lists.fd.io]

Hi Rene,

The homegateway configuration of NAT is bit outdated. I am working on a newer 
configuration. 

What the old configuration doesn't account for is the split of nat plugins. In 
configuration nat is setup as endpoint independent.
Current nat is split into plugins nat44-ed nat44-ei 
(endpoint-dependent/independent).

I would suggest using endpoint-dependent plugn. All of the cli commands are the 
same "nat44 etc.". Plugin has to be enabled before configuration using the 
appropriate command. Use nat44 ? in vppctl to explore all of the commands.

NAT44-ED uses inside/outside interfaces or output feature. Output feature NAT's 
all the outgoing trafic on the specified interface - from all LAN interfaces / 
all local traffic if you have one WAN interface.

If you have multiple LAN interfaces and you want to NAT only specific one you 
should  instead manually set inside nat interface for a each LAN interface and 
outside interface for WAN.

Pool address configuration is the last step to configure working nat.

Best regards,
Filip Varga

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of René Weiss
Sent: Monday, January 17, 2022 10:45 PM
To: vpp-dev 
Subject: Re: [vpp-dev] linux-cp + nat44 possible?
Importance: High

Hi Dave

Thanks, this looks like an extended version of 
https://fd.io/docs/vpp/v2009/usecases/homegateway.html

I will take a closer look, especially at the ipv6 section because I was already 
wondering how to do the things I'm currently doing with 
radvd/wide-dhcpv6-client with VPP.

Another open questions (and the main reason I was asking for linux-cp) is 
receiving IPTV over multicast.

On Linux I'm using imgpproxy and udpxy (http://www.udpxy.com/, a multicast-UDP 
to HTTP relay) for this and while it's not essential for me, it still would be 
nice to continue to be able to use it.

Maybe you (or anyone else here) can give me a pointer in the right direction?

Thanks.
René

Am 17.01.22 um 20:58 schrieb Dave Barach:
> https://s3-docs.fd.io/vpp/22.02/usecases/home_gateway.html - I've used vpp as 
> a home gateway for years.
> 
> HTH... Dave
> 
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of René 
> Weiss
> Sent: Monday, January 17, 2022 12:47 PM
> To: vpp-dev 
> Subject: [vpp-dev] linux-cp + nat44 possible?
> 
> Hi
> 
> Is it possible to use linux-cp and nat44 together?
> 
> I'm currently playing with VPP to see if I would be able to use it on my 
> Linux home gateway (mostly as an iptables/nftables replacement).
> 
> And because there I have an external and (at least) one internal interface I 
> tried (unsuccessfully) to replicate that with VPP.
> 
> I was able to create a basic NAT setup based on "VPP_Home_Gateway" from the 
> wiki where you get a single "lstack" interface in Linux.
> 
> Likewise, I also was able to pass VPP interfaces to Linux with "lcp create 
>  host-if ".
> 
> But as soon as I try to combine the two (setup nat44 while using the 
> forwarded interfaces) the external interface stops working for the Linux on 
> the machine.
> 
> Is this something that simply will not work with VPP, or have I just not 
> found the right settings yet?
> 
> Thanks,
> René
> 
> 
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20781): https://lists.fd.io/g/vpp-dev/message/20781
Mute This Topic: https://lists.fd.io/mt/88490068/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Unable to configure mixed NAT and non-NAT traffic

[Filip Varga via lists.fd.io]

Hi Rajith,

If you are not using output feature you have to configure both inside and 
outside nat interfaces separately.

So got with input + output or output feature.

Output feature is not an nat interface. it is a feature that handles 
configuration of both outside and inside nat interfaces and together with pool 
address creates complete configuratoin of basic NAT box == home router gateway.

To nat specific local interface traffic only you have to configure inside 
interface for the lan interface and outside interface for the wan interface.

If you have any further questions please ask.

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io  On Behalf Of Rajith PR via 
lists.fd.io
Sent: Saturday, January 15, 2022 8:02 AM
To: alekc...@googlemail.com
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Unable to configure mixed NAT and non-NAT traffic
Importance: High

Hi all,

Just to add to the query, I have observed that in interface configuration is 
optional for NAT to work. All traffic get NATed if out interface is set with 
output-feature.

Thanks,
Rajith

On Thu, 13 Jan 2022 at 7:06 AM, alekcejk via lists.fd.io 
mailto:googlemail@lists.fd.io>> wrote:
Hi all,

I am trying to get setup for mixed NAT and non-NAT traffic.

In GNS3 I created VPP VM with three interfaces (1 external, 2 internal).

External interface GigabitEthernet0/5/0 with public IP address
203.0.113.1/30 connected to host with IP 
203.0.113.2/30 and route to
198.51.100.0/24 via 203.0.113.1
Internal interface GigabitEthernet0/6/0 with private IP address
172.16.0.1/24 connected to host with IP 
172.16.0.2/24
Internal interface GigabitEthernet0/7/0 with public IP address
198.51.100.1/25 connected to host with IP 
198.51.100.2/25

Internal traffic from/to 198.51.100.0/25 should be 
forwarded without NAT.
NAT address 198.51.100.128 should be applied on external interface
only for internal traffic from 172.16.0.0/24.

Here my setup for VPP 21.01.1 (running on CentOS 8)

/etc/vpp/startup.conf:
unix {
  nodaemon
  startup-config /etc/vpp/startup-config
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen /run/vpp/cli.sock
  cli-history-limit 100
  cli-no-banner
  poll-sleep-usec 10
  gid vpp
}

api-trace {
  on
}

api-segment {
  gid vpp
}

dpdk {
  dev :00:05.0
  dev :00:06.0
  dev :00:07.0
}

plugins {
  plugin default { disable }
  plugin dpdk_plugin.so { enable }
  plugin nat_plugin.so { enable }
  plugin arping_plugin.so { enable }
  plugin ping_plugin.so { enable }
}

logging {
   default-log-level debug
   default-syslog-log-level debug
}

ethernet {
  default-mtu 1500
}

/etc/vpp/startup-config:
set interface state GigabitEthernet0/5/0 up
set interface state GigabitEthernet0/6/0 up
set interface state GigabitEthernet0/7/0 up
set interface ip address GigabitEthernet0/5/0 
203.0.113.1/30
set interface ip address GigabitEthernet0/6/0 
172.16.0.1/24
set interface ip address GigabitEthernet0/7/0 
198.51.100.1/25
nat44 enable sessions 5 endpoint-dependent
nat44 forwarding enable
nat44 add address 198.51.100.128
set interface nat44 in GigabitEthernet0/6/0 output-feature
set interface nat44 out GigabitEthernet0/5/0 output-feature

If I run ping from internal host 172.16.0.2 to external host
203.0.113.2 then translation works correctly
02:44:23.420497 IP 198.51.100.128 > 203.0.113.2: ICMP echo 
request, id
64233, seq 4, length 64
02:44:23.420516 IP 203.0.113.2 > 198.51.100.128: ICMP 
echo reply, id
64233, seq 4, length 64

But if I run ping 203.0.113.2 from internal host 198.51.100.2 then NAT
also applied even though I didn't set nat in on the
GigabitEthernet0/7/0
02:47:15.242598 IP 198.51.100.128 > 203.0.113.2: ICMP echo 
request, id
22324, seq 127, length 64
02:47:15.242620 IP 203.0.113.2 > 198.51.100.128: ICMP 
echo reply, id
22324, seq 127, length 64

vpp# show nat44 interfaces
NAT44 interfaces:
 GigabitEthernet0/6/0 output-feature in
 GigabitEthernet0/5/0 output-feature out

If I remove "output-feature" then translation not applied at all with
enabled "nat44 forwarding enable".



In setup for VPP 21.10 I removed "endpoint-dependent" but if
"output-feature" will stay on internal interface GigabitEthernet0/6/0
I see new problem.

Only one correct response received on internal host 172.16.0.2 when
running ping 203.0.113.2, second response comes with source IP
198.51.100.128 instead of 203.0.113.2.
03:06:18.420787 IP 172.16.0.2 > 203.0.113.2: ICMP echo 
request, id
405, seq 1, length 64
03:06:18.427246 IP 203.0.113.2 > 172.16.0.2: ICMP echo 
reply, id 405,
seq 1, length 64
03:06:19.424157 IP 172.16.0.2 > 

Re: [vpp-dev] SNAT, nat44 with static mapping of whole subnet

[Filip Varga via lists.fd.io]

Hi,

From the first look i can see you are not enabling deterministic plugin like 
you are nat44 plugin. Secondly mixing both plugins isn't fully supported. There 
could be probably some issues. I am not completely sure about your use cases 
but using static mappings in this kind of scenario isn't viable solution.

If you just want each subnet to have different outside address. You should 
definitely use PAT - aka dynamic mapping and put all of those inside subnets in 
different VRF's after that add nat address for each vrf.

VRF1 192.168.0/24 -> 10.0.0.1
VRF2 192.168.1/24 - > 10.0.02
etc.

Be sure to set inside - vrf interfaces as inside and outside interface as 
outside.

Use nat44-ed plugin.

Best regards,
Filip

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Adrian Imboden
Sent: Thursday, January 20, 2022 1:55 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] SNAT, nat44 with static mapping of whole subnet
Importance: High

Hi all

I'm using vpp only recently and I am very happy with the way it works.

I am currently trying to replace my linux based router with a linux based vpp 
router.

- I use version release 21.10
- I have a small ipv4 /24 network and an additional static ip
- I have multiple subnets (test1, test2 in the example)

What I want to do is:
- Do SNAT
- Map each subnet to a single ipv4 address
- For my test: assume 10.10.100.50-10.10.100.52 are my public ips

What I have now is the following:
```
comment { == setting up testnet1 } create tap id 1 
host-if-name testnet1 host-ip4-addr 192.168.10.1/23 create loopback interface 
instance 1 set interface ip address loop1 192.168.10.255/23

set int l2 bridge tap1 1
set int l2 bridge loop1 1 bvi
set int l2 bridge TenGigabitEthernet8/0/0 1

comment { == setting up testnet2 } create tap id 2 
host-if-name testnet2 host-ip4-addr 192.168.12.1/23 create loopback interface 
instance 2 set interface ip address loop2 192.168.12.255/23

set int l2 bridge tap2 2
set int l2 bridge loop2 2 bvi


set interface ip address TenGigabitEtherneta/0/3 10.10.100.50/23 set interface 
ip address TenGigabitEtherneta/0/3 10.10.100.51/23 set interface ip address 
TenGigabitEtherneta/0/3 10.10.100.52/23

nat44 enable

nat44 add address 10.10.100.50
nat44 add address 10.10.100.51
nat44 add address 10.10.100.52
set interface nat44 out TenGigabitEtherneta/0/3 set interface nat44 in loop1 
set interface nat44 in loop2

ip route add 10.10.100.0/23 via TenGigabitEtherneta/0/3

comment { = port forwarding }
det44 add static mapping udp local 102.168.10.33 1234 external
10.10.100.51 1234


comment { enable all interfaces }
set int state tap1 up
set int state loop1 up
set int state TenGigabitEthernet8/0/0 up
set int state tap2 up
set int state loop2 up
set int state TenGigabitEtherneta/0/3 up
```



Now I have the problem that only 10.10.100.50 gets used (or until the 
ports are used up I assume).
det44 would support my use case with:
```
det44 add in 192.168.10.0/23 out 10.10.100.51/32
det44 add in 192.168.20.0/23 out 10.10.100.52/32
```

but det44 does not support port forwarding.

and nat44 only supports mapping each host one by one.
In my case I could add all IPs (512 IPs per net). Not pretty, but doable.


My questions:
Do I have an error in my thoughts?

Is there something like this planned?:
```
nat44 add static mapping local 192.168.10.0/23 external 10.10.100.51
```

If not, would this addition be ok? If yes, I may invest some coding time :)


Thanks very much and greetings
Adrian

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20779): https://lists.fd.io/g/vpp-dev/message/20779
Mute This Topic: https://lists.fd.io/mt/88549403/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] About nat44 translation with multi-tenancy

[Filip Varga via lists.fd.io]

Hello,

Not exactly. No vrf if nat plugin means ~0 vrf (all vrf’s) so it really depends 
on what vrfs are inside and outside nat interfaces placed on. There is support 
for multi-tenancy in the plugin. There are though some limitations in the 
plugin.

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io  On Behalf Of samjen
Sent: Friday, November 5, 2021 8:59 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] About nat44 translation with multi-tenancy
Importance: High

Hi all,

I’m testing multiple tenants using nat44-plugin.

The app version uses 21.10-release.
```
vpp# sh version
vpp v21-10-release built by root on 912b4c35bcd7 at 2021-10-27T12:54:43
```

I understand that when nat44 is enabled, one inside-vrf and one outside-vrf are 
specified (If not specified, vrf0 is specified).
In the following document:
https://docs.fd.io/vpp/21.10/dd/d6d/clicmd_src_plugins_nat_nat44-ed.html

Given that only one inside-vrf, outside-vrf can be specified, I think that 1vrf 
+ 1tenant and that nat44 translation can not accommodate multi-tenancy.

Here may be the question:
Is there a way to do nat44 translation in multi-tenancy without specifying 
ports?

Any suggestion would be helpful. Thank you.

—
Kyosuke Hori (@samjen)
VPP Development List Member
mail: kyosuke1117su...@gmail.com
—



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20475): https://lists.fd.io/g/vpp-dev/message/20475
Mute This Topic: https://lists.fd.io/mt/86964010/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] det44 plugin

[Filip Varga via lists.fd.io]

Hi Ben,

Thank you for pointing out the issue. Indeed it looks like the node runs just 
once. I will provide a patch shortly.

Best regards,
Filip Varga


-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Ben McKeegan
Sent: Monday, November 1, 2021 7:24 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] det44 plugin

Hello,

I am fairly new to VPP so please bear with me.  I am trying to use the
det44 NAT plugin on 21.06 but I am experiencing some difficulties with 
running out of ports.   It would appear that my det44 sessions are never 
removed despite passing the expire time.   For example, I have the 
following setting:

show det44 timeout
udp timeout: 300sec
tcp established timeout: 7440sec
tcp transitory timeout: 240sec
icmp timeout: 60sec

However, if I generate a series of ICMP pings from my test host and then run 
'show det44 sessions' I have a session listed for every individual ping packet, 
as expected, but these remain long after the 60 second timeout configured.  For 
example on my last test I sent a flood of 100 pings which generated 100 
sessions in the lists, all "state: icmp-active 
expire" with expiry times ranging from 171 to 173.   I have just sent 
another 100 pings and now have another 100 sessions with expiry times ranging 
from 2647 to 2650, and the original 100 sessions are still there still with 
expiry times from 171 to 173 so these have not been refreshed or expired.

I have taken a look at the source code of the plugin and I can that
det44_create_expire_walk_process() is called from det44_plugin_enable(). 
  This function appears to start a new vlib process with the 'main loop' 
  function det44_expire_walk_fn().

According to the documentation here
https://docs.fd.io/vpp/21.06/dd/d64/vlib__process__doc_8h.html I understand 
these despatch functions should be implemented as a while (1) {} loop that 
never ends.  However, my reading of the
det44_expire_walk_fn() function code is that it will only perform a single walk 
of the det44 data structures before returning to its caller.

Is this a bug in det44_expire_walk_fn(), is the documentation wrong or 
am I misreading it?   My hypothesis is that det44_expire_walk_fn() runs 
just once, when the plugin is first enabled (and the session table is already 
empty), and does not get run again thereafter.  Therefore, the sessions never 
get expired.


Regards,
Ben.





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20408): https://lists.fd.io/g/vpp-dev/message/20408
Mute This Topic: https://lists.fd.io/mt/86748366/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Assertion failure: thread_index == ed_value_get_thread_index ()

[Filip Varga via lists.fd.io]

Hello,

Are you on current master ?
Can you please specify configuraiton parameters APi/CLI.

Best regards,
Filip

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of S Rajkumar
Sent: Tuesday, April 27, 2021 2:03 PM
To: vpp-dev 
Subject: [vpp-dev] Assertion failure: thread_index == ed_value_get_thread_index 
()

Hello list,

I am trying to get NAT working in latest VPP and occasionally VPP hits the 
following assert:

ASSERT (thread_index == ed_value_get_thread_index ());

Here ed_value_get_thread_index () is 1  and thread_index is 2

Not sure how the packet is getting into thread 2, if it is not expected?

If any hints can be given, as to where I should be focusing, I can dig deeper 
and even submit a patch if I figure out whats wrong.

bt is as follows:

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x747b1921 in __GI_abort () at abort.c:79
#2  0x7574c1e9 in os_panic () at
/root/raj/vpp_up/vpp/src/vppinfra/unix-misc.c:177
#3  0x75690079 in debugger () at
/root/raj/vpp_up/vpp/src/vppinfra/error.c:84
#4  0x7568fdf7 in _clib_error (how_to_die=2, function_name=0x0, 
line_number=0,
 fmt=0x7fff6b84b31f "%s:%d (%s) assertion `%s' fails") at
/root/raj/vpp_up/vpp/src/vppinfra/error.c:143
#5  0x7fff6b772a1c in nat44_ed_out2in_fast_path_node_fn_inline
(vm=0x7fff79f6d440, node=0x7fff9020e680, frame=0x7fff79fee9c0,
 is_multi_worker=1) at
/root/raj/vpp_up/vpp/src/plugins/nat/nat44-ed/nat44_ed_out2in.c:896
#6  0x7fff6b77220a in nat44_ed_out2in_node_fn_hsw (vm=0x7fff79f6d440, 
node=0x7fff9020e680, frame=0x7fff79fee9c0)
 at /root/raj/vpp_up/vpp/src/plugins/nat/nat44-ed/nat44_ed_out2in.c:1354
#7  0x7605b586 in dispatch_node (vm=0x7fff79f6d440, 
node=0x7fff9020e680, type=VLIB_NODE_TYPE_INTERNAL,
 dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff79fee9c0,
last_time_stamp=3626105361929035)
 at /root/raj/vpp_up/vpp/src/vlib/main.c:1217
#8  0x7605bef7 in dispatch_pending_node (vm=0x7fff79f6d440, 
pending_frame_index=0, last_time_stamp=3626105361929035)
 at /root/raj/vpp_up/vpp/src/vlib/main.c:1376
#9  0x76055951 in vlib_main_or_worker_loop (vm=0x7fff79f6d440,
is_main=0) at /root/raj/vpp_up/vpp/src/vlib/main.c:1904
#10 0x760547f7 in vlib_worker_loop (vm=0x7fff79f6d440) at
/root/raj/vpp_up/vpp/src/vlib/main.c:2038
#11 0x760acfbd in vlib_worker_thread_fn (arg=0x7fff74db6fc0) at 
/root/raj/vpp_up/vpp/src/vlib/threads.c:1868
#12 0x756b6c64 in clib_calljmp () at
/root/raj/vpp_up/vpp/src/vppinfra/longjmp.S:123
#13 0x7fff5fffecd0 in ?? ()
#14 0x760a51e3 in vlib_worker_thread_bootstrap_fn
(arg=0x7fff74db6fc0) at /root/raj/vpp_up/vpp/src/vlib/threads.c:585
#15 0x7fff6f9d049a in eal_thread_loop ()
from
/root/raj/vpp_up/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/dpdk_plugin.so
#16 0x77bbb6db in start_thread (arg=0x7fff5700) at
pthread_create.c:463
#17 0x7489271f in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) f 5
#5  0x7fff6b772a1c in nat44_ed_out2in_fast_path_node_fn_inline
(vm=0x7fff79f6d440, node=0x7fff9020e680, frame=0x7fff79fee9c0,
 is_multi_worker=1) at
/root/raj/vpp_up/vpp/src/plugins/nat/nat44-ed/nat44_ed_out2in.c:896
896   ASSERT (thread_index == ed_value_get_thread_index ());

Some more exploration with gdb:

(gdb) p thread_index
$12 = 2
(gdb) p value0
$13 = {key = {72372793148251820, 18411084712597520390}, value = 4295018919}
(gdb) p lookup.saddr
$14 = {data = "\254\036\001\003", data_u32 = 50405036, as_u8 = 
"\254\036\001\003", as_u16 = {7852, 769}, as_u32 = 50405036}
(gdb) p lookup.daddr
$15 = {data = "\254\036\001\001", data_u32 = 16850604, as_u8 = 
"\254\036\001\001", as_u16 = {7852, 257}, as_u32 = 16850604}
(gdb) p lookup.sport
$16 = 20480
(gdb) p lookup.dport
$17 = 65409
(gdb) p/x lookup.sport
$18 = 0x5000
(gdb) p/x lookup.dport
$19 = 0xff81





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19290): https://lists.fd.io/g/vpp-dev/message/19290
Mute This Topic: https://lists.fd.io/mt/82401959/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Memory requirements for CGN

[Filip Varga via lists.fd.io]

Hi Macros,

Yes it might be possible.

Best regards,
Filip

From: mar...@mgiga.com.br 
Sent: Thursday, April 15, 2021 5:20 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
Cc: vpp-dev@lists.fd.io
Subject: RES: [vpp-dev] Memory requirements for CGN
Importance: High

Hi Filip,

Is it correct to think that "out of ports" error is a consequence of this 
limitation in number of simultaneous sessions?

Best Regars

Marcos

De: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> Em nome de Filip Varga via 
lists.fd.io
Enviada em: quinta-feira, 15 de abril de 2021 10:47
Para: mar...@mgiga.com.br<mailto:mar...@mgiga.com.br>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Assunto: Re: [vpp-dev] Memory requirements for CGN

Hi Marcos,

Q1.
Yes exactly.

Q2.
Yes there is difference between 20.05 and master. DET44 was previously one of 
the many features of NAT plugin now it is separate plugin. Still there was 
never possibility to change any inner configuration of memory allocation 
through API/CLI. NAT plugin was never well documented and for user it might not 
have bean obvious which configuration parameters would apply to which features. 
The parameter you are mentioning is used for Endpoint-Independnet NAT44 - 
previously default NAT mode.

Best regards,
Filip

From: mar...@mgiga.com.br<mailto:mar...@mgiga.com.br> 
mailto:mar...@mgiga.com.br>>
Sent: Thursday, April 15, 2021 2:52 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
Subject: RES: [vpp-dev] Memory requirements for CGN
Importance: High

Hi Filip,

When you say: "DET44 has a preddefined number of sessions per "user" and that 
is 1000." Does it mean that it doesn't matter the size of the pool, each user 
will be limited to 1000 session?

I'm running VPP 20.05, so I believe there is some differences from the master 
branch. In that case is "mode max translations per user" an available parameter 
?

Best Regards
De: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
Enviada em: quinta-feira, 15 de abril de 2021 04:58
Para: Marcos - Mgiga mailto:mar...@mgiga.com.br>>
Assunto: RE: [vpp-dev] Memory requirements for CGN

Hi Marcos,

Determnistic NAT - DET44 is a separate plugin. In the CLI you can find all 
commands containing det44 string.

For example:
det44
set det44
show det44

Before using DET44 plugin you will have to enable it via command line / api.
det44 plugin enable

  *   you can add optional arguments specifying inside and outside vrf
Except of configuring vrf's the plugin doesn't support any other configuration 
like for example specifying the size of hash tables.
Yes it is correct there is no hash table involved all sessions are prealocated 
in a vector.
DET44 has a preddefined number of sessions per "user" and that is 1000. You can 
specify only the size of the pool mapping.

rough estimate for determining sizeof session vektor would look something like 
this:
 *  * 
1000 *  * 24 bytes

Best regards,
Filip

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Marcos - Mgiga
Sent: Wednesday, April 14, 2021 9:57 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] Memory requirements for CGN

Hello VPP community,

According to Wiki page "Deterministic NAT prealocate vector with 1000 session 
slots (one session 15B) for each host from inside network range".

In case a expect to have about 16 k users, with an address share ratio of 1 to 
64 (1000 ports per user) is it correct to expect up to 2000 sessions per user ( 
1000 tcp and 1000 udp), so this prealocated slots can be an issue?

In the previous scenario is it correct to expect up to 480MB of memory usage ( 
2000 sessions * 16000 users * 15)?

When using deterministic mode is "nat mode max translations per user" a 
parameter that must be specified or is it calculated based on address sharing 
ratio?

It is  correct to claim that since deterministic mode have an static mapping 
from a range of internal IP/port to a range of external ip / port it doesnt 
require a bi hash table so "user hash memory", "user hash buckets"," 
translation hash memory" and translation hash memory" are not available to work 
in this mode. Does it make sense?


Thank you very Much

Best Regards



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19224): https://lists.fd.io/g/vpp-dev/message/19224
Mute This Topic: https://lists.fd.io/mt/82101219/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Memory requirements for CGN

[Filip Varga via lists.fd.io]

Hi Marcos,

Q1.
Yes exactly.

Q2.
Yes there is difference between 20.05 and master. DET44 was previously one of 
the many features of NAT plugin now it is separate plugin. Still there was 
never possibility to change any inner configuration of memory allocation 
through API/CLI. NAT plugin was never well documented and for user it might not 
have bean obvious which configuration parameters would apply to which features. 
The parameter you are mentioning is used for Endpoint-Independnet NAT44 - 
previously default NAT mode.

Best regards,
Filip

From: mar...@mgiga.com.br 
Sent: Thursday, April 15, 2021 2:52 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
Subject: RES: [vpp-dev] Memory requirements for CGN
Importance: High

Hi Filip,

When you say: "DET44 has a preddefined number of sessions per "user" and that 
is 1000." Does it mean that it doesn't matter the size of the pool, each user 
will be limited to 1000 session?

I'm running VPP 20.05, so I believe there is some differences from the master 
branch. In that case is "mode max translations per user" an available parameter 
?

Best Regards
De: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
Enviada em: quinta-feira, 15 de abril de 2021 04:58
Para: Marcos - Mgiga mailto:mar...@mgiga.com.br>>
Assunto: RE: [vpp-dev] Memory requirements for CGN

Hi Marcos,

Determnistic NAT - DET44 is a separate plugin. In the CLI you can find all 
commands containing det44 string.

For example:
det44
set det44
show det44

Before using DET44 plugin you will have to enable it via command line / api.
det44 plugin enable

  *   you can add optional arguments specifying inside and outside vrf
Except of configuring vrf's the plugin doesn't support any other configuration 
like for example specifying the size of hash tables.
Yes it is correct there is no hash table involved all sessions are prealocated 
in a vector.
DET44 has a preddefined number of sessions per "user" and that is 1000. You can 
specify only the size of the pool mapping.

rough estimate for determining sizeof session vektor would look something like 
this:
 *  * 
1000 *  * 24 bytes

Best regards,
Filip

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Marcos - Mgiga
Sent: Wednesday, April 14, 2021 9:57 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Memory requirements for CGN

Hello VPP community,

According to Wiki page "Deterministic NAT prealocate vector with 1000 session 
slots (one session 15B) for each host from inside network range".

In case a expect to have about 16 k users, with an address share ratio of 1 to 
64 (1000 ports per user) is it correct to expect up to 2000 sessions per user ( 
1000 tcp and 1000 udp), so this prealocated slots can be an issue?

In the previous scenario is it correct to expect up to 480MB of memory usage ( 
2000 sessions * 16000 users * 15)?

When using deterministic mode is "nat mode max translations per user" a 
parameter that must be specified or is it calculated based on address sharing 
ratio?

It is  correct to claim that since deterministic mode have an static mapping 
from a range of internal IP/port to a range of external ip / port it doesnt 
require a bi hash table so "user hash memory", "user hash buckets"," 
translation hash memory" and translation hash memory" are not available to work 
in this mode. Does it make sense?


Thank you very Much

Best Regards



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19211): https://lists.fd.io/g/vpp-dev/message/19211
Mute This Topic: https://lists.fd.io/mt/82101219/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] TCP timeout

[Filip Varga via lists.fd.io]

Hi,

Yes you should be able to use the command on v20.05 vpp build.
I don't think the Out of ports error is timeout related. I would check mapping 
configurations.

nat44 deterministic add in / out /

Best regards,
Filip Varga

From: mar...@mgiga.com.br 
Sent: Tuesday, April 6, 2021 2:16 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
vpp-dev@lists.fd.io
Subject: RES: [vpp-dev] TCP timeout
Importance: High

Hello Filip, thats great to know about this update .

Do you mind telling me if am I able to change this timeouts values in 20.05 
using the following sintax:  set nat timeout udp 300  tcp-established 900 
tcp-transitory 240 icmp 90 ?

Another point is I'm getting the "out of ports" error in the output of show 
error command. I suspect that is because of the default timeout value is too 
large . Do you have any thoughts about how can I work this out?


Best Regards

De: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> Em nome de Filip Varga via 
lists.fd.io
Enviada em: terça-feira, 6 de abril de 2021 08:00
Para: Marcos - Mgiga mailto:mar...@mgiga.com.br>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Assunto: Re: [vpp-dev] TCP timeout

Hello Marcos,

I am currently working on updating docs for all NAT plugins. These docs & 
introduction should be than located in the appropriate plugin directory in 
markdown file.

The wiki docs is outdated.
To use DET44 nat (it is a separate plugin now) you would have to first enable 
the plugin and then set the timeout. For example like this:

det44 plugin enable
set interface det44 inside <> outside <>
set det44 timeouts udp  tcp established  tcp transitory  icmp 

det44 add in / out /

  *   to add maps

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Marcos - Mgiga
Sent: Monday, April 5, 2021 9:56 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [vpp-dev] TCP timeout

Hello VPP experts,

I would like to set a certain timeout value  for TCP established connections. 
According to VPP Wiki ( https://wiki.fd.io/view/VPP/NAT) in deterministic mode, 
that would be achieved by using the following sintax:

set nat44 deterministic timeout [udp  | tcp-established  | 
tcp-transitory  | icmp  | reset]

In my enviroment ( Using VPP 20.05) I just was able to change that value using 
: set nat timeout udp 300  tcp-established 900 tcp-transitory 240 icmp 90. I 
would like to confirm is this behavior is correct.

Best Regards

Marcos

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19110): https://lists.fd.io/g/vpp-dev/message/19110
Mute This Topic: https://lists.fd.io/mt/81873610/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] TCP timeout

[Filip Varga via lists.fd.io]

Hello Marcos,

I am currently working on updating docs for all NAT plugins. These docs & 
introduction should be than located in the appropriate plugin directory in 
markdown file.

The wiki docs is outdated.
To use DET44 nat (it is a separate plugin now) you would have to first enable 
the plugin and then set the timeout. For example like this:

det44 plugin enable
set interface det44 inside <> outside <>
set det44 timeouts udp  tcp established  tcp transitory  icmp 

det44 add in / out /

  *   to add maps

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io  On Behalf Of Marcos - Mgiga
Sent: Monday, April 5, 2021 9:56 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] TCP timeout

Hello VPP experts,

I would like to set a certain timeout value  for TCP established connections. 
According to VPP Wiki ( https://wiki.fd.io/view/VPP/NAT) in deterministic mode, 
that would be achieved by using the following sintax:

set nat44 deterministic timeout [udp  | tcp-established  | 
tcp-transitory  | icmp  | reset]

In my enviroment ( Using VPP 20.05) I just was able to change that value using 
: set nat timeout udp 300  tcp-established 900 tcp-transitory 240 icmp 90. I 
would like to confirm is this behavior is correct.

Best Regards

Marcos

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19107): https://lists.fd.io/g/vpp-dev/message/19107
Mute This Topic: https://lists.fd.io/mt/81873610/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] nat-ed and max translations per user

[Filip Varga via lists.fd.io]

Hi Xuo,

That is a fair point but i don’t it should be NAT’s responsibility implementing 
DoS prevention mechanisms. This would require having some sort of list of ip 
addresses for all dynamic clients and that would greatly decrease performance 
of NAT. This kind of protection could be achieved through some other plugin 
preceding NAT making it more modular.

Best regards,
Filip

From: Xuo Guoto 
Sent: Tuesday, March 30, 2021 10:26 AM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
Cc: vpp-dev@lists.fd.io
Subject: RE: [vpp-dev] nat-ed and max translations per user
Importance: High

Thanks Filip!

Its clear that ED do not support per ip session limiting, but one question 
remaining is that what if one private IP generates so many sessions that entire 
sessions of that thread is taken by sessions from that IP?

This can happen in the case of a virus or worm  infected machine in LAN.

How can VPP defend itself in this case?

X.

‐‐‐ Original Message ‐‐‐
On Monday, March 29, 2021 9:27 PM, Filip Varga -X (fivarga - PANTHEON TECH SRO 
at Cisco) mailto:fiva...@cisco.com>> wrote:


Hello,



Max translations per user is a NAT44 EI (endpoint independent) plugin concept. 
EI plugin was previously mode of NAT. NAT would run either EI or ED (endpoint 
dependent). If you are interested in running EI mode please use the plugin 
configuration as follows:



nat44 ei enable

nat44 ei add interface

nat44 ei add static interface

...



All nat44 ei plugin commands are prefixed with ei.



In NAT44 ed plugin you are not able to specify session limiting based on 
internal ip address in othre words user.

You can only specify per vrf limit if you like so:

set nat44 session limit



P.S. nat44 ed commands will be also prefixed with ed in the near future.



Best regards,

Filip Varga





From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Xuo Guoto via 
lists.fd.io
Sent: Monday, March 29, 2021 5:48 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] nat-ed and max translations per user
Importance: High



Hello,



While going through the nat configuration of latest VPP, I find that max 
translations per user is missing and is kind of replaced by "nat44 enable  
sessions 40 endpoint-dependent" which limit max translations per thread.



Is there any equivalent config of max translations per user in latest VPP? If 
not, how to prevent one user (possibly infected) from using up all the sessions 
of a thread and create a DoS situation?



X.


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19057): https://lists.fd.io/g/vpp-dev/message/19057
Mute This Topic: https://lists.fd.io/mt/81699736/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] nat-ed and max translations per user

[Filip Varga via lists.fd.io]

Hello,

Max translations per user is a NAT44 EI (endpoint independent) plugin concept. 
EI plugin was previously mode of NAT. NAT would run either EI or ED (endpoint 
dependent). If you are interested in running EI mode please use the plugin 
configuration as follows:

nat44 ei enable
nat44 ei add interface
nat44 ei add static interface
...

All nat44 ei plugin commands are prefixed with ei.

In NAT44 ed plugin you are not able to specify session limiting based on 
internal ip address in othre words user.
You can only specify per vrf limit if you like so:
set nat44 session limit

P.S. nat44 ed commands will be also prefixed with ed in the near future.

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io  On Behalf Of Xuo Guoto via 
lists.fd.io
Sent: Monday, March 29, 2021 5:48 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] nat-ed and max translations per user
Importance: High

Hello,

While going through the nat configuration of latest VPP, I find that max 
translations per user is missing and is kind of replaced by "nat44 enable  
sessions 40 endpoint-dependent" which limit max translations per thread.

Is there any equivalent config of max translations per user in latest VPP? If 
not, how to prevent one user (possibly infected) from using up all the sessions 
of a thread and create a DoS situation?

X.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19048): https://lists.fd.io/g/vpp-dev/message/19048
Mute This Topic: https://lists.fd.io/mt/81699736/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] NAT44 how to control external address assignment from pool?

[Filip Varga via lists.fd.io]

Hello,

For clarification i will explain how the nat is devided.

At this point NAT functionality is devided in multiple sub plugins because of 
it's previous complexity and issues with it.
We have det44 and nat44 plugins that are completely separate. The whole 
separation is still in progress. So changes in nat44 like picking up pool 
allocation algorithm or anything else will not affect det44 plugin. These two 
plugins operate completely independently and share just some NAT library for 
common stuff.

Regarding the det44 allocation algorithm. No at this point it is not supported 
to pick up a new randomly selected address as you are asking. Det44 is / should 
act in predetermined way so logging is not required.

I will look further in the code and plugins if i can help you find some 
solution.

Best regards,
Filip

From: Юрий Иванов 
Sent: Wednesday, February 10, 2021 8:47 AM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
vpp-dev@lists.fd.io
Subject: RE: [vpp-dev] NAT44 how to control external address assignment from 
pool?
Importance: High

Hi Filip, thanks for reply.

This is only for host mapping and looks that it can be done with det44 plugin - 
very strange btw that it operates separatly from standard nat44 (meaning that I 
do need to configure nat at all to use it).

My problem is different, when I set pool i.e. 1.0.0.1-1.0.0.100 all clients 
always get the last address from the pool (.100) until external IP run out of 
ports and only after that client will get .99 IP untile this IP will run out of 
ports and etc.

Is there way to select new random address from pool for new client and after 
that use this randomly selected same source-/destination-address for each 
client connection.

Now it leads to problems with  Google 'Unusual Traffic' Block/Captcha, because 
it utilizes several IP addresses where most IP from pool leave unused.


От: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
Отправлено: 9 февраля 2021 г. 13:54
Кому: Юрий Иванов mailto:format_...@outlook.com>>; 
vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>>
Тема: RE: [vpp-dev] NAT44 how to control external address assignment from pool?


Hi,



If you are looking for option to specify exact outside translation address from 
a specific pool. You should try :



nat44 add static mapping ... exact 



Also supported by API.

This will give you exact address picked from pool.



Best regards,

Filip Varga



From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of  ??
Sent: Monday, February 8, 2021 11:04 AM
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] NAT44 how to control external address assignment from 
pool?
Importance: High



Just an update, to perform manual translation we should do:



det44 plugin enable

det44 add in 10.0.1.0/29 out 1.0.0.7/32

det44 add in 10.0.1.16/29 out 1.0.0.5/32

...

set interface det44 inside GigabitEthernet0/5/0 outside GigabitEthernet0/4/0



Ignoring specific nat configuration.



Nevertheless, maybe there is some option to select different IP addres from 
pool?



От: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> от имени Юрий Иванов 
mailto:format_...@outlook.com>>
Отправлено: 7 февраля 2021 г. 12:10
Кому: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>>
Тема: [vpp-dev] NAT44 how to control external address assignment from pool?



Hi,



I've got basic nat setup in a lab which works as expected except extarnat IP 
address allocation.



My clients, behind NAT, has addresses:

vpp# show ip neighbors

Time   IPFlags  Ethernet
  Interface

   1421.592910.0.1.30  D00:50:79:66:68:00 
GigabitEthernet0/5/0

   1424.360910.0.1.55  D00:50:79:66:68:02 
GigabitEthernet0/5/0

   1423.165010.0.1.41  D00:50:79:66:68:01 
GigabitEthernet0/5/0

   1389.292910.0.1.20  D00:50:79:66:68:04 
GigabitEthernet0/5/0

   1377.144910.0.1.2   D00:50:79:66:68:03 
GigabitEthernet0/5/0



All can reach external 1.0.0.2 IP which looks good, but for all client external 
address is 1.0.0.100 (the last IP in a range).

I think it will be more better to randomly select different address in range 
like nftables do with this configuration stranza:

"ip saddr 10.0.0.0/8 oif "vlan2" snat to 1.0.0.2-1.0.0.100 persistent"



After changing

vpp# nat addr-port-assignment-alg map-e psid 10 psid-offset 6 psid-len 6

the only difference that external address become 1.0.0.1 for all clients.



I of cource can map host to external address with:

vpp# nat44 add static mapping  local 10.0.1.2 external 1.0.0.10

But with 30 thousands 

Re: [vpp-dev] NAT44 how to control external address assignment from pool?

[Filip Varga via lists.fd.io]

Hi,

If you are looking for option to specify exact outside translation address from 
a specific pool. You should try :

nat44 add static mapping ... exact 

Also supported by API.
This will give you exact address picked from pool.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of  ??
Sent: Monday, February 8, 2021 11:04 AM
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] NAT44 how to control external address assignment from 
pool?
Importance: High

Just an update, to perform manual translation we should do:

det44 plugin enable
det44 add in 10.0.1.0/29 out 1.0.0.7/32
det44 add in 10.0.1.16/29 out 1.0.0.5/32
...
set interface det44 inside GigabitEthernet0/5/0 outside GigabitEthernet0/4/0

Ignoring specific nat configuration.

Nevertheless, maybe there is some option to select different IP addres from 
pool?

От: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> от имени Юрий Иванов 
mailto:format_...@outlook.com>>
Отправлено: 7 февраля 2021 г. 12:10
Кому: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>>
Тема: [vpp-dev] NAT44 how to control external address assignment from pool?

Hi,

I've got basic nat setup in a lab which works as expected except extarnat IP 
address allocation.

My clients, behind NAT, has addresses:
vpp# show ip neighbors
Time   IPFlags  Ethernet
  Interface
   1421.592910.0.1.30  D00:50:79:66:68:00 
GigabitEthernet0/5/0
   1424.360910.0.1.55  D00:50:79:66:68:02 
GigabitEthernet0/5/0
   1423.165010.0.1.41  D00:50:79:66:68:01 
GigabitEthernet0/5/0
   1389.292910.0.1.20  D00:50:79:66:68:04 
GigabitEthernet0/5/0
   1377.144910.0.1.2   D00:50:79:66:68:03 
GigabitEthernet0/5/0

All can reach external 1.0.0.2 IP which looks good, but for all client external 
address is 1.0.0.100 (the last IP in a range).
I think it will be more better to randomly select different address in range 
like nftables do with this configuration stranza:
"ip saddr 10.0.0.0/8 oif "vlan2" snat to 1.0.0.2-1.0.0.100 persistent"

After changing
vpp# nat addr-port-assignment-alg map-e psid 10 psid-offset 6 psid-len 6
the only difference that external address become 1.0.0.1 for all clients.

I of cource can map host to external address with:
vpp# nat44 add static mapping  local 10.0.1.2 external 1.0.0.10
But with 30 thousands clients it will very strange idea to map every host route 
manually.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18706): https://lists.fd.io/g/vpp-dev/message/18706
Mute This Topic: https://lists.fd.io/mt/80449794/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Static NAT rule only match specific source ip or port

[Filip Varga via lists.fd.io]

Hi Date,

Ok now i understand where you are trying to get. At this point this kind of 
matching is not supported. At this point you can either get match combination 
of external-host external-host-port && external external-port in out2in-only 
twice-nat scenario. I will try to look into it and do some testying of the 
configuraiton. At the moment though we are in a situation of simplyfing NAT 
because of it’s complexity. That means separating features into sub plugins. We 
were though working on policy based matching for NAT this is on hold right now.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Date Huang
Sent: Wednesday, September 30, 2020 1:29 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
Nathan Skrzypczak 
Cc: vpp-dev 
Subject: Re: [vpp-dev] Static NAT rule only match specific source ip or port
Importance: High

Continued from previous mail

For example this rule
vpp# nat44 add static mapping tcp local Z.Z.Z.Z 5566 external Y.Y.Y.Y 8080 
external-host X.X.X.X 1234

  1.  check if match source IP and Port in packet with rule "external-host 
X.X.X.X 1234"
  2.  check if match dst IP and Port in packet "external Y.Y.Y.Y 8080"
  3.  If 1 & 2 match success, do NAT translate dst IP and Port to "local 
Z.Z.Z.Z 5566"
  4.  If no match, do nothing.

Regards,
Date


寄件者: 黃 宇強 mailto:tjjh89...@hotmail.com>>
寄件日期: 2020年9月30日 下午 07:17
收件者: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>; Nathan Skrzypczak 
mailto:nathan.skrzypc...@gmail.com>>
副本: vpp-dev mailto:vpp-dev@lists.fd.io>>
主旨: RE: [vpp-dev] Static NAT rule only match specific source ip or port

Hi Filip


(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566

  *   outside host address should be uniquelly matched to the NAT rule for the 
(NAT ADDRESS) to (INSIDE HOST) translation ?

Yes
Other OUTSIDE HOST will not be matched to this NAT rule and will not perform 
NAT to translate.

If below

Rules: (using VPP debug CLI-like, Schematic command, not existing)
and we can specify only external-host(outside host)=X.X.X.X will match this 
function.
vpp# nat44 add static mapping tcp local Z.Z.Z.Z 5566 external Y.Y.Y.Y 8080 
external-host X.X.X.X

Action:

(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566

(OUTSIDE HOST) X2.X2.X2.X2 :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 (DO NOTHING)


Rules:
or we can sepcify external-host ip and port
vpp# nat44 add static mapping tcp local Z.Z.Z.Z 5566 external Y.Y.Y.Y 8080 
external-host X.X.X.X 1234

Action:

(OUTSIDE HOST) X.X.X.X :1234 -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to 
(OUTSIDE HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566\

(OUTSIDE HOST) X.X.X.X :(other port) -> (NAT ADDRESS)Y.Y.Y.Y:8080 (DO NOTHING)

(OUTSIDE HOST) X2.X2.X2.X2 :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 (DO NOTHING)

It will a little bit like this:
nat44 add static mapping tcp|udp|icmp local  [] external 
(|) [] [external-host (|all) []] [vrf 
] [twice-nat] [out2in-only] [del]
to assign source ip and port (or "external-host" used in session tables) to 
static NAT rule directly.


Have you also tried twice-nat out2in-only ? I think there should be this 
feature if ofc. i correctly understand what you are trying to accomplish.
twice-nat didn't meet my requirement.
I want to keep source ip and port
but I want to add more match entries to NAT rule, source ip, source port, dst 
ip, dst port match this rule, and them perform this rule. (source ip and source 
port can be "don't care" attribute)
For now, I only saw match "dst ip and port" and then perform NAT rule to 
translate.


Thanks and appreciate
Regards,
Date




寄件者: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
寄件日期: 2020年9月30日 下午 06:52
收件者: 黃 宇強 mailto:tjjh89...@hotmail.com>>; Nathan 
Skrzypczak mailto:nathan.skrzypc...@gmail.com>>
副本: vpp-dev mailto:vpp-dev@lists.fd.io>>
主旨: RE: [vpp-dev] Static NAT rule only match specific source ip or port


Hi Date,



Just to verify you want something like this ?



(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566

  *   outside host address should be uniquelly matched to the NAT rule for the 
(NAT ADDRESS) to (INSIDE HOST) translation ?



Have you also tried twice-nat out2in-only ? I think there should be this 
feature if ofc. i correctly understand what you are trying to accomplish.



Best regards,

Filip Varga



From: 黃 宇強 mailto:tjjh89...@hotmail.com>>
Sent: Wednesday, September 30, 2020 12:48 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>; Nathan Skrzypczak 
mailto:nathan.skrzypc...@gmail.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: 回覆: [vpp-dev] Static NAT rule only match specific source ip or port
Importance: High



Hi Filip

Re: [vpp-dev] Static NAT rule only match specific source ip or port

[Filip Varga via lists.fd.io]

Hi Date,

Just to verify you want something like this ?


(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566

  *   outside host address should be uniquelly matched to the NAT rule for the 
(NAT ADDRESS) to (INSIDE HOST) translation ?

Have you also tried twice-nat out2in-only ? I think there should be this 
feature if ofc. i correctly understand what you are trying to accomplish.

Best regards,
Filip Varga

From: 黃 宇強 
Sent: Wednesday, September 30, 2020 12:48 PM
To: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
Nathan Skrzypczak 
Cc: vpp-dev 
Subject: 回覆: [vpp-dev] Static NAT rule only match specific source ip or port
Importance: High

Hi Filip

We already used ED mode to re-use the 8080 port

But we still cannot limit this rule only for specific source IP.
Do you have any idea or advice for this?

Thanks and appreciate
Regards,
Date


寄件者: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>
寄件日期: 2020年9月30日 下午 06:41
收件者: Date Huang mailto:tjjh89...@hotmail.com>>; Nathan 
Skrzypczak mailto:nathan.skrzypc...@gmail.com>>
副本: vpp-dev mailto:vpp-dev@lists.fd.io>>
主旨: RE: [vpp-dev] Static NAT rule only match specific source ip or port


Hi Date,



I would suggest looking into ED NAT out2in only translations.



Just to point out

(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566



Let’s go with NAT in VPP



set interface nat44 in LAN out WAN

nat44 add address (NAT ADDRESS)

nat44 add static mapping tcp local (INSIDE HOST) (INSIDE PORT) external (NAT 
ADDRESS) 8080 out2in-only .



ED – endpoint dependent NAT will let you reuse 8080 port for other connections.



Best regards,

Filip Varga





From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Date Huang
Sent: Wednesday, September 30, 2020 9:55 AM
To: Nathan Skrzypczak 
mailto:nathan.skrzypc...@gmail.com>>
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] Static NAT rule only match specific source ip or port
Importance: High



Hi Nathan



[sorry send this mail again, I found that I didn't cc vpp-dev mailing list]



so glad to see your reply.

My English is poor, so if there is some confusing term, please tell me.
I will try my best to describe it with some graph.

Just to be sure to understand your use case, you want to have the following 
translations happening :

*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> Z.Z.Z.Z:5566

* (not X.X.X.X):* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> W.W.W.W:1234

Do you also need other NATing to happen (e.g. deterministic / sourceNATing ?)

Also are you in a home-gateway scenario a.k.a do you have inside & outside 
interfaces respectively

for Y.Y.Y.Y and Z.Z.Z.Z / W.W.W.W or should those be reachable from all 
interfaces ?

If I used netfilter/iptables to explain.
It will more like this.

iptables -t nat -D PREROUTING -p tcp -s X.X.X.X -d Y.Y.Y.Y --dport 8080 -j DNAT 
--to-destination Z.Z.Z.Z:5566

when (not x.x.x.x) source IP comes in, don't do any DNAT for that.
Only do DNAT for src_IP=X.X.X.X
I also need this for LAN device to be out.
"iptables -t nat -A POSTROUTING -o WAN_INTERFACE -j MASQUERADE"

I only have two interface (named LAN, WAN)
X.X.X.X, X2.X2.X2.X2 will be device ip outside WAN

Y.Y.Y.Y will be WAN's ip
Z.Z.Z.Z and W.W.W.W will be device inside LAN.

And I want to re-use "--dport 8080"

Detail:
1. First I want to translate Y.Y.Y.Y:8080 mapping to Z.Z.Z.Z:5566 if source ip 
is X.X.X.X
setup Static NAT rule "match source ip=X.X.X.X and translate dst ip and port 
from `Y.Y.Y.Y:8080` to `Z.Z.Z.Z:5566`"
*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> Z.Z.Z.Z:5566 (same 
as your description)
*. (no X.X.X.X):* -> Y.Y.Y.Y:8080 NAT will not do anything.

2. X.X.X.X established connection to Z.Z.Z.Z:5566 with TCP, VPP will keep 
session in session tables
and record translated rule "Y.Y.Y.Y:8080 -> Z.Z.Z.Z:5566" and "source IP need 
to be X.X.X.X:port_xx"
So even there is no rule, VPP will use session to determine translate rule 
first before searching Static NAT rule.
TCP connection will not be closed or denied by NAT function.

3. Remove Static NAT rule after establishing connection.
remove Static NAT rule "match source ip=X.X.X.X and translate dst ip and port 
from `Y.Y.Y.Y:8080` to `Z.Z.Z.Z:5566`"
TCP connection still need to be kept.
X.X.X.X:port_xx still can connect Z.Z.Z.Z:5566 (same established connection)

4. I want to translate Y.Y.Y.Y:8080 mapping to W.W.W.W:1234 if source ip is 
X2.X2.X2.X2
setup Static NAT rule "match source ip=X2.X2.X2.X2 and translate dst ip and 
port from `Y.Y.Y.Y:8080` to `W.W.W.W:1234`"
*  X2.X2.X2.X2 :* -> Y.Y.Y.Y:8080 translated to X2.X2.X2.X2:* -> 
W.W.W.W:1234 (port overloading via conntrack if netfilter)
*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> 

Re: [vpp-dev] Static NAT rule only match specific source ip or port

[Filip Varga via lists.fd.io]

Hi Date,

I would suggest looking into ED NAT out2in only translations.

Just to point out
(OUTSIDE HOST) X.X.X.X :* -> (NAT ADDRESS)Y.Y.Y.Y:8080 translated to (OUTSIDE 
HOST)X.X.X.X:* -> (INSIDE HOST)Z.Z.Z.Z:5566

Let’s go with NAT in VPP

set interface nat44 in LAN out WAN
nat44 add address (NAT ADDRESS)
nat44 add static mapping tcp local (INSIDE HOST) (INSIDE PORT) external (NAT 
ADDRESS) 8080 out2in-only .

ED – endpoint dependent NAT will let you reuse 8080 port for other connections.

Best regards,
Filip Varga


From: vpp-dev@lists.fd.io  On Behalf Of Date Huang
Sent: Wednesday, September 30, 2020 9:55 AM
To: Nathan Skrzypczak 
Cc: vpp-dev 
Subject: Re: [vpp-dev] Static NAT rule only match specific source ip or port
Importance: High

Hi Nathan

[sorry send this mail again, I found that I didn't cc vpp-dev mailing list]

so glad to see your reply.
My English is poor, so if there is some confusing term, please tell me.
I will try my best to describe it with some graph.
Just to be sure to understand your use case, you want to have the following 
translations happening :
*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> Z.Z.Z.Z:5566
* (not X.X.X.X):* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> W.W.W.W:1234
Do you also need other NATing to happen (e.g. deterministic / sourceNATing ?)
Also are you in a home-gateway scenario a.k.a do you have inside & outside 
interfaces respectively
for Y.Y.Y.Y and Z.Z.Z.Z / W.W.W.W or should those be reachable from all 
interfaces ?
If I used netfilter/iptables to explain.
It will more like this.
iptables -t nat -D PREROUTING -p tcp -s X.X.X.X -d Y.Y.Y.Y --dport 8080 -j DNAT 
--to-destination Z.Z.Z.Z:5566
when (not x.x.x.x) source IP comes in, don't do any DNAT for that.
Only do DNAT for src_IP=X.X.X.X
I also need this for LAN device to be out.
"iptables -t nat -A POSTROUTING -o WAN_INTERFACE -j MASQUERADE"
I only have two interface (named LAN, WAN)
X.X.X.X, X2.X2.X2.X2 will be device ip outside WAN
Y.Y.Y.Y will be WAN's ip
Z.Z.Z.Z and W.W.W.W will be device inside LAN.

And I want to re-use "--dport 8080"

Detail:
1. First I want to translate Y.Y.Y.Y:8080 mapping to Z.Z.Z.Z:5566 if source ip 
is X.X.X.X
setup Static NAT rule "match source ip=X.X.X.X and translate dst ip and port 
from `Y.Y.Y.Y:8080` to `Z.Z.Z.Z:5566`"
*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> Z.Z.Z.Z:5566 (same 
as your description)
*. (no X.X.X.X):* -> Y.Y.Y.Y:8080 NAT will not do anything.

2. X.X.X.X established connection to Z.Z.Z.Z:5566 with TCP, VPP will keep 
session in session tables
and record translated rule "Y.Y.Y.Y:8080 -> Z.Z.Z.Z:5566" and "source IP need 
to be X.X.X.X:port_xx"
So even there is no rule, VPP will use session to determine translate rule 
first before searching Static NAT rule.
TCP connection will not be closed or denied by NAT function.

3. Remove Static NAT rule after establishing connection.
remove Static NAT rule "match source ip=X.X.X.X and translate dst ip and port 
from `Y.Y.Y.Y:8080` to `Z.Z.Z.Z:5566`"
TCP connection still need to be kept.
X.X.X.X:port_xx still can connect Z.Z.Z.Z:5566 (same established connection)

4. I want to translate Y.Y.Y.Y:8080 mapping to W.W.W.W:1234 if source ip is 
X2.X2.X2.X2
setup Static NAT rule "match source ip=X2.X2.X2.X2 and translate dst ip and 
port from `Y.Y.Y.Y:8080` to `W.W.W.W:1234`"
*  X2.X2.X2.X2 :* -> Y.Y.Y.Y:8080 translated to X2.X2.X2.X2:* -> 
W.W.W.W:1234 (port overloading via conntrack if netfilter)
*  X.X.X.X :* -> Y.Y.Y.Y:8080 translated to X.X.X.X:* -> Z.Z.Z.Z:5566 
(X still can connect to Z, even there is no Static NAT rule, if session is 
kept. this translation rule will be stored in session not Static NAT rule until 
TCP connection is timeout)
* (no X.X.X.X) or (no X2.X2.X2.X2):* -> Y.Y.Y.Y:8080, NAT will not do anything.

5. X2.X2.X2.X2 established connection to W.W.W.W:1234 with TCP, VPP will keep 
session in session tables
and record translated rule "Y.Y.Y.Y:8080 -> W.W.W.W:1234" and "source IP need 
to be X2.X2.X2.X2:port_xx2"
So even there is no rule, VPP will use session to determine translate rule 
first before searching Static NAT rule.
TCP connection will not be closed or denied by NAT function.

6. and so on.

---

for now, we can only do this in NAT ED mode
(any ip):* -> Y.Y.Y.Y:8080 translated to (any ip):* -> W.W.W.W:1234 (we cannot 
limit specific source ip to apply NAT translation, and deny other source ip for 
security issue)

Thanks and appreciate for your reply.
Regards,
Date


寄件者: Nathan Skrzypczak 
mailto:nathan.skrzypc...@gmail.com>>
寄件日期: 2020年9月30日 下午 03:17
收件者: Date Huang mailto:tjjh89...@hotmail.com>>
副本: vpp-dev mailto:vpp-dev@lists.fd.io>>
主旨: Re: [vpp-dev] Static NAT rule only match specific source ip or port

Hi Date,

Sorry for the late reply,
I'm not sure this will be supported by the existing NAT plugin but it might be 
doable with few additions.

Just to be sure to understand your use case, you want to have the 

Re: [vpp-dev] VPP Deterministic NAT Same in/out Interface Not Matching Session

[Filip Varga via lists.fd.io]

Hi Joshua,

Try to run your setup on master branch with recent changes. Few weeks ago i 
have moved deterministic feature out of snat plugin. Now deterministic feature 
is running in it’s separate plugin. Please check det44 sub plugin. If you have 
any issues feel free to write me back.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Joshua Moore
Sent: Tuesday, September 29, 2020 11:38 PM
To: Joshua Moore 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] VPP Deterministic NAT Same in/out Interface Not Matching 
Session
Importance: High

Yep, definitely looks like this is unsupported. I moved to separated in/out 
interfaces and packets started flowing appropriately.



On Tue, Sep 29, 2020 at 2:35 PM Joshua Moore via 
lists.fd.io 
mailto:jcm...@lists.fd.io>> wrote:
Hello,

Do we know if the same in/out interface for NAT in deterministic mode is 
supported in VPP? I am seeing a strange behavior where return traffic is not 
matching the session. For example, see session below where a DNS request is 
initially captured outbound to 8.8.8.8: 
http://jcm.me/session.txt

As you can see, this is recorded as 1.1.1.0:2325 for the 
outside translated IP/port:

in 100.65.0.2:35573 out 
1.1.1.0:2325 external host 8.8.8.8:53 
state: udp-active expire: 869

When reply comes back from 8.8.8.8 though to 1.1.1.0:2325 
the packet is dropped. I captured this in the trace: http://jcm.me/trace.txt

The only thing I can think of here that may be a little odd with my setup is 
that I am using the same interface for inside and outside. See my VPP config 
below:
jmoore@test:~$ cat /etc/vpp/setup.gate
set interface ip address loop0 1.1.1.1/29
set interface state loop0 up
set interface ip address GigabitEthernet3/0/0 
172.16.30.250/24
set int nat44 in GigabitEthernet3/0/0 out GigabitEthernet3/0/0
nat44 deterministic add in 100.65.0.0/22 out 
1.1.1.0/29
set interface state GigabitEthernet3/0/0 up
ip route add 0.0.0.0/0 via 172.16.30.1

Any reason that the trace is showing the below?
00:09:23:047897: drop
  nat44-det-in2out: No translation



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17606): https://lists.fd.io/g/vpp-dev/message/17606
Mute This Topic: https://lists.fd.io/mt/77203973/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] nat44 static mapping does not work in endpoint-dependent mode and workers > 1

[Filip Varga via lists.fd.io]

Hi,

Yes snat plugin is still maintained but it is undergoing big refactor and 
separation of features. Some of them already happend some will be done just in 
a few days. If you still encounter the issue reported here in the community 
feel free to provide a patch and put me on review though i have it on my TODO 
list.

Best regards,
Filip

From: vpp-dev@lists.fd.io  On Behalf Of Dengfeng Liu
Sent: Tuesday, September 29, 2020 3:57 AM
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] nat44 static mapping does not work in endpoint-dependent 
mode and workers > 1

thanks for your kindly response.
I don't know whether vpp dev-team still maintian snat plugin, I found some bug 
about snat plugin and also submit it in this forum. anyway no furhter response 
about my message. It really confuse me, does vpp community still welcome us to 
join the game ?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17574): https://lists.fd.io/g/vpp-dev/message/17574
Mute This Topic: https://lists.fd.io/mt/77054012/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] nat44 static mapping does not work in endpoint-dependent mode and workers > 1

[Filip Varga via lists.fd.io]

Hi,

I will look into it.

Best regards,
Filip

From: vpp-dev@lists.fd.io  On Behalf Of Dengfeng Liu
Sent: Thursday, September 24, 2020 11:44 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] nat44 static mapping does not work in endpoint-dependent 
mode and workers > 1
Importance: High

Dear all,

It seems that nat44 static mapping feature does not work in ed mode and workers 
>1.

my setup is :

vpp  20.09

startup.conf
unix {
interactive
nodaemon
cli-listen  /run/vpp/cli.sock
cli-no-pager
poll-sleep-usec 100
}
api-trace {
on
}
api-segment {
gid vpp
}
socksvr {
default
}
cpu {
main-core 1
corelist-workers 2,3
}
dpdk {
uio-driver vfio-pci
dev :02:05.0 { name G0}
}
nat {
endpoint-dependent
}

vpp config:
set int state G0 up
create tap id 1
set int state tap1 up
set int ip addr tap1 1.1.1.1/24
set dhcp client intfc G0

set int nat44 out G0 output-feature
nat44 add int address G0
nat44 forwarding enable

nat44 add static map tcp local 1.1.1.2 80 external 192.168.1.155 80

after config, show all information
DBGvpp# sh int addr
G0 (up):
  L3 192.168.1.155/24
local0 (dn):
tap1 (up):
  L3 1.1.1.1/24
DBGvpp# sh nat44 int
interfaces  interface
DBGvpp# sh nat44 interfaces
NAT44 interfaces:
 G0 output-feature out
DBGvpp# sh nat44 addresses
NAT44 pool addresses:
192.168.1.155
  tenant VRF independent
  0 busy other ports
  0 busy udp ports
  1 busy tcp ports
  0 busy icmp ports
NAT44 twice-nat pool addresses:
DBGvpp# sh nat44 static mappings
NAT44 static mappings:
 tcp local 1.1.1.2:80 external 192.168.1.155:80 vrf 0

when http client start to request ,  device 1.1.1.2 can receive tcp syn and 
respond syn+ack but drop by vpp for reason of  "non-SYN packet try to create 
session
"
I read the code and find in node nat44-in2out-output-worker-handoff, it should 
use snat_static_mapping_match to check whether the packet is static mapping or 
not, otherwise it will give wrong thread_index and when packet is processed in 
node  nat44-ed-in2out-output-slowpath, it will raise error of "non-SYN packet 
try to create session".
the following patch will fix above bug, If anybody find problem of the patch or 
have better idea, please let me know, thanks.
diff --git a/src/plugins/nat/nat.c b/src/plugins/nat/nat.c
index 15c767c..f4147f2 100644
--- a/src/plugins/nat/nat.c
+++ b/src/plugins/nat/nat.c
@@ -1599,8 +1599,8 @@ nat44_lb_static_mapping_add_del_local (ip4_address_t 
e_addr, u16 e_port,
  .src_address = local->addr,
};
tsm = vec_elt_at_index (sm->per_thread_data,
-   sm->worker_in2out_cb (, m->fib_index,
- 0));
+   sm->worker_in2out_cb (,
+ m->fib_index, 0));
  }
   else
  tsm = vec_elt_at_index (sm->per_thread_data, sm->num_workers);
@@ -3254,8 +3254,8 @@ format_ed_session_kvp (u8 * s, va_list * args)
 }

 static u32
-snat_get_worker_in2out_cb (ip4_header_t * ip0, u32 rx_fib_index0,
-u8 is_output)
+snat_get_worker_in2out_cb (ip4_header_t * ip0,
+u32 rx_fib_index0, u8 is_output)
 {
   snat_main_t *sm = _main;
   u32 next_worker_index = 0;
@@ -3358,8 +3358,8 @@ snat_get_worker_out2in_cb (vlib_buffer_t * b, 
ip4_header_t * ip0,
 }

 static u32
-nat44_ed_get_worker_in2out_cb (ip4_header_t * ip, u32 rx_fib_index,
-u8 is_output)
+nat44_ed_get_worker_in2out_cb (ip4_header_t * ip,
+u32 rx_fib_index, u8 is_output)
 {
   snat_main_t *sm = _main;
   u32 next_worker_index = sm->first_worker_index;
@@ -3383,8 +3383,6 @@ nat44_ed_get_worker_in2out_cb (ip4_header_t * ip, u32 
rx_fib_index,
  ,
   };

-  udp = ip4_next_header (ip);
-
   switch (vec_len (sm->outside_fibs))
  {
  case 0:
@@ -3411,8 +3409,27 @@ nat44_ed_get_worker_in2out_cb (ip4_header_t * ip, u32 
rx_fib_index,
break;
  }

-  init_ed_k (, ip->src_address, udp->src_port, ip->dst_address,
-  udp->dst_port, fib_index, ip->protocol);
+  udp = ip4_next_header (ip);
+
+  ip4_address_t sm_addr;
+  u16 sm_port;
+  u32 sm_fib_index;
+  u32 nat_proto = ip_proto_to_nat_proto (ip->protocol);
+  u16 sport = udp->src_port;
+  u16 dport = udp->dst_port;
+  if (snat_static_mapping_match
+   (sm, ip->src_address, sport, rx_fib_index, nat_proto,
+_addr, _port, _fib_index, 0, 0, 0, 0, 0, 0, 0))
+ {
+
+   init_ed_k (, ip->src_address, sport, ip->dst_address,
+  dport, fib_index, ip->protocol);
+ }
+  else
+ {
+   init_ed_k (, sm_addr, sm_port, ip->dst_address, dport,
+  sm_fib_index, ip->protocol);
+ }

   if (PREDICT_TRUE (!clib_bihash_search_16_8 (>out2in_ed,
, )))
diff --git a/src/plugins/nat/out2in_ed.c b/src/plugins/nat/out2in_ed.c
index 8eef1e4..205947f 100644
--- a/src/plugins/nat/out2in_ed.c
+++ b/src/plugins/nat/out2in_ed.c
@@ -300,9 +300,8 @@ create_session_for_static_mapping_ed (snat_main_t * sm,
o2i_fib_index, ip->protocol, thread_index, s - tsm->sessions);
   ctx.now = now;
   ctx.thread_index = thread_index;
-  if (clib_bihash_add_or_overwrite_stale_16_8 (>out2in_ed, ,

Re: [vpp-dev] ping not able to receive replies on vpp interface

[Filip Varga via lists.fd.io]

Hi Pack,

I am currently working on larger patch. If you need the fix as soon as possible 
feel free to contribute and add me as reviewer in gerrit. I will be able to 
look into it on the end of the weekend.

Best regards,
Filip

From: Pac Ette 
Sent: Wednesday, September 23, 2020 7:47 PM
To: Pac Ette 
Cc: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
Dave Barach (dbarach) ; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface

Hi Filip,

Sorry to bother you.

If you have any update on this please let me know. I can perform any test you 
need.

Thanks!

On Thu, Sep 17, 2020 at 10:05 AM Pac Ette via lists.fd.io 
mailto:gmail@lists.fd.io>> wrote:
Hi Filip,

Sounds good.

Thanks!

On Thu, Sep 17, 2020 at 1:27 AM Filip Varga -X (fivarga - PANTHEON TECH SRO at 
Cisco) mailto:fiva...@cisco.com>> wrote:
Hi Pac,

Yes the idea is that you should be able to ping NAT outside interface from the 
adjacent side of the connection. This was previously working. I will create a 
ticket for the issue and look in to it shortly.

If you have any other issues with NAT please let me know.

Best regards,
Filip

From: Pac Ette mailto:pktno...@gmail.com>>
Sent: Wednesday, September 16, 2020 10:16 PM
To: pktno...@gmail.com
Cc: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) 
mailto:fiva...@cisco.com>>; Dave Barach (dbarach) 
mailto:dbar...@cisco.com>>; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface
Importance: High

Hi Filip,

sorry, amendment to my last email:

Your suggestion worked. But ping from linux stack (tap interface) is not 
working because there is no NAT in/out anymore.

I am using VPP with NAT. Is it not possible for ping to work with NAT?

I noticed something. If I add back:

set interface nat44 in loop0 in loop1
set interface nat44 out wan1

In addition to vpp-to-server not working, I am also unable to ping from linux 
server to vpp box.

$ ping 10.200.1.7
PING 10.200.1.7 (10.200.1.7) from 10.200.1.1 eth2: 56(84) bytes of data.
--- 10.200.1.7 ping statistics ---
80 packets transmitted, 0 received, 100% packet loss, time 80882ms

Thanks!

On Wed, Sep 16, 2020 at 1:10 PM Pac Ette via lists.fd.io 
mailto:gmail@lists.fd.io>> wrote:
Hi Filip,

Your suggestion worked. But I am using VPP with NAT. Is it not possible for 
ping to work with NAT?

I noticed something. When NAT is set, in addition to vpp to server not working, 
I am also unable to ping from linux server to vpp box.

If I add back:
set interface nat44 in loop0 in loop1
set interface nat44 out wan1

linux (10.200.1.1) --> to --> wan1 (vpp) does not work either.

Thanks!


On Wed, Sep 16, 2020 at 12:35 PM Filip Varga -X (fivarga - PANTHEON TECH SRO at 
Cisco) mailto:fiva...@cisco.com>> wrote:
Hi Pac,

Try removing from your configuration following two lines:

set interface nat44 in loop0 in loop1
set interface nat44 out wan1

Now try to ping from linux & vpp. From linux ping vpp wan interface 10.200.1.7 
and from vpp linux host 10.200.1.1 (don’t forget to specify the source 
interface, this should be your wan interface in vpp).

# ping  source 

Best regards,
Filip

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Pac Ette
Sent: Wednesday, September 16, 2020 6:14 PM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface

Hi Dave,

I had a suspicion that my configs might be wrong. But, why would packets go 
through the NAT plugin when I am pinging from vppctl and directly on the wan1 
port - I was thinking these packets don't need to be NAT-ed. Here are my 
configs and testing results.

## Configs
cat vpp.conf
set interface state wan1 up
set interface state lan3 up
set interface state lan2 up
set interface state lan0 up

set dhcp client intfc wan1 hostname test-ccB

loopback create
set interface l2 bridge loop0 1 bvi
set interface ip address loop0 10.90.0.1/16
set interface state loop0 up

create tap id 0 host-ip4-addr 10.90.0.2/16 host-if-name 
lstack1
set interface l2 bridge tap0 1
set interface state tap0 up

loopback create
set interface l2 bridge loop1 2 bvi
set interface ip address loop1 10.100.0.1/16
set interface state loop1 up

set interface l2 bridge lan0 2
set interface l2 bridge lan2 2
set interface l2 bridge lan3 2

create tap id 1 host-ip4-addr 10.100.0.2/16 host-if-name 
lstack2
set interface l2 bridge tap1 2
set interface state tap1 up

nat44 add interface address wan1
set interface nat44 in loop0 in loop1
set interface nat44 out wan1
--
vpp# sh nat44 addresses
NAT44 pool addresses:
10.200.1.7
  tenant VRF independent
  0 busy other ports
 

Re: [vpp-dev] ping not able to receive replies on vpp interface

[Filip Varga via lists.fd.io]

Hi Pac,

Yes the idea is that you should be able to ping NAT outside interface from the 
adjacent side of the connection. This was previously working. I will create a 
ticket for the issue and look in to it shortly.

If you have any other issues with NAT please let me know.

Best regards,
Filip

From: Pac Ette 
Sent: Wednesday, September 16, 2020 10:16 PM
To: pktno...@gmail.com
Cc: Filip Varga -X (fivarga - PANTHEON TECH SRO at Cisco) ; 
Dave Barach (dbarach) ; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface
Importance: High

Hi Filip,

sorry, amendment to my last email:

Your suggestion worked. But ping from linux stack (tap interface) is not 
working because there is no NAT in/out anymore.

I am using VPP with NAT. Is it not possible for ping to work with NAT?

I noticed something. If I add back:

set interface nat44 in loop0 in loop1
set interface nat44 out wan1

In addition to vpp-to-server not working, I am also unable to ping from linux 
server to vpp box.

$ ping 10.200.1.7
PING 10.200.1.7 (10.200.1.7) from 10.200.1.1 eth2: 56(84) bytes of data.
--- 10.200.1.7 ping statistics ---
80 packets transmitted, 0 received, 100% packet loss, time 80882ms

Thanks!

On Wed, Sep 16, 2020 at 1:10 PM Pac Ette via lists.fd.io 
mailto:gmail@lists.fd.io>> wrote:
Hi Filip,

Your suggestion worked. But I am using VPP with NAT. Is it not possible for 
ping to work with NAT?

I noticed something. When NAT is set, in addition to vpp to server not working, 
I am also unable to ping from linux server to vpp box.

If I add back:
set interface nat44 in loop0 in loop1
set interface nat44 out wan1

linux (10.200.1.1) --> to --> wan1 (vpp) does not work either.

Thanks!


On Wed, Sep 16, 2020 at 12:35 PM Filip Varga -X (fivarga - PANTHEON TECH SRO at 
Cisco) mailto:fiva...@cisco.com>> wrote:
Hi Pac,

Try removing from your configuration following two lines:

set interface nat44 in loop0 in loop1
set interface nat44 out wan1

Now try to ping from linux & vpp. From linux ping vpp wan interface 10.200.1.7 
and from vpp linux host 10.200.1.1 (don’t forget to specify the source 
interface, this should be your wan interface in vpp).

# ping  source 

Best regards,
Filip

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Pac Ette
Sent: Wednesday, September 16, 2020 6:14 PM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface

Hi Dave,

I had a suspicion that my configs might be wrong. But, why would packets go 
through the NAT plugin when I am pinging from vppctl and directly on the wan1 
port - I was thinking these packets don't need to be NAT-ed. Here are my 
configs and testing results.

## Configs
cat vpp.conf
set interface state wan1 up
set interface state lan3 up
set interface state lan2 up
set interface state lan0 up

set dhcp client intfc wan1 hostname test-ccB

loopback create
set interface l2 bridge loop0 1 bvi
set interface ip address loop0 10.90.0.1/16
set interface state loop0 up

create tap id 0 host-ip4-addr 10.90.0.2/16 host-if-name 
lstack1
set interface l2 bridge tap0 1
set interface state tap0 up

loopback create
set interface l2 bridge loop1 2 bvi
set interface ip address loop1 10.100.0.1/16
set interface state loop1 up

set interface l2 bridge lan0 2
set interface l2 bridge lan2 2
set interface l2 bridge lan3 2

create tap id 1 host-ip4-addr 10.100.0.2/16 host-if-name 
lstack2
set interface l2 bridge tap1 2
set interface state tap1 up

nat44 add interface address wan1
set interface nat44 in loop0 in loop1
set interface nat44 out wan1
--
vpp# sh nat44 addresses
NAT44 pool addresses:
10.200.1.7
  tenant VRF independent
  0 busy other ports
  18 busy udp ports
  0 busy tcp ports
  0 busy icmp ports
NAT44 twice-nat pool addresses:

vpp# sh nat44 interfaces
NAT44 interfaces:
 loop0 in
 loop1 in
 wan1 out

vpp# sh dhcp client
[0] wan1 state DHCP_BOUND installed 1 addr 10.200.1.7/24 
gw 10.200.1.1 server 10.200.1.1 dns 8.8.8.8

linux stack
$ ip route
default via 10.90.0.1 dev  lstack1
10.90.0.0/16 dev lstack1 proto kernel scope link src 
10.90.0.2
10.100.0.0/16 dev lstack2 proto kernel scope link src 
10.100.0.2
--
## Testing

pings via linux stack works but pings via vppctl do not.

ping via vppctl:
vpp# ping 10.200.1.1
Statistics: 5 sent, 0 received, 100% packet loss

ping via linux stack:
$ ping 10.200.1.1
PING 10.200.1.1 (10.200.1.1) 56(84) bytes of data.
64 bytes from 10.200.1.1: icmp_seq=1 ttl=63 time=1.01 ms
64 bytes from 10.200.1.1: icmp_seq=2 ttl=63 time=0.321 ms
--- 10.200.1.1 

Re: [vpp-dev] ping not able to receive replies on vpp interface

[Filip Varga via lists.fd.io]

Hi Pac,

Try removing from your configuration following two lines:

set interface nat44 in loop0 in loop1
set interface nat44 out wan1

Now try to ping from linux & vpp. From linux ping vpp wan interface 10.200.1.7 
and from vpp linux host 10.200.1.1 (don’t forget to specify the source 
interface, this should be your wan interface in vpp).

# ping  source 

Best regards,
Filip

From: vpp-dev@lists.fd.io  On Behalf Of Pac Ette
Sent: Wednesday, September 16, 2020 6:14 PM
To: Dave Barach (dbarach) 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] ping not able to receive replies on vpp interface

Hi Dave,

I had a suspicion that my configs might be wrong. But, why would packets go 
through the NAT plugin when I am pinging from vppctl and directly on the wan1 
port - I was thinking these packets don't need to be NAT-ed. Here are my 
configs and testing results.

## Configs
cat vpp.conf
set interface state wan1 up
set interface state lan3 up
set interface state lan2 up
set interface state lan0 up

set dhcp client intfc wan1 hostname test-ccB

loopback create
set interface l2 bridge loop0 1 bvi
set interface ip address loop0 10.90.0.1/16
set interface state loop0 up

create tap id 0 host-ip4-addr 10.90.0.2/16 host-if-name 
lstack1
set interface l2 bridge tap0 1
set interface state tap0 up

loopback create
set interface l2 bridge loop1 2 bvi
set interface ip address loop1 10.100.0.1/16
set interface state loop1 up

set interface l2 bridge lan0 2
set interface l2 bridge lan2 2
set interface l2 bridge lan3 2

create tap id 1 host-ip4-addr 10.100.0.2/16 host-if-name 
lstack2
set interface l2 bridge tap1 2
set interface state tap1 up

nat44 add interface address wan1
set interface nat44 in loop0 in loop1
set interface nat44 out wan1
--
vpp# sh nat44 addresses
NAT44 pool addresses:
10.200.1.7
  tenant VRF independent
  0 busy other ports
  18 busy udp ports
  0 busy tcp ports
  0 busy icmp ports
NAT44 twice-nat pool addresses:

vpp# sh nat44 interfaces
NAT44 interfaces:
 loop0 in
 loop1 in
 wan1 out

vpp# sh dhcp client
[0] wan1 state DHCP_BOUND installed 1 addr 10.200.1.7/24 
gw 10.200.1.1 server 10.200.1.1 dns 8.8.8.8

linux stack
$ ip route
default via 10.90.0.1 dev  lstack1
10.90.0.0/16 dev lstack1 proto kernel scope link src 
10.90.0.2
10.100.0.0/16 dev lstack2 proto kernel scope link src 
10.100.0.2
--
## Testing

pings via linux stack works but pings via vppctl do not.

ping via vppctl:
vpp# ping 10.200.1.1
Statistics: 5 sent, 0 received, 100% packet loss

ping via linux stack:
$ ping 10.200.1.1
PING 10.200.1.1 (10.200.1.1) 56(84) bytes of data.
64 bytes from 10.200.1.1: icmp_seq=1 ttl=63 time=1.01 ms
64 bytes from 10.200.1.1: icmp_seq=2 ttl=63 time=0.321 ms
--- 10.200.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.321/0.670/1.019/0.349 ms

Thanks!

On Wed, Sep 16, 2020 at 4:32 AM Dave Barach (dbarach) 
mailto:dbar...@cisco.com>> wrote:
The dispatch pcap trace shows that “nat44-ed-out2in-slowpath” drops the reply. 
Since the request never visits the nat plugin, there is no translation set up 
to process the reply.

Please check your configuration.

Dave

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Pac Ette
Sent: Tuesday, September 15, 2020 11:40 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] ping not able to receive replies on vpp interface

Hi folks,

vpp is unable to ping on an interface:
vpp# ping 10.200.1.1 source wan1
Statistics: 5 sent, 0 received, 100% packet loss

current setup:
vpp 20.05

linux machine (A) <> vpp machine (B)
  10.200.1.1  <> 10.200.1.7

[cid:image002.png@01D68C71.35D2D150]

So the server at 10.200.1.1 is replying but vpp is showing 100% packet loss.

I am also attaching a vpp dispatch pcap file.

Thanks for the help!


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17425): https://lists.fd.io/g/vpp-dev/message/17425
Mute This Topic: https://lists.fd.io/mt/76880903/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Q about VPP NAT

[Filip Varga via lists.fd.io]

Hi Nick,

The behavior you are seeing is correct. Based on specificatoin TCP sessions / 
tcp ports should not get reused before transitory timeout passes. WAIT-CLOSED 
means that these sessions are closed but still waiting for timeout to expire 
before address and port can be reused. The are not able to be reused by timed 
out connections or by new connections until they enter CLOSED state. NAT44 
implementation doesn’t do scavenging rather maintains LRU list logically order 
for cheap reuse of CLOSED sessions. The reaping you are talking about 
(scavenging) looks fine at small number of sessions but exponentionally 
increases by number of sessions – if we are talking in thousand or milion 
sessions for exmaple.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Nick Zavaritsky
Sent: Thursday, September 10, 2020 10:53 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Q about VPP NAT
Importance: High

Dear VPP hackers,

I need your advice concerning configuring and possibly extremely ending the NAT 
in VPP.

We are currently using nat44 in endpoint-dependent mode. We are witnessing TCP 
sessions piling up even though clients close connections gracefully. These 
lingering sessions are categorised as WAIT-CLOSING by show nat44 summary. After 
a timeout they are considered CLOSED and could get reaped (lazily).

I suspect that this behaviour is actually correct, since the NAT seeing FIN/ACK 
passing by doesn't imply that the packets were actually delivered. Please 
confirm.

It looks like RST doesn't terminate a NAT session (doesn't put it in 
WAIT-CLOSING state), are there reasons for that as well?

Best,
N
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#17361): https://lists.fd.io/g/vpp-dev/message/17361
Mute This Topic: https://lists.fd.io/mt/76751794/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] nat44 bug - created nat sessions aren't automatically cleaned up

[Filip Varga via lists.fd.io]

Hi Venkat,

Yes this is resolved. At this point NAT44 ED is using port overloading 
algorithm with LRU list of session for reusability of the expired ones.

Best regards,
Filip Varga

From: vpp-dev@lists.fd.io  On Behalf Of Venkat
Sent: Thursday, September 3, 2020 5:25 PM
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] nat44 bug - created nat sessions aren't automatically 
cleaned up
Importance: High

was there any resolution provided for this issue>?
I noticed https://jira.fd.io/browse/VPP-1795  

does show any fix or activity. Please comment.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#17333): https://lists.fd.io/g/vpp-dev/message/17333
Mute This Topic: https://lists.fd.io/mt/3887/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] NAT bugix related to in2out/out2in handoff node index

[Filip Varga via Lists.Fd.Io]

Hello,

Thank you for your contribution Elias.

Best regards,
Filip

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Elias Rudberg
Sent: Friday, March 13, 2020 3:00 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] NAT bugix related to in2out/out2in handoff node index

Hello,

While working on moving from VPP 19.08 to 20.01 we found that NAT was no longer 
working and it seems to be due to a bug in src/plugins/nat/nat.c for the 
dynamic endpoint-independent case, here:

sm->handoff_out2in_index = snat_in2out_node.index;
sm->handoff_in2out_index = snat_out2in_node.index;

As I understand it, handoff_out2in_index is supposed to be the node index of 
the out2in node, but it is set to the in2out node index instead. And the other 
way around, in2out/in2out are mixed up in those two lines.

I pushed a fix to gerrit, it's just those two lines that are changed:
https://gerrit.fd.io/r/c/vpp/+/25856

If you agree, can this fix please be accepted into master and also into the 
stable/2001 branch?

Best regards,
Elias
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15774): https://lists.fd.io/g/vpp-dev/message/15774
Mute This Topic: https://lists.fd.io/mt/71926127/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Doubts about NAT #nat44 #nat

[Filip Varga via Lists.Fd.Io]

Hi,

Could you please send me output of `show version` command and your NAT 
configuration. There is a use case when this behavior is expected.

Best regards,
Filip

[https://www.cisco.com/c/dam/m/en_us/signaturetool/images/logo/Cisco_Logo_no_TM_Cisco_Blue-RGB_43px.png]
Filip Varga
Engineer - Software
fiva...@cisco.com
Tel:




Cisco Systems, Inc.



Slovakia
cisco.com
[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]
Think before you print.
This email may contain confidential and privileged material for the sole use of 
the intended recipient. Any review, use, distribution or disclosure by others 
is strictly prohibited. If you are not the intended recipient (or authorized to 
receive for the recipient), please contact the sender by reply email and delete 
all copies of this message.
Please click 
here
 for Company Registration Information.


From: vpp-dev@lists.fd.io  On Behalf Of 
baixiaop...@ekean.cn
Sent: Tuesday, December 17, 2019 8:25 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Doubts about NAT #nat44 #nat

I'm testing the NAT feature, and after configuring the NAT, the trace 
information is as follows below，
Why is the nat44-in2out-worker-handoff node followed by the nat44-out2in node 
instead of the nat44-in2out node?

P.S: trace information
Packet 1

02:23:07:173961: af-packet-input
  af_packet: hw_if_index 7 next-index 4
tpacket2_hdr:
  status 0x2001 len 98 snaplen 98 mac 66 net 80
  sec 0x5df85360 nsec 0x2e9e94c8 vlan 0 vlan_tpid 0
02:23:07:173970: ethernet-input
  IP4: 42:64:0a:23:df:a3 -> de:ad:00:00:00:00
02:23:07:173975: l2-input
  l2-input: sw_if_index 7 dst de:ad:00:00:00:00 src 42:64:0a:23:df:a3
02:23:07:173979: l2-learn
  l2-learn: sw_if_index 7 dst de:ad:00:00:00:00 src 42:64:0a:23:df:a3 bd_index 3
02:23:07:173983: l2-fwd
  l2-fwd:   sw_if_index 7 dst de:ad:00:00:00:00 src 42:64:0a:23:df:a3 bd_index 
3result [0x7000a, 10] static age-not bvi
02:23:07:173987: ip4-input
  ICMP: 10.10.0.1 -> 10.10.1.1
tos 0x00, ttl 64, length 84, checksum 0xcdaa
fragment id 0x57e9, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x5686
02:23:07:173991: nat44-in2out-worker-handoff
  NAT44_IN2OUT_WORKER_HANDOFF : next-worker 1 trace index 0
02:23:07:173999: nat44-out2in
  NAT44_OUT2IN: sw_if_index 10, next index 0, session index -1
02:23:07:174005: error-drop
  rx:loop0
02:23:07:174008: drop
  nat44-out2in: no translation


Best Regards!!!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14908): https://lists.fd.io/g/vpp-dev/message/14908
Mute This Topic: https://lists.fd.io/mt/68755389/21656
Mute #nat44: https://lists.fd.io/mk?hashtag=nat44=1480452
Mute #nat: https://lists.fd.io/mk?hashtag=nat=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-