Re: [vpp-dev] VXLAN Tunnel IF Names

[Florin Coras]

.. aand then one of the calls in the batch fails. Why not use a shim layer that 
on the one hand fits what Neale was saying and on the other it accepts batches 
and handles failures. Say .. netconf? :-)

Florin

> On Feb 7, 2018, at 2:10 PM, Ed Warnicke <hagb...@gmail.com> wrote:
> 
> Neale,
> 
> I think this really really fundamentally comes down to the ability to 
> configure in batch.  If I know I want to send something to vpp that creates 
> an interface and configures it in a *batch*, I can't know when I send that 
> batch configuration what sw_index will be.  So I either have to break up the 
> batch or have a way of getting a deterministic handle (name) to configure the 
> interface being created.When you are writing a local control plane, you 
> often just say "fuck it, I'm not going to be able to batch.   But if you are 
> writing a remote control plane (high latency per message) or have a user copy 
> pasting over configs via a CLI (not our CLI, but another CLI you write) 
> etc... batching is crucially important.
> 
> Ed
> 
> 
> 
> 
> On Wed, Feb 7, 2018 at 7:18 AM Neale Ranns (nranns) <nra...@cisco.com 
> <mailto:nra...@cisco.com>> wrote:
>  
> 
> But all control planes need to know the sw_if_index provided by VPP as it is 
> this value (and not the interface name) that one needs to subsequently 
> configure the interface. So why the reliance of VPP’s given name? AFIAK VPP’s 
> given name is only important when interpreting CLI output.
> 
> All control planes are free to choose whatever naming schemes they like, but 
> they must maintain a control-plane-name to sw_if_index mapping. And to make 
> VPP CLI output more useful, also maintain a control-plane-name to VPP-name 
> mapping. In other words, X, is always the control-plane-name.
> 
>  
> 
> /neale
> 
>  
> 
>  
> 
> From: "John Lo (loj)" <l...@cisco.com <mailto:l...@cisco.com>>
> Date: Wednesday, 7 February 2018 at 15:34
> To: "Neale Ranns (nranns)" <nra...@cisco.com <mailto:nra...@cisco.com>>, Jon 
> Loeliger <j...@netgate.com <mailto:j...@netgate.com>>
> Cc: vpp-dev <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>>
> Subject: RE: [vpp-dev] VXLAN Tunnel IF Names
> 
>  
> 
> I guess Jon’s point was the control plane need to know exactly what X would 
> be and not rely on VPP to provide the sw_if_index or query VPP for the name 
> after the VXLAN create request.  This makes the control plane CLI 
> script/template support more straight forward.  In the case of mixed usage 
> where a requested instance is not available, I wonder how control plane 
> should handle the error condition if it does not check the result of the 
> VXLAN create request…   -John
> 
>  
> 
> From: Neale Ranns (nranns) 
> Sent: Wednesday, February 07, 2018 5:55 AM
> To: Jon Loeliger <j...@netgate.com <mailto:j...@netgate.com>>; John Lo (loj) 
> <l...@cisco.com <mailto:l...@cisco.com>>
> Cc: vpp-dev <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>>
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
>  
> 
> Hi Jon,
> 
>  
> 
> I feel I must ask J
> 
> “That  has to match a VPP interface by name.  But what name?”
> 
> why does it need to match a VPP interface name? shouldn’t  = MyVxlan23. 
> Your control plane maintains mappings between MyVxlan23 and VPP’s sw_if_index.
> 
>  
> 
> /neale
> 
>  
> 
>  
> 
> From: <vpp-dev-boun...@lists.fd.io <mailto:vpp-dev-boun...@lists.fd.io>> on 
> behalf of Jon Loeliger <j...@netgate.com <mailto:j...@netgate.com>>
> Date: Friday, 2 February 2018 at 22:08
> To: "John Lo (loj)" <l...@cisco.com <mailto:l...@cisco.com>>
> Cc: vpp-dev <vpp-dev@lists.fd.io <mailto:vpp-dev@lists.fd.io>>
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
>  
> 
> On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) <l...@cisco.com 
> <mailto:l...@cisco.com>> wrote:
> 
> Hi Jon,
> 
>  
> 
> Hi John, 
> 
>  
> 
> Thanks for taking the time to get back to me on this!
> 
>  
> 
> All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
> created tunnel.
> 
>  
> 
> I know.  I get that.  It works.
> 
>  
> 
> The name of the tunnel is then followed by a number being the instance or 
> index to the tunnel struct vector. Thus, the first VXLAN tunnel created is 
> called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is 
> incrementing unless tunnels are deleted and created again where a newly 
> created tunnel will reuse the last deleted tunnel, hence its name. If a

Re: [vpp-dev] VXLAN Tunnel IF Names

[Ed Warnicke]

Neale,

I think this really really fundamentally comes down to the ability to
configure in batch.  If I know I want to send something to vpp that creates
an interface and configures it in a *batch*, I can't know when I send that
batch configuration what sw_index will be.  So I either have to break up
the batch or have a way of getting a deterministic handle (name) to
configure the interface being created.When you are writing a local
control plane, you often just say "fuck it, I'm not going to be able to
batch.   But if you are writing a remote control plane (high latency per
message) or have a user copy pasting over configs via a CLI (not our CLI,
but another CLI you write) etc... batching is crucially important.

Ed




On Wed, Feb 7, 2018 at 7:18 AM Neale Ranns (nranns) <nra...@cisco.com>
wrote:

>
>
> But all control planes need to know the sw_if_index provided by VPP as it
> is this value (and not the interface name) that one needs to subsequently
> configure the interface. So why the reliance of VPP’s given name? AFIAK
> VPP’s given name is only important when interpreting CLI output.
>
> All control planes are free to choose whatever naming schemes they like,
> but they must maintain a control-plane-name to sw_if_index mapping. And to
> make VPP CLI output more useful, also maintain a control-plane-name to
> VPP-name mapping. In other words, X, is always the control-plane-name.
>
>
>
> /neale
>
>
>
>
>
> *From: *"John Lo (loj)" <l...@cisco.com>
> *Date: *Wednesday, 7 February 2018 at 15:34
> *To: *"Neale Ranns (nranns)" <nra...@cisco.com>, Jon Loeliger <
> j...@netgate.com>
> *Cc: *vpp-dev <vpp-dev@lists.fd.io>
> *Subject: *RE: [vpp-dev] VXLAN Tunnel IF Names
>
>
>
> I guess Jon’s point was the control plane need to know exactly what X
> would be and not rely on VPP to provide the sw_if_index or query VPP for
> the name after the VXLAN create request.  This makes the control plane CLI
> script/template support more straight forward.  In the case of mixed usage
> where a requested instance is not available, I wonder how control plane
> should handle the error condition if it does not check the result of the
> VXLAN create request…   -John
>
>
>
> *From:* Neale Ranns (nranns)
> *Sent:* Wednesday, February 07, 2018 5:55 AM
> *To:* Jon Loeliger <j...@netgate.com>; John Lo (loj) <l...@cisco.com>
> *Cc:* vpp-dev <vpp-dev@lists.fd.io>
> *Subject:* Re: [vpp-dev] VXLAN Tunnel IF Names
>
>
>
> Hi Jon,
>
>
>
> I feel I must ask J
>
> “That  has to match a VPP interface by name.  But what name?”
>
> why does it need to match a VPP interface name? shouldn’t  = MyVxlan23.
> Your control plane maintains mappings between MyVxlan23 and VPP’s
> sw_if_index.
>
>
>
> /neale
>
>
>
>
>
> *From: *<vpp-dev-boun...@lists.fd.io> on behalf of Jon Loeliger <
> j...@netgate.com>
> *Date: *Friday, 2 February 2018 at 22:08
> *To: *"John Lo (loj)" <l...@cisco.com>
> *Cc: *vpp-dev <vpp-dev@lists.fd.io>
> *Subject: *Re: [vpp-dev] VXLAN Tunnel IF Names
>
>
>
> On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) <l...@cisco.com> wrote:
>
> Hi Jon,
>
>
>
> Hi John,
>
>
>
> Thanks for taking the time to get back to me on this!
>
>
>
> All VPP tunnel creation uses the mechanism of returning a sw_if_index of
> the created tunnel.
>
>
>
> I know.  I get that.  It works.
>
>
>
> The name of the tunnel is then followed by a number being the instance or
> index to the tunnel struct vector. Thus, the first VXLAN tunnel created is
> called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is
> incrementing unless tunnels are deleted and created again where a newly
> created tunnel will reuse the last deleted tunnel, hence its name. If all
> previously deleted tunnels are used up, then the tunnel name number will
> start incrementing again.
>
>
>
> Right.
>
>
>
> I am not sure if it is feasible to follow this behavior to “predict”
> tunnel name.
>
>
>
> It is not.
>
>
>
> The problem is not just "predict it", but the user has to be
>
> able to know the associated SW IF name at the time that
>
> the (vxlan tunnel) create API call happens.
>
>
>
> The reason for that is because the very next thing the
>
> API call user is going to want to do is configure that interface.
>
> Short of interrogating the system, there is no way to know
>
> the IF name.  (I understand that the name can be obtained
>
> from the sw_if_index.  That's not what I mean.)
>
>
>
> Consider this series of PEUDO API calls that ar

Re: [vpp-dev] VXLAN Tunnel IF Names

[Neale Ranns (nranns)]

But all control planes need to know the sw_if_index provided by VPP as it is 
this value (and not the interface name) that one needs to subsequently 
configure the interface. So why the reliance of VPP’s given name? AFIAK VPP’s 
given name is only important when interpreting CLI output.
All control planes are free to choose whatever naming schemes they like, but 
they must maintain a control-plane-name to sw_if_index mapping. And to make VPP 
CLI output more useful, also maintain a control-plane-name to VPP-name mapping. 
In other words, X, is always the control-plane-name.

/neale


From: "John Lo (loj)" <l...@cisco.com>
Date: Wednesday, 7 February 2018 at 15:34
To: "Neale Ranns (nranns)" <nra...@cisco.com>, Jon Loeliger <j...@netgate.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: RE: [vpp-dev] VXLAN Tunnel IF Names

I guess Jon’s point was the control plane need to know exactly what X would be 
and not rely on VPP to provide the sw_if_index or query VPP for the name after 
the VXLAN create request.  This makes the control plane CLI script/template 
support more straight forward.  In the case of mixed usage where a requested 
instance is not available, I wonder how control plane should handle the error 
condition if it does not check the result of the VXLAN create request…   -John

From: Neale Ranns (nranns)
Sent: Wednesday, February 07, 2018 5:55 AM
To: Jon Loeliger <j...@netgate.com>; John Lo (loj) <l...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

Hi Jon,

I feel I must ask ☺
“That  has to match a VPP interface by name.  But what name?”
why does it need to match a VPP interface name? shouldn’t  = MyVxlan23. Your 
control plane maintains mappings between MyVxlan23 and VPP’s sw_if_index.

/neale


From: <vpp-dev-boun...@lists.fd.io<mailto:vpp-dev-boun...@lists.fd.io>> on 
behalf of Jon Loeliger <j...@netgate.com<mailto:j...@netgate.com>>
Date: Friday, 2 February 2018 at 22:08
To: "John Lo (loj)" <l...@cisco.com<mailto:l...@cisco.com>>
Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) 
<l...@cisco.com<mailto:l...@cisco.com>> wrote:
Hi Jon,

Hi John,

Thanks for taking the time to get back to me on this!

All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
created tunnel.

I know.  I get that.  It works.

The name of the tunnel is then followed by a number being the instance or index 
to the tunnel struct vector. Thus, the first VXLAN tunnel created is called 
vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is incrementing unless 
tunnels are deleted and created again where a newly created tunnel will reuse 
the last deleted tunnel, hence its name. If all previously deleted tunnels are 
used up, then the tunnel name number will start incrementing again.

Right.

I am not sure if it is feasible to follow this behavior to “predict” tunnel 
name.

It is not.

The problem is not just "predict it", but the user has to be
able to know the associated SW IF name at the time that
the (vxlan tunnel) create API call happens.

The reason for that is because the very next thing the
API call user is going to want to do is configure that interface.
Short of interrogating the system, there is no way to know
the IF name.  (I understand that the name can be obtained
from the sw_if_index.  That's not what I mean.)

Consider this series of PEUDO API calls that are being
issued by some client front-end.  Think of this as a batch
of CLI commands:

vxlan MyVxlan23
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface 
do stuff to interface like apply an ACL or whatever
exit

There is no way to batch-run those as there is a step
needed to determine what  is.  It could be "vxlan_tunnel..."
anything.

That  has to match a VPP interface by name.  But what name?
The entire transactional  history of VXLANs must be known in
order to guess the next name.

BTW, GRE tunnels are created and named the same way. You added TEB (transparent 
ethernet bridging) support to GRE. Have you not been using it and bothered by 
how to predict name of created GRE tunnels?

I've not touched GRE yet.  However, it is next on my list to fix!
My current plan is to submit essentially the same patch there too.

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to 
identify VXLAN tunnels. It is quite common for existence of multiple VXLAN 
tunnels with the same VNI with different remote end point IP addresses. A 
bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to several 
remote servers where the same overlay network exist. It is quite common to use 
the ID for BD or VLAN as the VNI for all its VXLAN tunnels to make it easie

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

I guess Jon’s point was the control plane need to know exactly what X would be 
and not rely on VPP to provide the sw_if_index or query VPP for the name after 
the VXLAN create request.  This makes the control plane CLI script/template 
support more straight forward.  In the case of mixed usage where a requested 
instance is not available, I wonder how control plane should handle the error 
condition if it does not check the result of the VXLAN create request…   -John

From: Neale Ranns (nranns)
Sent: Wednesday, February 07, 2018 5:55 AM
To: Jon Loeliger <j...@netgate.com>; John Lo (loj) <l...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

Hi Jon,

I feel I must ask ☺
“That  has to match a VPP interface by name.  But what name?”
why does it need to match a VPP interface name? shouldn’t  = MyVxlan23. Your 
control plane maintains mappings between MyVxlan23 and VPP’s sw_if_index.

/neale


From: <vpp-dev-boun...@lists.fd.io<mailto:vpp-dev-boun...@lists.fd.io>> on 
behalf of Jon Loeliger <j...@netgate.com<mailto:j...@netgate.com>>
Date: Friday, 2 February 2018 at 22:08
To: "John Lo (loj)" <l...@cisco.com<mailto:l...@cisco.com>>
Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) 
<l...@cisco.com<mailto:l...@cisco.com>> wrote:
Hi Jon,

Hi John,

Thanks for taking the time to get back to me on this!

All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
created tunnel.

I know.  I get that.  It works.

The name of the tunnel is then followed by a number being the instance or index 
to the tunnel struct vector. Thus, the first VXLAN tunnel created is called 
vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is incrementing unless 
tunnels are deleted and created again where a newly created tunnel will reuse 
the last deleted tunnel, hence its name. If all previously deleted tunnels are 
used up, then the tunnel name number will start incrementing again.

Right.

I am not sure if it is feasible to follow this behavior to “predict” tunnel 
name.

It is not.

The problem is not just "predict it", but the user has to be
able to know the associated SW IF name at the time that
the (vxlan tunnel) create API call happens.

The reason for that is because the very next thing the
API call user is going to want to do is configure that interface.
Short of interrogating the system, there is no way to know
the IF name.  (I understand that the name can be obtained
from the sw_if_index.  That's not what I mean.)

Consider this series of PEUDO API calls that are being
issued by some client front-end.  Think of this as a batch
of CLI commands:

vxlan MyVxlan23
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface 
do stuff to interface like apply an ACL or whatever
exit

There is no way to batch-run those as there is a step
needed to determine what  is.  It could be "vxlan_tunnel..."
anything.

That  has to match a VPP interface by name.  But what name?
The entire transactional  history of VXLANs must be known in
order to guess the next name.

BTW, GRE tunnels are created and named the same way. You added TEB (transparent 
ethernet bridging) support to GRE. Have you not been using it and bothered by 
how to predict name of created GRE tunnels?

I've not touched GRE yet.  However, it is next on my list to fix!
My current plan is to submit essentially the same patch there too.

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to 
identify VXLAN tunnels. It is quite common for existence of multiple VXLAN 
tunnels with the same VNI with different remote end point IP addresses. A 
bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to several 
remote servers where the same overlay network exist. It is quite common to use 
the ID for BD or VLAN as the VNI for all its VXLAN tunnels to make it easier to 
track their usage.

Excellent!  Thank you for this insight!  That makes total sense now!

I suppose it is feasible to provide another set of VXLAN tunnel create/delete 
API to allow caller to specify instance or number of the tunnel, similar to 
what you did for loopback interface.

This is the route I was headed:  Just like loopback interface naming!

In this case, the above example would be, perhaps, something like this:

vxlan MyVxlan23
instance 101
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface vxlan_tunnel101
do stuff to interface like apply an ACL or whatever
exit

  One complication for the new API is that, upon VXLAN tunnel deletion, the 
interface created for the tunnel is not really deleted by the current code but 
put into a reuse pool.

I solved that by making a trivial  vxl

Re: [vpp-dev] VXLAN Tunnel IF Names

[Neale Ranns (nranns)]

Hi Jon,

I feel I must ask ☺
“That  has to match a VPP interface by name.  But what name?”
why does it need to match a VPP interface name? shouldn’t  = MyVxlan23. Your 
control plane maintains mappings between MyVxlan23 and VPP’s sw_if_index.

/neale


From: <vpp-dev-boun...@lists.fd.io> on behalf of Jon Loeliger <j...@netgate.com>
Date: Friday, 2 February 2018 at 22:08
To: "John Lo (loj)" <l...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) 
<l...@cisco.com<mailto:l...@cisco.com>> wrote:
Hi Jon,

Hi John,

Thanks for taking the time to get back to me on this!

All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
created tunnel.

I know.  I get that.  It works.

The name of the tunnel is then followed by a number being the instance or index 
to the tunnel struct vector. Thus, the first VXLAN tunnel created is called 
vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is incrementing unless 
tunnels are deleted and created again where a newly created tunnel will reuse 
the last deleted tunnel, hence its name. If all previously deleted tunnels are 
used up, then the tunnel name number will start incrementing again.

Right.

I am not sure if it is feasible to follow this behavior to “predict” tunnel 
name.

It is not.

The problem is not just "predict it", but the user has to be
able to know the associated SW IF name at the time that
the (vxlan tunnel) create API call happens.

The reason for that is because the very next thing the
API call user is going to want to do is configure that interface.
Short of interrogating the system, there is no way to know
the IF name.  (I understand that the name can be obtained
from the sw_if_index.  That's not what I mean.)

Consider this series of PEUDO API calls that are being
issued by some client front-end.  Think of this as a batch
of CLI commands:

vxlan MyVxlan23
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface 
do stuff to interface like apply an ACL or whatever
exit

There is no way to batch-run those as there is a step
needed to determine what  is.  It could be "vxlan_tunnel..."
anything.

That  has to match a VPP interface by name.  But what name?
The entire transactional  history of VXLANs must be known in
order to guess the next name.

BTW, GRE tunnels are created and named the same way. You added TEB (transparent 
ethernet bridging) support to GRE. Have you not been using it and bothered by 
how to predict name of created GRE tunnels?

I've not touched GRE yet.  However, it is next on my list to fix!
My current plan is to submit essentially the same patch there too.

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to 
identify VXLAN tunnels. It is quite common for existence of multiple VXLAN 
tunnels with the same VNI with different remote end point IP addresses. A 
bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to several 
remote servers where the same overlay network exist. It is quite common to use 
the ID for BD or VLAN as the VNI for all its VXLAN tunnels to make it easier to 
track their usage.

Excellent!  Thank you for this insight!  That makes total sense now!

I suppose it is feasible to provide another set of VXLAN tunnel create/delete 
API to allow caller to specify instance or number of the tunnel, similar to 
what you did for loopback interface.

This is the route I was headed:  Just like loopback interface naming!

In this case, the above example would be, perhaps, something like this:

vxlan MyVxlan23
instance 101
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface vxlan_tunnel101
do stuff to interface like apply an ACL or whatever
exit

  One complication for the new API is that, upon VXLAN tunnel deletion, the 
interface created for the tunnel is not really deleted by the current code but 
put into a reuse pool.

I solved that by making a trivial  vxlan_name_renumber()  function,
and then calling vnet_interface_name_remumber(sw_if_index, instance)
to update the previously captured "vxlan_tunnel" name and rename
it using the new instance number.

When more tunnels are created, any interface from reuse pool with its existing 
interface name will be used for the new tunnel, before new interfaces are 
created. If a interface is reused upon tunnel creation, its interface name may 
not match the specified tunnel instance/number of the new tunnel creation API.  
One way to overcome this may be to not keep interface in reused pool on tunnel 
deletion. Thus, tunnel creation would always create new interface.  For 
backward compatibility, I suppose we can keep the tunnel create/delete API as 
is so interfaces of deleted tunnels are kept for reuse. The new API will then 
always cre

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

John,

On Mon, Feb 5, 2018 at 2:14 PM, John Lo (loj)  wrote:

> Thanks Jon.  Seems we have an agreed upon an approach now.  Just to be
> sure, let me list the specifics:
>
>
>
> 1.  We will get rid of custom_instance_by_real_instance vector in
> vxlan_main_t
>
See #3 below.

> 2.  We will add something like user_instance_by_real_instance hash in
> vxlan_main_t to track instance usage.
>
Yes.

> 3.  We will add u32 instance in vxlan_tunnel_t to keep user specified
> instance, which will be the same as real instance or index of vxlan tunnel
> in the tunnels vector in vxlan_main.
>
> I am happy to do this, yes.  However, there is some reason that this
number can not be used directly within the format() of the vxlan_tunnel%d
name.
At the time that the reuse of the HW IF happens, the necessary
name_renumber()
call fails to get the right instance number as the hw_if_index isn't set in
the hi struct
properly yet.  This is why the custom_instance_by_real_instance mechanism
works:
it is always available.

I will endeavor to use just the hash and the instance on the tunnel struct.
I will attempt to resolve the issue described in the previous paragraph in
more detail.
I know that was a bit vague, so I'll attempt to nail-down the real issue I
was seeing.

4.  We will replace the 64 byte if_name in the vxlan_tunnel_details API
> with u32 instance.
>
Yes.

> 5.  We will generate instance-in-use API error if vxlan tunnel
> creation cannot use the desired instance, either natural or specified.
>
Yes.

We also have to guarantee that: in the case of default allocation (ie, not
specified by the user/API), that there still always a successful allocation.
(Ie, a bit of searching for a valid instance might take place.)  I don't
think
this will be a problem in general.  it will likely only be noticed when both
default and specified allocations are intermixed.


> Will review your new patch set when it shows up.  Once you patch is
> merged, I will copy the same mechanism into the GRE tunnel update I am
> currently working. I will submit my GRE patch for review sometime later and
> add you as one of the reviewers.
>

Awesome!  Sounds great!


> Thanks,
>
> John
>

Thanks,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Thanks Jon.  Seems we have an agreed upon an approach now.  Just to be sure, 
let me list the specifics:


1.  We will get rid of custom_instance_by_real_instance vector in 
vxlan_main_t.

2.  We will add something like user_instance_by_real_instance hash in 
vxlan_main_t to track instance usage.

3.  We will add u32 instance in vxlan_tunnel_t to keep user specified 
instance, which will be the same as real instance or index of vxlan tunnel in 
the tunnels vector in vxlan_main.

4.  We will replace the 64 byte if_name in the vxlan_tunnel_details API 
with u32 instance.

5.  We will generate instance-in-use API error if vxlan tunnel creation 
cannot use the desired instance, either natural or specified.

Will review your new patch set when it shows up.  Once you patch is merged, I 
will copy the same mechanism into the GRE tunnel update I am currently working. 
I will submit my GRE patch for review sometime later and add you as one of the 
reviewers.

Thanks,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Monday, February 05, 2018 2:26 PM
To: John Lo (loj) <l...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Mon, Feb 5, 2018 at 11:48 AM, John Lo (loj) 
<l...@cisco.com<mailto:l...@cisco.com>> wrote:
One more thang, I would prefer to have a u32 instance number instead of the 64 
bytes if_name in the vxlan_tunnel_details API.   -John

Hi John,

I'll work up a patch re-spin using a hash-table, and I will
include the if_name -> sw_if_index change mentioned above!

Thanks,
jdl


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Mon, Feb 5, 2018 at 11:48 AM, John Lo (loj)  wrote:

> One more thang, I would prefer to have a u32 instance number instead of
> the 64 bytes if_name in the vxlan_tunnel_details API.   -John
>

Hi John,

I'll work up a patch re-spin using a hash-table, and I will
include the if_name -> sw_if_index change mentioned above!

Thanks,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

One more thang, I would prefer to have a u32 instance number instead of the 64 
bytes if_name in the vxlan_tunnel_details API.   -John

From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of John Lo (loj)
Sent: Monday, February 05, 2018 12:43 PM
To: Jon Loeliger <j...@netgate.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

Hi Jon,

I think a good approach would be to store the insurance number in the tunnel 
struct and use a hash table to track instance number usage. This is a more 
effective way to not waste memory even for user who like to use big numbers for 
instance while not using that many tunnels.  The instance number in the tunnel 
struct would be the natural one if not specified by user. We will need an 
“instance number in-use” API error.  For users who always use default or always 
specify instance number, there should not be any problem and the current 
default instance behavior is preserved. For users that use mixed mode to create 
tunnels, if a natural one is not available, the API error will force the user 
to specify a usable instance number.  What do you think?

About the renumber param in API, I suppose it is better in the sense that 
Python binding always default non-specified parameters to 0 and thus more user 
friendly. An alternative would be to use instance 0 as “not specified” so any 
user specified instance number must be > 0.

As for GRE tunnel, I am in the process of making a major change to it to 
support ERSPAN encap that is built on top of TEB support.  There were two 
interface names used, greN or teb-greN, depending on payload type (and also 
tunnel interface type) while sharing the same instance space for N. This is 
kind of awkward to me and I have changed this to use greN interface name only. 
I removed the interface reuse logic on tunnel creation to prevent the problem 
caused by having to reuse the right interface type. Thus tunnel create/delete 
always create and delete associated interfaces. Once we agreed on instance 
mechanism for VXLAN, I will also include it in my change to GRE since I am 
changing the API anyway.

Regards,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Monday, February 05, 2018 11:49 AM
To: John Lo (loj) <l...@cisco.com<mailto:l...@cisco.com>>
Cc: Dave Barach (dbarach) <dbar...@cisco.com<mailto:dbar...@cisco.com>>; 
vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Mon, Feb 5, 2018 at 9:55 AM, Jon Loeliger 
<j...@netgate.com<mailto:j...@netgate.com>> wrote:


I think we must check and reject instance misuse for this to be acceptable. I 
doubt the renumber is really used much for vhost and tap interface (others 
please correct me if not true).

We do use them on every call we make to these functions.
As well as on loopbacks.  Knowing what object (by name) will b
created is essential for us.


In your previous patch set #1, the usage of a bitmap to track instance usage 
seem a reasonable solution. It does imply we will never use “natural” instance 
and would always use the instance allocated from the bitmap and stored in the 
tunnel struct.

That said, we can combine the two approaches.  That is, use the
show_instance_by_real_instance vector for knowing what name to use,
but also use a bitmap to record what  was used and allocated.

One question on the bitmap approach.  It had a hard-coded upper limit of 16K
bits in my previous patch.  Are people OK with that?  It is arbitrary, and could
be larger.  It simply depends on how much memory folks are willing to spend
recording in-use bits.  That or a search of the show_by_real vector.  Or we
could convert to a hash-map to check "in-use" status.

Preference?

John,

Also as a point of information, I'd like to follow this approach for GRE as well
as soon as we agree on an approach here!

Thanks!
jdl

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Hi Jon,

I think a good approach would be to store the insurance number in the tunnel 
struct and use a hash table to track instance number usage. This is a more 
effective way to not waste memory even for user who like to use big numbers for 
instance while not using that many tunnels.  The instance number in the tunnel 
struct would be the natural one if not specified by user. We will need an 
“instance number in-use” API error.  For users who always use default or always 
specify instance number, there should not be any problem and the current 
default instance behavior is preserved. For users that use mixed mode to create 
tunnels, if a natural one is not available, the API error will force the user 
to specify a usable instance number.  What do you think?

About the renumber param in API, I suppose it is better in the sense that 
Python binding always default non-specified parameters to 0 and thus more user 
friendly. An alternative would be to use instance 0 as “not specified” so any 
user specified instance number must be > 0.

As for GRE tunnel, I am in the process of making a major change to it to 
support ERSPAN encap that is built on top of TEB support.  There were two 
interface names used, greN or teb-greN, depending on payload type (and also 
tunnel interface type) while sharing the same instance space for N. This is 
kind of awkward to me and I have changed this to use greN interface name only. 
I removed the interface reuse logic on tunnel creation to prevent the problem 
caused by having to reuse the right interface type. Thus tunnel create/delete 
always create and delete associated interfaces. Once we agreed on instance 
mechanism for VXLAN, I will also include it in my change to GRE since I am 
changing the API anyway.

Regards,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Monday, February 05, 2018 11:49 AM
To: John Lo (loj) <l...@cisco.com>
Cc: Dave Barach (dbarach) <dbar...@cisco.com>; vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Mon, Feb 5, 2018 at 9:55 AM, Jon Loeliger 
<j...@netgate.com<mailto:j...@netgate.com>> wrote:


I think we must check and reject instance misuse for this to be acceptable. I 
doubt the renumber is really used much for vhost and tap interface (others 
please correct me if not true).

We do use them on every call we make to these functions.
As well as on loopbacks.  Knowing what object (by name) will b
created is essential for us.


In your previous patch set #1, the usage of a bitmap to track instance usage 
seem a reasonable solution. It does imply we will never use “natural” instance 
and would always use the instance allocated from the bitmap and stored in the 
tunnel struct.

That said, we can combine the two approaches.  That is, use the
show_instance_by_real_instance vector for knowing what name to use,
but also use a bitmap to record what  was used and allocated.

One question on the bitmap approach.  It had a hard-coded upper limit of 16K
bits in my previous patch.  Are people OK with that?  It is arbitrary, and could
be larger.  It simply depends on how much memory folks are willing to spend
recording in-use bits.  That or a search of the show_by_real vector.  Or we
could convert to a hash-map to check "in-use" status.

Preference?

John,

Also as a point of information, I'd like to follow this approach for GRE as well
as soon as we agree on an approach here!

Thanks!
jdl

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Mon, Feb 5, 2018 at 9:55 AM, Jon Loeliger  wrote:

>
>
>
>> I think we must check and reject instance misuse for this to be
>> acceptable. I doubt the renumber is really used much for vhost and tap
>> interface (others please correct me if not true).
>>
>
> We do use them on every call we make to these functions.
> As well as on loopbacks.  Knowing what object (by name) will b
> created is essential for us.
>
>
>>
>>
> In your previous patch set #1, the usage of a bitmap to track instance
>> usage seem a reasonable solution. It does imply we will never use “natural”
>> instance and would always use the instance allocated from the bitmap and
>> stored in the tunnel struct.
>>
>
> That said, we can combine the two approaches.  That is, use the
> show_instance_by_real_instance vector for knowing what name to use,
> but also use a bitmap to record what  was used and allocated.
>
> One question on the bitmap approach.  It had a hard-coded upper limit of
> 16K
> bits in my previous patch.  Are people OK with that?  It is arbitrary, and
> could
> be larger.  It simply depends on how much memory folks are willing to spend
> recording in-use bits.  That or a search of the show_by_real vector.  Or we
> could convert to a hash-map to check "in-use" status.
>
> Preference?
>

John,

Also as a point of information, I'd like to follow this approach for GRE as
well
as soon as we agree on an approach here!

Thanks!
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Mon, Feb 5, 2018 at 9:29 AM, John Lo (loj)  wrote:

> Hi Jon,
>

Hi John,


> I don’t think the renumber scheme, as used by vhost and tap interfaces, is
> foolproof and can cause problems if used extensively for vxlan.  On
> creating an interface with renumber, it does not check if the instance
> being requested is already used by another device. On creating an interface
> without renumber, it also does not check if the “natural instance’ may have
> already been used by another device’s renumber.  Thus, on instance
> collision, last interface will take over the interface name and leave the
> previous interface  not accessible by the duplicated interface name.
>

I was solving that the UI level, but it could be added into the
vxlan_add_del_interface() layer as well.


> I think we must check and reject instance misuse for this to be
> acceptable. I doubt the renumber is really used much for vhost and tap
> interface (others please correct me if not true).
>

We do use them on every call we make to these functions.
As well as on loopbacks.  Knowing what object (by name) will b
created is essential for us.


>   I see a “FIXME” warning in the existing vhost renumber function:
>
>
>
> static int
>
> vhost_user_name_renumber (vnet_hw_interface_t * hi, u32 new_dev_instance)
>
> {
>
>   // FIXME: check if the new dev instance is already used
>
>   vhost_user_main_t *vum = _user_main;
>
>   vec_validate_init_empty (vum->show_dev_instance_by_real_dev_instance,
>
>  hi->dev_instance, ~0);
>

Hmmm, yes, I see ...


> In your previous patch set #1, the usage of a bitmap to track instance
> usage seem a reasonable solution. It does imply we will never use “natural”
> instance and would always use the instance allocated from the bitmap and
> stored in the tunnel struct.
>

This procedure fails for some reason on the 5th test in VXLAN tests.
I am not certain, but I think it is because the renumbering effort was
trying to get the desired instance number out of the tunnel struct,
which lead to finding the corresponding vnet_hw_interface, and when
it needed to renumber, the hi->hw_if_index or hi->dev_instance wasn't
yet set, and it ended up creating IFs with wrong names (always 0?).

That said, we can combine the two approaches.  That is, use the
show_instance_by_real_instance vector for knowing what name to use,
but also use a bitmap to record what  was used and allocated.

One question on the bitmap approach.  It had a hard-coded upper limit of 16K
bits in my previous patch.  Are people OK with that?  It is arbitrary, and
could
be larger.  It simply depends on how much memory folks are willing to spend
recording in-use bits.  That or a search of the show_by_real vector.  Or we
could convert to a hash-map to check "in-use" status.

Preference?

As for the API/CLI, I though another parameter “instance” would suffice as
> ~0 can be used to state “not specified” rather than having an additional
> parameter “renumber”, although I can accept either way.
>

Two things here.  I'm fine with either as well.  I was following the
existing
pattern for the "renumber" approach.  Using ~0 effectively means that the
message constructors must all explicitly initialize this field to ~0.  I'm
good
with that (previous patch!) if you guys are as well.

Regards,
>
> John
>

Thanks for your feedback!

jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Hi Jon,

I don’t think the renumber scheme, as used by vhost and tap interfaces, is 
foolproof and can cause problems if used extensively for vxlan.  On creating an 
interface with renumber, it does not check if the instance being requested is 
already used by another device. On creating an interface without renumber, it 
also does not check if the “natural instance’ may have already been used by 
another device’s renumber.  Thus, on instance collision, last interface will 
take over the interface name and leave the previous interface  not accessible 
by the duplicated interface name.

I think we must check and reject instance misuse for this to be acceptable. I 
doubt the renumber is really used much for vhost and tap interface (others 
please correct me if not true).  I see a “FIXME” warning in the existing vhost 
renumber function:

static int
vhost_user_name_renumber (vnet_hw_interface_t * hi, u32 new_dev_instance)
{
  // FIXME: check if the new dev instance is already used
  vhost_user_main_t *vum = _user_main;
  vec_validate_init_empty (vum->show_dev_instance_by_real_dev_instance,
 hi->dev_instance, ~0);

In your previous patch set #1, the usage of a bitmap to track instance usage 
seem a reasonable solution. It does imply we will never use “natural” instance 
and would always use the instance allocated from the bitmap and stored in the 
tunnel struct.

As for the API/CLI, I though another parameter “instance” would suffice as ~0 
can be used to state “not specified” rather than having an additional parameter 
“renumber”, although I can accept either way.

Regards,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Sunday, February 04, 2018 3:38 PM
To: John Lo (loj) <l...@cisco.com>; Dave Barach (dbarach) <dbar...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj) 
<l...@cisco.com<mailto:l...@cisco.com>> wrote:
Hi Jon,

All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
created tunnel. The name of the tunnel is then followed by a number being the 
instance or index to the tunnel struct vector. Thus, the first VXLAN tunnel 
created is called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is 
incrementing unless tunnels are deleted and created again where a newly created 
tunnel will reuse the last deleted tunnel, hence its name. If all previously 
deleted tunnels are used up, then the tunnel name number will start 
incrementing again.  I am not sure if it is feasible to follow this behavior to 
“predict” tunnel name.

John and Dave,

I have uploaded a new patch set.  It is being verified at the time of this 
writing.

This patch has a different implementation than the previous version.
The API changes, including VAT-n-friends, have been updated a bit,
but the basic mechanism of obtaining predictable SW IF names is
different from my original patch.

In the current form, I followed an existing "custom device instance" pattern
that was already present on several other objects already.  Much simpler,
and I expect it to verify now. :-)

So, I guess I realize my mistake in my previous discussions of this general 
problem
I apparently wasn't using the right words.  I was saying something like "The 
User
needs to be able to know or predict the SW IF name."  The phrase that seems to
be used specifically for this purpose in many other places is "Let the user 
assign
a custom device instance id."  And that is achieved via the use of the so-called
*_name_renumber() class function.

So my earlier plea for a "can we somehow solve this at a higher level" might
be more properly stated as:  Where objects are created as a side-effect of
some other creation request, a "renumber flag and custom device instance"
would be really nice on the Create API call.

HTH,
jdl

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj)  wrote:

> Hi Jon,
>
>
>
> All VPP tunnel creation uses the mechanism of returning a sw_if_index of
> the created tunnel. The name of the tunnel is then followed by a number
> being the instance or index to the tunnel struct vector. Thus, the first
> VXLAN tunnel created is called vxlan_tunnel0 followed by vxlan_tunnel1,
> etc. The number is incrementing unless tunnels are deleted and created
> again where a newly created tunnel will reuse the last deleted tunnel,
> hence its name. If all previously deleted tunnels are used up, then the
> tunnel name number will start incrementing again.  I am not sure if it is
> feasible to follow this behavior to “predict” tunnel name.
>

John and Dave,

I have uploaded a new patch set.  It is being verified at the time of this
writing.

This patch has a different implementation than the previous version.
The API changes, including VAT-n-friends, have been updated a bit,
but the basic mechanism of obtaining predictable SW IF names is
different from my original patch.

In the current form, I followed an existing "custom device instance" pattern
that was already present on several other objects already.  Much simpler,
and I expect it to verify now. :-)

So, I guess I realize my mistake in my previous discussions of this general
problem
I apparently wasn't using the right words.  I was saying something like
"The User
needs to be able to know or predict the SW IF name."  The phrase that seems
to
be used specifically for this purpose in many other places is "Let the user
assign
a custom device instance id."  And that is achieved via the use of the
so-called
*_name_renumber() class function.

So my earlier plea for a "can we somehow solve this at a higher level" might
be more properly stated as:  Where objects are created as a side-effect of
some other creation request, a "renumber flag and custom device instance"
would be really nice on the Create API call.

HTH,
jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

There are even more filtering options, allowing you to specify a class or a 
single test function within a test case if needed.

Please consult `make test-help`.

Regards,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of John Lo (loj)
> Sent: Friday, February 2, 2018 10:40 PM
> To: Jon Loeliger <j...@netgate.com>
> Cc: vpp-dev <vpp-dev@lists.fd.io>
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> Hi Jon,
> 
> 
> 
> You can easily run just vxlan tests by “make test TEST=vxlan”, or better yet
> “make test-debug TEST=vxlan”, to run just test_vxlan.py script in VPP normal
> or debug mode. Similarly for other test cases.
> 
> 
> 
> Regards,
> 
> John
> 
> 
> 
> From: Jon Loeliger [mailto:j...@netgate.com]
> Sent: Friday, February 02, 2018 4:18 PM
> To: John Lo (loj) <l...@cisco.com>
> Cc: vpp-dev <vpp-dev@lists.fd.io>
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> 
> 
> On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger <j...@netgate.com
> <mailto:j...@netgate.com> > wrote:
> 
> 
> 
>   I have submitted a patch to Gerrit.  When this passes "verify", I'll add
> you to be
> 
>   a Reviewer!
> 
> 
> 
> John,
> 
> 
> 
> How do I run just these tests locally?  I need to determine if
> 
> these failures are due to my patch or not.
> 
> 
> 
> Specifically, I am not able to succesfully "make test" on a CentOS system
> 
> due to Python having a memory corruption failure and dumping core on me.
> 
> As many tests until that point were passing though.
> 
> 
> 
> I'll add you as a patch reviewer anyway -- maybe you can spot something
> 
> brain-dead that I did in the patch...?
> 
> 
> 
> Thanks,
> 
> jdl
> 
> 
> 
> 
> 
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases*
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT
> | PASS |
> 20:46:28 20:34:06 
> ---
> -
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases* | 
> PASS |
> 20:46:28 20:34:06 1 critical test, 1 passed, 0 failed
> 20:46:28 20:34:06 1 test total, 1 passed, 0 failed
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func ::
> *IPv6 Router Advertisement test cases*
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top]
> TG-DUT1-DUT2-TG.  | PASS |
> 20:46:28 20:34:20 
> ---
> -
> 20:46:28 20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a
> set interval :: [Top] TG-DUT1-DUT2-TG.   | PASS |
> 20:46:28 20:34:36 
> ---
> -
> 20:46:28 20:34:49 TC03: DUT responds to Router Solicitation request :: [Top]
> TG-DUT1-DUT2-TG. | FAIL |
> 20:46:28 20:34:49 Traffic script execution failed
> 20:46:28 20:34:49 
> ---
> -
> 20:46:28 20:35:02 TC04: DUT responds to Router Solicitation request sent
> from link local address :: [Top] TG-DUT1-DUT2-TG.| 
> FAIL |
> 20:46:28 20:35:02 Traffic script execution failed
> 20:46:28 20:35:0

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Hi Jon,

You can easily run just vxlan tests by “make test TEST=vxlan”, or better yet 
“make test-debug TEST=vxlan”, to run just test_vxlan.py script in VPP normal or 
debug mode. Similarly for other test cases.

Regards,
John

From: Jon Loeliger [mailto:j...@netgate.com]
Sent: Friday, February 02, 2018 4:18 PM
To: John Lo (loj) <l...@cisco.com>
Cc: vpp-dev <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VXLAN Tunnel IF Names

On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger 
<j...@netgate.com<mailto:j...@netgate.com>> wrote:

I have submitted a patch to Gerrit.  When this passes "verify", I'll add you to 
be
a Reviewer!

John,

How do I run just these tests locally?  I need to determine if
these failures are due to my patch or not.

Specifically, I am not able to succesfully "make test" on a CentOS system
due to Python having a memory corruption failure and dumping core on me.
As many tests until that point were passing though.

I'll add you as a patch reviewer anyway -- maybe you can spot something
brain-dead that I did in the patch...?

Thanks,
jdl



20:46:28 20:33:41 


20:46:28 20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6 
Multipath routing test cases*

20:46:28 20:33:41 


20:46:28 20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT   
  | PASS |

20:46:28 20:34:06 


20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6 
Multipath routing test cases* | PASS |

20:46:28 20:34:06 1 critical test, 1 passed, 0 failed

20:46:28 20:34:06 1 test total, 1 passed, 0 failed

20:46:28 20:34:06 


20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6 
Router Advertisement test cases*

20:46:28 20:34:06 


20:46:28 20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top] 
TG-DUT1-DUT2-TG.  | PASS |

20:46:28 20:34:20 


20:46:28 20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a 
set interval :: [Top] TG-DUT1-DUT2-TG.   | PASS |

20:46:28 20:34:36 


20:46:28 20:34:49 TC03: DUT responds to Router Solicitation request :: [Top] 
TG-DUT1-DUT2-TG. | FAIL |

20:46:28 20:34:49 Traffic script execution failed

20:46:28 20:34:49 


20:46:28 20:35:02 TC04: DUT responds to Router Solicitation request sent from 
link local address :: [Top] TG-DUT1-DUT2-TG.| FAIL |

20:46:28 20:35:02 Traffic script execution failed

20:46:28 20:35:02 


20:46:28 20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6 
Router Advertisement test cases*| PASS |

20:46:28 20:35:02 0 critical tests, 0 passed, 0 failed

20:46:28 20:35:02 4 tests total, 2 passed, 2 failed

20:46:28 20:35:02 


___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger  wrote:

>
> I have submitted a patch to Gerrit.  When this passes "verify", I'll add
> you to be
> a Reviewer!
>

John,

How do I run just these tests locally?  I need to determine if
these failures are due to my patch or not.

Specifically, I am not able to succesfully "make test" on a CentOS system
due to Python having a memory corruption failure and dumping core on me.
As many tests until that point were passing though.

I'll add you as a patch reviewer anyway -- maybe you can spot something
brain-dead that I did in the patch...?

Thanks,
jdl


*20:46:28* 20:33:41
*20:46:28*
20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6
Multipath routing test cases*
   *20:46:28* 20:33:41
*20:46:28*
20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT
  |
PASS |*20:46:28* 20:34:06
*20:46:28*
20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func :: *Ipv6
Multipath routing test cases* |
PASS |*20:46:28* 20:34:06 1 critical test, 1 passed, 0
failed*20:46:28* 20:34:06 1 test total, 1 passed, 0 failed*20:46:28*
20:34:06 
*20:46:28*
20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6
Router Advertisement test cases*
 *20:46:28* 20:34:06
*20:46:28*
20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top]
TG-DUT1-DUT2-TG.
| PASS |*20:46:28* 20:34:20
*20:46:28*
20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a
set interval :: [Top] TG-DUT1-DUT2-TG.   |
PASS |*20:46:28* 20:34:36
*20:46:28*
20:34:49 TC03: DUT responds to Router Solicitation request :: [Top]
TG-DUT1-DUT2-TG. |
FAIL |*20:46:28* 20:34:49 Traffic script execution failed*20:46:28*
20:34:49 
*20:46:28*
20:35:02 TC04: DUT responds to Router Solicitation request sent from
link local address :: [Top] TG-DUT1-DUT2-TG.|
FAIL |*20:46:28* 20:35:02 Traffic script execution failed*20:46:28*
20:35:02 
*20:46:28*
20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func :: *IPv6
Router Advertisement test cases*|
PASS |*20:46:28* 20:35:02 0 critical tests, 0 passed, 0
failed*20:46:28* 20:35:02 4 tests total, 2 passed, 2 failed*20:46:28*
20:35:02 

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VXLAN Tunnel IF Names

[Jon Loeliger]

On Wed, Jan 31, 2018 at 6:41 PM, John Lo (loj)  wrote:

> Hi Jon,
>

Hi John,

Thanks for taking the time to get back to me on this!

All VPP tunnel creation uses the mechanism of returning a sw_if_index of
> the created tunnel.
>

I know.  I get that.  It works.


> The name of the tunnel is then followed by a number being the instance or
> index to the tunnel struct vector. Thus, the first VXLAN tunnel created is
> called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is
> incrementing unless tunnels are deleted and created again where a newly
> created tunnel will reuse the last deleted tunnel, hence its name. If all
> previously deleted tunnels are used up, then the tunnel name number will
> start incrementing again.
>

Right.


> I am not sure if it is feasible to follow this behavior to “predict”
> tunnel name.
>

It is not.

The problem is not just "predict it", but the user has to be
able to know the associated SW IF name at the time that
the (vxlan tunnel) create API call happens.

The reason for that is because the very next thing the
API call user is going to want to do is configure that interface.
Short of interrogating the system, there is no way to know
the IF name.  (I understand that the name can be obtained
from the sw_if_index.  That's not what I mean.)

Consider this series of PEUDO API calls that are being
issued by some client front-end.  Think of this as a batch
of CLI commands:

vxlan MyVxlan23
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface 
do stuff to interface like apply an ACL or whatever
exit

There is no way to batch-run those as there is a step
needed to determine what  is.  It could be "vxlan_tunnel..."
anything.

That  has to match a VPP interface by name.  But what name?
The entire transactional  history of VXLANs must be known in
order to guess the next name.

BTW, GRE tunnels are created and named the same way. You added TEB
> (transparent ethernet bridging) support to GRE. Have you not been using it
> and bothered by how to predict name of created GRE tunnels?
>

I've not touched GRE yet.  However, it is next on my list to fix!
My current plan is to submit essentially the same patch there too.

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to
> identify VXLAN tunnels. It is quite common for existence of multiple VXLAN
> tunnels with the same VNI with different remote end point IP addresses. A
> bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to
> several remote servers where the same overlay network exist. It is quite
> common to use the ID for BD or VLAN as the VNI for all its VXLAN tunnels to
> make it easier to track their usage.
>

Excellent!  Thank you for this insight!  That makes total sense now!

I suppose it is feasible to provide another set of VXLAN tunnel
> create/delete API to allow caller to specify instance or number of the
> tunnel, similar to what you did for loopback interface.
>

This is the route I was headed:  Just like loopback interface naming!

In this case, the above example would be, perhaps, something like this:

vxlan MyVxlan23
instance 101
source 1.2.3.4
destination 10.11.12.13
vni 19
exit

interface vxlan_tunnel101
do stuff to interface like apply an ACL or whatever
exit


  One complication for the new API is that, upon VXLAN tunnel deletion, the
> interface created for the tunnel is not really deleted by the current code
> but put into a reuse pool.
>

I solved that by making a trivial  vxlan_name_renumber()  function,
and then calling vnet_interface_name_remumber(sw_if_index, instance)
to update the previously captured "vxlan_tunnel" name and rename
it using the new instance number.

When more tunnels are created, any interface from reuse pool with its
> existing interface name will be used for the new tunnel, before new
> interfaces are created. If a interface is reused upon tunnel creation, its
> interface name may not match the specified tunnel instance/number of the
> new tunnel creation API.  One way to overcome this may be to not keep
> interface in reused pool on tunnel deletion. Thus, tunnel creation would
> always create new interface.  For backward compatibility, I suppose we can
> keep the tunnel create/delete API as is so interfaces of deleted tunnels
> are kept for reuse. The new API will then always create/delete interface on
> tunnel create/delete.  This would put a restriction that user should not
> mix the usage of new or old APIs.
>

To me, that sounds like a whole bunch of really unnecessary code replication
and fragile separation of API results that will invariably get mixed up and
cause mysterious failures.

Instead, I maintain the original instance allocation scheme when there is
no requested Instance Id made by the CLI or API user.

I have submitted a patch to Gerrit.  When this passes "verify", I'll add
you to be
a Reviewer!

Re: [vpp-dev] VXLAN Tunnel IF Names

[John Lo (loj)]

Hi Jon,

All VPP tunnel creation uses the mechanism of returning a sw_if_index of the 
created tunnel. The name of the tunnel is then followed by a number being the 
instance or index to the tunnel struct vector. Thus, the first VXLAN tunnel 
created is called vxlan_tunnel0 followed by vxlan_tunnel1, etc. The number is 
incrementing unless tunnels are deleted and created again where a newly created 
tunnel will reuse the last deleted tunnel, hence its name. If all previously 
deleted tunnels are used up, then the tunnel name number will start 
incrementing again.  I am not sure if it is feasible to follow this behavior to 
“predict” tunnel name.

BTW, GRE tunnels are created and named the same way. You added TEB (transparent 
ethernet bridging) support to GRE. Have you not been using it and bothered by 
how to predict name of created GRE tunnels?

It is not a good idea to tie tunnel name to VNI as it is not a unique ID to 
identify VXLAN tunnels. It is quite common for existence of multiple VXLAN 
tunnels with the same VNI with different remote end point IP addresses. A 
bridge domain (BD) may have multiple VXLAN tunnels with the same VNI to several 
remote servers where the same overlay network exist. It is quite common to use 
the ID for BD or VLAN as the VNI for all its VXLAN tunnels to make it easier to 
track their usage.

I suppose it is feasible to provide another set of VXLAN tunnel create/delete 
API to allow caller to specify instance or number of the tunnel, similar to 
what you did for loopback interface.  One complication for the new API is that, 
upon VXLAN tunnel deletion, the interface created for the tunnel is not really 
deleted by the current code but put into a reuse pool. When more tunnels are 
created, any interface from reuse pool with its existing interface name will be 
used for the new tunnel, before new interfaces are created. If a interface is 
reused upon tunnel creation, its interface name may not match the specified 
tunnel instance/number of the new tunnel creation API.  One way to overcome 
this may be to not keep interface in reused pool on tunnel deletion. Thus, 
tunnel creation would always create new interface.  For backward compatibility, 
I suppose we can keep the tunnel create/delete API as is so interfaces of 
deleted tunnels are kept for reuse. The new API will then always create/delete 
interface on tunnel create/delete.  This would put a restriction that user 
should not mix the usage of new or old APIs.

Regards,
John

From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
Behalf Of Jon Loeliger
Sent: Wednesday, January 31, 2018 3:13 PM
To: vpp-dev 
Subject: [vpp-dev] VXLAN Tunnel IF Names

Hey Developers,

I've turned my attention to the wondrous world of VXLANs!
And with that, I stare into the Vast VXLAN API Abyss...

My first question, as usual, involves the SW IF that is created
by the vxlan tunnel create API call.   The IF has a name that
is once-again unknowable by the User who creates the tunnel
unless direct inspection takes place.

Can we somehow invert the naming responsibility here so that
the API states either the name or the unique integer used to
form the the SW IF tunnel name?  Or perhaps it can be formed
using the VNI number?

Inspecting the reply return value isn't good enough; we need to
be able to know, in advance, what the tunnel SW IF name will be.
This is, again, the same naming problem we've had in the past
with Loopback IFs, bridges, and recently memif socket names.

If directly using the VNI won't work, I can work up a patch to
allow user numbering in the style of, say, bridges, if that would
be helpful.

Thanks,
jdl

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev