Re: [vpp-dev] goodbye Scapy...

[Paul Vinciguerra]

Hi Hermant,

With some effort, it could be.  It is not as comprehensive as scapy.

Paul

On Sat, May 29, 2021 at 7:42 PM  wrote:

> Paul,
>
>
>
> I thought VPP wanted to replace using scapy, so why would dpkt not work?
>
>
>
> Thanks,
>
>
>
> Hemant
>
>
>
> *From:* Paul Vinciguerra 
> *Sent:* Saturday, May 29, 2021 7:04 PM
> *To:* hem...@mnkcg.com
> *Cc:* vpp-dev 
> *Subject:* Re: [vpp-dev] goodbye Scapy...
>
>
>
> Hi Hermant,
>
>
>
> Thanks for the info, but snappi does not address this issue for us.
>
>
>
> If you look at the sample in the repo [0], you will see that they use
> dpkt[1] instead of scapy.
>
>
>
> Paul
>
>
>
> [0]
> https://github.com/open-traffic-generator/ixia-c/blob/main/docs/hello-snappi.md
>
> [1] https://github.com/kbandla/dpkt
>
>
>
> On Sat, May 29, 2021 at 1:50 PM hemant via lists.fd.io  mnkcg@lists.fd.io> wrote:
>
> In recent past, folks discussed the licensing issue with Scapy and VPP.
> At the recently concluded 2021 P4 Workshop, Ixia has snappi as a common
> api.  Please watch video below.
>
>
>
> https://www.youtube.com/watch?v=Db7Cx1hngVY
>
>
>
> Hemant
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19507): https://lists.fd.io/g/vpp-dev/message/19507
Mute This Topic: https://lists.fd.io/mt/83174836/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] goodbye Scapy...

[Paul Vinciguerra]

Hi Hermant,

Thanks for the info, but snappi does not address this issue for us.

If you look at the sample in the repo [0], you will see that they use
dpkt[1] instead of scapy.

Paul

[0]
https://github.com/open-traffic-generator/ixia-c/blob/main/docs/hello-snappi.md
[1] https://github.com/kbandla/dpkt

On Sat, May 29, 2021 at 1:50 PM hemant via lists.fd.io  wrote:

> In recent past, folks discussed the licensing issue with Scapy and VPP.
> At the recently concluded 2021 P4 Workshop, Ixia has snappi as a common
> api.  Please watch video below.
>
>
>
> https://www.youtube.com/watch?v=Db7Cx1hngVY
>
>
>
> Hemant
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19505): https://lists.fd.io/g/vpp-dev/message/19505
Mute This Topic: https://lists.fd.io/mt/83174836/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vnet_rename_interface()

[Paul Vinciguerra]

Hi Dave,

I started out saying that I think this is a valuable change.  That being
said, this is going to have implications for folks.
Sometimes concrete examples help.  I did not have to look far.
Here is code from VOM.  Yes, it is deprecated, but it is an example for
folks to follow.

/**
 * A DB of all the interfaces, key on the name
 */
singular_db interface::m_db;

/**
 * A DB of all the interfaces, key on VPP's handle
 */
std::map> interface::m_hdl_db;


If you look at tests, the name is a cached property and used as an identity
property

class VppInterface(metaclass=abc.ABCMeta):

@property
def name(self):
"""Name of the interface."""
return self._name


r = self.test.vapi.sw_interface_dump(sw_if_index=self.sw_if_index)
for intf in r:
if intf.sw_if_index == self.sw_if_index:
self._name = intf.interface_name
self._local_mac = intf.l2_address
self._dump = intf
break
else:

...


Matt has shown himself to be quite capable, and I'm sure he could easily
write a few extra lines of code to protect existing implementations in
less time than we have gone back and forth.  I'm not tied to my suggestion
as the only way to address this, but rather to not invalidate assurances
that other folks rely upon.  It has the side effect of making *other*
existing api's backward incompatible.

Ole's comments are aspirational at this point and I 100% agree that we
should strive for all access to be keyed/ filtered on sw_if_index, but
that is not the reality of the state of the api today.

Paul

On Fri, May 14, 2021 at 12:40 AM David Cornejo  wrote:

> Ole,
> As you note, internally only the index is used. As far as I can tell, the
> ability to store the name in VPP is a convenience. The ability to change it
> shouldn't impact VPP but makes it much easier for those of us that have to
> deal with the human side of a product. The change to expose it is minor and
> it is very useful, so I hope that we will be allowed to commit it.
>
> Paul,
> The availability of an API doesn't mean that you have to use it or expose
> it to your users, so I don't see it as an issue. The idea of enabling or
> disabling every possible way to shoot your foot in VPP is daunting, and
> IMO, this is the job of the control plane. I could interpret Ole's comment
> in that light too.
>
> Nathan, thanks for the support!
>
> dave c
>
>
> On Wed, May 5, 2021 at 3:25 AM Nathan Skrzypczak <
> nathan.skrzypc...@gmail.com> wrote:
>
>> Hi Paul,
>>
>> From my understanding - but I might be mistaken here - I thought this was
>> about an explicit `rename` cli & api,
>> so foot shooting would be quite explicit from the user's perspective.
>> On ability to query/disable `show int` and equivalent apis would return
>> this modified state,
>> and not calling it would preserve the existing auto-generated names, so
>> existing users shouldn't be disrupted.
>>
>> Anyway, I'm pushing for this change, as this would push quite useful for
>> container interfaces in Calico/VPP :)
>>
>> Cheers
>> -Nathan
>>
>>
>> Le mer. 5 mai 2021 à 14:46, Paul Vinciguerra 
>> a écrit :
>>
>>> Hi Matt, Ole, Nathan,
>>>
>>> Matt,
>>> I have no objection to the feature, I would just ask that you provide a
>>> startup.conf option to disable it and an api to query that state.  The
>>> reason I say this is that there are configuration models that depend on the
>>> stability of the device name. Many yang models as an example, expect the
>>> interface name to be stable and take on the role of candidate
>>> configuration.  The name filter functionality within the api may be
>>> invalidated once you execute your change.
>>>
>>> Ole,
>>> I agree with the factual aspect of your comment, but the interface index
>>> is not an idempotent property of the system, but dependent on the
>>> operational history of the device.  We have interface instances which can
>>> be user specified, and apis that rely on the instance identifier
>>> irrespective of the format name.  We also do not yet support all actions
>>> via interface index via the api.  I'm just saying that this could break
>>> existing api clients' assumptions.
>>>
>>> Nathan,
>>> I too see the utility of the change, but also the disruptive nature.  I
>>> just ask that we provide a means to keep folks from inadvertently shooting
>>> themselves in the foot, especially considering the diversity of use cases.
>>>
>>> Paul
>>>
>>> On Wed, May 5, 2021 at 6:15 AM Nathan Skrzypczak <
>>> nathan.skrzypc...@gmail.com> wrote:
>&

Re: [vpp-dev] dhcp proxy implementation in vpp #vpp

[Paul Vinciguerra]

Consider the case where a subnet spans a distributed bridge group with an
anycast gateway configured.  Each proxy has the identical ip/mac
configured.  The src-address is the address of the device originating the
proxy request.  It would be my guess that the clixon command is looking up
the src-address of the eth0 for you as a convenience.
There is an h2cvpp yang model here:  [0].

[0]
https://github.com/FDio/hc2vpp/blob/master/dhcp/dhcp-api/src/main/yang/dhcp%402018-06-29.yang

On Fri, May 7, 2021 at 11:34 PM  wrote:

> Hi,all:
>   I found yang model for dhcp proxy in openconfig-relay-agent, but it
> can't match with vpp implementation well. I set a dhcp proxy server in vpp
> command line like this:
>
>   vpp$  set dhcp proxy server 192.168.16.9 src-address 192.168.16.24
>
>   do the same thing via clixon command like this:
>
> clixon$ set relay-agent dhcp interfaces eth0 config helper-address
> 192.168.16.9
>
>  why vpp need "src-address" parameter ? and it's mandatory ?
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19367): https://lists.fd.io/g/vpp-dev/message/19367
Mute This Topic: https://lists.fd.io/mt/82671920/21656
Mute #vpp:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vnet_rename_interface()

[Paul Vinciguerra]

Hi Matt, Ole, Nathan,

Matt,
I have no objection to the feature, I would just ask that you provide a
startup.conf option to disable it and an api to query that state.  The
reason I say this is that there are configuration models that depend on the
stability of the device name. Many yang models as an example, expect the
interface name to be stable and take on the role of candidate
configuration.  The name filter functionality within the api may be
invalidated once you execute your change.

Ole,
I agree with the factual aspect of your comment, but the interface index is
not an idempotent property of the system, but dependent on the operational
history of the device.  We have interface instances which can be user
specified, and apis that rely on the instance identifier irrespective of
the format name.  We also do not yet support all actions via interface
index via the api.  I'm just saying that this could break existing api
clients' assumptions.

Nathan,
I too see the utility of the change, but also the disruptive nature.  I
just ask that we provide a means to keep folks from inadvertently shooting
themselves in the foot, especially considering the diversity of use cases.

Paul

On Wed, May 5, 2021 at 6:15 AM Nathan Skrzypczak <
nathan.skrzypc...@gmail.com> wrote:

> Hi Matt, Hi Ole,
>
> I think adding a cli & api call for renaming interfaces would be quite
> useful.
> I agree we shouldn't refer to interfaces by name in the API thought
> (sw_if_index is definitely the way to go)
>
> Several non-api use-cases would imho benefit from this. I can see writing
> cli scripts without having to sed $interface_name all over the place.
> Also towards user friendliness, when exposing vppctl in a production
> environment, `show interface`
> can be a bit hard to understand and could benefit from context-related
> names.
>
> Anyway, would definitely use such a feature :)
>
> Cheers
> -Nathan
>
> Le mer. 5 mai 2021 à 10:18, Ole Troan  a écrit :
>
>> Matt,
>>
>> > Source file src/vnet/interface.c has a function
>> vnet_rename_interface(). It only appears to be called by the lisp plugin
>> currently. It would be handy to be able to rename a DPDK interface without
>> having to change startup.conf and restart VPP. I am wondering if I could do
>> that by adding a sw_interface_rename API and calling
>> vnet_rename_interface() in the handler function. Before I spend much time
>> working on that, I want to find out if there are any known issues which
>> would prevent that from working or if anyone has any objections to doing it.
>>
>> As a general principle, VPP doesn't have interface names. It has
>> interface indices and you can in your application choose the index to name
>> mapping yourself.
>>
>> Best regards,
>> Ole
>>
>>
>>
>>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19336): https://lists.fd.io/g/vpp-dev/message/19336
Mute This Topic: https://lists.fd.io/mt/82588728/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] sw_interface_dump currently cann't dump interface which sw_if_index == 0 #vapi

[Paul Vinciguerra]

Hi Andrew,

I agree with you, we should revert the comparison back to  ~0 and set the
default to ~0.  The larger issue is setting a false narrative in the commit
message to relax the review process.  "This is a temporary measure, ". It
may not have been false at the time it was authored, but the expectation
made by the author is false after two years. Is it not incumbent on the
author of a temporary measure to address it before the next release? A
manager at one of my clients always says "explain that to me: You don't
have the time to do it right, but you have the time to do it again?"

The larger issue is whether high level languages are going to be considered
first class citizens with VPP. You want to mentor someone to do rust
bindings. Florin reached out for someone to do Go bindings. I have a change
[0] that was submitted before the crc process. You -1 it "to test the crc
process" and it took Ole to remove your -1 months later after the crc check
was put in place. It too, is a bug-fix, because Jakub updated the .api with
the wrong typedef, needed by the high-level language bindings to be correct.

Paul


[0] https://gerrit.fd.io/r/c/vpp/+/26832

On Mon, May 3, 2021 at 5:40 AM Andrew Yourtchenko 
wrote:

> I would argue that this case can be classified as "bugfix" - there was
> no good reason to use the 0 as a wildcard value in the first place,
> since it is a valid sw_if_index, and there is a perfectly good
> "wildcard value" of ~0 that already works, right ?
>
> So I would say this discussion should serve as the announcement (Or
> there can be another separate thread with the explicit subject of it),
> and post-branch of stable/2106, we can apply the fix to consider the 0
> to be a valid interface ID.
>
> And of course bump the semver for those who look at it.
>
> It's always useful to keep in mind "why" the process is in place, when
> evaluating how to apply it:
>
> the reason for it is minimize the amount of forced work downstream,
> caused by the API changes, and minimize the element of surprise.
>
> In this case, having the "buggy" clients use ~0 in place of 0 (if they
> ever used that) is strictly less work than having *all* clients to
> switch to using a new message name even if they used the ~0 to begin
> with.
>
> We can take care of minimizing the "element of surprise" by this
> discussion, or maybe a separate mail - and giving the folks a few
> weeks (namely, until after RC1), to adapt.
>
> This way the spirit of why the process there in the first place will
> be fulfilled, without incurring the unnecessary effort for everyone.
>
> Does this make sense ?
>
> --a
>
> On 5/3/21, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
> via lists.fd.io  wrote:
> >> Is there any plan for support selecting only index==0 ?
> >
> > Good news first.
> > I added the TODO here [0], but since then
> > CSIT stopped using the VAT command in [1],
> > and other uses based on PAPI should be ready since [2].
> >
> > The bad news is VPP now has more strict process [3]
> > regarding changes to API.
> > This particular case is debatable, as the change
> > will not affect the message CRC value,
> > only the behavior for a particular argument value.
> > But I think it would be an API change overall,
> > so not directly allowed (as sw_interface_dump is a production API).
> >
> > The indirect fix is to add a new API message
> > (sw_interface_dump_v2) and mark the old one as deprecated.
> >
> > Vratko.
> >
> > [0] https://gerrit.fd.io/r/c/vpp/+/19106
> > [1] https://gerrit.fd.io/r/c/csit/+/21149
> > [2] https://gerrit.fd.io/r/c/vpp/+/23530
> > [3] https://wiki.fd.io/view/VPP/ApiChangeProcess
> >
> > From: vpp-dev@lists.fd.io  On Behalf Of
> > jiangxiaom...@outlook.com
> > Sent: Thursday, 2021-April-29 13:41
> > To: vpp-dev@lists.fd.io
> > Subject: [vpp-dev] sw_interface_dump currently cann't dump interface
> which
> > sw_if_index == 0 #vapi
> >
> > Hi  Vratko Polak,
> >
> > currently sw_interface_dump will dump all interfaces if sw_if_index ==
> 0. In
> > my case, i need get interface information by id without by dumping all,
> but
> > there is no way for this.
> > I notice there is a TODO comment for sw_interface_dump in interface.api
> > file.
> > /** \brief Request all or filtered subset of sw_interface_details
> > @param client_index - opaque cookie to identify the sender
> > @param context - sender context, to match reply w/ request
> > @param sw_if_index - index of the interface to dump info on, 0 or ~0
> if
> > on all
> >   TODO: Support selecting only index==0 when CSIT is ready.
> > @param name_filter_valid - 1 if requesting a filtered subset of
> records
> > else 0
> >   if name filter is set as valid, sw_if_index value is ignored and
> all
> > interfaces are examined
> > @param name_filter - interface name substring filter. Eg. loop1
> returns
> > [loop1, loop10]
> > */
> > define sw_interface_dump
> > Is there any plan for support selecting only index==0 ?
> >
>
> 
>
>

Re: [vpp-dev] vpp_papi.py error

[Paul Vinciguerra]

Please see [0] from Dec.

This proposed change has a breaking impact further down the line.
It has been a more reliable fix for me at least to instead modify the
makefile to do `python3 -m pip install cmake`, which is a proven technique
and has been used with older distros in the CI for a good while now.

Paul

https://gerrit.fd.io/r/c/vpp/+/30401/6/src/vpp-api/python/CMakeLists.txt


On Fri, Apr 30, 2021 at 5:45 PM hemant via lists.fd.io  wrote:

> Govind,
>
>
>
> Thanks!
>
>
>
> Hemant
>
>
>
> *From:* Govindarajan Mohandoss 
> *Sent:* Friday, April 30, 2021 5:43 PM
> *To:* hem...@mnkcg.com; vpp-dev@lists.fd.io
> *Cc:* Zachary Leaf ; nd 
> *Subject:* RE: [vpp-dev] vpp_papi.py error
>
>
>
> Hi Hemant,
>
>   We saw this issue and submitted a patch.
>
>https://gerrit.fd.io/r/c/vpp/+/32172
>
>
>
> Thanks
>
> Govind
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *hemant
> via lists.fd.io
> *Sent:* Friday, April 30, 2021 3:12 PM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] vpp_papi.py error
>
>
>
> I git cloned vpp gerrit repo yesterday and during using “sudo make
> build-release” I run into this error.  Is a bug filed for this issue?
> Python 2.7 doesn’t like a “*” for an arg.
>
>
>
> make[3]: Entering directory
> '/home/hemant/vpp/build-root/build-vpp-native/vpp'
>
>   File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 389
>
> def __init__(self, *, apifiles=None, testmode=False,
> async_thread=True,
>
> ^
>
> SyntaxError: invalid syntax
>
>
>
> Regards,
>
>
>
> Hemant
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19308): https://lists.fd.io/g/vpp-dev/message/19308
Mute This Topic: https://lists.fd.io/mt/82490885/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Changes to .api files

[Paul Vinciguerra]

Hi Everyone.

I'd like to propose three more changes/cleanup to the api files/vppapigen.

1. I'd like to remove the logic in vpppigen that imperatively declares the
service definitions.  Instead of calculating the values, we would output
the necessary values to be added to the api file.

2. I'd like to change the existing message definitions to use the
corresponding enums in place of the primitives that they decay to.
u8 dscp -> vl_api_dscp_t dscp

3. I'd like to change the flag type enums to use the api enumflag type.

Objections?

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19204): https://lists.fd.io/g/vpp-dev/message/19204
Mute This Topic: https://lists.fd.io/mt/82107376/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] Faster PAPI

[Paul Vinciguerra]

Transport after connect
> to stop message_thread and read from socket directly.
> Or use a different VppTransport implementation
> that does not start the thread (nor multiprocessing queues) in first place.
> 26019 does the former, 31920 does the latter.
> This bottleneck is the primary reason I started this conversation.
>
> Third bottleneck: Serialization of many commands (and deserialization of
> responses).
> I do not recall if the slower part is VPP PAPI code (vpp_serializer)
> or CSIT preparing arguments to serialize. Say, it is both.
> Many solutions to this one, important from CSIT reviewers point of view,
> but not that important for VPP developers (PAPI or otherwise).
>
> Fourth bottleneck: SSH forwarding volume of data between
> socket endpoints on different machines.
> Solution: Forward just a single command, and have a utility/plugin/whatever
> on VPP machine to execute the implied bulk work quickly.
> This sidesteps all the previous bottlenecks,
> so it is the secondary reason for this conversation.
>
> Personally, I do not like putting CSIT utilities on VPP machine,
> but sometimes it is the least evil (e.g. we do that for reading stats
> segment).
>
> 1a) with VAT1 is a fourth bottleneck solution,
> that is why we still use it.
> 6a) is also a fourth bottleneck solution, but with stable API.
>
> 26019 contains solutions for first, second and third bottleneck.
> 31920 is a solution for second bottleneck, assuming first bottleneck
> is solved as in 26019 and for third bottleneck we use 26019 or something
> else.
>
> 1b) solves (avoids) first and second bottleneck,
> but it has its own third bottleneck.
> Luckily, many binary data serialization solutions
> could be adapted for json data serialization.
>
> Vratko.
>
> -Original Message-
> From: otr...@employees.org 
> Sent: Thursday, 2021-February-04 08:47
> To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <
> vrpo...@cisco.com>
> Cc: Paul Vinciguerra ; Peter Mikus -X (pmikus
> - PANTHEON TECH SRO at Cisco) ; csit-...@lists.fd.io;
> vpp-dev@lists.fd.io
> Subject: Re: [csit-dev] Faster PAPI
>
> > 1. Keep the status quo.
> > That means using VAT for some operations (e.g. adding multiple routes
> [5]),
> > and creating "exec scripts" [6] for operations without VAT one-liner.
> > Pros: No work needed, good speed, old VPP versions are supported.
> > Cons: Relying on VAT functionality (outside API compatibility rules).
>
> 1b) Pre-generate a JSON file with all commands and load that into VPP with
> VAT2.
> 1c) Use a debug CLI script file and "exec"
>
> > 2. Support "vector operations" in VPP via binary API.
> > This will probably need a new VPP plugin to host the implementations.
> > Pros: Fast speed, small CSIT work, guarded by API compatibility rules.
> > Cons: New VPP plugin of questionable usefulness outside CSIT,
> > plugin maintainer needed, old VPP versions not supported.
>
> Could you ellaborate on this idea?
>
> > 3. VPP PAPI improvements only.
> > No changes to VPP API, just changes to PAPI to allow better speed for
> socket interactions.
> > CSIT would need a fast way to synthetize binary messages.
> > Pros: Small VPP work, good speed, only "official" VPP API is used.
> > Cons: Brittle CSIT message handling, old VPP versions not supported.
>
> What would this be?
> E.g. we could do serializations in C with a small Python wrapper.
>
> > 4. CSIT hacks only (Gerrit 26019).
> > No changes to VPP API nor PAPI. CSIT code messes with PAPI internals.
> > CSIT needs a fast way to synthetize binary messages.
> > Pros: Code is ready, good speed, old VPP versions are supported.
> > Cons: Brittle CSIT message handling, risky with respect to VPP PAPI
> changes.
>
> This is the idea explained below, where you serialize message once and
> replays right?
> In addition to tweaking the reader thread etc?
>
> 5) Paul's proposal. I don't know if he has measured performance impact on
> that.
>
> 6) The add 4 million routes that CSIT uses, generates routes according to
> a set of parameters.
>We could add a small plugin on the VPP side that exposes an API of the
> sort
>"create  routes from ".
>
> Is it the serialisation, or the message RTT or both?
> If serialisation, doing it in C should help.
> If it's the RTT, larger messages is one option.
> Uploading all messages to VPP and then ask VPP to process them, receiving
> a single reply is also an option.
> Is this your "vector" idea?
>
> > The open questions:
> > Do you see any other options?
> > Did I miss some important pros or cons?

Re: [vpp-dev] Faster PAPI

[Paul Vinciguerra]

>
> Do you mean this line?
>   vpp_transport_shmem.VppTransport.read = read_map_domains_get_mock
>
> I do not like it that much.
> I mean, it works for make test, as it is in the same git repository as
> PAPI code,
> so make test can be accommodated for any big PAPI edit in the same Gerrit
> Change.
> We want to avoid such tight coupling with any code outside VPP repo
> (which includes code inside CSIT repo).


That is actually not tightly coupled.  I just selected that as a sample of
a stream type message so folks could see I tested samples of all the
variants:
"map_domains_get": {
"reply": "map_domains_get_reply",
"stream": true,
"stream_msg": "map_domain_details"
}

We could rename them as NOT_map_domains_get.  It was selected because the
map api uses the new stream/cursor logic.

Yes, the monkey patching is tightly coupled to the transport code. It could
be replaced with a MockTransport but that code isn't present here.


On Wed, Apr 7, 2021 at 11:38 AM Vratko Polak -X (vrpolak - PANTHEON TECH
SRO at Cisco)  wrote:

> Finally, this got back high enough in my TODO list.
>
>
>
> > Have you looked at [0]?
>
> > [0] https://gerrit.fd.io/r/c/vpp/+/30350
>
>
>
> Do you mean this line?
>
>   vpp_transport_shmem.VppTransport.read = read_map_domains_get_mock
>
>
>
> I do not like it that much.
>
> I mean, it works for make test, as it is in the same git repository as
> PAPI code,
>
> so make test can be accommodated for any big PAPI edit in the same Gerrit
> Change.
>
> We want to avoid such tight coupling with any code outside VPP repo
>
> (which includes code inside CSIT repo).
>

>
> As you said:
>
> Poking in the values is an ingenious way to go about solving the problem,
> but it is going to make the systems too tightly coupled and create a
> maintenance headache.
>
>
>
> > Have you seen [0]?
>
> > [0] https://gerrit.fd.io/r/c/vpp/+/30361
>
>
>
> That looks better.
>
> You can have MakeTestMockTransport defined somewhere in tests/,
>
> and use that when appropriate.
>
> It may take a while to make vpp_papi APIs clean,
>
> as different transports have different requirements and capabilities,
>
> but definitely a step in the right direction.
>
>
>
> For a socket transport variant suitable for high performance,
>
> I imagine something like [3].
>
> Marked as WiP, as it is poorly documented and not tested at all yet.
>
> Based on master, as 30361 is not merged yet (and needs a rebase).
>
>
>
> With 30361, the new transport code from 31920 could be in CSIT,
>
> but I imagine other users may want to use it too
>
> (not probable for MakeTestMockTransport).
>
>
>
> What do you think?
>
>
>
> > Can I suggest that if you want to use the binary api (vapi) instead of
> papi, that you look at swig.  [1]
>
> > [1] http://www.swig.org/
>
>
>
> Not sure vapi+swig gives big enough performance benefit.
>
> In CSIT we want to keep all Python stuff in one machine (with Robot tests),
>
> and using socket (forwarded via SSH) to talk to a plain VPP machine.
>
>
>
> If we consider putting custom code onto the VPP machine,
>
> the range of possibilities increase considerably
>
> (extreme example: CSIT could start building its own VPP plugin
>
> to do direct C calls, similarly to what CLI and VAT code in VPP does).
>
>
>
> Vratko.
>
>
>
> [3] https://gerrit.fd.io/r/c/vpp/+/31920
>
>
>
> *From:* Paul Vinciguerra 
> *Sent:* Thursday, 2021-February-04 05:09
> *To:* Paul Vinciguerra 
> *Cc:* Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <
> vrpo...@cisco.com>; Ole Troan (otroan) ; Peter Mikus -X
> (pmikus - PANTHEON TECH SRO at Cisco) ;
> csit-...@lists.fd.io; vpp-dev@lists.fd.io
> *Subject:* Re: [vpp-dev] Faster PAPI
>
>
>
> Following up on Peter's comments in the changeset,
>
>
>
> Have you seen [0]?  It separates out the transports, so that you can
> implement/subclass your own without poking in values from your executor.
> Poking in the values is an ingenious way to go about solving the problem,
> but it is going to make the systems too tightly coupled and create a
> maintenance headache.  Even if we write unit tests against papi, we cannot
> anticipate how you might redefine the logic.  It looks relatively
> straightforward to move your methods into a new or subclassed transport.
>
>
>
> Can I suggest that if you want to use the binary api (vapi) instead of
> papi, that you look at swig.  [1]
>
>
>
> [0] https://gerrit.fd.io/r/c

Re: [vpp-dev] New Committer Proposal

[Paul Vinciguerra]

+1

> On Mar 31, 2021, at 1:58 PM, Damjan Marion via lists.fd.io 
>  wrote:
> 
> 
> Dear VPP Committers,
> 
> I would like to propose Roy Fan Zhang from Intel as a new VPP committer.
> Fan made significant contributions to the VPP including the async crypto
> infrastructure and crypto scheduler.
> Beside that I found that Fan is active in the community, and willing to help.
> 
> Please let me know if you agree/neutral/disagree with +1/0/-1 (committers 
> only please).
> 
> My +1 is here.
> 
> Thanks,
> 
> Damjan
> 
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19079): https://lists.fd.io/g/vpp-dev/message/19079
Mute This Topic: https://lists.fd.io/mt/81756505/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] keeping tests outside of src/

[Paul Vinciguerra]

There are 2 separate issues

>From your example:

def create_stream(self, src_if):
"""Create input packet stream for defined interface.
:param VppInterface src_if: Interface to create packet stream for.
"""
good_request = (Ether(dst=src_if.local_mac, src=src_if.remote_mac) /
IP(src=src_if.remote_ip4) /
UDP(sport=1234, dport=53) /
DNS(rd=1, qd=DNSQR(qname="bozo.clown.org")))

bad_request = (Ether(dst=src_if.local_mac, src=src_if.remote_mac) /
IP(src=src_if.remote_ip4) /
UDP(sport=1234, dport=53) /
DNS(rd=1, qd=DNSQR(qname="no.clown.org")))
pkts = [good_request, bad_request]
return pkts

Why do we generate this at runtime?  Create the packet stream once at test
development, not at runtime, every time the test is run.
Scapy is for python and is licensed gpl and requires the tests to be
licensed as gpl due to the imports.
libpnet is a rust crate and licensed MIT/Apache2.  If the packet is read as
a .pcap file, we have no licensing issues with packet creation and the test
creator is free to use whatever tool he/she chooses to craft the packet.
It also reduces the work to move to a different language for testing if
someone wishes to.

Dave Barach has shown, more than once on this list, how a packet can be
created from a text file by the vpp packet generator.  My point is that we
have options.

I agree, python review *should be* possible.  These are the same folks that
Andrew asserts should be the owners of the python tests.

Why are jvpp and govpp separate projects but vpp_papi requires someone to
have to download the sources?  Yes, there is a 3 year old package on pypi
[0], but it does not work on any supported version of VPP.  Python users
want to do python3 -m pip install vpp_papi and go.  Even if someone didn't
want to cut official releases, if it were stand alone, folks could use any
version they chose with just pip.  If folks want it in the source tree, use
a submodule.  We have the skillset in house. Florin does that in [1].

Paul

[0] https://pypi.org/project/vpp-papi/
[1] https://github.com/FDio/vsap

Paul

On Tue, Mar 30, 2021 at 3:40 PM  wrote:

> Doesn’t take much to find a Python example that uses vapi.  See
> https://github.com/FDio/vpp/blob/master/src/plugins/dns/test/test_dns.py
>
>
>
> What is wrong with using Scapy to specify packets that pcap or libpnet is
> being discussed?
>
>
>
> There is also lua and Snabb:  https://github.com/snabbco/snabb
>
>
>
> I suspect more engineers know Python than golang or rust, so code review
> should be possible.
>
>
>
> Hemant
>
>
>
> *From:* Paul Vinciguerra 
> *Sent:* Tuesday, March 30, 2021 12:00 PM
> *To:* hem...@mnkcg.com
> *Cc:* Damjan Marion ; Dave Wallace ;
> vpp-dev 
> *Subject:* Re: [vpp-dev] keeping tests outside of src/
>
>
>
> Hi Damjan,
>
>
>
> Funny you say that.  I *have* been looking into exactly that.  The issue
> is that the library ecosystem is scarce.  I have actually started adding
> some of our protocols to the libpnet crate to see if it is viable.
>
>
>
> #[packet]
>
> pub struct Snap {
> #[length="3"]
> org_code: Vec,
> ethertype: u16be,
> #[payload]
> data: Vec,
> }
> #[test]
> fn test_llc_builder() {
> let mut buff = [0; 3];
> let mut l = MutableLlcPacket();
> l.set_dsap(0xaa);
> l.set_ssap(0xbb);
> l.set_control(3);
> assert_eq! (buff, [0xaa, 0xbb, 3]);
> }
>
> I don't think it is viable to move to for tests yet, but it has promise
> and the licensing of libpnet is promising.  I would support moving to it,
> but there are some of the same issues that need to be addressed.
>
> If we move the packet creation to saved .pcap files, it facilitates moving
> tests to another language or framework.
> Some of the tests rely on external python packages today, presumably
> because the developer did not want to roll the code on their own.  Moving
> to a new language needs to address that.  The problems that exist with the
> tests is that imperative rules are forced on the test developers. Move
> configuration of VPP to a declarative mechanism and let the test developers
> focus on the action they wish to test.
>
> I see no issue with python as a language.  I see issues with how the group
> used python.  Compare/contrast the python use in /test to that of CSIT.
> The CSIT code is orders of magnitude better in my opinion.  Clean
> interfaces, proper documentation. Kudos to them.
>
> I don't care that the tests are in python. I care that there is an example
> of how to implement VPP in python, because it is, for the time being, the
> ling

Re: [vpp-dev] keeping tests outside of src/

[Paul Vinciguerra]

Hi Damjan,

Funny you say that.  I *have* been looking into exactly that.  The issue is
that the library ecosystem is scarce.  I have actually started adding some
of our protocols to the libpnet crate to see if it is viable.

#[packet]
pub struct Snap {
#[length="3"]
org_code: Vec,
ethertype: u16be,
#[payload]
data: Vec,
}
#[test]
fn test_llc_builder() {
let mut buff = [0; 3];
let mut l = MutableLlcPacket();
l.set_dsap(0xaa);
l.set_ssap(0xbb);
l.set_control(3);
assert_eq! (buff, [0xaa, 0xbb, 3]);
}

I don't think it is viable to move to for tests yet, but it has promise and
the licensing of libpnet is promising.  I would support moving to it, but
there are some of the same issues that need to be addressed.

If we move the packet creation to saved .pcap files, it facilitates moving
tests to another language or framework.
Some of the tests rely on external python packages today, presumably
because the developer did not want to roll the code on their own.  Moving
to a new language needs to address that.  The problems that exist with the
tests is that imperative rules are forced on the test developers. Move
configuration of VPP to a declarative mechanism and let the test developers
focus on the action they wish to test.

I see no issue with python as a language.  I see issues with how the group
used python.  Compare/contrast the python use in /test to that of CSIT.
The CSIT code is orders of magnitude better in my opinion.  Clean
interfaces, proper documentation. Kudos to them.

I don't care that the tests are in python. I care that there is an example
of how to implement VPP in python, because it is, for the time being, the
lingua franca of the netdevops world.  As it exists today, however, it is
really not usable.  The majority of the tests use vpp_ objects to
isolate consumers from the shortcomings of the api from a consumer's
perspective.

If you embrace Andew's position that the component maintainer "own's" the
tests for that component, maybe we should embrace it fully.  Let the
maintainer write the test code in any language they choose.

There is a whole spectrum of options available.  I'm not against rust.  I
have also been asked about moving the tests to go.  My opinion is the same.
The issues around tests exist irrespective of the language.  As a
maintainer of /test, the most frequent response I received is that "I don't
know python enough to review your change". How does moving to
rust/golang/ address that?

Is the problem that the tests are written in python, or that the tests rely
on papi, or that python vpp_objects are written in python to replace vom?
They're all considerations if we're thinking about starting over.

Paul


On Mon, Mar 29, 2021 at 7:45 PM  wrote:

> Hmm, most folks have moved from Python to Golang, not Rust.  Rust is
> better suited to replace C for data plane programming.
>
>
>
> Hemant
>
>
>
> *From:* Damjan Marion 
> *Sent:* Monday, March 29, 2021 4:27 PM
> *To:* Paul Vinciguerra 
> *Cc:* hem...@mnkcg.com; Dave Wallace ; vpp-dev <
> vpp-dev@lists.fd.io>
> *Subject:* Re: [vpp-dev] keeping tests outside of src/
>
>
>
>
>
> Dear Paul,
>
>
>
> I think we should stop using python and move to rust :)
>
>
>
> —
>
> Damjan
>
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19061): https://lists.fd.io/g/vpp-dev/message/19061
Mute This Topic: https://lists.fd.io/mt/81611239/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] keeping tests outside of src/

[Paul Vinciguerra]

Hi Damjan,

Moving the files has tradeoffs.  The relative value depends on the user's
workflow.  Most tests have their own launcher at the bottom of the
test_ file and at some point were able to run outside of the
Makefile/run_tests.
Relocating the files essentially breaks that, but if it is not used by
anyone, that could be ok.  We just need to clean up the code to remove
those remnants.

If we were to clean up and package framework.py and related files as a
python module, then the tests could be run from anywhere in the file
system.  If papi and the test framework were separated out of the repo,
building tests and applications would become 50x easier.  Why? Because
python would handle the files and their locations/namespaces within the
python venv.

It would look something like:

$ cd 
$ python3 -m pip install .
installing 
installing 
installing vpp_papi
$ python3 ./test_


None of these are insurmountable issues.  We have to agree, and we have to
agree to not punt on the associated hygiene.

I agree with Florin, I don't have a strong opinion as to where the files
reside.  Unless, we can agree to move the tests to a separate repository,
then I'd prefer to have them all in a single directory.  Why? Because it
would be trivial to run the stable release's tests against vpp master and
know what code I need to update across versions.

While we are on the subject of tests, I would like to socialize a few ideas
for improvement.

   1. Can we stop building packets in the test files?  Yes, I know we need
   packets to test against, but can we just have the test file pass an
   associated pcap file, since that is what we send to the pg to run the
   test.   A side effect of this is that the need for scapy in the tests is
   reduced.  There is a video somewhere where all the different methods to sum
   up all the numbers from 1..1_000_000 are compared in python and the final
   statement of the video was to just use n(n + 1) / 2.
   2. Can we make the testing packets more useful?  We typically take a
   single packet and replicate it a number of times to ensure that the packet
   takes each path in the node.  Additionally, the packet value choices aren't
   always useful.  When the source port == the dest port, for example, typos
   in the code under test can be missed.  Even better would be to increment
   ephemeral port values.
   3. I would like to see vpp add a stride width value (1,2,4,8) to a
   node's registration info that was accessible to the test via an api, even
   if the api were only in the unit test plugin.  This way, the tests can
   calculate the optimal packet stream characteristics, or at the very least
   warn that a given test doesn't sufficiently test the component.


Paul


On Fri, Mar 26, 2021 at 3:15 PM hemant via lists.fd.io  wrote:

> Damjan,
>
> Thanks for clarifying - I agree with you on vnet tests.
>
> Hemant
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Damjan
> Marion via lists.fd.io
> Sent: Friday, March 26, 2021 12:38 PM
> To: Dave Wallace 
> Cc: vpp-dev 
> Subject: Re: [vpp-dev] keeping tests outside of src/
>
>
>
> > On 25.03.2021., at 21:14, Dave Wallace  wrote:
> >
> > Hi Damjan,
> >
> > This initiative originated with the wider adoption of plugin development
> at the request of Dave Barach to allow the development of plugins outside
> the VPP repo.  After completing the job for plugins, there were several
> requests to extend that to all of the features.  Presumably this was
> coupled with the desire to migrate feature source code from vnet into the
> plugin arena, but I don't recall all of the details of the discussion.
>
> OK, there are 2 different things. One is testing of out-of-tree plugins,
> another one is VPP tree layout.
> I fully agree that we need to support testing of out-of-tree plugins. Than
> can be fixed as simple as ‘make test
> TEST_DIR=/path/to/out-of-tree-plugin/test’.
>
> I don’t see how those two things relate.
>
> >
> > Unfortunately, this effort stalled across several releases due to lack
> of cycles and I'm just now in the process of completing the job.
> >
> > I'm perfectly ok accepting a -2 for test code that maintainers prefer to
> leave in .../vpp/test, but I don't see the original requirement to
> co-locate plugin source & test code going away.  So the majority of the
> feature source & test code will remain structured that way and the end
> result will be inconsistent at best.
>
> I’m not trying ot say we should -2, and I know that you submitted those
> patches believing that this is the right thing to do. I am just under
> impression that we are all not on the same page what is right thing to do.
>
>
> >
> > Personally, I think that it makes sense to continue to move features
> source, test code, and documentation to be co-located in a modular and
> consistent sub-tree structure.  I also see value in migrating features out
> of vnet into the plugin sub-tree.
>
> I disagree here, I believe it should stay separate. But 

Re: [vpp-dev] keeping tests outside of src/

[Paul Vinciguerra]

Hi Damian.

Why do you care?  I just looked in the flowprobe folder, for example, and
there is a flowprobe_test.c.  We have tests in the src tree.  I'm not
trying to argue for or against this, I'm trying to understand why.

We can make out of tree tests work without moving the files.  The test
runner has a custom discovery method. We just need to agree on how we want
it to behave.  What is the process used for out of tree plugins?  What has
worked for me is to create a new repository and add the repo as a git
submodule under plugins.

I'm glad Dave provided some background.  From my vantage point, Andrew said
on the list a while back that the maintainers owned the tests and shortly
thereafter, the tests started to be rearranged under their modules.

I do however want to raise the scapy licensing issue.  If I recall
correctly, there was some talk from the tsc about tagging the test folder
to indicate that the use of scapy required different licensing, so a new
plan would need to be devised.  I was doing some research on this, and the
FRR folks seem to have addressed the licensing issue around the scapy code,
by not importing it but rather by moving it to a single file that they fork
to run.  See:
https://github.com/FRRouting/frr/blob/master/tests/topotests/lib/send_bsr_packet.py

Paul

On Thu, Mar 25, 2021 at 4:13 PM Dave Wallace  wrote:

> Hi Damjan,
>
> This initiative originated with the wider adoption of plugin development
> at the request of Dave Barach to allow the development of plugins outside
> the VPP repo.  After completing the job for plugins, there were several
> requests to extend that to all of the features.  Presumably this was
> coupled with the desire to migrate feature source code from vnet into the
> plugin arena, but I don't recall all of the details of the discussion.
>
> Unfortunately, this effort stalled across several releases due to lack of
> cycles and I'm just now in the process of completing the job.
>
> I'm perfectly ok accepting a -2 for test code that maintainers prefer to
> leave in .../vpp/test, but I don't see the original requirement to
> co-locate plugin source & test code going away.  So the majority of the
> feature source & test code will remain structured that way and the end
> result will be inconsistent at best.
>
> Personally, I think that it makes sense to continue to move features
> source, test code, and documentation to be co-located in a modular and
> consistent sub-tree structure.  I also see value in migrating features out
> of vnet into the plugin sub-tree.
>
> For what its worth, the changes to test/Makefile gather all of the source
> as soft links into the build tree (.../vpp/build-root/build-test/src), but
> I understand that is not the same your original plan.
>
> Thanks,
> -daw-
>
> On 3/25/2021 3:16 PM, Damjan Marion via lists.fd.io wrote:
>
> Hi,
>
> It may be that it is not discussed or i was just ignorant, but I noticed that 
> there is ongoing activity to scatter tests all across the src/.
> When I started "make test" long long time ago i intentionally put it to 
> separate tree following the pattern from other projects and to be honest
> it makes me more sense that all tests are contained in the separate tree.
>
> Are we sure that this test file scatter activity is right thing to do?
> Anyone aware of any other project doing the same?
>
> —
> Damjan
>
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19029): https://lists.fd.io/g/vpp-dev/message/19029
Mute This Topic: https://lists.fd.io/mt/81611239/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] tests - attach debug option available

[Paul Vinciguerra]

Hi Klement,

I disagree with the implementation.  Naveen proposed similar functionality
a while back.  I objected to that for the same reason.

It does not make sense to use a testcase written to popen an instance of
vpp, then mock out the vpp instance with dummy values and conditionally
check and skip the logic distributed across the class.

To accomplish your goal, the baseclass should be refactored from the popen
code/logic.  Your change would then use the baseclass and the CI would use
the subclass, or a mixin.

If you look at the csit python code, they use a papi executor class. The
test cases depend on the vppapiclient and the stats client. They should be
the only dependencies in the base class, anything else is a concrete
implementation detail.

Paul



On Tue, Mar 16, 2021 at 12:35 PM Klement Sekera via lists.fd.io  wrote:

> Hi all,
>
> I implemented a new debug option for make test called ‘attach’. This has
> been requested a couple of times over the last few months and while it has
> some drawbacks, it also has advantages.
>
> It’s not merged yet.
>
> https://gerrit.fd.io/r/c/vpp/+/31663
>
> As always make test-help is full of clues on how to use it.
>
> TLDR:
>
> window 1: make test-start-vpp-debug-in-gdb (set breakpoints, … whatever
> and then run vpp from within gdb)
> window 2: make test
> TEST=test_nat44_ed.TestNAT44ED.test_outside_address_distribution
>
> NOTE: a lot of tests rely on having a fresh VPP, so repeat test runs
> without restarting vpp are questionable at best. Running more than one test
> class will most probably work only by accident.
> NOTE #2: it doesn’t matter if it’s make test or make test-debug in window
> 2 as the binary is selected in window 1. Use test-start-vpp-in-gdb to debug
> release binary.
>
> Feedback is much appreciated.
>
> Regards,
> Klement
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18971): https://lists.fd.io/g/vpp-dev/message/18971
Mute This Topic: https://lists.fd.io/mt/81380736/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [vpp-api-dev] vppapigen: Backwards compatible API dictionary removal

[Paul Vinciguerra]

Hi Ole,

Thank you for this, but it is not enough.  I don't mean that to come across
ungratefully, but it doesn't fix the problem.

We need to also remove backward compatible tags from the enums in the .api
files.  The need for this arose from the fact that enums and enum flags got
mixed together a while back.  An enum flag is backward compatible, in that
it's size never changes without a redefinition in the .api file.  An enum,
on the other hand, is a list of values, and 5 elements != 6 elements.

A manifest is not a CRC.  Vppapigen could trivially generate a separate
manifest with custom logic without breaking the metadata of the api.  I
would not recommend that route.  I already did the work of supporting
plugins with vppapigen which was already merged, as well as turning
vppapigen into a module [0] which is importable.
I stopped pursuing it because the community feedback was against replacing
the existing symlink in the build.  I instead published a fork of it on
pypi:  [1].  I would prefer that there was only one code base.

The rationale behind refactoring and contributing the code was to empower
anyone (like say CSIT) to have control over how they wanted to consume the
data.  The csit job could run its own manifest generating logic based on
their own needs and the two stay loosely coupled.

[0] https://gerrit.fd.io/r/c/vpp/+/30080
[1] https://pypi.org/project/vppapigen/

Paul

On Fri, Mar 5, 2021 at 4:22 AM Ole Troan  wrote:

> API users,
>
> This summer we found an error in the crc generation and to avoid forcing
> API users to update their manifests on short notice I put in 9f84e70c6d.
> Which "cheats" and returns the old crc value instead of the correct crc
> value for a set of messages.
>
> https://git.fd.io/vpp/tree/src/tools/vppapigen/vppapigen.py#n1143
>
>
> In time for the next release I'd like to remove the "crc compatibility
> table" from the API compiler.
>
> Paul has already a patch for that:
> https://gerrit.fd.io/r/c/vpp/+/30345
>
> The APIs themselves will not change, only the CRCs. This would affect
> anyone maintaining manifests.
> I know CSIT is affected, but are there others too that will be affected by
> such a change?
>
> I'll leave this issue open for 2 weeks or so, and unless there are
> objections I'd like to have 30345 merged before next release.
>
> Best regards,
> Ole
>
>
>
> 
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18867): https://lists.fd.io/g/vpp-dev/message/18867
Mute This Topic: https://lists.fd.io/mt/81108231/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Performance of PAPI request in multi threaded VPP

[Paul Vinciguerra]

Hi Xuo,

Neale is the maintainer of the policer code.  Maybe he can chime in if it
is safe to simply mark the binary apis as multiple parallel vector safe
`am->is_mp_safe`.  There is a brief reference to this in the developer docs
[0].


[0]
https://fd.io/docs/vpp/master/gettingstarted/developers/binary_api_support.html


On Thu, Feb 25, 2021 at 10:20 AM Xuo Guoto via lists.fd.io  wrote:

> Hi List,
>
> We have been using policer_add_del and classify_add_del_session in single
> threaded VPP (ie one main thread only) and both API were giving decent
> performance, but after switching to multi thread VPP the performance seems
> be drastically less.
>
> To test this out a small test program was written which will add 10,000
> policer and classify table entries and measure the speed.
>
> In single threaded VPP the program took 2.19 sec while with 1 main and 2
> worker threads it took 115.89 sec. The tests were conducted without any
> traffic flowing through VPP.
>
> The python test program too is attached for reference.
>
> Platform and version are:
>
> vpp# sh version
> vpp v21.01.0-1~gfa065f96d built by root on ubuntu20-04 at
> 2021-02-24T09:00:32
>
> vpp# sh cpu
> Model name:   Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz
> Microarch model (family): [0x6] Skylake ([0x55] Skylake X/SP) stepping 0x7
> Flags:sse3 pclmulqdq ssse3 sse41 sse42 avx rdrand avx2
> pqm pqe avx512f rdseed aes avx512_vnni invariant_tsc
> Base frequency:   2.09 GHz
> vpp#
>
> vpp# sh thread
> ID NameTypeLWP Sched Policy (Priority)
> lcore  Core   Socket State
> 0  vpp_main4230other (0)
> 1  7  0
> 1  vpp_wk_0workers 4243other (0)
> 2  1  0
> 2  vpp_wk_1workers 4244other (0)
> 3  6  0
> vpp#
>
> corresponding classify table : classify table mask l3 ip4 src miss-next
> drop memory-size 800M
>
> Is this behavior expected? Can some thing be done to achieve performance
> similar to single threaded VPP while running VPP with multiple threads?
>
> X.
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18823): https://lists.fd.io/g/vpp-dev/message/18823
Mute This Topic: https://lists.fd.io/mt/80903834/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Make test failures on ubuntu-20.04 AARCH64

[Paul Vinciguerra]

Hi Dave,

test_cli.py verifies the timeout checking code is working.

The groundwork to clean up the framework can be found here.  [0]

The test framework cannot distinguish between a timeout and a more
serious problem, so for now we assume vpp has died.

For test, I would have liked to do something similar to what you do with
growing your vectors.  If it is a true Timeout, rerun the test with say 3/2
the failing value and output the timeout with the run.

We could have the testcases use different values for arm, but when load or
hw specs change, the tests will break again.

Paul

[0] https://gerrit.fd.io/r/c/vpp/+/24085



On Tue, Feb 9, 2021 at 8:37 AM Dave Barach  wrote:

> On aarch64, "make TEST=test_bihash test-debug" involves a debug CLI
> timeout which is right on the hairy edge of being too short.
>
>
>
> 05:29:45,470 Calling cli_inband('cmd':'test bihash threads 2 nbuckets
> 64000 careful 0 verbose 0\n','context':2,'_vl_msg_id':574)
>
> 05:29:45,471 TIMEOUT:: 5
>
>
>
> This seems like the right incantation to fix it in
> src/vppinfra/test/test_bihash.py, but apparently it is not:
>
>
>
> error = self.vapi.cli("test bihash threads 2 nbuckets" +
>
>   " 64000 careful 0 verbose 0", timeout=15)
>
>
>
> Please advise... Thanks... Dave
>
>
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Benoit Ganne
> (bganne) via lists.fd.io
> Sent: Tuesday, February 9, 2021 8:06 AM
> To: Dave Barach ; 'Ole Troan' ;
> 'Dave Wallace' ; vpp-dev@lists.fd.io
> Cc: 'Juraj Linkeš' 
> Subject: Re: [vpp-dev] Make test failures on ubuntu-20.04 AARCH64
>
>
>
> I am going to have a look at IKE.
>
>
>
> In case it is useful to others, thanks to QEMU you can run a Ubuntu 20.04
> ARM docker container on your x86 workstation:
>
>
>
> # step 0: add support for multiarch (must be done once after reboot)
> docker run --rm --privileged multiarch/qemu-user-static --reset
> --persistent yes --credential yes
>
>
>
> # step 1: create your chroot (must be done once - I am sharing my homedir
> with my chroot and same UID/GID) docker run --name aarch64_u2004
> --privileged --net host -v $HOME:$HOME -v /dev:/dev -td
> arm64v8/ubuntu:20.04 /bin/bash
>
>
>
> # step 2: update and add user inside container docker container exec
> aarch64_u2004 sh -c "apt -qy update && apt dist-upgrade -qy && apt install
> -qy vim sudo make git && groupadd -g $(id -rg) $USER && useradd -u $(id
> -ru) -g $(id -rg) -M -d $HOME -s /bin/bash $USER && echo '$USER ALL=(ALL)
> NOPASSWD:ALL' >> /etc/sudoers && echo aarch64_u2004 > /etc/debian_chroot"
>
>
>
> # step 3: log into ARM chroot
>
> docker container exec -it aarch64_deb10 /bin/login -f $USER
>
>
>
> Best
>
> ben
>
>
>
> > -Original Message-
>
> > From: vpp-dev@lists.fd.io  On Behalf Of Dave
>
> > Barach
>
> > Sent: mardi 9 février 2021 12:47
>
> > To: 'Ole Troan' ; 'Dave Wallace'
>
> > ; vpp-dev@lists.fd.io
>
> > Cc: 'Juraj Linkeš' 
>
> > Subject: Re: [vpp-dev] Make test failures on ubuntu-20.04 AARCH64
>
> >
>
> > Would be glad to take a look at the aarch64 bihash failure, modulo the
>
> > fact that the LF data center resources I've used before seem to have
>
> > had their credentials changed.
>
> >
>
> > Please unicast appropriate (ip, userid, password) tuples.
>
> >
>
> > Thanks... Dave
>
> >
>
> > ssh vppdev@10.30.51.65
>
> > Warning: Permanently added '10.30.51.65' (ECDSA) to the list of known
>
> > hosts.
>
> > vppdev@10.30.51.65's password:
>
> > Permission denied, please try again.
>
> > vppdev@10.30.51.65's password:
>
> > Permission denied, please try again.
>
> > vppdev@10.30.51.65's password:
>
> > Received disconnect from 10.30.51.65 port 22:2: Too many
>
> > authentication failures Disconnected from 10.30.51.65 port 22 $ ssh
>
> > vppdev@10.30.51.68
>
> > Warning: Permanently added '10.30.51.68' (ECDSA) to the list of known
>
> > hosts.
>
> > vppdev@10.30.51.68's password:
>
> > Permission denied, please try again.
>
> > vppdev@10.30.51.68's password:
>
> > Permission denied, please try again.
>
> > vppdev@10.30.51.68's password:
>
> > Received disconnect from 10.30.51.68 port 22:2: Too many
>
> > authentication failures Disconnected from 10.30.51.68 port 22
>
> >
>
> > -Original Message-
>
> > From: vpp-dev@lists.fd.io  On Behalf Of Ole Troan
>
> > Sent: Tuesday, February 9, 2021 5:20 AM
>
> > To: Dave Wallace ; vpp-dev@lists.fd.io
>
> > Cc: Juraj Linkeš 
>
> > Subject: Re: [vpp-dev] Make test failures on ubuntu-20.04 AARCH64
>
> >
>
> > And just to be clear, the failing tests are in the QUIC, bihash and
>
> > IKEv2 features.
>
> >
>
> > > [4]
>
> > https://logs.fd.io/sandbox/vex-yul-rot-jenkins-2/daw_30773_31019-vpp-
>
> > verify-
>
> > master-ubuntu2004-aarch64/4/
>
> > > vpp-unittest-QUICEchoIntMStreamTestCase-f5ywwo0l-FAILED
>
> > > vpp-unittest-TestBihash-_iohsbtr-FAILED
>
> > > vpp-unittest-TestInitiatorDelSAFromResponder-dgqdg78o-FAILED
>
> > > vpp-unittest-TestInitiatorKeepaliveMsg-0vo66yj2-FAILED
>
> 

Re: [vpp-dev] Scapy license in VPP

[Paul Vinciguerra]

This is good information.

Can we take a step back and approach this from a different perspective?
Why do we actually need scapy?
The general process is that we use scapy to create a separate pcap file
that is fed to the vpp packet generator.
If we instead create named pcap files that we pre-package with the tests,
we would 1) resolve this issue 2) speed up the actual execution of the
tests.

The only usage of scapy outside of the unit tests proper that comes to mind
is within test/utils.py

Paul

On Thu, Feb 4, 2021 at 9:54 AM Vratko Polak -X (vrpolak - PANTHEON TECH SRO
at Cisco)  wrote:

> Few quick comments and links.
>
>
>
> Turns out GPLv2-or-later is not by itself compatible with Apache [2],
>
> we need to use GPLv3 (which is compatible with both).
>
>
>
> The questions on dynamic linking are still open [3] [4] [5].
>
>
>
> We probably need to distinguish several Programs
>
> available from VPP repository.
>
> VPP PAPI (library) is a separate distribution
>
> from VPP test framework.
>
>
>
> When a user starts VPP test framework,
>
> linking with VPP PAPI in runtime,
>
> therefore creating a derived work.
>
> GPLv3 would apply to that Program,
>
> but it does not mean we need to mention GPLv3
>
> in VPP PAPI license headers.
>
>
>
> This only works because VPP PAPI does not share
>
> any code nor data structures specific to VPP test framework.
>
> Modules which link with Scapy do share data structures,
>
> so the "derived work" nature is apparent already in repo.
>
>
>
> Vratko.
>
>
>
> [2] https://www.gnu.org/licenses/license-list.html#apache2
>
> [3]
> https://tech.popdata.org/the-gpl-license-and-linking-still-unclear-after-30-years/
>
> [4] https://lwn.net/Articles/548216/
>
> [5]
> https://www.kilpatricktownsend.com/-/media/2019/IP-Desk-Reference-2015/A-Practical-Guide-to-Open-Source-Software.ashx?la=en=895293E77B462CF85E4D3E80AF8DC83143D0C165
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Vratko
> Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
> *Sent:* Friday, 2021-January-29 18:10
> *To:* Paul Vinciguerra 
> *Cc:* t...@lists.fd.io; Kinsella, Ray ;
> vpp-dev@lists.fd.io
> *Subject:* Re: [vpp-dev] Scapy license in VPP
>
>
>
> > Why do you say that vpp_papi need to be dual licensed?
>
>
>
> I think e-mail reply would be long,
>
> and I will need to address comments anyway,
>
> so I respond via a Gerrit change [1].
>
>
>
> Vratko.
>
>
>
> [1] https://gerrit.fd.io/r/c/vpp/+/31025
>
>
>
> *From:* Paul Vinciguerra 
> *Sent:* Friday, 2021-January-29 15:29
> *To:* Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <
> vrpo...@cisco.com>
> *Cc:* t...@lists.fd.io; Kinsella, Ray ;
> vpp-dev@lists.fd.io
> *Subject:* Re: [vpp-dev] Scapy license in VPP
>
>
>
> Why do you say that vpp_papi need to be dual licensed?
>
>
>
> On Thu, Jan 28, 2021 at 12:43 PM Vratko Polak -X (vrpolak - PANTHEON
> TECHNOLOGIES at Cisco) via lists.fd.io 
> wrote:
>
> First draft created [0] for the change that will switch
>
> licenses for Python files used together with Scapy.
>
>
>
> For some files, I was not sure whether they are used together with Scapy.
>
> One big detail is that vpp_papi needs to have dual license,
>
> as test framework integrates with it (and with scapy).
>
> If I understand the licensing logic correctly,
>
> CSIT tests can still choose to use vpp_papi under Apache license option.
>
> But we may need to discuss that with lawyers.
>
>
>
> Ray, you may need to upgrade your contributor-finding shell pipeline
>
> to cover all files I added the new license into.
>
>
>
> Vratko.
>
>
>
> [0] https://gerrit.fd.io/r/c/vpp/+/30998
>
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18673): https://lists.fd.io/g/vpp-dev/message/18673
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Faster PAPI

[Paul Vinciguerra]

Following up on Peter's comments in the changeset,

Have you seen [0]?  It separates out the transports, so that you can
implement/subclass your own without poking in values from your executor.
Poking in the values is an ingenious way to go about solving the problem,
but it is going to make the systems too tightly coupled and create a
maintenance headache.  Even if we write unit tests against papi, we cannot
anticipate how you might redefine the logic.  It looks relatively
straightforward to move your methods into a new or subclassed transport.

Can I suggest that if you want to use the binary api (vapi) instead of
papi, that you look at swig.  [1]

[0] https://gerrit.fd.io/r/c/vpp/+/30361
[1] http://www.swig.org/

On Wed, Feb 3, 2021 at 11:08 AM Paul Vinciguerra via lists.fd.io  wrote:

> Hi Vratko.
>
> Have you looked at [0]?
>
> [0] https://gerrit.fd.io/r/c/vpp/+/30350
>
> On Wed, Feb 3, 2021 at 9:44 AM Vratko Polak -X (vrpolak - PANTHEON TECH
> SRO at Cisco)  wrote:
>
>> > Hello people interested in PAPI (VPP's Python API client library).
>>
>>
>>
>> Hello again.
>>
>> This is an update e-mail, adding some information,
>>
>> while still asking basically the same questions.
>>
>>
>>
>> Since my first e-mail, there was some private communication,
>>
>> mostly related to reasons the vanilla performance is not good,
>>
>> and how improvements to VAT [2] can help.
>>
>>
>>
>> > The exact code change is [0], but that may be hard to review.
>>
>>
>>
>> The current patch set [3] is a little better.
>>
>>
>>
>> > For that I have created [1], which shows the changed VPP PAPI code.
>>
>>
>>
>> Still mostly unfinished, I need to familiarize better with shmem
>> transport.
>>
>>
>>
>> The main inputs came from Peter, who expressed
>>
>> dislike [4] on how brittle the fast binary message generation is,
>>
>> and he prefers "we will call a vector operation and PAPI just executes
>> it".
>>
>>
>>
>> Let me summarize the current options as I see them.
>>
>>
>>
>> 1. Keep the status quo.
>>
>> That means using VAT for some operations (e.g. adding multiple routes
>> [5]),
>>
>> and creating "exec scripts" [6] for operations without VAT one-liner.
>>
>> Pros: No work needed, good speed, old VPP versions are supported.
>>
>> Cons: Relying on VAT functionality (outside API compatibility rules).
>>
>>
>>
>> 2. Support "vector operations" in VPP via binary API.
>>
>> This will probably need a new VPP plugin to host the implementations.
>>
>> Pros: Fast speed, small CSIT work, guarded by API compatibility rules.
>>
>> Cons: New VPP plugin of questionable usefulness outside CSIT,
>>
>> plugin maintainer needed, old VPP versions not supported.
>>
>>
>>
>> 3. VPP PAPI improvements only.
>>
>> No changes to VPP API, just changes to PAPI to allow better speed for
>> socket interactions.
>>
>> CSIT would need a fast way to synthetize binary messages.
>>
>> Pros: Small VPP work, good speed, only "official" VPP API is used.
>>
>> Cons: Brittle CSIT message handling, old VPP versions not supported.
>>
>>
>>
>> 4. CSIT hacks only (Gerrit 26019).
>>
>> No changes to VPP API nor PAPI. CSIT code messes with PAPI internals.
>>
>> CSIT needs a fast way to synthetize binary messages.
>>
>> Pros: Code is ready, good speed, old VPP versions are supported.
>>
>> Cons: Brittle CSIT message handling, risky with respect to VPP PAPI
>> changes.
>>
>>
>>
>> The open questions:
>>
>> Do you see any other options?
>>
>> Did I miss some important pros or cons?
>>
>> Which option do you prefer?
>>
>>
>>
>> Vratko.
>>
>>
>>
>> [2] https://lists.fd.io/g/vpp-dev/topic/78362835#18092
>>
>> [3] https://gerrit.fd.io/r/c/csit/+/26019/140
>>
>> [4]
>> https://gerrit.fd.io/r/c/csit/+/26019/140#message-314d168d8951b539e588e644a875624f5ca3fb77
>>
>> [5]
>> https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/templates/vat/vpp_route_add.vat
>>
>> [6]
>> https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/libraries/python/TestConfig.py#L121-L150
>>
>>
>>
>> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Vratko
>> Polak -X (vrpolak - PANTHEON TECHNOLOGIE

Re: [vpp-dev] RFC: Enabling Gerrit Auto-Abandon job on VPP master

[Paul Vinciguerra]

Ben,

It was not intended toward you at all.  In fact, in my experience, you are
very helpful.  I was only looking to highlight cases that did not
specifically benefit me.

Would you like a different change for me to highlight?  How about
https://gerrit.fd.io/r/c/vpp/+/27349? That is mine, though.

Let me share with the list, another illustrative example of the actual
point I'm trying to make. I reached out to Florin and Ben about an ASAN
crash in the debug CI job.  I am the first to admit that I know nothing
about ASAN.  Ben promptly submitted a fix, but Florin said he was in the
process of a significant refactor and asked if it could be held back until
after then.

This is yet another reason why we should not auto-delete submissions.
Period.  Most of you have a VPP dev environment setup already, so tossing
in a changeset is no big thing.  But when folks go to the wiki, follow the
steps for all the hoops that they have to jump through to get a change
submitted, and for a change to be ignored is just wrong.  If someone cares
enough about the project to go through the effort, they deserve feedback.

A while back, Ole asked on the list, if he should set up a wiki page.  I
said I was against it, because anyone can edit a wiki page.  He, however,
is one of a very small group who can actually clear the backlog.  If we
truly want to grow the community, we have to act in ways that welcome folks
to the community.  Ignoring contributions and auto-deleting them is not
welcoming in my opinion.

If one thinks that auto-delete is a valid option, then auto-submit must be
equally valid.  Both address a MAINTAINERS lack of attention.  In an
auto-delete scenario, the maintainer is rewarded for ignoring the
changeset, is an auto-submit, he is not.

To your point, I agree with you.  I don't want to touch other MAINTAINERs
areas either, that's why I've been just leaving +1's, as I said in my
original post, as my practice lately.

Dave,
Let me know how I can help.





On Fri, Jan 29, 2021 at 9:51 AM Benoit Ganne (bganne) 
wrote:

> Hi Paul,
>
> as you refer to this specific story which is close to my heart (as I am
> the one who triggered the whole drama by -2'ed), let me clarify:
>
> > The Netgate folks had a changeset they were waiting a month or so for a
> > review, then they were told that it was too close to the release to merge
> > it.  It was merged, but the argument that "because we held you back,
> we're
> > going to hold you back some more" is very anti-community.
>
> I am not disagreeing we have an issue with the merging process, but this
> is not what happened for this specific change (see below).
> I apologized to Jon (patch author) and from the discussion I had with him
> we both agreed we were acting in good faith.
> Anyway, the net result of all this is I am now reluctant to review
> anything for which I am not a maintainer (so less review manpower).
>
> Here is the story from my point-of-view:
>  * I did a patch that I abandoned on Jon's request (because Jon's and my
> were trying to fix the same problem but Jon's was more complete) and then
> did initial reviews on Jon's patch when he pushed it: I was looking forward
> for it to be merged and spent time on it
>  * then I saw a number of updates by Jon (-2, -1, new patches) between
> patchset 2 on Nov 30 until the last significant change with Patchset 5 on
> Dec 7
> => I did not spent time to review waiting for the final version, because I
> understood Jon were still actively debugging/updating it. Reviews take
> time, so I'd rather review when it is considered as "done" by the author
>  * it is only when Jon pinged again that I looked at it again and asked
> Andrew and Dave, respectively because of their release manager hat and
> maintainer hat
>
> I am not saying it was the best way of managing it, and this is definitely
> not how it was received/understood. I understand that. But there was no
> nefarious anti-community intent.
>
> Best
> ben
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18630): https://lists.fd.io/g/vpp-dev/message/18630
Mute This Topic: https://lists.fd.io/mt/80169540/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Scapy license in VPP

[Paul Vinciguerra]

Why do you say that vpp_papi need to be dual licensed?

On Thu, Jan 28, 2021 at 12:43 PM Vratko Polak -X (vrpolak - PANTHEON
TECHNOLOGIES at Cisco) via lists.fd.io 
wrote:

> First draft created [0] for the change that will switch
>
> licenses for Python files used together with Scapy.
>
>
>
> For some files, I was not sure whether they are used together with Scapy.
>
> One big detail is that vpp_papi needs to have dual license,
>
> as test framework integrates with it (and with scapy).
>
> If I understand the licensing logic correctly,
>
> CSIT tests can still choose to use vpp_papi under Apache license option.
>
> But we may need to discuss that with lawyers.
>
>
>
> Ray, you may need to upgrade your contributor-finding shell pipeline
>
> to cover all files I added the new license into.
>
>
>
> Vratko.
>
>
>
> [0] https://gerrit.fd.io/r/c/vpp/+/30998
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18628): https://lists.fd.io/g/vpp-dev/message/18628
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] RFC: Enabling Gerrit Auto-Abandon job on VPP master

[Paul Vinciguerra]

I am a firm -1.
I already have a python script that generates maintainers, but I would
rather we look at the maintainer plugin which was contributed to gerrit by
Cisco.  https://gerrit.googlesource.com/plugins/maintainer/

As to Andrew's point, it is just as fair to propose that if a maintainer
hasn't addressed a verified +1 in T+30, it is merged and it is then it will
become a priority to the maintainer.

About a month or so ago, Ole +2'd and merged a change of mine that has been
sitting for 2 releases, because Andrew -1'd for doing some testing and then
left it.  Thank you, Ole.

When I first became a comitter, I submitted a change that Danjan -1'd that
the feedback was that it was agreed to not accept.  Damjan +2'd and
committed it on his own some 10 months later.  Thank you, Damjan.

There is a change in the system https://gerrit.fd.io/r/c/vpp/+/30559 that
allows a user to build vpp if the system language is Chinese.  The author
tagged the maintainer, and hasn't gotten a response.  I tested the patch
and +1'd it to help save some time for the maintainer.  A week or so later,
another comitter was tagged.  It is still sitting there.

When the question of adding worker tests to the framework came up, Klement
sent me a link to work he had done over a year earlier that was never
merged.

The Netgate folks had a changeset they were waiting a month or so for a
review, then they were told that it was too close to the release to merge
it.  It was merged, but the argument that "because we held you back, we're
going to hold you back some more" is very anti-community.

I have MANY changes that I rely on that I cherry pick into my own builds.
I had a change that added the names to the established unix sockets for
troubleshooting.  It was a good year old.  Neale saw it a month or so ago
and merged it.  The maintainers did not.

I have uncommitted changes right now that fix issues in the build system
that are blocking other changes from being merged.

Dave,
Let me know how I can help.

On Thu, Jan 28, 2021 at 11:36 AM Andrew Yourtchenko 
wrote:

> Dave,
>
> On 28 Jan 2021, at 16:59, Dave Wallace  wrote:
>
>  Andrew,
>
> The only issue I have with your proposal is that there is no way to
> implement it. Gerrit auto-abandon doesn't have a way to detect 'T'.
>
>
> We could approximate it by setting changeCleanup.abandonIfMergeable to
> false, it’s not perfect, of course, but I think it will take care of most
> of the true “dead” changes.
>
> It seems like we already do have gerrit do the indexing, so it’s no-cost
> operation.
>
>
>
> While I agree with your idea that 'T' should be based on a time other than
> the initial patch upload, I don't see a mechanism to do so.  It would
> appear to me that the only other solution would be to make the window
> larger and anything up to the release cycle duration (120 days) seems
> reasonable to me.
>
> The other point I would make is that auto-abandon is not deletion.  Thus
> it puts more ownership on the patch creator to poke the maintainer(s) for a
> review.  And I'm assuming that 'Restore' resets the clock on the
> auto-abandon trigger.
>
>
> True, and we could put the appropriate abandon message that would
> describes what to do and explains why, so people don’t get upset/annoyed.
>
>
>
> Lastly, I have not heard a counter-proposal from either you or Ole on how
> to clean up the current state of the queue.  We're wasting cycles and
> abusing the infra by letting the queue remain so large and it would be wise
> to address that sooner rather than later.
>
>
> If we decide to enable it, shout loudly on the list and give folks a month
> warning before actually enable it, to touch the changes they intend on
> working on. Then enable the plugin. Then start with whatever
> nagging/reporting process from clean slate on top of the plugin being
> already active.
>
> To be clear: I think that there are two separate things:
>
> 1)  having a boatload of super outdated open changes
> 2) avoiding such changes piling up in the future/improving the review time
>
> The enabling of the plugin proposal solves (1), which is an immediate
> technical problem.
>
> Ole’s proposal seems to me to be aimed to address the (1) via (2), so my
> comment was that I disagreed that the suggested approach of addressing (2)
> is going to work, thus the suggested tweak.
>
> I still think it’s fine to enable the plugin (with not deleting the
> mergeable changes config), and it will solve the (1), and then discuss how
> to tackle (2).
>
> —a
>
>
> Thanks,
> -daw-
>
> On 1/28/2021 4:29 AM, Andrew  Yourtchenko wrote:
>
> On 28 Jan 2021, at 10:10, Ole Troan  
>  wrote:
>
> My impression is that there is a disconnect between someone putting 
> something on gerrit and a vpp maintainer reviewing and contributor merging.
>
> Absolutely agree on that.
>
> As a project we certainly can do better on managing the stream of changes. 
> With my release manager hat on, I have seen a non-trivial number of cases 
> where the 

Re: [vpp-dev] Building on Ubuntu 20.10

[Paul Vinciguerra]

Hi Pim.

If you look at your change, it was labeled -1.  When you dig through the
logs, you will see that the commit message needs to be updated.
The title needs to start with build: and Type: should be improvement.
After it verifies with a +1, you are ready to reach out to a maintainer.

See .git_commit_template.txt for more info.

If you look in the maintainers file, you will see:

Build System
I: build
M: Damjan Marion 
F: Makefile
F: src/CMakeLists.txt
F: src/cmake/
F: build/


Paul

On Mon, Dec 28, 2020 at 6:21 PM Pim van Pelt  wrote:

> Hoi,
>
> After upgrading my workstation to `Groovy Gorilla`, I noticed that libffi
> got changed. To be able to compile on Ubuntu 20.10, adding a small
> OS_VERSION_ID override is necessary.
> See https://gerrit.fd.io/r/c/vpp/+/30552 for the diff on toplevel
> Makefile. This is my first change to VPP and I'm following the wiki
> 
>  but
> unsure who to contact (SMEs page
>  was useful, but not for the
> build subsystem itself). Closest activity I could find was ayourtch@ due
> to the upcoming 21.01 release.
>
> If somebody can help me navigate and assign a reviewer, I'd be grateful.
>
> groet,
> Pim
>
>
> --
> Pim van Pelt 
> PBVP1-RIPE - http://www.ipng.nl/
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18450): https://lists.fd.io/g/vpp-dev/message/18450
Mute This Topic: https://lists.fd.io/mt/79280624/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multipoint GRE support

[Paul Vinciguerra]

Hi Vijay.

Those cli commands come from the wiki url provided.
The wiki was actually incorrect. The command is:

create teib  peer  nh  [nh-table-id ]

(Which I just updated)



On Fri, Dec 25, 2020 at 11:19 AM Vijay Kumar  wrote:

> Hi Paul,
>
> Thanks for the response.
> Could you clarify the below two points based out of your response?
>
> 1) Can the TEIB entry be filled by CLI and does TEIB concept apply for GRE
> also(besides IPIP)?
>
> 2) I assume the address 192.168.0.2  points to the GRE destination that's
> dynamically discovered. While the 10.1.1.1 address in your example below
> refer to the tunnel src address hosted in VPP?
>
>
>
> Regards
>
>
>
> On Fri, 25 Dec 2020, 04:49 Paul Vinciguerra, 
> wrote:
>
>> Hi Vijay,
>>
>> The Tunnel Endpoint Information Base (TEIB), is the database of
>> mappings.  It was formerly called NHRP in VPP.
>> It can be populated:
>>
>>1. via the 'tieb ipip0 peer 192.168.0.2 nh 10.1.1.1' command
>>2. or when the SA is established, via 'ipsec tunnel protect ipip0
>>sa-in 20 sa-out 30 nh 192.168.0.2'
>>
>> See the ipsec wiki page [0] for more info.
>>
>> [0] https://wiki.fd.io/view/VPP/IPSec
>>
>> On Thu, Dec 24, 2020 at 1:59 PM Vijay Kumar 
>> wrote:
>>
>>> Hi Paul,
>>>
>>> If you know about the below two questions that I asked in my last reply,
>>> could you plz answer?
>>>
>>>
>>> Regards.
>>>
>>> On Thu, 24 Dec 2020, 00:02 Vijay Kumar via lists.fd.io, >> gmail@lists.fd.io> wrote:
>>>
>>>> Hi Paul,
>>>>
>>>> Question 1
>>>> By physical addresses you mean the routable public IPs that form the
>>>> tunnel src and tunnel dst addresses?
>>>>
>>>> In my use case there is a GRE traffic that is originating from 5G
>>>> mobiles and is received by the N3IWF (wifi gateway) as shown in the below
>>>> topo. The GRE traffic generated by UE is over IPSEC (the GRE pkts have ESP
>>>> encap). In the production, there can be several thousands of 5G UEs that
>>>> generate gre-over-ipsec traffic that will terminate on a single p2mp
>>>> (multipoint) GRE interface on the N3IWF gateway as shown below.
>>>>
>>>> Question 2
>>>> Since GRE is always encapsulated in IPSEC tunnel, do we still need to
>>>> need to map the GRE tunnel addresses to physical addresses. I thought we
>>>> would not be need GRE physical addresses in our use-case as there is the
>>>> IPSEC that is always outermost hdr and would be used for routing
>>>>
>>>>
>>>> Topo
>>>> 
>>>> UE --(gre-over-ipsec traffic)--N3IWF
>>>>
>>>> On Wed, Dec 23, 2020 at 10:12 AM Paul Vinciguerra <
>>>> pvi...@vinciconsulting.com> wrote:
>>>>
>>>>> Hi Vijay,
>>>>>
>>>>> How are you planning to map the tunnel addresses to the
>>>>> physical addresses?
>>>>>
>>>>> On Tue, Dec 22, 2020 at 9:04 PM Vijay Kumar 
>>>>> wrote:
>>>>>
>>>>>> Hi Paul,
>>>>>>
>>>>>> Thanks for the information.
>>>>>>
>>>>>> Your script is talking about nhrp protocol.
>>>>>> Is NHRP protocol mandatory to support mGRE?
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> On Wed, 23 Dec 2020, 01:24 Paul Vinciguerra, <
>>>>>> pvi...@vinciconsulting.com> wrote:
>>>>>>
>>>>>>> Hi Vijay.
>>>>>>>
>>>>>>> Does this help any?
>>>>>>> https://github.com/vpp-dev/vpp/blob/master/test/test_gre.py#L998
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Dec 22, 2020 at 12:47 PM Vijay Kumar 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Can someone help me understand if multipoint GRE (one gre interface
>>>>>>>> that can communicate with multiple peers) is supported in the fd.ip GRE
>>>>>>>> plugin?
>>>>>>>>
>>>>>>>> If yes, could you please share with me an example config for
>>>>>>>> multi-point GRE. In the fd.io wiki pages, I am only seeing *p2mp 
>>>>>>>> *configuration
>>>>>>>> (point to multipoint) for IP-in-IP but not for GRE
>>>>>>>>
>>>>>>>>
>>>>>>>> Please share me an example config that I can use to test
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>> 
>>>>
>>>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18430): https://lists.fd.io/g/vpp-dev/message/18430
Mute This Topic: https://lists.fd.io/mt/79154511/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP crashes when route is not reachable

[Paul Vinciguerra]

Hi Marcos,

I'm not sure what you're asking here.  Reinstalling VPP should give you the
same outcome.

The log you provided, points to vnet[2669]: #2 0x7f1e6cb16f11
ip4_lookup_node_fn + 0xad1.
To troubleshoot further, run gdb on the core file
 # gdb  

You may need to install the debug rpm of VPP, vpp-dbg, but do not run this
file in production.  My experience is that if you run the debug image
outside of a lab, it will likely crash.


root@ae2a2e85c5d4:/vpp/src# grep -r "ASSERT (0)" * | wc -l


127


On Thu, Dec 24, 2020 at 1:11 PM Marcos - Mgiga  wrote:

> Hi Paul,
>
>
>
> Ok, I will get the core file, but is this situation unusual ? Because if
> it is something so particular, maybe I should try reinstalling VPP.
>
>
>
> Best Regards
>
>
>
> *De:* vpp-dev@lists.fd.io  *Em nome de *Paul
> Vinciguerra
> *Enviada em:* quinta-feira, 24 de dezembro de 2020 14:26
> *Para:* Marcos - Mgiga 
> *Cc:* vpp-dev 
> *Assunto:* Re: [vpp-dev] VPP crashes when route is not reachable
>
>
>
> Hi Marcos,
>
>
>
> You can still edit the systemd service file.  That should give you a core
> file if vpp crashes.
>
>
>
>
>
>
>
> On Wed, Dec 23, 2020 at 12:46 PM Marcos - Mgiga 
> wrote:
>
> Hello Paul,
>
>
>
> No I haven’t. Actually I’ve installed VPP from packege manager. Do you
> think building VPP from source is a better way to deploy it?
>
>
>
> Best Regards
>
>
>
>
>
> Marcos
>
>
>
> *De:* vpp-dev@lists.fd.io  *Em nome de *Paul
> Vinciguerra
> *Enviada em:* quarta-feira, 23 de dezembro de 2020 13:33
> *Para:* Marcos - Mgiga 
> *Cc:* vpp-dev@lists.fd.io
> *Assunto:* Re: [vpp-dev] VPP crashes when route is not reachable
>
>
>
> Hi Marcos,
>
>
>
> Have you edited vpp.service and in commented the limitcore line?
>
>
>
>
> https://fdio-vpp.readthedocs.io/en/latest/troubleshooting/reportingissues/reportingissues.html
>
>
>
> On Dec 23, 2020, at 8:09 AM, Marcos - Mgiga  wrote:
>
> 
>
> Hi there,
>
>
>
> I’m trying to setup the attached topology (VPPTOPOLOGY.png) in order to
> work with VPP as CGN deterministic gateway.  VPP is intend to replace
> another CGN software, so the link between VPP and BGNRouter is not working
> yet.
>
>
>
> So I first linked VPP to Layer2 switch and I was able to ping border
> router from vpp and vice versa.
>
>
>
> When I add a static route on border router pointed to some of the networks
> specified as “VPP ROUTING TABLE” on the drawing and start a ping, VPP
> immediately crashes.
>
>
>
> I attached all VPP config files and a outfrom from “journalctl -u vpp” at
> the moment of the crash, called error vpp.txt.
>
>
>
> Hope that helps, I wasn’t able to get a trace since VPP stops working when
> that happens.
>
>
>
> Best Regards
>
>
>
> Marcos
>
> 
>
> 
>
> 
>
> 
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18429): https://lists.fd.io/g/vpp-dev/message/18429
Mute This Topic: https://lists.fd.io/mt/79238862/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multipoint GRE support

[Paul Vinciguerra]

Hi Vijay,

The Tunnel Endpoint Information Base (TEIB), is the database of mappings.
It was formerly called NHRP in VPP.
It can be populated:

   1. via the 'tieb ipip0 peer 192.168.0.2 nh 10.1.1.1' command
   2. or when the SA is established, via 'ipsec tunnel protect ipip0 sa-in
   20 sa-out 30 nh 192.168.0.2'

See the ipsec wiki page [0] for more info.

[0] https://wiki.fd.io/view/VPP/IPSec

On Thu, Dec 24, 2020 at 1:59 PM Vijay Kumar  wrote:

> Hi Paul,
>
> If you know about the below two questions that I asked in my last reply,
> could you plz answer?
>
>
> Regards.
>
> On Thu, 24 Dec 2020, 00:02 Vijay Kumar via lists.fd.io,  gmail@lists.fd.io> wrote:
>
>> Hi Paul,
>>
>> Question 1
>> By physical addresses you mean the routable public IPs that form the
>> tunnel src and tunnel dst addresses?
>>
>> In my use case there is a GRE traffic that is originating from 5G mobiles
>> and is received by the N3IWF (wifi gateway) as shown in the below topo. The
>> GRE traffic generated by UE is over IPSEC (the GRE pkts have ESP encap). In
>> the production, there can be several thousands of 5G UEs that generate
>> gre-over-ipsec traffic that will terminate on a single p2mp (multipoint)
>> GRE interface on the N3IWF gateway as shown below.
>>
>> Question 2
>> Since GRE is always encapsulated in IPSEC tunnel, do we still need to
>> need to map the GRE tunnel addresses to physical addresses. I thought we
>> would not be need GRE physical addresses in our use-case as there is the
>> IPSEC that is always outermost hdr and would be used for routing
>>
>>
>> Topo
>> 
>> UE --(gre-over-ipsec traffic)--N3IWF
>>
>> On Wed, Dec 23, 2020 at 10:12 AM Paul Vinciguerra <
>> pvi...@vinciconsulting.com> wrote:
>>
>>> Hi Vijay,
>>>
>>> How are you planning to map the tunnel addresses to the
>>> physical addresses?
>>>
>>> On Tue, Dec 22, 2020 at 9:04 PM Vijay Kumar 
>>> wrote:
>>>
>>>> Hi Paul,
>>>>
>>>> Thanks for the information.
>>>>
>>>> Your script is talking about nhrp protocol.
>>>> Is NHRP protocol mandatory to support mGRE?
>>>>
>>>>
>>>> Regards
>>>>
>>>> On Wed, 23 Dec 2020, 01:24 Paul Vinciguerra, <
>>>> pvi...@vinciconsulting.com> wrote:
>>>>
>>>>> Hi Vijay.
>>>>>
>>>>> Does this help any?
>>>>> https://github.com/vpp-dev/vpp/blob/master/test/test_gre.py#L998
>>>>>
>>>>>
>>>>> On Tue, Dec 22, 2020 at 12:47 PM Vijay Kumar 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Can someone help me understand if multipoint GRE (one gre interface
>>>>>> that can communicate with multiple peers) is supported in the fd.ip GRE
>>>>>> plugin?
>>>>>>
>>>>>> If yes, could you please share with me an example config for
>>>>>> multi-point GRE. In the fd.io wiki pages, I am only seeing *p2mp 
>>>>>> *configuration
>>>>>> (point to multipoint) for IP-in-IP but not for GRE
>>>>>>
>>>>>>
>>>>>> Please share me an example config that I can use to test
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>> 
>>
>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18427): https://lists.fd.io/g/vpp-dev/message/18427
Mute This Topic: https://lists.fd.io/mt/79154511/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP crashes when route is not reachable

[Paul Vinciguerra]

Hi Marcos,

You can still edit the systemd service file.  That should give you a core
file if vpp crashes.



On Wed, Dec 23, 2020 at 12:46 PM Marcos - Mgiga  wrote:

> Hello Paul,
>
>
>
> No I haven’t. Actually I’ve installed VPP from packege manager. Do you
> think building VPP from source is a better way to deploy it?
>
>
>
> Best Regards
>
>
>
>
>
> Marcos
>
>
>
> *De:* vpp-dev@lists.fd.io  *Em nome de *Paul
> Vinciguerra
> *Enviada em:* quarta-feira, 23 de dezembro de 2020 13:33
> *Para:* Marcos - Mgiga 
> *Cc:* vpp-dev@lists.fd.io
> *Assunto:* Re: [vpp-dev] VPP crashes when route is not reachable
>
>
>
> Hi Marcos,
>
>
>
> Have you edited vpp.service and in commented the limitcore line?
>
>
>
>
> https://fdio-vpp.readthedocs.io/en/latest/troubleshooting/reportingissues/reportingissues.html
>
>
>
> On Dec 23, 2020, at 8:09 AM, Marcos - Mgiga  wrote:
>
> 
>
> Hi there,
>
>
>
> I’m trying to setup the attached topology (VPPTOPOLOGY.png) in order to
> work with VPP as CGN deterministic gateway.  VPP is intend to replace
> another CGN software, so the link between VPP and BGNRouter is not working
> yet.
>
>
>
> So I first linked VPP to Layer2 switch and I was able to ping border
> router from vpp and vice versa.
>
>
>
> When I add a static route on border router pointed to some of the networks
> specified as “VPP ROUTING TABLE” on the drawing and start a ping, VPP
> immediately crashes.
>
>
>
> I attached all VPP config files and a outfrom from “journalctl -u vpp” at
> the moment of the crash, called error vpp.txt.
>
>
>
> Hope that helps, I wasn’t able to get a trace since VPP stops working when
> that happens.
>
>
>
> Best Regards
>
>
>
> Marcos
>
> 
>
> 
>
> 
>
> 
>
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18423): https://lists.fd.io/g/vpp-dev/message/18423
Mute This Topic: https://lists.fd.io/mt/79206267/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP crashes when route is not reachable

[Paul Vinciguerra]

Hi Marcos,

Have you edited vpp.service and in commented the limitcore line?

https://fdio-vpp.readthedocs.io/en/latest/troubleshooting/reportingissues/reportingissues.html

> On Dec 23, 2020, at 8:09 AM, Marcos - Mgiga  wrote:
> 
> 
> Hi there,
>
> I’m trying to setup the attached topology (VPPTOPOLOGY.png) in order to work 
> with VPP as CGN deterministic gateway.  VPP is intend to replace another CGN 
> software, so the link between VPP and BGNRouter is not working yet.
>
> So I first linked VPP to Layer2 switch and I was able to ping border router 
> from vpp and vice versa.
>
> When I add a static route on border router pointed to some of the networks 
> specified as “VPP ROUTING TABLE” on the drawing and start a ping, VPP 
> immediately crashes.
>
> I attached all VPP config files and a outfrom from “journalctl -u vpp” at the 
> moment of the crash, called error vpp.txt.
>
> Hope that helps, I wasn’t able to get a trace since VPP stops working when 
> that happens.
>
> Best Regards
>
> Marcos
> 
> 
> 
> 
> 
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18418): https://lists.fd.io/g/vpp-dev/message/18418
Mute This Topic: https://lists.fd.io/mt/79179463/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multipoint GRE support

[Paul Vinciguerra]

Hi Vijay,

How are you planning to map the tunnel addresses to the physical addresses?

On Tue, Dec 22, 2020 at 9:04 PM Vijay Kumar  wrote:

> Hi Paul,
>
> Thanks for the information.
>
> Your script is talking about nhrp protocol.
> Is NHRP protocol mandatory to support mGRE?
>
>
> Regards
>
> On Wed, 23 Dec 2020, 01:24 Paul Vinciguerra, 
> wrote:
>
>> Hi Vijay.
>>
>> Does this help any?
>> https://github.com/vpp-dev/vpp/blob/master/test/test_gre.py#L998
>>
>>
>> On Tue, Dec 22, 2020 at 12:47 PM Vijay Kumar 
>> wrote:
>>
>>> Hi,
>>>
>>> Can someone help me understand if multipoint GRE (one gre interface that
>>> can communicate with multiple peers) is supported in the fd.ip GRE plugin?
>>>
>>> If yes, could you please share with me an example config for multi-point
>>> GRE. In the fd.io wiki pages, I am only seeing *p2mp *configuration
>>> (point to multipoint) for IP-in-IP but not for GRE
>>>
>>>
>>> Please share me an example config that I can use to test
>>>
>>>
>>> Regards
>>>
>>> 
>>>
>>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18415): https://lists.fd.io/g/vpp-dev/message/18415
Mute This Topic: https://lists.fd.io/mt/79154511/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Message posting policy change

[Paul Vinciguerra]

+1

On Wed, Dec 16, 2020 at 8:26 AM Damjan Marion via lists.fd.io  wrote:

> makes sense to me. that is common practice.
>
> --
> Damjan
>
> On 16.12.2020., at 13:47, Dave Barach  wrote:
>
> 
>
> Folks,
>
>
>
> At the moment, we hold non-list-member posts in the vpp-dev moderation
> queue. Unfortunately, spam comprises almost all of the messages in the
> queue. I’ve been deleting 10-20 junk emails per day – “want to buy a list
> of conference attendees?” - and I would like to spend that time doing
> something else.
>
>
>
> Any objections to reconfiguring the vpp-dev@lists.fd.io group to reject
> messages from non-list-members? For those who participate sporadically,
> it’s easy to turn on/off message delivery. For regular list participants,
> nothing changes.
>
>
>
> Thanks... Dave
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18367): https://lists.fd.io/g/vpp-dev/message/18367
Mute This Topic: https://lists.fd.io/mt/78999410/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] MAINTAINER for extras/rpm?

[Paul Vinciguerra]

Hi Damjen,

Thanks for approving my change.

Hi Dave,

That was what I wanted to understand.  I think some folks here may use
rpm's.  If they don't want to take over the responsibility of maintaining
the spec, I'm +1 for removing the rpm build and saving the resources on the
CI.

I just know that it means that the docs and vpp_config would have to be
updated, but I don't know what else.

Paul

On Mon, Dec 14, 2020 at 6:50 PM Dave Wallace  wrote:

> I agree with Damjan.  It was orphaned by the RedHat folks who left the
> project a long time ago.
> I'd recommend removing centos support sooner rather than later as it is
> simply wasting energy at this point.
>
> Thanks,
> -daw-
>
> On 12/14/2020 1:57 PM, Paul Vinciguerra wrote:
>
> Is the maintenance burden the unwanted child, or the RPM itself?
>
> Now that it was announced that Centos 8 will be EOL'd at the end of 2021,
> yet Centos 7 EOL's in 2024, Should we go back to supporting Centos 7 or
> should we plan to drop the process of building rpms before Centos 8 ends
> support?
>
> On Mon, Dec 14, 2020 at 1:37 PM Damjan Marion  wrote:
>
>>
>> AFAIK it is unwanted child….
>>
>>
>> —
>> Damjan
>>
>>
>> On 14.12.2020., at 18:17, Paul Vinciguerra 
>> wrote:
>>
>> Is there a maintainer? There is no entry in the Maintainers file.
>>
>> https://gerrit.fd.io/r/c/vpp/+/30408
>> fixes failures on centos-8 due to an older cmake package.  I'd bet that
>> it probably breaks something that we're not testing, though.
>>
>>
>>
>>
>>
> 
>
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18337): https://lists.fd.io/g/vpp-dev/message/18337
Mute This Topic: https://lists.fd.io/mt/78955767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] MAINTAINER for extras/rpm?

[Paul Vinciguerra]

Is the maintenance burden the unwanted child, or the RPM itself?

Now that it was announced that Centos 8 will be EOL'd at the end of 2021,
yet Centos 7 EOL's in 2024, Should we go back to supporting Centos 7 or
should we plan to drop the process of building rpms before Centos 8 ends
support?

On Mon, Dec 14, 2020 at 1:37 PM Damjan Marion  wrote:

>
> AFAIK it is unwanted child….
>
>
> —
> Damjan
>
>
> On 14.12.2020., at 18:17, Paul Vinciguerra 
> wrote:
>
> Is there a maintainer? There is no entry in the Maintainers file.
>
> https://gerrit.fd.io/r/c/vpp/+/30408
> fixes failures on centos-8 due to an older cmake package.  I'd bet that it
> probably breaks something that we're not testing, though.
>
> 
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18334): https://lists.fd.io/g/vpp-dev/message/18334
Mute This Topic: https://lists.fd.io/mt/78955767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] MAINTAINER for extras/rpm?

[Paul Vinciguerra]

Is there a maintainer? There is no entry in the Maintainers file.

https://gerrit.fd.io/r/c/vpp/+/30408
fixes failures on centos-8 due to an older cmake package.  I'd bet that it
probably breaks something that we're not testing, though.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18332): https://lists.fd.io/g/vpp-dev/message/18332
Mute This Topic: https://lists.fd.io/mt/78955767/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Please review https://gerrit.fd.io/r/c/vpp/+/29754

[Paul Vinciguerra]

Thanks!

On Sun, Dec 13, 2020 at 6:48 AM Damjan Marion  wrote:

>
> I reverted that patch.
>
> —
> Damjan
>
> On 13.12.2020., at 05:23, Paul Vinciguerra 
> wrote:
>
> It looks like this change is causing vpp-verify-master-ubuntu1804-aarch64
> jobs to fail in the CI.
>
> Can someone take a look?
>
> ==
> FAIL: IPFIX logging maximum session entries exceeded
> --
> Traceback (most recent call last):
>   File 
> "/w/workspace/vpp-verify-master-ubuntu1804-aarch64/build-root/build-test/src/test_nat44.py",
>  line 2743, in test_ipfix_max_sessions
> self.verify_ipfix_max_sessions(data, max_sessions)
>   File 
> "/w/workspace/vpp-verify-master-ubuntu1804-aarch64/build-root/build-test/src/test_nat44.py",
>  line 762, in verify_ipfix_max_sessions
> self.assertEqual(struct.pack("I", 1), record[466])
> AssertionError: b'\x01\x00\x00\x00' != b'\x00\x00\x00\x01'
>
>
> 
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18330): https://lists.fd.io/g/vpp-dev/message/18330
Mute This Topic: https://lists.fd.io/mt/78921896/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Please review https://gerrit.fd.io/r/c/vpp/+/29754

[Paul Vinciguerra]

It looks like this change is causing vpp-verify-master-ubuntu1804-aarch64
jobs to fail in the CI.

Can someone take a look?

==
FAIL: IPFIX logging maximum session entries exceeded
--
Traceback (most recent call last):
  File 
"/w/workspace/vpp-verify-master-ubuntu1804-aarch64/build-root/build-test/src/test_nat44.py",
line 2743, in test_ipfix_max_sessions
self.verify_ipfix_max_sessions(data, max_sessions)
  File 
"/w/workspace/vpp-verify-master-ubuntu1804-aarch64/build-root/build-test/src/test_nat44.py",
line 762, in verify_ipfix_max_sessions
self.assertEqual(struct.pack("I", 1), record[466])
AssertionError: b'\x01\x00\x00\x00' != b'\x00\x00\x00\x01'

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18317): https://lists.fd.io/g/vpp-dev/message/18317
Mute This Topic: https://lists.fd.io/mt/78921896/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

On Wed, Dec 2, 2020 at 1:12 PM Andrew  Yourtchenko 
wrote:

> s/Got doesn’t have/Git doesn’t have/
>
> (iPhone autocorrect, sorry)
>
> --a
>
> On 2 Dec 2020, at 19:01, Andrew Yourtchenko via lists.fd.io  gmail@lists.fd.io> wrote:
>
> 
>
> On 2 Dec 2020, at 18:21, Paul Vinciguerra 
> wrote:
>
> 
>
>
> On Wed, Dec 2, 2020 at 10:57 AM Andrew  Yourtchenko 
> wrote:
>
>>
>>
>> On 2 Dec 2020, at 16:42, Paul Vinciguerra 
>> wrote:
>>
>> 
>>
>>
>> On Wed, Dec 2, 2020 at 5:38 AM Andrew  Yourtchenko 
>> wrote:
>>
>>>
>>>
>>> > On 2 Dec 2020, at 10:38, Klement Sekera via lists.fd.io >> cisco@lists.fd.io> wrote:
>>> >
>>> > I like the idea.
>>> >
>>> > Regarding maintainer pain:
>>> >
>>> > Q: are we now stuck forever with what we have because there will
>>> always be somebody facing some difficulties adopting?
>>>
>>> To me it is always a question of cost/benefit analysis - and admittedly
>>> the value of “cost” and “benefit” is always subjective, and not the same in
>>> time, so let me expand on why I suggested why suggested:
>>>
>>> - I can see a benefit in having a target “make test-fixstyle”, similar
>>> with C code, introduced sometime *just* before the LTS branch pull, such
>>> that we have a chance to be able to sync the changes.
>>>
>>> - I can see less benefit in introducing that target now, so soon after
>>> the LTS release - since that would mean the moment you run a .py file on
>>> master through the formatter, as a developer now you will have to manually
>>> deal with *two* versions of file - in master and LTS. And porting the diffs
>>> that don’t apply due to python formatting is a highly unhappy task.
>>>
>>
>> I don't know if git 2.23 helps you there.  If we make the change now
>> (figuratively), the issue goes away in theory this time next year as the
>> old releases go out of support.
>>
>>
>> Right. And until then nearly every change involving test code changes
>> will have to be backported to 20.09 manually by the feature developers. I
>> would it’s the net increase of work, with less trivial yet still very sad
>> job of dealing with reshuffling the patches. PEP501-type annoyance, but
>> with more lines and higher chances of error.
>>
>
> So, this is a different issue to me.  Because we don't like chained
> commits, this is an issue.  To do this properly in my mind, that is to make
> your job easier, there are dependent changes needed.  One tuning the build
> system to support the "black" pep8 output, the second with the changes
> introducing black and the file changes.
>
> If the first change were backported, then post black changes shouldn't
> cause the stable to barf on it.
>
>
> Got doesn’t have an idea about dependent changes.
>
> Git doesn't.  Gerrit and our workflow does.

>
>
>
>>
>>
>>> - Having this done automatically as a precommit hook and removing it
>>> from the checkstyle job means:
>>>
>>> 1) we trust the client to run the formatting themselves. This is a
>>> recipe to style inconsistency due to *whatever* unrelated things like
>>> merges, etc.
>>>
>>> 2) forcefully running it just seems like a unsound idea since then we
>>> are going to get arbitrary white space changes whenever someone of the
>>> formatter devs decides to change the python formatter. I tried formatters
>>> for C, golang, rust to name a few. *none* of them produce the identical
>>> result from differently white-spaced code, that is functionally identical,
>>> even if the formatter is part of the language.
>>>
>>> In addition this approach shares all the drawbacks of introducing the
>>> change at the inopportune moment.
>>>
>>
>> I'm not saying +2 the change today.  ;). I'm trying to socialize the idea
>> and see what folks think.
>>
>>
>> Ah ok :)
>>
>>
>>
>>>
>>> That’s the reason I suggested to just add the E501 to ignore and move on
>>> - on balance it seems like their relieves the maximum amount of real pain
>>> without introducing unknown amounts of new one.
>>>
>> Going to a line length of 88 (value taken from black) leaves us with only
>> 4-6 E501's (line too long) in test.
>>
>>
>> That is *today* with the existing “new meaningless variable” hacks
>> already in place.
>>
> Corr

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

On Wed, Dec 2, 2020 at 10:57 AM Andrew  Yourtchenko 
wrote:

>
>
> On 2 Dec 2020, at 16:42, Paul Vinciguerra 
> wrote:
>
> 
>
>
> On Wed, Dec 2, 2020 at 5:38 AM Andrew  Yourtchenko 
> wrote:
>
>>
>>
>> > On 2 Dec 2020, at 10:38, Klement Sekera via lists.fd.io > cisco@lists.fd.io> wrote:
>> >
>> > I like the idea.
>> >
>> > Regarding maintainer pain:
>> >
>> > Q: are we now stuck forever with what we have because there will always
>> be somebody facing some difficulties adopting?
>>
>> To me it is always a question of cost/benefit analysis - and admittedly
>> the value of “cost” and “benefit” is always subjective, and not the same in
>> time, so let me expand on why I suggested why suggested:
>>
>> - I can see a benefit in having a target “make test-fixstyle”, similar
>> with C code, introduced sometime *just* before the LTS branch pull, such
>> that we have a chance to be able to sync the changes.
>>
>> - I can see less benefit in introducing that target now, so soon after
>> the LTS release - since that would mean the moment you run a .py file on
>> master through the formatter, as a developer now you will have to manually
>> deal with *two* versions of file - in master and LTS. And porting the diffs
>> that don’t apply due to python formatting is a highly unhappy task.
>>
>
> I don't know if git 2.23 helps you there.  If we make the change now
> (figuratively), the issue goes away in theory this time next year as the
> old releases go out of support.
>
>
> Right. And until then nearly every change involving test code changes will
> have to be backported to 20.09 manually by the feature developers. I would
> it’s the net increase of work, with less trivial yet still very sad job of
> dealing with reshuffling the patches. PEP501-type annoyance, but with more
> lines and higher chances of error.
>

So, this is a different issue to me.  Because we don't like chained
commits, this is an issue.  To do this properly in my mind, that is to make
your job easier, there are dependent changes needed.  One tuning the build
system to support the "black" pep8 output, the second with the changes
introducing black and the file changes.

If the first change were backported, then post black changes shouldn't
cause the stable to barf on it.


>
>
>> - Having this done automatically as a precommit hook and removing it from
>> the checkstyle job means:
>>
>> 1) we trust the client to run the formatting themselves. This is a recipe
>> to style inconsistency due to *whatever* unrelated things like merges, etc.
>>
>> 2) forcefully running it just seems like a unsound idea since then we are
>> going to get arbitrary white space changes whenever someone of the
>> formatter devs decides to change the python formatter. I tried formatters
>> for C, golang, rust to name a few. *none* of them produce the identical
>> result from differently white-spaced code, that is functionally identical,
>> even if the formatter is part of the language.
>>
>> In addition this approach shares all the drawbacks of introducing the
>> change at the inopportune moment.
>>
>
> I'm not saying +2 the change today.  ;). I'm trying to socialize the idea
> and see what folks think.
>
>
> Ah ok :)
>
>
>
>>
>> That’s the reason I suggested to just add the E501 to ignore and move on
>> - on balance it seems like their relieves the maximum amount of real pain
>> without introducing unknown amounts of new one.
>>
> Going to a line length of 88 (value taken from black) leaves us with only
> 4-6 E501's (line too long) in test.
>
>
> That is *today* with the existing “new meaningless variable” hacks already
> in place.
>
Correct.


>
>
>
>
>>
>> That said: I think it is a good idea to  add the formatter as a
>> dependency, add a “make test-fixstyle” target and let the willing folks
>> experiment with it for the time being for the regular commits, this will
>> keep the white space changes contained on a per-feature-maintainer basis.
>>
>
> One monster change is supposedly easier to manage from the git side.
> Doing it piecemeal seems to have more problems associated with it.
>
>
> If it is done just before LTS branch fork - sure, I won’t mind a single
> commit which would both add black as a “make test-fixcheckstyle” and run it
> on all the files. With enough warning the folks can avoid having any
> inflight work related to it.
>
> See above,  2 commits may make life easier for everyone.

>
>
> As it is today, people frequently make formatting 

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

On Wed, Dec 2, 2020 at 5:38 AM Andrew  Yourtchenko 
wrote:

>
>
> > On 2 Dec 2020, at 10:38, Klement Sekera via lists.fd.io  cisco@lists.fd.io> wrote:
> >
> > I like the idea.
> >
> > Regarding maintainer pain:
> >
> > Q: are we now stuck forever with what we have because there will always
> be somebody facing some difficulties adopting?
>
> To me it is always a question of cost/benefit analysis - and admittedly
> the value of “cost” and “benefit” is always subjective, and not the same in
> time, so let me expand on why I suggested why suggested:
>
> - I can see a benefit in having a target “make test-fixstyle”, similar
> with C code, introduced sometime *just* before the LTS branch pull, such
> that we have a chance to be able to sync the changes.
>
> - I can see less benefit in introducing that target now, so soon after the
> LTS release - since that would mean the moment you run a .py file on master
> through the formatter, as a developer now you will have to manually deal
> with *two* versions of file - in master and LTS. And porting the diffs that
> don’t apply due to python formatting is a highly unhappy task.
>

I don't know if git 2.23 helps you there.  If we make the change now
(figuratively), the issue goes away in theory this time next year as the
old releases go out of support.


> - Having this done automatically as a precommit hook and removing it from
> the checkstyle job means:
>
> 1) we trust the client to run the formatting themselves. This is a recipe
> to style inconsistency due to *whatever* unrelated things like merges, etc.
>
> 2) forcefully running it just seems like a unsound idea since then we are
> going to get arbitrary white space changes whenever someone of the
> formatter devs decides to change the python formatter. I tried formatters
> for C, golang, rust to name a few. *none* of them produce the identical
> result from differently white-spaced code, that is functionally identical,
> even if the formatter is part of the language.
>
> In addition this approach shares all the drawbacks of introducing the
> change at the inopportune moment.
>

I'm not saying +2 the change today.  ;). I'm trying to socialize the idea
and see what folks think.


>
> That’s the reason I suggested to just add the E501 to ignore and move on -
> on balance it seems like their relieves the maximum amount of real pain
> without introducing unknown amounts of new one.
>
Going to a line length of 88 (value taken from black) leaves us with only
4-6 E501's (line too long) in test.


>
> That said: I think it is a good idea to  add the formatter as a
> dependency, add a “make test-fixstyle” target and let the willing folks
> experiment with it for the time being for the regular commits, this will
> keep the white space changes contained on a per-feature-maintainer basis.
>

One monster change is supposedly easier to manage from the git side.  Doing
it piecemeal seems to have more problems associated with it.
As it is today, people frequently make formatting changes in the same
changeset as something more valuable.  One of the benefits of black, is
that it formats the code to be more git friendly.


>
> Thoughts ?
>
> —a
>
> >
> > Thanks,
> > Klement
> >
> >>> On 2 Dec 2020, at 10:30, Andrew Yourtchenko 
> wrote:
> >>>
> >>>
> >>>> On 2 Dec 2020, at 10:27, Neale Ranns via lists.fd.io  cisco@lists.fd.io> wrote:
> >>>
> >>> 
> >>>
> >>> Hi Paul,
> >>>
> >>> Having to write code to conform to python linting is my number 1
> annoyance when writing tests. This is my usual hack:
> >>>  e = VppEnum.vl_api_tunnel_encap_decap_flags_t
> >>>  f = e.TUNNEL_API_ENCAP_DECAP_FLAG_ENCAP_COPY_DSCP
> >>
> >> +1 E501 specifically being a massive annoyance and having to use the
> exact same hack - if you use the flags multiple times it might forcing the
> somewhat better  readability, but then one still has to do f1, f2 and later
> combine *them*, which definitely doesn’t help understanding of the code by
> any later reader since now they have to keep these mappings in the head.
> >>
> >> —a
> >>
> >>>
> >>> I support having an auto-linter. I have no knowledge about what’s
> available, so I defer to your choice. All I ask is that it works  i.e.
> you don’t have to pepper code with /* *HERE-BE-DRAGONS* */
> >>>
> >>> IIUC the plan post 21.01 is to upgrade our default linux distro to
> 20.04, that brings git 2.25 (at least that’s what my VM has, but maybe I
> put that there for recent gerrit up-revs…)
> >>>
> >>

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

Copying the list this time ;)


> Has to be 2.23 or greater.
>
> On Wed, Dec 2, 2020 at 4:15 AM Andrew  Yourtchenko 
> wrote:
>
>> Can git in any version ignore the massive white space changes while
>> cherry-picking / merging?
>> If not - this sounds like a *massive* inconvenience to anyone not sitting
>> on the head of master, which is a lot of folks. At least that is why it’s
>> still on my WIP to deal with the C code indent... big white space-changing
>> changes are a *massive* pain to deal with for any maintainer.
>>
>> Also, 88 is not much bigger than 80, and won’t solve the particular case
>> mentioned - you will still need to chop that 97 char line into smaller one..
>>
>> So: What do you and others think of an alternative of just adding E501
>> to the existing “ignore” list for the current formatter and moving on ?
>>
>> --a
>>
>> On 1 Dec 2020, at 23:56, Paul Vinciguerra 
>> wrote:
>>
>> 
>> I'd like to propose that we make it easier for everyone by adding black
>> [0] as a pre-commit hook.  Black will automatically reformat your file to a
>> git friendly, pep-8 friendly file.
>> For those interested in the details, it moves to a line length of 88,
>> which helps us out with the lengthy VppEnum names we have.  We can keep it
>> at 80 if the community objects.
>> I can't do anything about:
>>
>>   /vpp/build-root/build-test/src/test_ipsec_esp.py:504:89: E501 line too
>> long (97 > 88 characters)
>>
>>   
>> VppEnum.vl_api_tunnel_encap_decap_flags_t.TUNNEL_API_ENCAP_DECAP_FLAG_ENCAP_COPY_DSCP
>>
>> ;)
>>
>> For those who want more details in the changes, see the black code style
>> [1]
>>
>> Saving time around python linting is the #1 request I have had from the
>> community.
>>
>> This is a MASSIVE whitespace change.  git blame can ignore whitespace
>> changes starting in git 2.23.
>>
>> The question is whether the community wants to upgrade their version of
>> git to ignore this change with git blame, in exchange for not having to
>> manually lint/fix their files.
>>
>> Thoughts?
>>
>> [0] https://github.com/psf/black
>> [1] https://github.com/psf/black/blob/master/docs/the_black_code_style.md
>>
>> 
>>
>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18222): https://lists.fd.io/g/vpp-dev/message/18222
Mute This Topic: https://lists.fd.io/mt/78647163/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

Hi Damjan.

>From their docs.
Migrating your code style without ruining git blame

A long-standing argument against moving to automated code formatters like
*Black* is that the migration will clutter up the output of git blame. This
was a valid argument, but since Git version 2.23, Git natively
supports ignoring
revisions in blame
<https://git-scm.com/docs/git-blame#Documentation/git-blame.txt---ignore-revltrevgt>
with
the --ignore-rev option. You can also pass a file listing the revisions to
ignore using the --ignore-revs-file option. The changes made by the
revision will be ignored when assigning blame. Lines modified by an ignored
revision will be blamed on the previous revision that modified those lines.

So when migrating your project's code style to *Black*, reformat everything
and commit the changes (preferably in one massive commit). Then put the
full 40 characters commit identifier(s) into a file.

# Migrate code style to Black
5b4ab991dede475d393e9d69ec388fd6bd949699

Afterwards, you can pass that file to git blame and see clean and
meaningful blame information.

$ git blame important.py --ignore-revs-file
.git-blame-ignore-revs7a1ae265 (John Smith 2019-04-15 15:55:13 -0400
1) def very_important_function(text, file):abdfd8b0 (Alice Doe
2019-09-23 11:39:32 -0400 2) text = text.lstrip()7a1ae265 (John
Smith 2019-04-15 15:55:13 -0400 3) with open(file, "r+") as
f:7a1ae265 (John Smith 2019-04-15 15:55:13 -0400 4)
f.write(formatted)

You can even configure git to automatically ignore revisions listed in a
file on every call to git blame.

On Wed, Dec 2, 2020 at 4:38 AM Damjan Marion  wrote:

>
>
> On 01.12.2020., at 23:55, Paul Vinciguerra 
> wrote:
>
> The question is whether the community wants to upgrade their version of
> git to ignore this change with git blame, in exchange for not having to
> manually lint/fix their files.
>
>
> Can you please explain how new version of git helps here?
>
> —
> Damjan
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18221): https://lists.fd.io/g/vpp-dev/message/18221
Mute This Topic: https://lists.fd.io/mt/78647163/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

Hi Neale,

The problem with that code in general is that the delayed binding with papi
means that you have to manually ensure that you only reference it *after*
papi has been initialized.  The other alternative is to put them in a
lambda or @property, so they are evaluated at the time of use instead of
when the module is loaded.

On Wed, Dec 2, 2020 at 4:27 AM Neale Ranns (nranns) 
wrote:

>
>
> Hi Paul,
>
>
>
> Having to write code to conform to python linting is my number 1 annoyance
> when writing tests. This is my usual hack:
>
>   e = VppEnum.vl_api_tunnel_encap_decap_flags_t
>
>   f = e.TUNNEL_API_ENCAP_DECAP_FLAG_ENCAP_COPY_DSCP
>
>
>
> I support having an auto-linter. I have no knowledge about what’s
> available, so I defer to your choice. All I ask is that it works  i.e.
> you don’t have to pepper code with /* **HERE-BE-DRAGONS** */
>
>
>
> IIUC the plan post 21.01 is to upgrade our default linux distro to 20.04,
> that brings git 2.25 (at least that’s what my VM has, but maybe I put that
> there for recent gerrit up-revs…)
>
>
>
> /neale
>
>
>
> *From: * on behalf of Paul Vinciguerra <
> pvi...@vinciconsulting.com>
> *Date: *Tuesday 1 December 2020 at 23:56
> *To: *vpp-dev 
> *Subject: *[vpp-dev] replacing make test-checkstyle with black
>
>
>
> I'd like to propose that we make it easier for everyone by adding black
> [0] as a pre-commit hook.  Black will automatically reformat your file to a
> git friendly, pep-8 friendly file.
>
> For those interested in the details, it moves to a line length of 88,
> which helps us out with the lengthy VppEnum names we have.  We can keep it
> at 80 if the community objects.
>
> I can't do anything about:
>
>   /vpp/build-root/build-test/src/test_ipsec_esp.py:504:89: E501 line too
> long (97 > 88 characters)
>
>
>   
> VppEnum.vl_api_tunnel_encap_decap_flags_t.TUNNEL_API_ENCAP_DECAP_FLAG_ENCAP_COPY_DSCP
>
> ;)
>
>
>
> For those who want more details in the changes, see the black code style
> [1]
>
>
>
> Saving time around python linting is the #1 request I have had from the
> community.
>
>
>
> This is a MASSIVE whitespace change.  git blame can ignore whitespace
> changes starting in git 2.23.
>
>
>
> The question is whether the community wants to upgrade their version of
> git to ignore this change with git blame, in exchange for not having to
> manually lint/fix their files.
>
>
>
> Thoughts?
>
>
>
> [0] https://github.com/psf/black
>
> [1] https://github.com/psf/black/blob/master/docs/the_black_code_style.md
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18220): https://lists.fd.io/g/vpp-dev/message/18220
Mute This Topic: https://lists.fd.io/mt/78647163/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] replacing make test-checkstyle with black

[Paul Vinciguerra]

I'd like to propose that we make it easier for everyone by adding black [0]
as a pre-commit hook.  Black will automatically reformat your file to a git
friendly, pep-8 friendly file.
For those interested in the details, it moves to a line length of 88, which
helps us out with the lengthy VppEnum names we have.  We can keep it at 80
if the community objects.
I can't do anything about:

  /vpp/build-root/build-test/src/test_ipsec_esp.py:504:89: E501 line too
long (97 > 88 characters)
  
VppEnum.vl_api_tunnel_encap_decap_flags_t.TUNNEL_API_ENCAP_DECAP_FLAG_ENCAP_COPY_DSCP

;)

For those who want more details in the changes, see the black code style [1]

Saving time around python linting is the #1 request I have had from the
community.

This is a MASSIVE whitespace change.  git blame can ignore whitespace
changes starting in git 2.23.

The question is whether the community wants to upgrade their version of git
to ignore this change with git blame, in exchange for not having to
manually lint/fix their files.

Thoughts?

[0] https://github.com/psf/black
[1] https://github.com/psf/black/blob/master/docs/the_black_code_style.md

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18207): https://lists.fd.io/g/vpp-dev/message/18207
Mute This Topic: https://lists.fd.io/mt/78647163/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] minor doc change

[Paul Vinciguerra]

Hi Hermant,

I had to go back and double check the thread.  Elias said we should make it
clear that it was a mirror, not that the use of Gerrit should be
deprecated.  I don't have the keys to the vpp github account, so I can't
update the message.  It's a good idea, though. +1

I would not support moving from Gerrit, it provides too much value.  You
can always point your upstream repo on your github fork to "
https://gerrit.fd.io/r/vpp; if that helps any.

Paul

On Sun, Nov 29, 2020 at 12:07 PM  wrote:

> Another issue.  The github vpp repo doesn't seem to be a mirror to me or
> the
> mirror is not updated frequently enough.  During the past months, I was
> writing a new VPP plugin.  The code crashed outside of my plugin.  After a
> day
> or two, I pulled a gerrit vpp repo and using my plugin, the code didn't
> crash.
> I aw significant differences between the github and gerrit repos at the
> time.
>
> I agree with Elias.  Long term, maybe use of gerrit is deprecated and
> github
> is used. Github is free for public repos.
>
> Hemant
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Elias Rudberg
> Sent: Sunday, November 29, 2020 5:14 AM
> To: pvi...@vinciconsulting.com; hem...@mnkcg.com
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] minor doc change
>
> I think it would be good if that could be clarified on the github page.
> When people search for "vpp source code" or similar, I think they will
> often
> end up on the github page and it's not immediately obvious from there that
> it's only a mirror. (People might get the wrong idea about some things,
> for
> example github shows a "contributors" list which I guess is not accurate
> as it
> only shows authors who happen to have github accounts that are linked to
> gerrit in some way?)
>
> The github page says, under "About" to the right, "No description,
> website, or
> topics provided." So there is apparently a possibility to enter a
> "description", perhaps that could be used to indicate that it is just a
> mirror?
>
> Best regards,
> Elias
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18192): https://lists.fd.io/g/vpp-dev/message/18192
Mute This Topic: https://lists.fd.io/mt/78559913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Removing [backwards_compatible] from the .api

[Paul Vinciguerra]

Hi Ole.

Comments inline.

On Wed, Nov 25, 2020 at 4:46 PM  wrote:

> Hi Paul,
>
> > No.
> >
> > crc is actually a misnomer here.  If you say crc, one assumes
> binascii.crc32.  That is not what crc is.  What is exposed as .crc is
> actually a custom function in vppapigen called foldup_blocks, which is
> dependent on the version of vppapigen that generated it.[0]. (Ask me how I
> know...) In fact, in the source is a fixup_crc_dict [1] with pre/post
> 6/25/20 crc's hardcoded.
>
> the crc is a crc of the string representation of an API message including
> all its dependent types.
>

The definition of what the crc covered changed on 6/25/2020.
[backward_compatable] is a knob that removes blocks of the string
representation from the crc.
If we look at the code:

self.block = block2
self.crc = str(block3).encode()

The crc is actually NOT the string representation of the dependent type.

>
> > backward_compatable is a misnomer.  If you marked existing fields as
> backward_compatable, which is true by common definition of the option's
> name, that value is removed from the crc and everything trusting the crc
> breaks.
> >
> > Having the same crc match against different sources is a real problem.
> It is a violation of a consumer's trust.
>
> there is a bit of history here.
> prior to more explicit types a flags field would be represented as "u32
> flags;".
> defining flags as an enum with explicit values makes the API more
> explicit, but also limits how easy it is to evolve.
> the backwards compatible option for enums try to strike a balance between
> those two.
> it does of course depend on the developers doing the right thing.
>
This is where I have an issue.  I'm not saying that developers are trying
to do the wrong thing, but rather when code is being written, it can not
always be known how the code is going to be used, so it cannot be "right".


>
> it would indeed make sense to time-limit the backwards_compatible option,
> otherwise as you say users cannot depend on the new messages tagged as such.
> with the current process that would likely require a _2 of the message.
>
> You know what I am working on with enums/enumflags. As you and I have
discussed, for some languages, we should support None = 0, for others it is
not relevant.  For me to make the api consistent, I may need to add NONE=0
to emuns with the missing value.  It clearly states that
[backward_compatable] additions can only be at the end.


> > History has already shown that you have made the process overly
> difficult.  Let's go back and revisit [2]. We have a version (a semver)
> attached to the .api.  The refusal in [2]  was to continue to follow the
> rules of semantic versioning in the name of expediency. (For those who are
> having trouble following along, if a bug is fixed, the version is to be
> incremented[3]. In this case, it was decided to not increment the semver,
> to not trigger the overly painful process)  A single semver with differing
> results violates our contract with the consumer.  The semver is different
> than the wire format.  Under [2], the semver was intentionally not bumped,
> because it was going to trigger changing the crc.  That's a sign to me that
> the process is *broken*.
>
> are you arguing that you would have a semver and only a semver?
> the current scheme is automatic with no dependency of the developer having
> to bump the semantic version. it also has the granularity of the
> per-message level, allowing the user of the API to specify a manifest of
> the messages with versions (aka crcs) of the messages used.
>

No.  I'm saying that when you change the endianess of an api handler, even
though the wire format doesn't change, the semver and therefore your crc
changes.  I'm saying that when we enter into a contract with a consumer by
publishing a semver and/or a crc, we need to make sure it is accurate.  If
you publish crc's, they need to be accurate and consistent, and a consumer
should have a way to know that the definition of the crc was doctored, by
removing fields so that an external process doesn't fail.

In my code, If the crc doesn't match, I report the api option['version'].
When the crc doesn't change, I can't catch the change you allowed to be
hidden via [backward_compatable]. When it does change, I can't point the
user to where the info was sourced from because we have multiple api's with
identical semvers.

I just looked at Andrew's release notes page, and it is full of
inaccurate data with regard to api changes, when there aren't any.  Just
because code was moved from core to a plugin doesn't mean that the api
changed.  Before someone emails me that I'm attacking Andrew, I'm not.  I'm
reasserting my earlier point that a developer can't do the right thing when
he can't predict every possible way that his code may be consumed.

If I am arguing for anything, I am arguing for providing accurate
information to the consumer and that all the [backward_compatable] options
need to be limited to the 

Re: [vpp-dev] minor doc change

[Paul Vinciguerra]

Hi Hermant,

We don't see pull requests.  Github is just a mirror of the gerrit repo.

You can submit your change to the gerrit repo.  See: [0]
Bugs can be filed at [1]

Paul

[0]
https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#Pushing

[1] https://jira.fd.io/projects/VPP/issues

On Sat, Nov 28, 2020 at 12:58 PM  wrote:

> Paul,
>
>
>
> For all my work I use github.  I used VPP github repo to make the doc
> change in a PR.   See
>
>
>
> https://github.com/FDio/vpp/pull/32
>
>
>
> If anyone can send me a link where bugs are filed against VPP gerrit repo,
> I can use the link to file issues.  I haven’t used gerrit before but in
> future I can try and use it.
>
>
>
> Thanks,
>
>
>
> Hemant
>
>
>
> *From:* Paul Vinciguerra 
> *Sent:* Friday, November 27, 2020 10:47 PM
> *To:* hem...@mnkcg.com
> *Cc:* vpp-dev 
> *Subject:* Re: [vpp-dev] minor doc change
>
>
>
> Hi Hermant.
>
>
>
> The document is in the sources here:
>
> docs/gettingstarted/developers/infrastructure.md
>
>
>
> The best way to get the docs updated is to submit a change.
>
>
>
> Paul
>
>
>
> On Fri, Nov 27, 2020 at 8:37 PM hemant via lists.fd.io  mnkcg@lists.fd.io> wrote:
>
> At this link:
>
>
>
>
> https://my-vpp-docs.readthedocs.io/en/latest/gettingstarted/developers/infrastructure.html
>
>
>
> unformat_init_string() used two args but the latest VPP code uses three
> args.  We should change the doc.
>
>
>
> Thanks,
>
>
>
> Hemant
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18186): https://lists.fd.io/g/vpp-dev/message/18186
Mute This Topic: https://lists.fd.io/mt/78559913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] minor doc change

[Paul Vinciguerra]

Hi Hermant.

The document is in the sources here:
docs/gettingstarted/developers/infrastructure.md

The best way to get the docs updated is to submit a change.

Paul

On Fri, Nov 27, 2020 at 8:37 PM hemant via lists.fd.io  wrote:

> At this link:
>
>
>
>
> https://my-vpp-docs.readthedocs.io/en/latest/gettingstarted/developers/infrastructure.html
>
>
>
> unformat_init_string() used two args but the latest VPP code uses three
> args.  We should change the doc.
>
>
>
> Thanks,
>
>
>
> Hemant
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18184): https://lists.fd.io/g/vpp-dev/message/18184
Mute This Topic: https://lists.fd.io/mt/78559913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Policy on Makefile vs cmake usage

[Paul Vinciguerra]

>
> > My question is specifically about the carve out in your reply.
> > "assuming that all mandatory requirements are installed in the system:"
>
> Yes, it is not cmake project job to install dependencies.
>
That is the direction I was looking for.

[snip...]

>
> At the moment vppapigen is script directly called by cmake, if your change
> wil prevent cmake to call that script directly from the place where it is
> today, we will need to significantly refactor cmake project files and i
> will rather not go there….
>

  My change makes vppapigen look like a system executable.  If that cannot
be supported easily with cmake , I can update the target of the symlink.

>
> —
> Damjan
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18175): https://lists.fd.io/g/vpp-dev/message/18175
Mute This Topic: https://lists.fd.io/mt/78548158/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Policy on Makefile vs cmake usage

[Paul Vinciguerra]

Thanks Damjan.

My question is specifically about the carve out in your reply.
"assuming that all mandatory requirements are installed in the system:"

Today we have all these system package dependencies in the Makefile to add
ply.  These are all dependencies for vppapigen.  I have previously added
comments in the makefile why the package is there in the first place.

Why the changeset at all?
For the community, It is straightforward to have vppapigen manage its own
dependencies and we can clean up the makefile.  For me, I want to reuse
vppapigen from python, and launching it over and over again via subprocess
is suboptimal.  Furthermore, it is fragile in that it expects files to be
named in a certain template and placed in specific locations.

As a script only usable by the build system, vppapigen works.  But there
are uses for vppapigen outside of make.  My changeset works.  It's just not
integrated with VPP.

I expect that I can get it to pass the CI by adding a variant of a
single 'pip install vppapigen' and updating the symlink to the new install
location.  It just seems wrong to me to use a symlink to meet a
dependency.  If I'm already making changes in the code, I don't mind
updating it.


On Fri, Nov 27, 2020 at 10:04 AM Damjan Marion  wrote:

>
>
> On 27.11.2020., at 15:52, Paul Vinciguerra 
> wrote:
>
> Can someone memorialize the policy around changes to make and cmake?
> I have a change that replaces an existing symlink in the repo with a
> called command.  Since it is a symlink now, and therefore, by definition,
> already setup before cmake, can I address it in the Makefile, just like we
> set up system packages, or does it somehow need to be a formal dependency
> in cmake?
>
> The change is:  https://gerrit.fd.io/r/c/vpp/+/30080
>
>
> Not sure wha problem you are trying to solve with this patch and what is
> the benefit it brings,
> but general rule is that cmake project needs to be self-suffiicient to
> build running vpp tree.
>
> In other words following sequence needs to work assuming that all
> mandatory requirements are installed in the system:
>
> $ mkdir /some/random/path
> $ cd /some/random/path
> $ cmake -G Ninja /path/to/vpp/src
> $ ninja
>
> —
> Damjan
>
>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18173): https://lists.fd.io/g/vpp-dev/message/18173
Mute This Topic: https://lists.fd.io/mt/78548158/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Policy on Makefile vs cmake usage

[Paul Vinciguerra]

Can someone memorialize the policy around changes to make and cmake?
I have a change that replaces an existing symlink in the repo with a called
command.  Since it is a symlink now, and therefore, by definition, already
setup before cmake, can I address it in the Makefile, just like we set up
system packages, or does it somehow need to be a formal dependency in cmake?

The change is:  https://gerrit.fd.io/r/c/vpp/+/30080

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18170): https://lists.fd.io/g/vpp-dev/message/18170
Mute This Topic: https://lists.fd.io/mt/78548158/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Custom VxLAN port

[Paul Vinciguerra]

Hi Ole, Artem,

For the benefit of others,
VPP's vxlan may be great for in memory transfers between containers, but it
is hard on real network infrastructure, because it doesn't provide the
signals to upstream routers for ECMP.

*/* UDP header, randomize src port on something, maybe? */
*udp->src_port = clib_host_to_net_u16 (4789);


On Thu, Nov 26, 2020 at 8:35 AM Artem Glazychev 
wrote:

> Ole,
>
> I don't fully understand,
> If I want to support configurable UDP port per tunnel, I need to extend
> "vxlan_tunnel_t" or "vxlan_gpe_tunnel_t" or "vxlan_gbp_tunnel_t" or all of
> them. So this is what I meant in point 2. Why 2.5? :)
> Or can I only use the "vxlan_gbp_tunnel_t" for this purpose?
>
> Thanks
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18157): https://lists.fd.io/g/vpp-dev/message/18157
Mute This Topic: https://lists.fd.io/mt/78507623/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Removing [backwards_compatible] from the .api

[Paul Vinciguerra]

No.

crc is actually a misnomer here.  If you say crc, one
assumes binascii.crc32.  That is not what crc is.  What is exposed as .crc
is actually a custom function in vppapigen called foldup_blocks, which is
dependent on the version of vppapigen that generated it.[0]. (Ask me how I
know...) In fact, in the source is a fixup_crc_dict [1] with pre/post
6/25/20 crc's hardcoded.

backward_compatable is a misnomer.  If you marked existing fields
as backward_compatable, which is true by common definition of the option's
name, that value is removed from the crc and everything trusting the crc
breaks.

Having the same crc match against different sources is a real problem.  It
is a violation of a consumer's trust.

History has already shown that you have made the process overly difficult.
Let's go back and revisit [2]. We have a version (a semver) attached to
the .api.  The refusal in [2]  was to continue to follow the rules of
semantic versioning in the name of expediency. (For those who are having
trouble following along, if a bug is fixed, the version is to be
incremented[3]. In this case, it was decided to not increment the semver,
to not trigger the overly painful process)  A single semver with differing
results violates our contract with the consumer.

No matter how hard you make the process, in the end, we can not violate the
trust given to us by consumers of VPP by providing them "sometimes" false
information.  We owe it to the community to do better!

[0]
https://github.com/FDio/vpp/blob/master/src/tools/vppapigen/vppapigen.py#L1058
[1]
https://github.com/FDio/vpp/blob/master/src/tools/vppapigen/vppapigen.py#L1072
[2] https://lists.fd.io/g/vpp-dev/message/16406
[3] https://semver.org/


On Wed, Nov 25, 2020 at 11:51 AM Andrew  Yourtchenko 
wrote:

> A naive implementation of that will break all of the CRC for the affected
> messages, which effectively means abandoning the message stability process.
>
> --a
>
> On 25 Nov 2020, at 17:27, Paul Vinciguerra 
> wrote:
>
> 
> I would like to socialize the idea of removing the backwards_compatible
> option from the .api files before the upcoming and any future releases.
>
> The commit message states:
>
> This allows adding backwards compatible (as guaranteed by the developer)
> enums.
> The enums marked backwards compatible are not considered in the CRC
> calculation.
>
> Because it is the widespread practice of the VPP comitter community to
> merge commits without Maintainer approval, it is practically not possible
> for the developer to assert his/her "guarantee".  This is a potentially
> risky area.
>
> If we remove these tags from the .api files before the next release, we
> can guarantee a "golden image" of sorts, for lack of a better analogy, of
> crc values.
>
> After the release, backwards_compatible could be used for subsequent
> additions, again being removed before the next release is cut.
>
> Does a repo exist with the code used for verification while "cutting" a
> release?
>
> I don't mind putting up a changeset.
>
> Paul
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18141): https://lists.fd.io/g/vpp-dev/message/18141
Mute This Topic: https://lists.fd.io/mt/78503177/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Removing [backwards_compatible] from the .api

[Paul Vinciguerra]

I would like to socialize the idea of removing the backwards_compatible
option from the .api files before the upcoming and any future releases.

The commit message states:

This allows adding backwards compatible (as guaranteed by the developer)
enums.
The enums marked backwards compatible are not considered in the CRC
calculation.

Because it is the widespread practice of the VPP comitter community to
merge commits without Maintainer approval, it is practically not possible
for the developer to assert his/her "guarantee".  This is a potentially
risky area.

If we remove these tags from the .api files before the next release, we can
guarantee a "golden image" of sorts, for lack of a better analogy, of crc
values.

After the release, backwards_compatible could be used for subsequent
additions, again being removed before the next release is cut.

Does a repo exist with the code used for verification while "cutting" a
release?

I don't mind putting up a changeset.

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18139): https://lists.fd.io/g/vpp-dev/message/18139
Mute This Topic: https://lists.fd.io/mt/78503177/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] vppapigen types

[Paul Vinciguerra]

vppapigen has 4 types:
  ['Enum', 'Typedef', 'Using', 'Union']

Using and Union seem to be hints to vpp_papi for converting to/from the
wire format.  Is there a use case where they need to be exposed to a
high-level client?

If I am creating too much noise for the list, let me know.  This is
interesting to me and may be someone else, but I understand that it is not
core VPP.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18101): https://lists.fd.io/g/vpp-dev/message/18101
Mute This Topic: https://lists.fd.io/mt/78368783/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Changing the syntax in the .api files

[Paul Vinciguerra]

Ole,

Can we clarify the .api (syntax) so that enums are separate from flags?
Today, everything is rendered (at least in Python) as an
(enum|aenum).IntFlag.

Here is a sample of how that an enum looks like in python:

class rx_mode(enum.IntFlag):   # u32:
UNKNOWN = 0
POLLING = 1
INTERRUPT = 2
ADAPTIVE = 3
DEFAULT = 4


What is 3?  Is it ADAPTIVE, or is it ( POLLING | INTERRUPT)  ?  I can
*assume* that if the block has a 0 value, I can downcast the class, but it
seems like a kludge.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18098): https://lists.fd.io/g/vpp-dev/message/18098
Mute This Topic: https://lists.fd.io/mt/78367407/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] why tunnel interfaces do not support device-input feature?

[Paul Vinciguerra]

I don't know if this is useful for anyone or not.

I posted https://wiki.fd.io/images/a/a6/VPP_node_graph.svg

It is impossible to view at once, but you can search for a node and go from
there.



On Wed, Nov 18, 2020 at 11:05 AM Damjan Marion via lists.fd.io  wrote:

>
> device-input feature arch is build for features which needs to deal with
> raw ethernet frames
> before they are processed and before we know to which software interface
> packet belongs (including the information if interface is in l2 or l3
> mode). So there is no many tunnelling protocols which will even be
> candidates for this arc. Tunnel encaps which doesn’t carry inner ethernet
> header will simply not work.
> People should think twice before deciding to build any feature which hangs
> on that arc, as likely there is more appropriate one to use.
> Pretty much the same applies to interface-output.
>
>
> > On 18.11.2020., at 11:15, Neale Ranns via lists.fd.io  cisco@lists.fd.io> wrote:
> >
> >
> > Hi Ye,
> >
> > Some comments inline...
> >
> > On 17/11/2020 02:34, "vpp-dev@lists.fd.io on behalf of 叶东岗" <
> vpp-dev@lists.fd.io on behalf of y...@wangsu.com> wrote:
> >
> >Hi all:
> >
> >why tunnel interfaces do not support device-input feature?
> >
> > No one has asked for/contributed such support.  If you're volunteering,
> here's some advice.
> >
> > Taking the feature arc always costs performance, but we accept that.
> What is harder to accept is a performance degradation when there are no
> features configured.
> >
> > Devices are 'physical' interfaces, they represent an interface from VPP
> to the external world. This means they are read by nodes in poll mode, one
> device at a time. They therefore have the luxury of knowing that all
> packets in the vector/frame come from the same device. Virtual interfaces
> don't have that luxury, so the check for 'are there features on the arc'
> would be per buffer, not per-packet, this would be a noticeable performance
> cost.
> >
> >why  esp packets  do not go through ipsec interface's
> "interface-output"
> >node?
> >
> > The actions for TX on a virtual interface are different. The equivalent
> node is 'adj-midchain-tx'. Running the 'interface-output' arc here would be
> possible, with a negligible performance cost because the adj can cache the
> feature arc's state.
> >
> > /neale
> >
> >I think it's no bad idea to keep some features consistency of all
> >interface in spite of an little performance degradation?
> >
> >
> >Best regards,
> >Ye Donggang
> >
> >
> >
> >
> >
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18086): https://lists.fd.io/g/vpp-dev/message/18086
Mute This Topic: https://lists.fd.io/mt/78307484/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Supported distributions.

[Paul Vinciguerra]

Expanding the audience of https://gerrit.fd.io/r/c/vpp/+/28721

> Patch Set 2:
>
> Am I correct if I summarize your point as:
>  1) we should document the distribution we officially support (this was
the goal of the wiki page you setup)
>  2) as long as we do not break this list we should be allowed to use new
tools etc. (which might be breaking other, unsupported distribution)
>
> If so, I agree with you but I think I disagree about the list of
supported distribution :)
> Right now we run CI on:
>  - Ubuntu 18.04 and 20.04
>  - Debian 9 and 10
>  - CentOS 7 and 8
> I'd say this is what we currently support :) (but again: it is my own
personal view here, and obviously I am a bit biased on the deb9 subject ;)
).
> There was discussion during last community call about dropping CentOS 7
support and I think the consensus was to keep it for now, but we might drop
it for 21.01 (or something along that line).
>
> Probably the best should be to have that discussion directly on the ml
and during the community calls instead of in this ticket.

Everything on the wiki is wrong!  It was right at some point in time, but
that time has passed.  If it is right, it is a coincidence.  The info was
put there to try to form a consensus, but no one agreed.

If we can get a consensus, it should be memorialized in a README type doc
in the repo and the wiki can be removed or pointed to the repo.

I don't know the history with deb9 for you, or who needs/wants it.  I have
nothing against it.  We should know how long we are to maintain it for,
yes?  Is there a reason we can't support it by building a current python in
the makefile instead?

I'd characterize #2 somewhat differently.  People should be
allowed/encouraged to introduce new tools, but they need to be tested.
Last year, The folks at Netgate started pushing changes for Centos 8.  I
wasn't approving them, not because I wanted to hold anyone up, but because
they didn't work for me on a fresh Centos 8 container.  I was told that as
long as they didn't fail the CI, I could/should approve them.  They
shouldn't be blocked from having the feature, but it has to still be
tested/work.

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18085): https://lists.fd.io/g/vpp-dev/message/18085
Mute This Topic: https://lists.fd.io/mt/78344343/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Python API modules

[Paul Vinciguerra]

On Wed, Nov 18, 2020 at 4:38 AM  wrote:

> > I think that generating the python code makes the most sense.  The body
> of the code can still defer to the dynamically generated object, since it's
> not available until runtime anyway, but the user no longer has to resort to
> using lambdas because of the deferred bindings.
>
> Could you ellaborate for why you need to use lambdas?
>

Today, in the tests, all references to vpp_papi. have to happen after
connect, otherwise they don't exist.  Let's say I want to set up a test and
use an enum from vpp_papi, it may not exist due to the deferred binding.
Instead, you have to wrap it in a lamba so that it is evaluated at use time
instead of at module load time.  You could also make it a @property and
force it evaluated at run time.


> [SNIP]
> >
> > but the typedefs could become data classes and the encode/decode methods
> in the test code become unnecessary.
>
> What are you thinking about regarding the encode/decode in tests?
> E.g. for the IP address ones, they can just be deleted. Without any
> changes to PAPI...
>
> As an example:

class VppGbpContractNextHop():
def __init__(self, mac, bd, ip, rd):
self.mac = mac
self.ip = ip
self.bd = bd
self.rd = rd

def encode(self):
return {'ip': self.ip,
'mac': self.mac.packed,
'bd_id': self.bd.bd.bd_id,
'rd_id': self.rd.rd_id}



> > vl_api_ip_ecn_t.IP_API_ECN_NONE could become ip_ecn.NONE
> >
> > The IDE's could then provide type checking and auto completion. (So one
> would know that NONE was an option without referring to the sources.)
> >
> > We would need to play around with it to see how well it works.
>
> That seems like a good idea.
> Let me give you a quick Python plugin for vppapigen that does this.
>
> The packaging is going to be a challenge I guess.
>
> The packaging could be as simple as we write the files to a folder in
vpp_papi, or we could have its own setup.py and make it a plugin to
vpp_papi.


> Best regards,
> Ole

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18083): https://lists.fd.io/g/vpp-dev/message/18083
Mute This Topic: https://lists.fd.io/mt/78229638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Help clearing a -2 review.

[Paul Vinciguerra]

reposting to the list.

2400 for each 'make test' run.

On Tue, Nov 17, 2020 at 10:27 PM Paul Vinciguerra <
pvi...@vinciconsulting.com> wrote:

> 2400 for each 'make test' run.
>
> On Tue, Nov 17, 2020 at 5:48 PM Andrew  Yourtchenko 
> wrote:
>
>> Paul, is it 2400 comparisons per single test or 2400 comparisons in total
>> ?
>>
>> If the latter, I would rather optimize for readability, since it’s
>> probably less than a second of run time.
>>
>> Specifically about the example with debugging of internals - replacing
>> the hooks with subclassing hinders the intent imho - for that particular
>> case.
>>
>> --a
>>
>> On 17 Nov 2020, at 19:50, Paul Vinciguerra 
>> wrote:
>>
>> 
>> Ok.  This is a backwards request.  I'm asking for help trying to explain
>> properly why I've -2'd a change.  The code is both useful to the community
>> and cleanly written.  I think the plumbing needs some help.  When we find
>> someone who is willing and able to contribute, I'd like to not frustrate
>> them away.
>>
>> At a high level, when we run tests, the makefile sets up a specific
>> environment that is passed to run_tests.py (which is a re-implementation of
>> the python stdlib unittest. test runner).  The test runner does discovery,
>> that is that it finds all the tests that match a customized string, and
>> builds a list of tests which are either run serially or forked in parallel.
>>
>> What people have done is put conditional logic in the test case and
>> change the behavior after the test has started.  I consider this analogous
>> to you unrolling a loop and me coming by and testing if 1==2 for each
>> element of your unrolled loop.
>>
>> The test should instead be done once in the runner, instead of 2400 or so
>> times for every submission into the gate.
>>
>> To explain this, I cherry-picked some of my code and submitted it as an
>> example.  My example is here:  https://gerrit.fd.io/r/c/vpp/+/29938. It'
>> not something I planned to contribute, but I changed it enough to get it to
>> pass the date.
>>
>> The commit I am blocking is here: https://gerrit.fd.io/r/c/vpp/+/29921 .
>>
>> How do I, with limited cycles, convey what needs to be done without
>> writing sample code or going in and patching over someone's work.
>> The code is well written and I'd rather +2 it and try to coax some more
>> contributions.  ;)
>>
>> Paul
>>
>>
>>
>>
>>
>> 
>>
>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18075): https://lists.fd.io/g/vpp-dev/message/18075
Mute This Topic: https://lists.fd.io/mt/78323128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Requirements for Test

[Paul Vinciguerra]

Dave W. and I were chatting about putting together a collection of agreed
upon improvements for the test infrastructure.

If people want to respond to this thread or email me, I will start a .rst
that we can maintain in the repo.

I want to start the thread off with my disagreement with previous comments
that the module owner owns the tests.  The module owner owns the test
objectives.  I never commented until now because I wanted to see where
things went.

Back in May, Neale fixed this bug: https://gerrit.fd.io/r/c/vpp/+/27187

It was a typo that was slipping by because in the test framework, we send
the *same* packet over and over again.  So, by design, b[0] == b[1] and the
test didn't actually test the code.

We have never addressed the failing in the test framework.  In this case, I
think we actually have to make changes to VPP to make it more testable.
Say if we add a .stride_width to a node registration, the test could query
the .stride_width and make sure there were always (stride_width + x)
distinct packets.
(I guess that's my long winded way of saying that to fix the tests, we may
need to fix VPP)

I'm also curious as to what you think what the goal of the test
framework is.  Even though it relies on the python.unittest library, it
rarely runs unit tests.  We have cases where it runs and reports on c unit
tests, but mainly my experience is that it is a sort of regression test.  A
while back when the api was being changed, the tests were just adjusted to
match the new values.

I'm curious to hear what others have to add.

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18068): https://lists.fd.io/g/vpp-dev/message/18068
Mute This Topic: https://lists.fd.io/mt/78323792/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Help clearing a -2 review.

[Paul Vinciguerra]

Ok.  This is a backwards request.  I'm asking for help trying to explain
properly why I've -2'd a change.  The code is both useful to the community
and cleanly written.  I think the plumbing needs some help.  When we find
someone who is willing and able to contribute, I'd like to not frustrate
them away.

At a high level, when we run tests, the makefile sets up a specific
environment that is passed to run_tests.py (which is a re-implementation of
the python stdlib unittest. test runner).  The test runner does discovery,
that is that it finds all the tests that match a customized string, and
builds a list of tests which are either run serially or forked in parallel.

What people have done is put conditional logic in the test case and change
the behavior after the test has started.  I consider this analogous to you
unrolling a loop and me coming by and testing if 1==2 for each element of
your unrolled loop.

The test should instead be done once in the runner, instead of 2400 or so
times for every submission into the gate.

To explain this, I cherry-picked some of my code and submitted it as an
example.  My example is here:  https://gerrit.fd.io/r/c/vpp/+/29938. It'
not something I planned to contribute, but I changed it enough to get it to
pass the date.

The commit I am blocking is here: https://gerrit.fd.io/r/c/vpp/+/29921 .

How do I, with limited cycles, convey what needs to be done without writing
sample code or going in and patching over someone's work.
The code is well written and I'd rather +2 it and try to coax some more
contributions.  ;)

Paul

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18067): https://lists.fd.io/g/vpp-dev/message/18067
Mute This Topic: https://lists.fd.io/mt/78323128/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Python API modules

[Paul Vinciguerra]

I think that generating the python code makes the most sense.  The body of
the code can still defer to the dynamically generated object, since it's
not available until runtime anyway, but the user no longer has to resort to
using lambdas because of the deferred bindings.

I imagined it would work like a regular module:

>>> import vpp_papi.plugins.map as map
>>> dir(map)
...
>>> help(map)

vppapigen would generate stubs such as:

vpp_papi.plugins.map.py:

def map_add_domain(client_index: u32, context: u32, ip6_prefix:
IPV6Network, ip4_prefix: IPV6Network, ip6_src: IPV6Interface, ea_bits_len :
u8, psid_offset : u8, psid_length : u8, mtu: u16, tag : str  ) -> i32:
   """docstring from .api block comment
   """
 >

The user, of course, still needs to connect to VPP for their code to run.

but the typedefs could become data classes and the encode/decode methods in
the test code become unnecessary.
vl_api_ip_ecn_t.IP_API_ECN_NONE could become ip_ecn.NONE

The IDE's could then provide type checking and auto completion. (So one
would know that NONE was an option without referring to the sources.)

We would need to play around with it to see how well it works.

On Sat, Nov 14, 2020 at 1:16 PM  wrote:

> Hi Paul,
>
> Picking off one issue at the time.
>
> > Why won't Ole, as maintainer, allow it?
> >
> > The build system uses something called vppapigen to generate the c
> include files and the json files.  It could as easily generate static stubs
> so that development with python could be usable without a running vpp
> instance.
>
> I certainly didn't intend for my opinion to come off as a "don't allow".
>
> Initially the Python code was fully generated from vppapigen. With a class
> per .api file and fully formed methods. Later it evolved to dynamically
> create all functions based on the API representations in JSON.
>
> (This latter change made the Python binding independent of VPP version
> which is how it was possible to individually package it and put it on PyPI,
> but lets do that on a separate thread.)
>
> Generating Python code from vppapigen is simple. But just how do you want
> these stubs to look like and how should they be packaged and used?
>
> If I have pushed back on this it's likely more out of ignorance of the
> problem rather than ill will. ;-)
>
> Best regards,
> Ole

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18065): https://lists.fd.io/g/vpp-dev/message/18065
Mute This Topic: https://lists.fd.io/mt/78229638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Python API modules

[Paul Vinciguerra]

Hi Marcos.

Yes. Your assumption is correct. A library user expects access to the
library.

I'm going to ask the question that you haven't.

Why won't Ole, as maintainer, allow it?

The build system uses something called vppapigen to generate the c include
files and the json files.  It could as easily generate static stubs so that
development with python could be usable without a running vpp instance.

The thing is that there are other trivial issues that need to be addressed
first.

Wouldn't you love to be able to do 'pip install vpp_papi' and get going?
Yes, vpp builds artifacts for vpp_api_python, but who actually wants
vpp_api installed globally?  The python best practice is to install a venv,
and do pip install.
Even 'make test' follows this pattern.

vpp_papi has changed significantly, yet the version number is never, ever
increased.

To address this, on April 5, 2019, I attempted to automate versioning of
vpp_papi to the git tag.  https://gerrit.fd.io/r/c/vpp/+/18694.

There is a version on pypi, it is woefully out of date, and honestly not
usable.  On June 18, 2019, I contributed
https://gerrit.fd.io/r/c/vpp/+/20035, to automate the process of updating
pypi, with a simple command 'tox -epypi' see:src/vpp-api/python/tox.ini

I have also floated the idea of moving vpp_papi to a submodule, so that it
could be easily developed against if pypi were too burdensome to add to the
release process.  Just include the submodule in your code, pin it to any
commit you like, and you're off to the races!

To be 100% clear, this is not an attack on Ole.  I have nothing but respect
for him.  He is extremely talented and has been *extremely* generous with
his time to me.  As for me, as anyone here can tell you, I'm not a C
programmer.  The only reason, I ever considered touching the c code, was
because of Ole's help and guidance and his suggestion to fix the API
instead of conforming the test.

He/Vratko removed me as one of the maintainers of papi.
https://gerrit.fd.io/r/c/vpp/+/22672.  I'm cool with not being a
maintainer, kinda funny that it was stuffed into another changeset.  I have
been called out repeatedly for submitting unrelated changes ;)

All my changesets are still out there, so that should others become
blocked, they can still get work done.  'git-review -X 12345' is your
friend. (If you use -X instead of -x, you can remove the changeset with
'git rebase -i' if you want to submit a contribution)

In that spirit, if you want a python module with static methods, let me
know.

Paul


On Fri, Nov 13, 2020 at 10:57 AM Vratko Polak -X (vrpolak - PANTHEON
TECHNOLOGIES at Cisco)  wrote:

> > All available api are represented in python.
>
>
>
> That is one part of it.
>
> Not every "available" message is useable,
>
> as some messages need corresponding VPP plugins,
>
> which may be disabled.
>
>
>
> >> all available Python modules to use in VPP API.
>
>
>
> It depends on what do you mean by "Python module".
>
> If you mean "usable messages",
>
> you can examine the message table [0]
>
> after PAPI has connected to a running VPP.
>
>
>
> Vratko.
>
>
>
> [0]
> https://github.com/FDio/vpp/blob/66d10589f412d11841c4c8adc0a498b5527e88cb/src/vpp-api/python/vpp_papi/vpp_papi.py#L834-L836
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Ole Troan
> *Sent:* Friday, 2020-November-13 15:51
> *To:* Marcos - Mgiga 
> *Cc:* vpp-dev 
> *Subject:* Re: [vpp-dev] Python API modules
>
>
>
> Hi Marcos,
>
>
>
> On 13 Nov 2020, at 15:08, Marcos - Mgiga  wrote:
>
> 
>
>
>
>
>
> Hello There,
>
>
>
> I believe this is a trivial question, but where / how can I get a list of
> all avaialble Python modules to use in VPP API.
>
>
>
>
>
> You can’t.
>
> They are auto generated from the available json representations of .api
> files.
>
> All available api are represented in python.
>
>
>
> Cheers
>
> Ole
>
>
>
>
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18024): https://lists.fd.io/g/vpp-dev/message/18024
Mute This Topic: https://lists.fd.io/mt/78229638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP-1946

[Paul Vinciguerra]

Hi All,

I just wanted to take a sec., and say thank you to Damjan/Ben for fixing 
VPP-1946.  I documented an issue, and it was fixed in 2 days.  That was an 
example of a great user experience!

Thank you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18017): https://lists.fd.io/g/vpp-dev/message/18017
Mute This Topic: https://lists.fd.io/mt/78233745/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP committers: VPP PTL vote

[Paul Vinciguerra]

+1

On Fri, Sep 25, 2020 at 3:14 PM Dave Barach via lists.fd.io  wrote:

> Folks,
>
>
>
> The self-nomination period closed yesterday. We had one self-nomination,
> from Damjan Marion. At this point, we can proceed with a vote.
>
>
>
> I’m sure that Damjan will do a great job, so let me start:
>
>
>
> Damjan Marion as VPP PTL: +1
>
>
>
> Please vote +1, 0, -1. For once, the “reply-all” button is everyone’s
> friend.
>
>
>
> Thanks... Dave
>
>
>
> -Original Message-
> From: d...@barachs.net 
> Sent: Thursday, September 17, 2020 10:32 AM
> To: 'vpp-dev@lists.fd.io' ; 't...@lists.fd.io' <
> t...@lists.fd.io>
> Subject: Happy Trails to Me...
>
>
>
> Folks,
>
>
>
> I’m departing the employment rolls towards the end of next month. Although
> I intend to remain active in the fd.io vpp community as a coder,
> committer, and resident greybeard, it’s time for the community to pick a
> new PTL.
>
>
>
> According to the project governance document,
> https://fd.io/docs/tsc/FD.IO-Technical-Community-Document-12-12-2017.pdf:
>
>
>
>
> 3.2.3.1 Project Technical Leader Candidates Candidates for the project’s
> PTL will be derived from the Committers of the Project. Candidates must
> self-nominate.
>
>
>
> I'd like to invite any interested vpp project committer to self-nominate
> for the PTL role. Please email vpp-dev@lists.fd.io.
>
>
>
> Let's close the self-nomination period in one week: more specifically, by
> 5pm EDT on Thursday, September 24, 2020; committer vote to follow
> thereafter.
>
>
>
> I'll be glad to answer unicast questions about the PTL role from eligible
> committers.
>
>
>
> Thanks... Dave
>
>
>
>
>
>
>
>
>
>
>
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17527): https://lists.fd.io/g/vpp-dev/message/17527
Mute This Topic: https://lists.fd.io/mt/77123394/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] test/requirements-3.txt snnoyance

[Paul Vinciguerra]

Hi Damjan,

My personal solution is 'git-review -X 21208', 'make wipe-test test.
HTH,

On Tue, Jun 23, 2020 at 12:27 PM Damjan Marion via lists.fd.io  wrote:

> Folks,
>
> Every time I run “make test”, i need to deal with modifications in
> test/requirements-3.txt
>
> Even worse, if I do “git checkout -f”, i cannot run “make test” anymore
> until remove build-root/build-test.
>
> Any idea how to get a rid of that?
>
>
> $ git diff
> diff --git a/test/requirements-3.txt b/test/requirements-3.txt
> index a17c49fc3..896480387 100644
> --- a/test/requirements-3.txt
> +++ b/test/requirements-3.txt
> @@ -99,10 +99,6 @@ imagesize==1.2.0 \
>
>  
> --hash=sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1
> \
>
>  
> --hash=sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1
> \
>  # via sphinx
> -importlib-metadata==1.6.0 \
> -
> --hash=sha256:2a688cbaa90e0cc587f1df48bdc97a6eadccdcd9c35fb3f976a09e3b5016d90f
> \
> -
> --hash=sha256:34513a8a0c4962bc66d35b359558fd8a5e10cd472d37aec5f66858addef32c1e
> \
> -# via flake8
>  jinja2==2.11.2 \
>
>  
> --hash=sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0
> \
>
>  
> --hash=sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035
> \
> @@ -272,10 +268,6 @@ urllib3==1.25.9 \
>
>  
> --hash=sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527
> \
>
>  
> --hash=sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115
> \
>  # via requests
> -zipp==3.1.0 \
> -
> --hash=sha256:aa36550ff0c0b7ef7fa639055d797116ee891440eac1a56f378e2d3179e0320b
> \
> -
> --hash=sha256:c599e4d75c98f6798c509911d08a22e6c021d074469042177c8c86fb92eefd96
> \
> -# via importlib-metadata
>
>  # WARNING: The following packages were not pinned, but pip requires them
> to be
>  # pinned when the requirements file includes hashes. Consider using the
> --allow-unsafe flag.
>
>
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16798): https://lists.fd.io/g/vpp-dev/message/16798
Mute This Topic: https://lists.fd.io/mt/75063401/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Building with fresh Ubuntu Focal

[Paul Vinciguerra]

Hi Keith,

Yes.  The build assumes that vppapigen runs in the global python
environment such as if in a fresh container.

It is the same issue we have with generating vpp papi packages.  It
installs into the global environment and people have no option to pip
install it without downloading the sources and doing a pip install -e
.  There is a vpp_papi package on pypi, but it hasn't been updated in
almost 2 years.

To fix the issue properly, vppapigen should become a python package
encapsulating its dependency on ply, so that it works independent of the
python version/location.

The same holds true of the test framework.

Paul


On Mon, Jun 1, 2020 at 9:46 AM Keith Burns  wrote:

> Had some issues building with fresh Ubuntu Focal install around python3.
>
> To fix I had to:
> sudo apt install python3-venv
> python3 -m venv "~/pyenv/vpp"
> source ~/pyenv/vpp/bin/activate
>
> # build fails without manually installing
> pip install ply
>
> cd 
> make install-dep build
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16600): https://lists.fd.io/g/vpp-dev/message/16600
Mute This Topic: https://lists.fd.io/mt/74604485/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] generic TCP MSS clamping

[Paul Vinciguerra]

Yes, please do.  My changeset only freshens the api code to take advantage
of the refactorings that have been put in place since that code was last
submitted and fix the tests to work properly under python3 in anticipation
of your changes.

Paul

On Sun, May 31, 2020 at 5:23 AM Miklós Tirpák 
wrote:

> Hi Paul,
>
> I just saw the updated patch from you in
> https://gerrit.fd.io/r/c/vpp/+/15144/2.
> We discussed on the mailing list few days ago that I am working on
> updating this plugin, and your patch is a bit conflicting with my changes
> unfortunately.
>
> If you do not mind I would still update the patch set in this code review
> with some more changes: RX support, bugfix in clamping (mss is changed even
> if it is low enough), some more tests, cli command updates. I hope I can do
> this quickly.
>
> Thanks,
> Miklos
> --
> *From:* vpp-dev@lists.fd.io  on behalf of Miklos
> Tirpak via lists.fd.io 
> *Sent:* Thursday, May 28, 2020 4:51 PM
> *To:* otr...@employees.org ; Mohsin Kazmi (sykazmi)
> 
> *Cc:* vpp-dev@lists.fd.io 
> *Subject:* Re: [vpp-dev] generic TCP MSS clamping
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
> Thank you for the pointer, this is exactly what I was looking for. I will
> rebase the patch and add RX support.
>
> Thanks,
> Miklos
> --
> *From:* otr...@employees.org 
> *Sent:* Thursday, May 28, 2020 12:43 PM
> *To:* Mohsin Kazmi (sykazmi) 
> *Cc:* Miklós Tirpák ; vpp-dev@lists.fd.io <
> vpp-dev@lists.fd.io>
> *Subject:* Re: [vpp-dev] generic TCP MSS clamping
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> Good find Mohsin. So it's only missing clamping on RX. I'm sure Miklos can
> add that.
>
> Cheers,
> Ole
>
> > On 28 May 2020, at 12:23, Mohsin Kazmi (sykazmi) 
> wrote:
> >
> > Hi Miklos,
> >
> > May be, it will help https://gerrit.fd.io/r/c/vpp/+/15144
> >
> > -br
> > Mohsin
> > From:  on behalf of Ole Troan  >
> > Date: Thursday, May 28, 2020 at 11:23 AM
> > To: Miklos Tirpak 
> > Cc: "vpp-dev@lists.fd.io" 
> > Subject: Re: [vpp-dev] generic TCP MSS clamping
> >
> > Hi Miklos,
> >
> > > I see the NAT plugin already supports TCP MSS clamping but it is
> implemented only in in2out direction.
> > >
> > > We have endpoints with wrong MTUs behind tunnels and not all the
> traffic is NATed. Hence, it would be very nice to have generic support for
> MSS clamping that could be enabled on the tunnel interface.
> > >
> > > Do you think implementing this as a feature arch would make sense?
> Then it would not be limited to NAT or to one kind of tunnel for example.
> > > If so, what is the best place? A new plugin?
> >
> > A bidirectional TCP MSS adjust would be fine.
> > Putting it in a plugin is likely the simplest.
> >
> > I'm unsure if it should be generic or not. E.g. the NAT also needs to
> adjust the TCP checksum, and it's likely better to do it only once.
> >
> > Best regards,
> > Ole
> >
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16586): https://lists.fd.io/g/vpp-dev/message/16586
Mute This Topic: https://lists.fd.io/mt/74499850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Fix in set_ipfix_exporter_command_fn() to avoid segmentation fault crash

[Paul Vinciguerra]

A few weeks back, I became aware of the following issue with the LISP tests:

/vpp/build-root/install-vpp_debug-native/vpp/bin/vpp[63989]:
vnet_lisp_add_del_locator_set:2140: Can't delete a locator that supports a
mapping!
/vpp/build-root/install-vpp_debug-native/vpp/bin/vpp[63989]: received
signal SIGSEGV, PC 0xa0a0a00, faulting address 0xa0a0a00
14:32:44,290 Child test runner process unresponsive and core-file exists in
test temporary directory (last test running was `Test case for basic
encapsulation' in `/tmp/vpp-unittest-TestLisp-wtuvdu4q')!

which seems to be triggered by the api trace commands in tearDown in
framework.py.  I see it while running tests in a docker container.



On Thu, May 28, 2020 at 4:51 AM Andrew Yourtchenko 
wrote:

> Hi Elias,
>
> Yeah it all does point to something like uninitialized data - I ran
> yesterday the tests on two different machines for a while, apparently
> without the issues...
>
> The CI runtime environment is much more dynamic - it’s an ephemeral docker
> container that is orchestrated by the nomad and is destroyed after the job
> is run.
>
> Could you push as a separate change the code that reliably gives you the
> error in the LISP unit test in the CI, and let me know the change# ?
>
>
>  I will then test some tooling enhancement ideas I had for a while - to
> check within the job whether the core exists, and if it does, to load it
> into gdb and do some scripted processing of it and output the results...
> (Iterate over the call stack and issue stuff like ‘info locals’, ‘info
> regs’, etc).
>
> I did some experiments with that approach earlier and it seemed like a
> rather scalable technique for most of the issues, which should also save
> disk space and the developer time ...
>
> --a
>
> > On 28 May 2020, at 10:33, Elias Rudberg 
> wrote:
> >
> > Hi Andrew,
> >
> > In my case it failed several times and appeared to be triggered by
> > seemingly harmless code changes, but it seemed like the problem was
> > reproducible for a given version of the code. What seemed to matter was
> > when I changed things related to local variables inside the
> > set_ipfix_exporter_command_fn() function. The test logs said "Core-file
> > exists" which I suppose means that vpp crashed. The testing framework
> > repeats the test several times, saying "3 attempt(s) left", then "2
> > attempt(s) left" and so on, all those repeated attempts seemed to crash
> > in the same way.
> >
> > It could be something with uninitialized variables, e.g. something that
> > is assumed to be zero but is never explicitly initialized so it can
> > work when it happens to be zero but depending on platform and compiler
> > details there could be some garbage there causing a problem. Then
> > unrelated code changes like adding variables somewhere making things
> > end up at slightly different memory ocations could make the error come
> > and go. This is just guessing of course.
> >
> > Is it possible to get login access to the machine where the
> > gerrit/jenkins tests are run, to debug it there where the issue can be
> > reproduced?
> >
> > / Elias
> >
> >
> >> On Wed, 2020-05-27 at 19:03 +0200, Andrew  Yourtchenko wrote:
> >> Yep, so it looks like we have an issue...
> >>
> >> https://gerrit.fd.io/r/c/vpp/+/27305 has the same failures, I am
> >> rerunning it now to see how intermittent it is - as well as testing
> >> the latest master locally
> >>
> >> --a
> >>
> >>> On 27 May 2020, at 18:56, Elias Rudberg 
> >>> wrote:
> >>>
> >>> Hi Andrew,
> >>>
> >>> Yes, it was Basic LISP test. It looked like this in the
> >>> console.log.gz
> >>> for vpp-verify-master-ubuntu1804:
> >>>
> >>> ===
> >>> 
> >>> ===
> >>> TEST RESULTS:
> >>>Scheduled tests: 1177
> >>> Executed tests: 1176
> >>>   Passed tests: 1039
> >>>  Skipped tests: 137
> >>> Not Executed tests: 1
> >>> Errors: 1
> >>> FAILURES AND ERRORS IN TESTS:
> >>> Testcase name: Basic LISP test
> >>> ERROR: Test case for basic encapsulation
> >>> [test_lisp.TestLisp.test_lisp_basic_encap]
> >>> TESTCASES WHERE NO TESTS WERE SUCCESSFULLY EXECUTED:
> >>> Basic LISP test
> >>> ===
> >>> 
> >>> ===
> >>>
> >>> / Elias
> >>>
> >>>
> >>>
> >>> On Wed, 2020-05-27 at 18:42 +0200, Andrew  Yourtchenko wrote:
>  Basic LISP test - was it the one that was failing for you ?
>  That particular test intermittently failed a couple of times for
>  me
>  as well, on a doc-only change, so we have an unrelated issue.
>  I am running it locally to see what is going on.
>  --a
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16555): https://lists.fd.io/g/vpp-dev/message/16555
Mute This Topic: https://lists.fd.io/mt/74491544/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  

[vpp-dev] vpp-merge-master-centos7 jobs are broken

[Paul Vinciguerra]

I'm not sure who this should go to, nor the impact, so I'm posting it here.

vpp-merge-master-centos7 is failing due to libmemif and [ -Wstringop-overflow=] 
see [0].

[0] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-merge-master-centos7/9496/console.log.gz
 ( 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-merge-master-centos7/9496/console.log.gz
 )
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16422): https://lists.fd.io/g/vpp-dev/message/16422
Mute This Topic: https://lists.fd.io/mt/74243105/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: (Q about fixing endianness bugs in handlers) Re: [vpp-dev] Proposal for VPP binary API stability

[Paul Vinciguerra]

I chimed in initially, because admittedly, I am usually the one breaking
the api, (with Vratko telling me to stop breaking CSIT ;/ ) and I learned a
lot in the process.

1. The thing we don't have today that would have caught these endian issues
are api tests.
  a. Send in a create and verify all the fields in the _details record.
  b. Remove the magic from the vpp_papi_provider, thich "adapts" the api,
and test the defaults.
  I know the api is tested indirectly via its use, but it is not the same.

2.  The tests are frequently "massaged" to get the code through the gate.
I recently submitted https://gerrit.fd.io/r/c/vpp/+/26703 because the tests
were changed so that python prefixes (with 0'd host bits) were being used
in place of interface addresses.  I don't know if my fix is ok, but I don't
believe the existing code is currently correct.

As to Andrew's comments, above,
I agree with the decision to cherry pick to 2001 to avoid api changes,
although my understanding is that it was agreed that changes would go to
master before being cherry picked.  That was why my original suggestion was
for 2 dependent changes, one for 2001, and one based off of that to correct
the master branch.  A consumer shouldn't have to "bunny-hop" through each
version to get to the latest branch.

With respect to https://gerrit.fd.io/r/c/vpp/+/26879,  I disagree with
Ole's -2.  If you move from a 19.x branch, it is transparent.  But
moreover, the feedback to just change the handler is the issue for me.  I
understand that the implemented flag day process is heavy, but it has been
specified that the version is the semantic version, and therefore changing
the handler requires bumping the semver, which triggers the crc.  I know
the api is heavily modelled on protobuf, but if the will is not present in
the community, maybe we shouldn't use semver for the version number and
just tie the version to the git release tag.  Such a decision could of
course, be reverted in the future if the process ever becomes lighter, or
whatever may be.

Let's consider the api changing process for a moment.  There were a
number of lisp api changes made recently.
https://gerrit.fd.io/r/c/vpp/+/24663
https://gerrit.fd.io/r/c/vpp/+/26932
https://gerrit.fd.io/r/c/vpp/+/27044

The first was Jakub's.  It was substantial, changed the crc and merged.
The 2nd was mine, a trivial cleanup that as I understand it, should change
the crc and was not merged.
The last was Onong's, a trivial cleanup that did not change the crc and was
merged.

If api changing changes need to meet some threshold, we should chain them
until they meet the threshold and can be merged.  I don't think they should
be squashed to reduce the impact in case something individually needs to be
reverted.  That implies that the version bump would then be tied to the
last commit in the chain.

Thoughts?

Paul

On Fri, May 15, 2020 at 11:47 AM Christian Hopps  wrote:

> If I read you correctly, this bug went unnoticed, but has never seen an
> LTS release. LTS might be the only release people are actually developing
> products with. :)
>
> I also sent some mail about brokenness in the auto-generated endian
> functions. I noticed this while I was investigating a post 19.08 change
> that actually made the u8 declared enums be __packed__ (vs just being uint
> anyway), which was then breaking my endian conversion code.
>
> FWIW These changes w/ breakages that had to be RCAd and fixed were
> frustrating b/c they seemed like "cleanup" going on in the API w/o enough
> support from the API infrastructure to avoid user pain downstream. Why did
> we need any of these cleanup changes? It would be one thing if the cleanup
> was done at zero cost to the users (i.e., transparently handled by the API
> infrastructure), but that's not the case.
>
> So with that in mind (and understanding I'm only interested in LTS
> releases :) my vote is:
>
> - Back out the u8 change
> - Leave the u32 fix.
>
> More importantly:
>
> 1) Stop making enum u32->u8 changes until this can be handled better by
> the API infrastructure.
> 2) Improve the API infrastructure before making anymore of these enum (or
> any field) size changes.
>
> To accomplish 2) the generated API functions which try to do the endian
> conversion, should be fixed, and more importantly the whole thing needs to
> be made usable by providing non-callback functionality for the synchronous
> calls.
>
> When all that is done then people can actually use the generated API
> function calls, the endian stuff won't be being done by hand then, so the
> size changes will be handled automatically.
>
> Thanks,
> Chris.
>
> > On May 15, 2020, at 10:20 AM, Andrew Yourtchenko 
> wrote:
> >
> > There's a very interesting couple of gerrit changes that just came in
> > today that is worth discussing,
> > and they are a perfect case study to further clarify the process - so
> > I tweaked the subject accordingly..
> > The API message itself is relatively minor, but regardless of what is
> 

[vpp-dev] Resources for learning about VOM

[Paul Vinciguerra]

Does anything exist beyond the description on the wiki?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16235): https://lists.fd.io/g/vpp-dev/message/16235
Mute This Topic: https://lists.fd.io/mt/73983633/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] 11th hour changes and code reviews

[Paul Vinciguerra]

I was -2'd this morning for submitting a change with a "blast radius" this 
close to api freeze [0].  It was suggested that it should be discussed on the 
dev list and that multiple parties outside of vpp should have to +1 it.  I am 
glad to start the conversation.

My change is dependent on another developer's change has been sitting waiting. 
for a response from csit-dev almost a week.  I asked last week if it would be 
ok to refactor out the non-csit related code, but got no response.   I 
refactored the dependent code out of the other change, but it is still 
outstanding.

I agree that the change I submitted is a monster.  Honestly, there are 2-5 
git-friendly changes that could be refactored out of it.  Its written and used 
by me, but it's not worth investing the time to make mergeable if there is no 
desire for it.  I only submitted it at all because I had mentioned last week to 
someone that I had essentially stopped contributing and I was asked if I could 
be enticed to start re-submitting code.  There is no intent on my part to 
subvert the release.

Everyone complains about the tests.  I was "asked" last year to clean up the 
tests.  It's not that code hasn't been written to clean it up.  The reality is 
that code becomes a monster when simple refactorings aren't merged.

If you all tell me you want the changes,  I'm glad to clean it up.

If, in the meantime, someone wants to +2 Neale's change [2], that would be 
great!

[0] https://gerrit.fd.io/r/c/vpp/+/26833 ( https://gerrit.fd.io/r/c/vpp/+/26833 
)
[1] https://lists.fd.io/g/csit-dev/message/4009 ( 
https://lists.fd.io/g/csit-dev/message/4009 )
[2] https://gerrit.fd.io/r/c/vpp/+/26820/4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16231): https://lists.fd.io/g/vpp-dev/message/16231
Mute This Topic: https://lists.fd.io/mt/73980355/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] api_format.c

[Paul Vinciguerra]

I've run into an api where the api was changed and the SCRIPT: output wasn't 
updated to match.
I'd like to clean it up.  What is the best way to verify that the api_format 
code is correct?

Is a self.vapi.cli("binary-api foo") in a python unit test sufficient?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16217): https://lists.fd.io/g/vpp-dev/message/16217
Mute This Topic: https://lists.fd.io/mt/73395667/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP Multi-versioning changes on x86

[Paul Vinciguerra]

Hi Damjan,

Code was recently added for vpp to specify the variant to use.
Is there a minimum hardware spec for the CI tests, and what are your
thoughts on testing the variants?

Paul

On Wed, Apr 29, 2020 at 12:05 PM Damjan Marion via lists.fd.io  wrote:

>
> As you probably know, VPP build system is compiling selected C files
> multiple times with
> different compiler flags and there is runtime infra which detect CPU type
> on runtime and
> selects optimal binary.
>
> On x86 today we have following variants:
> - baseline (-march=core-i7)
> - avx2 (-march=core-avx2)
> - avx512 (-march=skylake-avx512)
>
> There are 2 issues today which forced us to prefer avx2 even on
> skylake/cascadelake server CPUs:
>
> 1) there is bug in binutils version used on ubuntu 18.04 which causes
> avx512 code to be broken
>
> 2) On Skylake/CascadeLake on the 1st sign of instructions which are
> dealing with ZMM registers (512-bit) CPU must request power
> licence and change frequency. This procedure can take up to 500
> microseconds and during that time core
> operates in degraded mode. Same happens again if there is no 512-bit
> instructions for ~2ms.
> That means that sparse use of 512-bit register will cause more harm than
> benefit.
> This is expected to be significantly improved on Icelake CPUs.
>
>
> So i’m planning to do following change:
>
> Replace variants above with following:
>
> - baseline - no changes
> - hsw - Haswell / Broadwell - AVX2 instruction set
> - skx - Skylake Server CPUs/ Cascadelake - AVX512 instruction set without
> use of 512-bit registers
> - icl - Icelake - AVX512 instruction set with use of 512-bit registers +
> new instructions (avx512 bit manipulation, vaes)
>
> Any comments, thoughts?
>
> —
> Damjan
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16204): https://lists.fd.io/g/vpp-dev/message/16204
Mute This Topic: https://lists.fd.io/mt/73355888/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ACL question

[Paul Vinciguerra]

See: src/plugins/acl/test/test_acl_plugin.py

On Tue, Apr 28, 2020 at 7:19 PM Govindarajan Mohandoss <
govindarajan.mohand...@arm.com> wrote:

> Sure Andrew. Is there a unit test case for ACL plugin ?
>
>
>
> *From:* Andrew  Yourtchenko 
> *Sent:* Tuesday, April 28, 2020 4:57 PM
> *To:* Govindarajan Mohandoss 
> *Cc:* vpp-dev@lists.fd.io; nd ; Lijian Zhang <
> lijian.zh...@arm.com>; Jieqiang Wang 
> *Subject:* Re: [vpp-dev] ACL question
>
>
>
> 1-3: no.
>
> 4: please make a “make test” test case illustrating the problem and share
> it.
>
> --a
>
>
>
> On 28 Apr 2020, at 22:37, Govindarajan Mohandoss <
> govindarajan.mohand...@arm.com> wrote:
>
> 
>
> Hi Andrew,
>
>   I am working on ACL plugin SF+SL optimization on ARM servers.
>
>   I am finding prefetches in ACL node is becoming bottle neck. I see
> performance improvements on both SL & SF mode, when SF mode bihash table
> related prefetching is disabled.
>
>   I need some help with right ACL config to verify my patch.
>
>
>
>  I did the testing with Ingress ACL -- 1 Rule and 50 Rules (Rule:  DIP, UDP, SPORT, DPORT> - DPORT is incremented). The Traffic match all the
> 50 rules.
>
>
>
>   When I tried to add 100 rules on the same rule set in SF mode:
>
>   "acl_add_replace -1 ipv4 permit+reflect src 192.81.1.1/32 dst
> 192.82.1.1/32 proto 17 sport 100 dport 1,
>
>... ,
>
>ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport
> 100 dport 100",
>
>
>
>I see only 48 rules in show tables and 48th rule is added as “permit”
> all and not “permit + reflect”. Does it mean <0 – 47> rules will be SF and
> the rest will be in SL mode ?
>
>
>
> "
>
> vpp# show acl-plugin acl
>
> acl-index 0 count 49 tag {}
>
>0: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto
> 17 sport 100 dport 1
>
>
>
>   47: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto
> 17 sport 100 dport 48
>
>   48: ipv4 permit src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport 0-65535
> dport 0-65535
>
>   applied inbound on sw_if_index: 1
>
>   used in lookup context index: 0
>
> "
>
>
>
>1. Is there a limit of 48 on number of rules that can be added into
>the Rule table (acl-index 0) in SF mode ?
>2. Whether 48 rules in a ruleset is good enough to verify my
>optimization patch (Traffic flow will match all the 48 rules) ?
>3. Can I associate more than 1 ACL rule set to an ingress interface
>(like “vat# acl_interface_set_acl_list TenGigabitEthernet1/0/0 input 0 1
>2”) ? Each Rule set 0, 1, 2 will have different ACL rules. Do I need to
>test this case also to study the performance gain ?
>4. In SL mode, When I tried to add 100 rules, only 53 rules are seen
>in show table. 53rd rule is added as permit all (Should I read it as
>permit all ?). Is there a limit on number of rules in SL mode ?
>
> “
>
> vpp# show acl-plugin acl
>
> acl-index 0 count 54 tag {}
>
>   0: ipv4 permit src 192.81.1.1/32 dst 192.82.1.1/32 proto 17
> sport 100 dport 1
>
>   ….
>
>  52: ipv4 permit src 192.81.1.1/32 dst 192.82.1.1/32 proto 17
> sport 100 dport 53
>
>  53: ipv4 permit src 0.0.0.0/0 dst 0.0.0.0/0 proto 0 sport
> 0-65535 dport 0-65535
>
>   applied inbound on sw_if_index: 1
>
>   used in lookup context index: 0
>
> “
>
>
>
> Thanks
>
> Govind
>
>
>
> > -Original Message-
>
> > From: vpp-dev@lists.fd.io  On Behalf Of
> Govindarajan
>
> > Mohandoss via Lists.Fd.Io
>
> > Sent: Friday, March 27, 2020 11:32 AM
>
> > To: Andrew  Yourtchenko 
>
> > Cc: vpp-dev@lists.fd.io
>
> > Subject: Re: [vpp-dev] ACL question
>
> >
>
> > Thank you very much Andrew !! I will do some benchmarks and get back to
>
> > you to understand it better.
>
> >
>
> > Thanks
>
> > Govind
>
> >
>
> > > -Original Message-
>
> > > From: Andrew  Yourtchenko 
>
> > > Sent: Friday, March 27, 2020 7:52 AM
>
> > > To: Govindarajan Mohandoss 
>
> > > Cc: vpp-dev@lists.fd.io; nd 
>
> > > Subject: Re: [vpp-dev] ACL question
>
> > >
>
> > > > On 27 Mar 2020, at 00:47, Govindarajan Mohandoss
>
> > >  wrote:
>
> > > >
>
> > > >
>
> > > >
>
> > > > Hi Andrew,
>
> > > >
>
> > > >I just found out that ACL action differentiates SF or SL.
>
> > > > Following
>
> > > command enables SF and provides better performance.
>
> > > >
>
> > > >“acl_add_replace -1 ipv4 permit+reflect dst 192.82.1.1/32”
>
> > > >
>
> > > >
>
> > > >
>
> > > >Few more questions:
>
> > > >
>
> > > >=
>
> > > >
>
> > > >Choosing between VPP Classifiers and ACL Plugin:
>
> > > >
>
> > > >
>
> > > > https://lists.fd.io/g/vpp-dev/message/5716?p=,,,20,0,0,0::relevance,
>
> > > > ,A
>
> > > > CL,20,2,60,10641995
>
> > > >
>
> > > > You mentioned that VPP classifiers are faster than ACL plugin.
>
> > > > For  field based classification, which one provides
>
> > > > better data
>
> > > plane perf ?
>
> > >
>
> > >
>
> > > It depends. If you wanna simultaneously match on all three, there is
>
> > > currently 

Re: [vpp-dev] vpp project committer nomination: Benoit Ganne

[Paul Vinciguerra]

+1

> On Apr 21, 2020, at 7:45 AM, Dave Barach via lists.fd.io 
>  wrote:
> 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16128): https://lists.fd.io/g/vpp-dev/message/16128
Mute This Topic: https://lists.fd.io/mt/73170252/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Issues with counters in statistics segment

[Paul Vinciguerra]

Hi Bin,

'show stats seg' shows the layout of the data in the stats segment.
' /err/af-packet-input/partial packet ' is at index number 211.

I think you are looking for:
DBGvpp# create loop interface
loop0
DBGvpp# set interface state loop0 up
DBGvpp# set interface lldp loop0
DBGvpp# show node counters
Count                    Node                  Reason
1               lldp-input               good lldp packets (processed)
DBGvpp# show errors
Count                    Node                  Reason
1               lldp-input               good lldp packets (processed)
DBGvpp#

vpp# show statistics seg

Name
 Type  Value

/buffer-pools/default-numa-0/available 
ScalarPtr   16800

/buffer-pools/default-numa-0/cached    
ScalarPtr   0

/buffer-pools/default-numa-0/used  
ScalarPtr   0

/err/af-packet-input/partial packet    
ErrIndex  211
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16049): https://lists.fd.io/g/vpp-dev/message/16049
Mute This Topic: https://lists.fd.io/mt/72920085/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Centos-7 jobs failing in the CI

[Paul Vinciguerra]

Jobs are consistently failing in the CI now.

make[6]: warning:  Clock skew detected.  Your build may be incomplete.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16048): https://lists.fd.io/g/vpp-dev/message/16048
Mute This Topic: https://lists.fd.io/mt/72947521/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Jobs are failing due to inspect.py

[Paul Vinciguerra]

Thanks,
But that only tells me where the traceback was written, I'm looking for the
contents of that file.

Paul

On Mon, Apr 6, 2020 at 11:43 AM Steven Luong (sluong) 
wrote:

> master
>
> *https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/19049/console
> <https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/19049/console>*
>
>
>
> 20.01
>
> https://jenkins.fd.io/job/vpp-make-test-docs-verify-2001/61/console
>
>
>
> Steven
>
>
>
> *From: *Paul Vinciguerra 
> *Date: *Monday, April 6, 2020 at 8:35 AM
> *To: *Paul Vinciguerra 
> *Cc: *"Steven Luong (sluong)" , "vpp-dev@lists.fd.io" <
> vpp-dev@lists.fd.io>
> *Subject: *Re: [vpp-dev] Jobs are failing due to inspect.py
>
>
>
> I have not been able to reproduce the problem from a fresh ubuntu 18.04
> container and for me, the build succeeds.
>
>
>
> build succeeded, 240 warnings.
>
> The HTML pages are in ../../build-root/build-test/doc/html.
> make[2]: Leaving directory '/vpp/test/doc'
>
> If someone can send me the error log:
>
> The full traceback has been saved in /tmp/sphinx-err-o2xo4j0j.log
>
> I will gladly look into it.
>
>
>
> Paul
>
>
>
> On Mon, Apr 6, 2020 at 11:15 AM Paul Vinciguerra via lists.fd.io  vinciconsulting@lists.fd.io> wrote:
>
> Andrew submitted a changeset that backs out the updated Sphinx package.  I
> am building the target 'test-doc' to try to learn the root cause.
>
>
>
> On Mon, Apr 6, 2020 at 11:03 AM steven luong via lists.fd.io  cisco@lists.fd.io> wrote:
>
> Folks,
>
>
>
> It looks like jobs for all branches, 19.08, 20.01, and master, are failing
> due to this inspect.py error. Could somebody who is familiar with the issue
> please take a look at it?
>
>
>
> *18:59:12* Exception occurred:
>
> *18:59:12*   File "/usr/lib/python3.6/inspect.py", line 516, in unwrap
>
> *18:59:12* raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
>
> *18:59:12* ValueError: wrapper loop when unwrapping scapy.fields.BitEnumField
>
> *18:59:12* The full traceback has been saved in /tmp/sphinx-err-o2xo4j0j.log, 
> if you want to report the issue to the developers.
>
> *18:59:12* Please also report this if it was a user error, so that a better 
> error message can be provided next time.
>
> *18:59:12* A bug report can be filed in the tracker at 
> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>
> *18:59:12* Makefile:71: recipe for target 'html' failed
>
> *18:59:12* make[2]: *** [html] Error 2
>
> *18:59:12* make[2]: Leaving directory 
> '/w/workspace/vpp-make-test-docs-verify-1908/test/doc'
>
> *18:59:12* Makefile:237: recipe for target 'doc' failed
>
> *18:59:12* make[1]: *** [doc] Error 2
>
> *18:59:12* make[1]: Leaving directory 
> '/w/workspace/vpp-make-test-docs-verify-1908/test'
>
> *18:59:12* Makefile:449: recipe for target 'test-doc' failed
>
> *18:59:12* make: *** [test-doc] Error 2
>
> *18:59:12* Build step 'Execute shell' marked build as failure
>
> *18:59:12* $ ssh-agent -k
>
>
>
>
>
> Steven
>
>
>
> 
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16012): https://lists.fd.io/g/vpp-dev/message/16012
Mute This Topic: https://lists.fd.io/mt/72813354/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Jobs are failing due to inspect.py

[Paul Vinciguerra]

I have not been able to reproduce the problem from a fresh ubuntu 18.04
container and for me, the build succeeds.

build succeeded, 240 warnings.
The HTML pages are in ../../build-root/build-test/doc/html.
make[2]: Leaving directory '/vpp/test/doc'
If someone can send me the error log:
The full traceback has been saved in /tmp/sphinx-err-o2xo4j0j.log
I will gladly look into it.

Paul

On Mon, Apr 6, 2020 at 11:15 AM Paul Vinciguerra via lists.fd.io  wrote:

> Andrew submitted a changeset that backs out the updated Sphinx package.  I
> am building the target 'test-doc' to try to learn the root cause.
>
> On Mon, Apr 6, 2020 at 11:03 AM steven luong via lists.fd.io  cisco@lists.fd.io> wrote:
>
>> Folks,
>>
>>
>>
>> It looks like jobs for all branches, 19.08, 20.01, and master, are
>> failing due to this inspect.py error. Could somebody who is familiar with
>> the issue please take a look at it?
>>
>>
>>
>> *18:59:12* Exception occurred:
>>
>> *18:59:12*   File "/usr/lib/python3.6/inspect.py", line 516, in unwrap
>>
>> *18:59:12* raise ValueError('wrapper loop when unwrapping 
>> {!r}'.format(f))
>>
>> *18:59:12* ValueError: wrapper loop when unwrapping scapy.fields.BitEnumField
>>
>> *18:59:12* The full traceback has been saved in 
>> /tmp/sphinx-err-o2xo4j0j.log, if you want to report the issue to the 
>> developers.
>>
>> *18:59:12* Please also report this if it was a user error, so that a better 
>> error message can be provided next time.
>>
>> *18:59:12* A bug report can be filed in the tracker at 
>> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>
>> *18:59:12* Makefile:71: recipe for target 'html' failed
>>
>> *18:59:12* make[2]: *** [html] Error 2
>>
>> *18:59:12* make[2]: Leaving directory 
>> '/w/workspace/vpp-make-test-docs-verify-1908/test/doc'
>>
>> *18:59:12* Makefile:237: recipe for target 'doc' failed
>>
>> *18:59:12* make[1]: *** [doc] Error 2
>>
>> *18:59:12* make[1]: Leaving directory 
>> '/w/workspace/vpp-make-test-docs-verify-1908/test'
>>
>> *18:59:12* Makefile:449: recipe for target 'test-doc' failed
>>
>> *18:59:12* make: *** [test-doc] Error 2
>>
>> *18:59:12* Build step 'Execute shell' marked build as failure
>>
>> *18:59:12* $ ssh-agent -k
>>
>>
>>
>>
>>
>> Steven
>>
>> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16010): https://lists.fd.io/g/vpp-dev/message/16010
Mute This Topic: https://lists.fd.io/mt/72813354/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Jobs are failing due to inspect.py

[Paul Vinciguerra]

Andrew submitted a changeset that backs out the updated Sphinx package.  I
am building the target 'test-doc' to try to learn the root cause.

On Mon, Apr 6, 2020 at 11:03 AM steven luong via lists.fd.io  wrote:

> Folks,
>
>
>
> It looks like jobs for all branches, 19.08, 20.01, and master, are failing
> due to this inspect.py error. Could somebody who is familiar with the issue
> please take a look at it?
>
>
>
> *18:59:12* Exception occurred:
>
> *18:59:12*   File "/usr/lib/python3.6/inspect.py", line 516, in unwrap
>
> *18:59:12* raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
>
> *18:59:12* ValueError: wrapper loop when unwrapping scapy.fields.BitEnumField
>
> *18:59:12* The full traceback has been saved in /tmp/sphinx-err-o2xo4j0j.log, 
> if you want to report the issue to the developers.
>
> *18:59:12* Please also report this if it was a user error, so that a better 
> error message can be provided next time.
>
> *18:59:12* A bug report can be filed in the tracker at 
> . Thanks!
>
> *18:59:12* Makefile:71: recipe for target 'html' failed
>
> *18:59:12* make[2]: *** [html] Error 2
>
> *18:59:12* make[2]: Leaving directory 
> '/w/workspace/vpp-make-test-docs-verify-1908/test/doc'
>
> *18:59:12* Makefile:237: recipe for target 'doc' failed
>
> *18:59:12* make[1]: *** [doc] Error 2
>
> *18:59:12* make[1]: Leaving directory 
> '/w/workspace/vpp-make-test-docs-verify-1908/test'
>
> *18:59:12* Makefile:449: recipe for target 'test-doc' failed
>
> *18:59:12* make: *** [test-doc] Error 2
>
> *18:59:12* Build step 'Execute shell' marked build as failure
>
> *18:59:12* $ ssh-agent -k
>
>
>
>
>
> Steven
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16008): https://lists.fd.io/g/vpp-dev/message/16008
Mute This Topic: https://lists.fd.io/mt/72813354/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP nat ipfix logging problem, need to use thread-specific vlib_main_t?

[Paul Vinciguerra]

Thanks Dave, Neale.

This is great information.

On Mon, Apr 6, 2020 at 6:14 AM Neale Ranns (nranns) 
wrote:

>
>
> In the test harness you can inject onto a given worker, e.g. see
> IpsecTun6HandoffTests.
>
>
>
> /neale
>
>
>
> *From: * on behalf of Paul Vinciguerra <
> pvi...@vinciconsulting.com>
> *Date: *Sunday 5 April 2020 at 17:24
> *To: *"Dave Barach (dbarach)" 
> *Cc: *Elias Rudberg , "vpp-dev@lists.fd.io" <
> vpp-dev@lists.fd.io>
> *Subject: *Re: [vpp-dev] VPP nat ipfix logging problem, need to use
> thread-specific vlib_main_t?
>
>
>
> How can we test scenarios like this?
>
> 'set interface rx-placement' doesn't support pg interfaces.
>
> DBGvpp# set interface rx-placement TenGigabitEthernet5/0/0 worker 2
> DBGvpp# set interface rx-placement pg0 worker 2
> set interface rx-placement: not found
>
> DBGvpp#
>
> Is there another command to bind a pg interface to a worker thread?
>
>
>
> On Sun, Apr 5, 2020 at 8:08 AM Dave Barach via lists.fd.io  cisco@lists.fd.io> wrote:
>
> If you have the thread index handy, that's OK. Otherwise, use
> vlib_get_main() which grabs the thread index from thread local storage.
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Elias Rudberg
> Sent: Sunday, April 5, 2020 4:58 AM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] VPP nat ipfix logging problem, need to use
> thread-specific vlib_main_t?
>
> Hello VPP experts,
>
> We have been using VPP for NAT44 for a while and it has been working fine,
> but a few days ago when we tried turing on nat ipfix logging, vpp crashed.
> It turned out that the problem went away if we used only a single thread,
> so it seemed related to how threading was handled in the ipfix logging
> code. The crash happened in different ways on different runs but often
> seemed related to the snat_ipfix_send() function in
> plugins/nat/nat_ipfix_logging.c.
>
> Having looked at the code in nat_ipfix_logging.c I have the following
> theory about what goes wrong (I might have misunderstood something, if so
> please correct me):
>
> In the the snat_ipfix_send() function, a vlib_main_t data structure is
> used, a pointer to it is fetched in the following way:
>
>vlib_main_t *vm = frm->vlib_main;
>
> So the frm->vlib_main pointer comes from "frm" which has been set to
> flow_report_main which is a global data structure from vnet/ipfix-
> export/flow_report.c that as far as I can tell only exists once in memory
> (not once per thread). This means that different threads calling the
> snat_ipfix_send() function are using the same vlib_main_t data structure.
> That is not how it should be, I think, instead each thread should be using
> its own thread-specific vlib_main_t data structure.
>
> A suggestion for how to fix this is to replace the line
>
>vlib_main_t *vm = frm->vlib_main;
>
> with the following line
>
>vlib_main_t *vm = vlib_mains[thread_index];
>
> in all places where worker threads are using such a vlib_main_t pointer.
> Using vlib_mains[thread_index] means that we are picking the
> thread-specific vlib_main_t data structure for the current thread, instead
> of all threads using the same vlib_main_t. I pushed such a change to
> gerrit, here: https://gerrit.fd.io/r/c/vpp/+/26359
>
> That fix seems to solve the issue in my tests, vpp does not crash anymore
> after the change. Please have a look at it and let me know if this seems
> reasonable or if I have misunderstood something.
>
> Best regards,
> Elias
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16003): https://lists.fd.io/g/vpp-dev/message/16003
Mute This Topic: https://lists.fd.io/mt/72786912/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP nat ipfix logging problem, need to use thread-specific vlib_main_t?

[Paul Vinciguerra]

How can we test scenarios like this?
'set interface rx-placement' doesn't support pg interfaces.

DBGvpp# set interface rx-placement TenGigabitEthernet5/0/0 worker 2
DBGvpp# set interface rx-placement pg0 worker 2
set interface rx-placement: not found
DBGvpp#

Is there another command to bind a pg interface to a worker thread?

On Sun, Apr 5, 2020 at 8:08 AM Dave Barach via lists.fd.io  wrote:

> If you have the thread index handy, that's OK. Otherwise, use
> vlib_get_main() which grabs the thread index from thread local storage.
>
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Elias Rudberg
> Sent: Sunday, April 5, 2020 4:58 AM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] VPP nat ipfix logging problem, need to use
> thread-specific vlib_main_t?
>
> Hello VPP experts,
>
> We have been using VPP for NAT44 for a while and it has been working fine,
> but a few days ago when we tried turing on nat ipfix logging, vpp crashed.
> It turned out that the problem went away if we used only a single thread,
> so it seemed related to how threading was handled in the ipfix logging
> code. The crash happened in different ways on different runs but often
> seemed related to the snat_ipfix_send() function in
> plugins/nat/nat_ipfix_logging.c.
>
> Having looked at the code in nat_ipfix_logging.c I have the following
> theory about what goes wrong (I might have misunderstood something, if so
> please correct me):
>
> In the the snat_ipfix_send() function, a vlib_main_t data structure is
> used, a pointer to it is fetched in the following way:
>
>vlib_main_t *vm = frm->vlib_main;
>
> So the frm->vlib_main pointer comes from "frm" which has been set to
> flow_report_main which is a global data structure from vnet/ipfix-
> export/flow_report.c that as far as I can tell only exists once in memory
> (not once per thread). This means that different threads calling the
> snat_ipfix_send() function are using the same vlib_main_t data structure.
> That is not how it should be, I think, instead each thread should be using
> its own thread-specific vlib_main_t data structure.
>
> A suggestion for how to fix this is to replace the line
>
>vlib_main_t *vm = frm->vlib_main;
>
> with the following line
>
>vlib_main_t *vm = vlib_mains[thread_index];
>
> in all places where worker threads are using such a vlib_main_t pointer.
> Using vlib_mains[thread_index] means that we are picking the
> thread-specific vlib_main_t data structure for the current thread, instead
> of all threads using the same vlib_main_t. I pushed such a change to
> gerrit, here: https://gerrit.fd.io/r/c/vpp/+/26359
>
> That fix seems to solve the issue in my tests, vpp does not crash anymore
> after the change. Please have a look at it and let me know if this seems
> reasonable or if I have misunderstood something.
>
> Best regards,
> Elias
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15992): https://lists.fd.io/g/vpp-dev/message/15992
Mute This Topic: https://lists.fd.io/mt/72786912/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp_papi not updated

[Paul Vinciguerra]

A python app needs to import vpp_papi.
If you made changes to papi, then you need to reinstall the latest changes
to the library.
Tests are a consumer of papi, just like your app.

On Fri, Mar 27, 2020 at 6:54 PM Christian Hopps  wrote:

> Right but that's for running the unit tests right? I'm not running unit
> tests, I'm actually just using the python API in another application that
> controls/monitors VPP.
>
> Thanks,
> Chris.
>
> > On Mar 27, 2020, at 6:45 PM, Paul Vinciguerra <
> pvi...@vinciconsulting.com> wrote:
> >
> > From the test/Makefile:
> >   $(PYTHON_INTERP) -m pip install -e $(PAPI_PYTHON_SRC_DIR)"
> >
> > On Fri, Mar 27, 2020 at 6:42 PM Christian Hopps 
> wrote:
> > Hmm, at least on 19.08 doing the papi-wipe/wipe-papi is only removing
> things under test. I'm running an external python app (not under test) and
> it's looking in the install directory which isn't updated after the
> wipe-papi.
> >
> > I found a hack/workaround which is to touch the CMakeLists.txt file in
> src/vpp-api/python, though. That CMakeLists.txt just has an install command
> so I guess it's not setting up any dependencies?
> >
> > Thanks,
> > Chris.
> >
> > > On Mar 27, 2020, at 6:30 PM, Paul Vinciguerra <
> pvi...@vinciconsulting.com> wrote:
> > >
> > > try 'test-wipe-papi'.
> > > You may need:
> > >
> > > https://gerrit.fd.io/r/c/vpp/+/26177
> > >
> > > On Fri, Mar 27, 2020 at 6:26 PM Christian Hopps 
> wrote:
> > > [on 19.08]
> > >
> > > I'm seeing something odd building. When I "make build" after changing
> something in "src/vpp-api/python/vpp_papi/*.py" it does not get reinstalled
> in the build-root. In fact if I delete
> build-root/install-vpp_debug-native/lib64/python3.6/site-packages/vpp_papi*
> it also does not get reinstalled with a "make build".
> > >
> > > I guess vpp_papi is not fully integrated into the cmake build system?
> Short of fixing that, is there a way to force it to reinstall?
> > >
> > > Thanks,
> > > Chris.
> > > 
> >
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15916): https://lists.fd.io/g/vpp-dev/message/15916
Mute This Topic: https://lists.fd.io/mt/72599377/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp_papi not updated

[Paul Vinciguerra]

>From the test/Makefile:

  $(PYTHON_INTERP) -m pip install -e $(PAPI_PYTHON_SRC_DIR)"


On Fri, Mar 27, 2020 at 6:42 PM Christian Hopps  wrote:

> Hmm, at least on 19.08 doing the papi-wipe/wipe-papi is only removing
> things under test. I'm running an external python app (not under test) and
> it's looking in the install directory which isn't updated after the
> wipe-papi.
>
> I found a hack/workaround which is to touch the CMakeLists.txt file in
> src/vpp-api/python, though. That CMakeLists.txt just has an install command
> so I guess it's not setting up any dependencies?
>
> Thanks,
> Chris.
>
> > On Mar 27, 2020, at 6:30 PM, Paul Vinciguerra <
> pvi...@vinciconsulting.com> wrote:
> >
> > try 'test-wipe-papi'.
> > You may need:
> >
> > https://gerrit.fd.io/r/c/vpp/+/26177
> >
> > On Fri, Mar 27, 2020 at 6:26 PM Christian Hopps 
> wrote:
> > [on 19.08]
> >
> > I'm seeing something odd building. When I "make build" after changing
> something in "src/vpp-api/python/vpp_papi/*.py" it does not get reinstalled
> in the build-root. In fact if I delete
> build-root/install-vpp_debug-native/lib64/python3.6/site-packages/vpp_papi*
> it also does not get reinstalled with a "make build".
> >
> > I guess vpp_papi is not fully integrated into the cmake build system?
> Short of fixing that, is there a way to force it to reinstall?
> >
> > Thanks,
> > Chris.
> > 
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15913): https://lists.fd.io/g/vpp-dev/message/15913
Mute This Topic: https://lists.fd.io/mt/72599377/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp_papi not updated

[Paul Vinciguerra]

try 'test-wipe-papi'.
You may need:

https://gerrit.fd.io/r/c/vpp/+/26177

On Fri, Mar 27, 2020 at 6:26 PM Christian Hopps  wrote:

> [on 19.08]
>
> I'm seeing something odd building. When I "make build" after changing
> something in "src/vpp-api/python/vpp_papi/*.py" it does not get reinstalled
> in the build-root. In fact if I delete
> build-root/install-vpp_debug-native/lib64/python3.6/site-packages/vpp_papi*
> it also does not get reinstalled with a "make build".
>
> I guess vpp_papi is not fully integrated into the cmake build system?
> Short of fixing that, is there a way to force it to reinstall?
>
> Thanks,
> Chris.
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15911): https://lists.fd.io/g/vpp-dev/message/15911
Mute This Topic: https://lists.fd.io/mt/72599377/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Unit Test Results in Random directories

[Paul Vinciguerra]

I'm ok with a change, but are you thinking about setting the tempdir to a
constant, or do you want to remove the unix { runtime-dir } option from the
startup?

On Tue, Mar 17, 2020 at 1:04 PM Klement Sekera via Lists.Fd.Io  wrote:

> Hi Neale,
>
> When originally designing this, I opted in for `mktemp` as that’s the
> usual way to avoid clashes as at that time I assumed that this might be
> used on boxes where there are multiple users and they might like to run
> tests in parallel. At that time I didn’t expect issues like multiple vpps
> causing problems and it all kinda stuck.
>
> The way I deal with this is using `less /tmp/vpp-unittest-*/log.txt’
> (which lately requires SANITY=no as it tends to pick log.txt from sanity
> run). For core I use DEBUG=core which gives me gdb without having to think
> about any of the paths.
>
> Having said all that, I don’t see any reason to keep random names. I’m not
> aware of any issues this change my cause…
>
> Regards,
> Klement
>
> > On 17 Mar 2020, at 17:56, Neale Ranns via Lists.Fd.Io  cisco@lists.fd.io> wrote:
> >
> >
> > Hi All,
> >
> > Am I the only one who finds the use of random directories for the
> unit-tests an unnecessary annoyance?
> >
> > I would suggest that random names are not needed for security purposes,
> since these files do not exist on a field system. Also, all directories are
> wiped before the next test run so it can't be to support saving runs nor
> multiple users.
> >
> > I find it annoying because I can't just reload log.txt or core in my
> editor or debugger.
> >
> > It's a simple change if there's consensus, or you can just call me and
> old grump and we can all move on __
> >
> > /neale
> >
> >
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15808): https://lists.fd.io/g/vpp-dev/message/15808
Mute This Topic: https://lists.fd.io/mt/72029031/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VRRP Unit Tests failing on Master Ubuntu

[Paul Vinciguerra]

Hi Dave,

Please don't add broken tests to extended tests.

See https://gerrit.fd.io/r/c/vpp/+/24567/12/test/test_bfd.py

Thanks!

On Fri, Mar 13, 2020 at 1:33 PM Dave Wallace  wrote:

> Matt,
>
> I will keep an eye on this gerrit and merge it once the verify jobs have
> completed.
> If there are other tests which fail, are you ok if I add them to this
> patch and turn it into a generic 'disable failing tests' gerrit change?
>
> The other possibility is that this is due to the recent disabling of the
> Naginator retry plugin.
>
> I'm going to investigate if this issue may have been masked by Naginator...
>
> Thanks for your help on keeping the CI operational!
> -daw-
>
> On 3/12/2020 12:09 PM, Matthew Smith via Lists.Fd.Io wrote:
>
>
> Change submitted - https://gerrit.fd.io/r/c/vpp/+/25834. Verification
> jobs are running. Hopefully they won't fail :)
>
> -Matt
>
>
> On Thu, Mar 12, 2020 at 10:22 AM Matthew Smith via Lists.Fd.Io  netgate@lists.fd.io> wrote:
>
>>
>> I don't have a solution yet, but one observation has popped up quickly
>>
>> In the 2 failed jobs Ray sent links for, one of them had a test fail
>> which was not related to VRRP. There is a BFD6 test failure for the NAT
>> change https://gerrit.fd.io/r/c/vpp/+/25462:
>>
>>
>> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/archives/
>>
>> Looking back through a couple of recent failed runs of that job, there is
>> also a DHCP6 PD test failure for rdma change
>> https://gerrit.fd.io/r/c/vpp/+/25823:
>>
>>
>> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2682/archives/
>>
>> The most obvious common thread between BFD6, DHCP6 and VRRP to me seems
>> to be that they all maintain state which is dependent on timers. There
>> could be a more general issue with timing-sensitive tests. I am going to
>> submit a change which will prevent the VRRP tests from running temporarily
>> while I can figure out a proper solution. Based on the above, other tests
>> may need the same treatment.
>>
>> -Matt
>>
>>
>>
>>
>> On Thu, Mar 12, 2020 at 8:57 AM Matthew Smith 
>> wrote:
>>
>>> Hi Ray,
>>>
>>> Thanks for bringing it to my attention. I'll look into it.
>>>
>>> -Matt
>>>
>>>
>>> On Thu, Mar 12, 2020 at 8:24 AM Ray Kinsella  wrote:
>>>
 Anyone else noticing seeming spurious failures related to the VRRP
 plugin's unit tests.
 Some examples from un-related commits.

 Ray K

 nat: timed out session scavenging upgrade (
 https://gerrit.fd.io/r/c/vpp/+/25462)

 https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/console.log.gz


 ==
 TEST RESULTS:
  Scheduled tests: 1138
   Executed tests: 1138
 Passed tests: 1021
Skipped tests: 112
 Failures: 3
   Errors: 2
 FAILURES AND ERRORS IN TESTS:
   Testcase name: IPv4 VRRP Test Case
 FAILURE: IPv4 Master VR does not reply for VIP w/ accept mode off
 [test_vrrp.TestVRRP4.test_vrrp4_accept_mode_disabled]
 FAILURE: IPv4 Master VR preempted by higher priority backup
 [test_vrrp.TestVRRP4.test_vrrp4_master_preempted]
   Testcase name: IPv6 VRRP Test Case
 FAILURE: IPv6 Master VR preempted by higher priority backup
 [test_vrrp.TestVRRP6.test_vrrp6_master_preempted]
   ERROR: IPv6 Backup VR preempts lower priority master
 [test_vrrp.TestVRRP6.test_vrrp6_backup_preempts]
   Testcase name: Bidirectional Forwarding Detection (BFD) (IPv6)
   ERROR: echo function [test_bfd.BFD6TestCase.test_echo]

 ==

 vlib: startup multi-arch variant configuration (
 https://gerrit.fd.io/r/c/vpp/+/25798_

 https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2675/console.log.gz


 ==
 TEST RESULTS:
  Scheduled tests: 22
   Executed tests: 22
 Passed tests: 21
 Failures: 1
 FAILURES AND ERRORS IN TESTS:
   Testcase name: IPv4 VRRP Test Case
 FAILURE: IPv4 Master VR preempted by higher priority backup
 [test_vrrp.TestVRRP4.test_vrrp4_master_preempted]

 ==




>>
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15782): https://lists.fd.io/g/vpp-dev/message/15782
Mute This Topic: https://lists.fd.io/mt/71901798/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VRRP Unit Tests failing on Master Ubuntu

[Paul Vinciguerra]

Yes.
Has been for a few days.

On Thu, Mar 12, 2020 at 9:25 AM Ray Kinsella  wrote:

> Anyone else noticing seeming spurious failures related to the VRRP
> plugin's unit tests.
> Some examples from un-related commits.
>
> Ray K
>
> nat: timed out session scavenging upgrade (
> https://gerrit.fd.io/r/c/vpp/+/25462)
>
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2678/console.log.gz
>
>
> ==
> TEST RESULTS:
>  Scheduled tests: 1138
>   Executed tests: 1138
> Passed tests: 1021
>Skipped tests: 112
> Failures: 3
>   Errors: 2
> FAILURES AND ERRORS IN TESTS:
>   Testcase name: IPv4 VRRP Test Case
> FAILURE: IPv4 Master VR does not reply for VIP w/ accept mode off
> [test_vrrp.TestVRRP4.test_vrrp4_accept_mode_disabled]
> FAILURE: IPv4 Master VR preempted by higher priority backup
> [test_vrrp.TestVRRP4.test_vrrp4_master_preempted]
>   Testcase name: IPv6 VRRP Test Case
> FAILURE: IPv6 Master VR preempted by higher priority backup
> [test_vrrp.TestVRRP6.test_vrrp6_master_preempted]
>   ERROR: IPv6 Backup VR preempts lower priority master
> [test_vrrp.TestVRRP6.test_vrrp6_backup_preempts]
>   Testcase name: Bidirectional Forwarding Detection (BFD) (IPv6)
>   ERROR: echo function [test_bfd.BFD6TestCase.test_echo]
>
> ==
>
> vlib: startup multi-arch variant configuration (
> https://gerrit.fd.io/r/c/vpp/+/25798_
>
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1804/2675/console.log.gz
>
>
> ==
> TEST RESULTS:
>  Scheduled tests: 22
>   Executed tests: 22
> Passed tests: 21
> Failures: 1
> FAILURES AND ERRORS IN TESTS:
>   Testcase name: IPv4 VRRP Test Case
> FAILURE: IPv4 Master VR preempted by higher priority backup
> [test_vrrp.TestVRRP4.test_vrrp4_master_preempted]
>
> ==
>
>
>
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15757): https://lists.fd.io/g/vpp-dev/message/15757
Mute This Topic: https://lists.fd.io/mt/71901798/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] how to catch packets on a specific TCP port

[Paul Vinciguerra]

Have you looked at the http_static plugin?

> On Mar 5, 2020, at 5:33 AM, Satya Murthy  wrote:
> 
> Hi,
> 
> We are looking for a way to get all the packets on a particular TCP port to 
> reach our plugin.
> For UDP ports, we could see that udp_register_dst_port() can be used to get 
> packets on a specific UDP port.
> But, for TCP, we could not see any registration function like this.
> 
> Any pointers on this pls ? 
>
> -- 
> Thanks & Regards,
> Murthy 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15693): https://lists.fd.io/g/vpp-dev/message/15693
Mute This Topic: https://lists.fd.io/mt/71746978/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp project committers: formal vote to add Matt Smith as a vpp committer

[Paul Vinciguerra]

+1

> On Mar 2, 2020, at 9:20 AM, d...@barachs.net wrote:
> 
> 
> VPP committers, please vote +1, 0, -1 on adding Matt Smith 
> (mgsm...@netgate.com) as a vpp project committer.
> Matt has contributed O(100) merged patches, and he recently contributed the 
> entire vrrp plugin. See https://gerrit.fd.io/r/q/owner:mgsmith%2540netgate.com
> Please vote (on vpp-dev@lists.fd.io) by the end of this Wednesday, 2/4/2020, 
> so we can put the results in front of the TSC this Thursday.
> Thanks... Dave
>
> 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15660): https://lists.fd.io/g/vpp-dev/message/15660
Mute This Topic: https://lists.fd.io/mt/71675525/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP configuration getting erased on VPP restart

[Paul Vinciguerra]

Hi Chinmaya.

Follow Dave's input.

I'm obsessive over typo's/transcription errors, so I've done it like this
in the past:

Edit your startup.conf to include:
unix {
  nodaemon
  *log /etc/vpp/startup-conf*
  full-coredump
  cli-listen /run/vpp/cli.sock
  gid vpp
}

start vpp and configure your systemas you like.
stop vpp (systemctl stop vpp)
edit /etc/vpp/startup-conf to remove the timestamps and place each command
on a new line.

then edit startup.conf to replace the 'log' keyword with 'startup-config'
to look like:

unix {
  nodaemon
  *startup-config /etc/vpp/startup-conf*
  full-coredump
  cli-listen /run/vpp/cli.sock
  gid vpp
}

and restart vpp (systemctl restart vpp)


On Fri, Feb 28, 2020 at 11:35 AM Dave Barach via Lists.Fd.Io  wrote:

> Please do exactly what I suggested. Here’s why: restarting vpp via “vppctl
> restart” re-executes vpp with *precisely* the same command line arguments
> passed to it e.g. via systemctl.
>
>
>
> Until you start vpp with the command line argument change I told you
> about, there’s no chance you’ll get the effect you’re looking for. None.
> Zero. Zip. Nada.
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Chinmaya
> Aggarwal
> *Sent:* Friday, February 28, 2020 11:23 AM
> *To:* vpp-dev@lists.fd.io
> *Subject:* Re: [vpp-dev] VPP configuration getting erased on VPP restart
>
>
>
> I did restart vpp using "vppctl restart" but I couldn't find the
> configuration. When i reboot the machine then only i could see my
> configuration. I am running VPP version 19.08. What could be the reason for
> this?
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15621): https://lists.fd.io/g/vpp-dev/message/15621
Mute This Topic: https://lists.fd.io/mt/71609857/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] route lookup api

[Paul Vinciguerra]

Those don't seem to be exposed via the api dump/details.

The pattern that is commonly used in the test framework is to call
vapi.cli_inline(cmd="show ip fib 1.2.3.4/24")

On Tue, Feb 18, 2020 at 5:53 PM Christian Hopps  wrote:

> In the CLI there's an option to lookup route for a given IP. Is there a
> similar interface in the binary API?
>
> The code I'm looking at now is doing an entire fib dump to look for this
> route, which seems problematic as all the logic VPP might us for what entry
> to select has to be attempted to be replicated in the client getting this
> dump. So I'm hoping theres a better way. :)
>
> Thanks,
> Chris.
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15459): https://lists.fd.io/g/vpp-dev/message/15459
Mute This Topic: https://lists.fd.io/mt/71382541/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] #vnet Any command to individually enable and disable IP forwarding per interface level

[Paul Vinciguerra]

Hi Neale,

Can you not also do:

comment { ingress direction }
set interface feature loop0 ip4-lookup arc ip4-unicast  disable

comment { egress direction }
set interface feature loop0 arp-reply arc arp disable


On Wed, Jan 22, 2020 at 12:16 PM Neale Ranns via Lists.Fd.Io  wrote:

>
>
> hi,
>
>
>
> to enables IP4 forwarding on an interface, either apply an address or make
> it unnumberered to another interface (that has an address);
>
>set int ip addr  
>
>set int unnumbered  use 
>
> to enables IP4 forwarding on an interface, either apply an address or
> enable ip6 on it.
>
>set int ip addr  
>
>enable ip6 
>
>
>
> /neale
>
>
>
>
>
> *From: * on behalf of "Gigo Thomas via Lists.Fd.Io"
> 
> *Reply to: *"gig...@thinkpalm.com" 
> *Date: *Wednesday 22 January 2020 at 23:38
> *To: *"vpp-dev@lists.fd.io" 
> *Cc: *"vpp-dev@lists.fd.io" 
> *Subject: *[vpp-dev] #vnet Any command to individually enable and disable
> IP forwarding per interface level
>
>
>
> May I know if VPP has any command to individually enable/disable ip
> forwarding per interface level.
>
> Thanks
> 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15228): https://lists.fd.io/g/vpp-dev/message/15228
Mute This Topic: https://lists.fd.io/mt/69978408/21656
Mute #vnet: https://lists.fd.io/mk?hashtag=vnet=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] python api over tcp?

[Paul Vinciguerra]

Hi Ole.

Where can we find the Telegraf plugin?

On Thu, Jan 9, 2020 at 5:44 AM Ole Troan  wrote:

> Christian,
>
> >> For exporting data out of the stats segment, I believe there is already
> quite a few solutions.
> >> There's the (naive) prometheus example in the repo, vpp_get_stats,
> there is a Telegraf plugin, a simple gNMI/gRPC plugin.
> >
> > Right I've used vpp_get_stats and may run that with ssh and awk. I guess
> it just seems odd on first encountering this that the CLI provided the
> data, but the binary API didn't. I suppose the view is that exposing the
> stats segment in shared memory *is* the binary API. :)
>
> What certainly would make sense to do, is to put a wrapper on top of
> vpp_stats.py that gives you a higher level API of accessing the stats.
> E.g. a get_interface_counters(). The stat segment also contains the name
> to interface index mapping (/if/names).
> Want to have a go?
>
> I am also exploring putting much more information into the stat segment,
> essentially making it into an operational data store (RFC8342). Don't hold
> your breath. But any help appreciated.
>
> Cheers,
> Ole
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15131): https://lists.fd.io/g/vpp-dev/message/15131
Mute This Topic: https://lists.fd.io/mt/69538850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] python api over tcp?

[Paul Vinciguerra]

My thought was to add a daemon that listened to tls and wrapped the shm 
transport/libapiclient, that would stay coresident with vpp then connect with 
the papi client from a remote system via tls.

> On Jan 9, 2020, at 1:38 PM, Ole Troan  wrote:
> 
> 
> Hi Paul,
> 
>>> On 9 Jan 2020, at 19:10, Paul Vinciguerra  
>>> wrote:
>>> 
>> 
>> Sounds like a little scope creep going on there.
>> If you provide protobuf3 encoded api messages to/from vpp, I'll add a grpc 
>> listener option to vpp_papi, in the interim, I'd be glad to add a tls 
>> wrapped listener if there is interest.
> 
> Were you thinking of adding a simple tls/tcp to uds proxy? Must be something 
> off the shelf that can be used. Netcat? Then you want to amend vpp_papi to be 
> able to sit at the other end of that? What did you plan to do with security?
> 
> The grpc idea isn’t too dissimilar. Just pass vpp api messages encapsulated 
> in grpc. One request, one reply and an event message. 
> 
> Cheers 
> Ole
> 
> 
>> 
>>> On Thu, Jan 9, 2020 at 12:00 PM Ole Troan  wrote:
>>> 
>>> 
>>>>> On 9 Jan 2020, at 16:50, Paul Vinciguerra  
>>>>> wrote:
>>>>> 
>>>> 
>>>> Is there any objection to adding a tls listener and an instance to the 
>>>> stats client to vpp_papi?
>>> 
>>> Use grpc as transport?
>>> 
>>> Cheers,
>>> Ole
>>> 
>>>> 
>>>>>> On Jan 9, 2020, at 6:45 AM, Vratko Polak -X (vrpolak - PANTHEON 
>>>>>> TECHNOLOGIES at Cisco) via Lists.Fd.Io  
>>>>>> wrote:
>>>>>> 
>>>>> 
>>>>> CSIT uses VPP API via socket (tunneled over SSH) for most interactions.
>>>>> 
>>>>>
>>>>> 
>>>>> We also read stats for just one (I think) purpose,
>>>>> 
>>>>> reading runtime stats (/sys/node).
>>>>> 
>>>>> The way we do that is historical and convoluted,
>>>>> 
>>>>> for the result see INFO line at [1].
>>>>> 
>>>>>
>>>>> 
>>>>> Looking at the result, the appropriate API way
>>>>> 
>>>>> would be to use send some _dump message
>>>>> 
>>>>> and process the _details responses, one per node name.
>>>>> 
>>>>>
>>>>> 
>>>>> Vratko.
>>>>> 
>>>>>
>>>>> 
>>>>> [1] 
>>>>> https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/58/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k9-k1-k1-k4-k1
>>>>> 
>>>>>
>>>>> 
>>>>> From: vpp-dev@lists.fd.io  On Behalf Of Christian 
>>>>> Hopps
>>>>> Sent: Thursday, January 9, 2020 12:05 PM
>>>>> To: Ole Troan 
>>>>> Cc: Christian Hopps ; vpp-dev 
>>>>> Subject: Re: [vpp-dev] python api over tcp?
>>>>> 
>>>>>
>>>>> 
>>>>> 
>>>>> 
>>>>> > On Jan 9, 2020, at 5:44 AM, Ole Troan  wrote:
>>>>> > 
>>>>> > Christian,
>>>>> > 
>>>>> >>> For exporting data out of the stats segment, I believe there is 
>>>>> >>> already quite a few solutions.
>>>>> >>> There's the (naive) prometheus example in the repo, vpp_get_stats, 
>>>>> >>> there is a Telegraf plugin, a simple gNMI/gRPC plugin.
>>>>> >> 
>>>>> >> Right I've used vpp_get_stats and may run that with ssh and awk. I 
>>>>> >> guess it just seems odd on first encountering this that the CLI 
>>>>> >> provided the data, but the binary API didn't. I suppose the view is 
>>>>> >> that exposing the stats segment in shared memory *is* the binary API. 
>>>>> >> :)
>>>>> > 
>>>>> > What certainly would make sense to do, is to put a wrapper on top of 
>>>>> > vpp_stats.py that gives you a higher level API of accessing the stats.
>>>>> > E.g. a get_interface_counters(). The stat segment also contains the 
>>>>> > name to interface index mapping (/if/names).
>>>>> > Want to have a go?
>>>>> 
>>>>> I'm actually going to use vpp_get_stats (run remotely using ssh) for now. 
>>>>> I'm using vpp_papi on a single testing server (so it connects to each of 
>>>>> the VPP /run/vpp/api.sock over ssh forwarded sockets), so it doesn't have 
>>>>> access to their shared memory segments.
>>>>> 
>>>>> > 
>>>>> > I am also exploring putting much more information into the stat 
>>>>> > segment, essentially making it into an operational data store 
>>>>> > (RFC8342). Don't hold your breath. But any help appreciated.
>>>>> 
>>>>> I will be looking at doing some YANG models later this year, so if the 
>>>>> timing aligns.. :)
>>>>> 
>>>>> Thanks,
>>>>> Chris.
>>>>> 
>>>>> 
>>>>> > Cheers,
>>>>> > Ole
>>>>> 
>>>>> 
>>>> 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15125): https://lists.fd.io/g/vpp-dev/message/15125
Mute This Topic: https://lists.fd.io/mt/69538850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] python api over tcp?

[Paul Vinciguerra]

Sounds like a little scope creep going on there.
If you provide protobuf3 encoded api messages to/from vpp, I'll add a grpc
listener option to vpp_papi, in the interim, I'd be glad to add a tls
wrapped listener if there is interest.

On Thu, Jan 9, 2020 at 12:00 PM Ole Troan  wrote:

>
>
> On 9 Jan 2020, at 16:50, Paul Vinciguerra 
> wrote:
>
> 
> Is there any objection to adding a tls listener and an instance to the
> stats client to vpp_papi?
>
>
> Use grpc as transport?
>
> Cheers,
> Ole
>
>
> On Jan 9, 2020, at 6:45 AM, Vratko Polak -X (vrpolak - PANTHEON
> TECHNOLOGIES at Cisco) via Lists.Fd.Io 
> wrote:
>
> 
>
> CSIT uses VPP API via socket (tunneled over SSH) for most interactions.
>
>
>
> We also read stats for just one (I think) purpose,
>
> reading runtime stats (/sys/node).
>
> The way we do that is historical and convoluted,
>
> for the result see INFO line at [1].
>
>
>
> Looking at the result, the appropriate API way
>
> would be to use send some _dump message
>
> and process the _details responses, one per node name.
>
>
>
> Vratko.
>
>
>
> [1]
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/58/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k9-k1-k1-k4-k1
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Christian
> Hopps
> *Sent:* Thursday, January 9, 2020 12:05 PM
> *To:* Ole Troan 
> *Cc:* Christian Hopps ; vpp-dev 
> *Subject:* Re: [vpp-dev] python api over tcp?
>
>
>
>
>
> > On Jan 9, 2020, at 5:44 AM, Ole Troan  wrote:
> >
> > Christian,
> >
> >>> For exporting data out of the stats segment, I believe there is
> already quite a few solutions.
> >>> There's the (naive) prometheus example in the repo, vpp_get_stats,
> there is a Telegraf plugin, a simple gNMI/gRPC plugin.
> >>
> >> Right I've used vpp_get_stats and may run that with ssh and awk. I
> guess it just seems odd on first encountering this that the CLI provided
> the data, but the binary API didn't. I suppose the view is that exposing
> the stats segment in shared memory *is* the binary API. :)
> >
> > What certainly would make sense to do, is to put a wrapper on top of
> vpp_stats.py that gives you a higher level API of accessing the stats.
> > E.g. a get_interface_counters(). The stat segment also contains the name
> to interface index mapping (/if/names).
> > Want to have a go?
>
> I'm actually going to use vpp_get_stats (run remotely using ssh) for now.
> I'm using vpp_papi on a single testing server (so it connects to each of
> the VPP /run/vpp/api.sock over ssh forwarded sockets), so it doesn't have
> access to their shared memory segments.
>
> >
> > I am also exploring putting much more information into the stat segment,
> essentially making it into an operational data store (RFC8342). Don't hold
> your breath. But any help appreciated.
>
> I will be looking at doing some YANG models later this year, so if the
> timing aligns.. :)
>
> Thanks,
> Chris.
>
>
> > Cheers,
> > Ole
>
>
> 
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15121): https://lists.fd.io/g/vpp-dev/message/15121
Mute This Topic: https://lists.fd.io/mt/69538850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] install-dep fails in RedHat (RHEL)

[Paul Vinciguerra]

Can you submit a patch to the Makefile that applies your fix?

On Thu, Jan 9, 2020 at 4:48 AM  wrote:

> Thanks for your response Benoit.
> After installing epel repo, it works fine.
>
> Regards,
> Vijayalakshmi 
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15114): https://lists.fd.io/g/vpp-dev/message/15114
Mute This Topic: https://lists.fd.io/mt/69523613/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-