Re: MX204 tunnel services BW

[Saku Ytti]

On Mon, 2 Oct 2023 at 20:21, Jeff Behrns via NANOG  wrote:

> Encountered an issue with an MX204 using all 4x100G ports and a logical
> tunnel to hairpin a VRF.  The tunnel started dropping packets around 8Gbps.
> I bumped up tunnel-services BW from 10G to 100G which made the problem
> worse; the tunnel was now limited to around 1.3Gbps.  To my knowledge with
> Trio PFE you shouldn't have to disable a physical port to allocate bandwidth
> for tunnel-services.  Any helpful info is appreciated.

You might have more luck in j-nsp.

But yes you don't need any physical interface in trio to do tunneling.
I can't explain your problem, and you probably need JTAC help. I would
appreciate it if you'd circle back and tell what the problem was.

How it works is that when PPE decides it needs to tunnel the packet,
you're going to send the packet back to MQ via SERDES (which will then
send it again to some PPE, not the same). I think what that bandwidth
command does is change the stream allocation, you should see it in
'show  <#> stream'.

In theory, because PPE can process packet forever (well, until
watchdog kills the PPE for thinking it is stuck) you could very
cheaply do outer+inner at the local PPE, but I think that would mean
that certain features like QoS would not work on the inner interface,
so I think all this expensive recirculation and SERDES consumption is
to satisfy quite limited need, and it should be possible to implement
some 'performance mode' for tunneling, where these MQ/XM provided
features are not available, but performance cost in most cases is
negligible.

In parallel to opening the JTAC case, you might want to try to
experiment in which FPC/PIC you set the tunneling bandwidth to. I
don't understand how the tunneling would work if the MQ/XM is remote,
like would you then also steal fabric capacity every time you tunnel,
not just  MQ>LU>MQ>LU SERDES, but MQ>LU>MQ>FAB>MQ>LU. So intuitively I
would recommend ensuring you have the bandwidth configured at the
local PFE, if you don't know what the local PFE is, just configure it
everywhere?
Also you could consult several counters to see if some stream or
fabric is congested, and these tunneled packets are being sent over
congested fabric every time with lower fabric qos.

I don't understand why the bandwidth command is a thing, and why you
can configure where it is. To me it seems obvious they should always
handle tunneling strictly locally, never over fabric, because you
always end up stealing more capacity if you send it to remote MQ. That
is, implicitly it should be on for every MQ, and every PPE tunnel via
local MQ.

-- 
  ++ytti

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Daniel Corbe]

On 10/3/2023 3:48 PM, Christopher Morrow wrote:

those are a bit of a false equivalence... but... ok.
I think: "Oh look, more spam, delete"
is basically how this sort of problem (email from randos trying to
sell me ED pills or 10Gs) should be treated.
I don't know that it's helpful to keep re-litigating that end state :(

I'm sure telling dave shaeffer: "Hey, your sales droids are being
rude" is going to end as well as sending him ED pill emails.


On the other hand, it's actually nice knowing Cogent are up to their 
same old tricks, so that when they try to end-around me to get a sale 
done, I have plenty of ammunition at my disposal to shoot them down.


Much like your ED pill E-Mail analogy above (and I think you might have 
been able to pick a less explicit example, but hey, edgy humor amirite?) 
it should be pretty trivial for you to nuke this thread so it doesn't 
keep appearing at the top of your inbox.


OpenPGP_signature.asc
Description: OpenPGP digital signature

RE: MX204 tunnel services BW

[Jeff Behrns via NANOG]

-Original Message-
From: Delong.com  
Sent: Monday, October 2, 2023 5:47 PM
To: behrnsj...@yahoo.com
Cc: nanog@nanog.org
Subject: Re: MX204 tunnel services BW

> “Tunnel gets whatever bandwidth is left after physical port packets are 
> processed” and likely some additional overhead for managing the sharing.

>Could that be what’s happening to you?

Aggregate throughput for the box was less than 100Gbps while the tunnel was 
being starved.

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Christopher Morrow]

On Tue, Oct 3, 2023 at 12:54 PM  wrote:
>
> * morrowc.li...@gmail.com (Christopher Morrow) [Tue 03 Oct 2023, 18:29 CEST]:
> >this sort of thing (provider X scrapes Y and mails Z for sales leads)
> >every ~18 months.
> >the same outrage and conversation happens every time.
> >the same protection mechanisms are noted every time.
> >
> >Is there a reason that: "killfileand move on" is not the answer
> >everytime for this?
> >(why do we need to keep rediscussing it)
>
> It's a vicious circle: provider scrapes operational addresses and
> spams them, providers stop putting useful addresses in public
> databases to avoid spam, everybody who needs to find operational
> contacts in a variety of situations loses in the end.

i agree this is a sad outcome of the internet ecosystem.

> We keep discussing it because we care about keeping the internet
> running. It's similar to why we keep looking for new security holes in
> existing software: we don't stop because inevitably we'll find more so
> it's a lost cause, we keep looking because inevitably we'll find more
> so the product becomes more secure.

those are a bit of a false equivalence... but... ok.
I think: "Oh look, more spam, delete"
is basically how this sort of problem (email from randos trying to
sell me ED pills or 10Gs) should be treated.
I don't know that it's helpful to keep re-litigating that end state :(

I'm sure telling dave shaeffer: "Hey, your sales droids are being
rude" is going to end as well as sending him ED pill emails.

ARIN Election

[Nicholas Warren]

Does anyone know how many people will be elected from each category?

I don't regularly keep up with everyone's business. So, are any of candidates 
overachievers or, well, underachievers?

Nich Warren

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[niels=nanog]

* morrowc.li...@gmail.com (Christopher Morrow) [Tue 03 Oct 2023, 21:50 CEST]:
I'm sure telling dave shaeffer: "Hey, your sales droids are being 
rude" is going to end as well as sending him ED pill emails.


Such outreach to technical contacts is counterproductive anyway. Which 
is more likely, that noc@ leads to a person with purchasing power, or 
that it leads to one who happens to be rabidly anti-spam who may then 
help get you thrown off the PeeringDB island for a few seasons?



-- Niels.

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[niels=nanog]

* morrowc.li...@gmail.com (Christopher Morrow) [Tue 03 Oct 2023, 18:29 CEST]:
this sort of thing (provider X scrapes Y and mails Z for sales leads) 
every ~18 months.

the same outrage and conversation happens every time.
the same protection mechanisms are noted every time.

Is there a reason that: "killfileand move on" is not the answer 
everytime for this?

(why do we need to keep rediscussing it)


It's a vicious circle: provider scrapes operational addresses and 
spams them, providers stop putting useful addresses in public 
databases to avoid spam, everybody who needs to find operational 
contacts in a variety of situations loses in the end.


We keep discussing it because we care about keeping the internet 
running. It's similar to why we keep looking for new security holes in 
existing software: we don't stop because inevitably we'll find more so 
it's a lost cause, we keep looking because inevitably we'll find more 
so the product becomes more secure.



-- Niels.

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Christopher Morrow]

this sort of thing (provider X scrapes Y and mails Z for sales leads)
every ~18 months.
the same outrage and conversation happens every time.
the same protection mechanisms are noted every time.

Is there a reason that: "killfileand move on" is not the answer
everytime for this?
(why do we need to keep rediscussing it)

On Tue, Oct 3, 2023 at 11:54 AM Bryan Fields  wrote:
>
> On 10/2/23 11:28 AM, Mel Beckman wrote:
> > I believe they got the contact information from ARIN
>
> I'd suggest everyone use an alias unique to ARIN for your POC and/or public
> email.  Makes it super simple to verify where it was sourced from.
>
> (and yes I've got the same spam)
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>

ARIN email address (was cogent spamming directly from ARIN records?)

[Bryan Fields]

On 10/2/23 11:28 AM, Mel Beckman wrote:

I believe they got the contact information from ARIN


I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
email.  Makes it super simple to verify where it was sourced from.


(and yes I've got the same spam)
--
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net

Re: maximum ipv4 bgp prefix length of /24 ?

[Chris Hills]

On 02/10/2023 14:19, t...@pelican.org wrote:

If the FIB is full, can we start making controlled and/or smart decisions about 
what to install, rather than either of the simple overflow conditions?


There is a project [1] that make use of sflow to install the top n 
prefixes by traffic, presuming there is a default route for the remainder.


Chris

[1] https://github.com/sflow-rt/active-routes

Re: ARIN Election

[John Curran]

Hello Nich!

On Oct 3, 2023, at 4:26 PM, Nicholas Warren  wrote:

Does anyone know how many people will be elected from each category?

This year’s ARIN election will fill four (4) seats on the ARIN Board of 
Trustees and seven (7) seats on the ARIN Advisory Council.

I don’t regularly keep up with everyone’s business. So, are any of candidates 
overachievers or, well, underachievers?

I don’t know the scope of appropriate discussions on the nanog mailing list, 
but I would be remiss if I didn’t point out that there’s ARIN general-members 
mailing list where such discussion is actively encouraged –


All candidates for the Board and Advisory Council will be offered the ability 
to subscribe to the General Members Mailing 
List to answer 
questions from the voting community. If your organization is a General Member 
at ARIN and plans to vote this year, we encourage you to join the mailing list 
to stay up to date on ARIN Election activity. Candidates have also been offered 
the opportunity to present themselves to the general membership in a brief, 
pre-recorded speech during ARIN 52, and those 
speeches will be made available online during the voting period.

I’ve attached the full ARIN Candidate Slate announcement below, for those 
interested in such matters.

Thanks!
/John

John Curran
President and CEO
American Registry for Internet Numbers


===


Begin forwarded message:

From: ARIN 
Subject: [arin-announce] The 2023 Slate of Candidates for ARIN Elections
Date: September 12, 2023 at 4:38:42 PM EDT
To: "arin-annou...@arin.net" 

The 2023 ARIN Slate of Candidates has been amended to reflect that Rebecca 
Naughton has withdrawn as a candidate from the election for the Board of 
Trustees.
Candidates for ARIN Elections
ARIN Elections will be held online from 3:00 PM EDT on Thursday, 19 October 
2023 through 7:00 PM EDT on Friday, 27 October 2023. These elections will fill 
four seats on the Board of Trustees and seven seats on the Advisory Council. 
The Nomination Committee (NomCom) has put forward the following slates of 
candidates with classifications for the terms beginning 1 January 2024:
Board of Trustees Candidates:

  *   Dan Alexander (Well Qualified)
  *   Nancy Carter (Well Qualified)
  *   Jack Cathey (Qualified)
  *   Philip Duclos (Qualifications not Demonstrated)
  *   Andrew Dul (Qualified)
  *   Khaled Koubaa (Well Qualified)
  *   Tina Morris (Well Qualified)
  *   William Sylvester (Well Qualified)
  *   Christian Tacit (Well Qualified)
  *   David Zumwalt (Well Qualified)

Advisory Council Candidates:

  *   Douglas Camin (Well Qualified)
  *   Anthony Delacruz (Qualified)
  *   Matthew Gamble (Well Qualified)
  *   Elizabeth Goodson (Qualified)
  *   Dean Hardy (Qualifications not Demonstrated)
  *   Roy Hoover (Qualified)
  *   Rob Johnstone (Qualifications not Demonstrated)
  *   Dustin Moses (Qualified)
  *   Kaitlyn Pellak (Qualified)
  *   Leif Sawyer (Qualified)
  *   Daniel Schatte (Qualified)
  *   Ibrahim (Ibro) Seremet (Qualifications not Demonstrated)
  *   Jason Weil (Qualified)
  *   Matthew Wilder (Well Qualified)

A compilation of candidate biographies and questionnaire responses is available 
at:
https://www.arin.net/participate/oversight/elections/candidate_bios.pdf
Each candidate for the Board of Trustees and Advisory Council, listed above, 
has a classification next to their name. The candidates were classified by an 
independent third-party vendor firm based on the Nominee Classification Process 
in the NomCom Charter. Every nominee is put forward on the initial slate unless 
the third-party vendor firm classifies a nominee as “Unable to Qualify.” No 
nominees were classified as “Unable to Qualify” this year. Fourteen nominations 
were received for the Advisory Council and twelve were received for the Board; 
all are present on the Final Slate of Candidates except for one Board nominee 
who withdrew their candidacy.
You are encouraged to submit and view Statements of Support for the candidates 
at: https://www.arin-elections.net/. Please note that all Statements of Support 
are automatically held for moderation and will be posted, upon approval, within 
one business day. All submissions are subject to the Statements of Support 
Acceptable Use 
Policy.
All candidates for the Board and Advisory Council will be offered the ability 
to subscribe to the General Members Mailing 
List to answer 
questions from the voting community. If your organization is a General Member 
at ARIN and plans to vote this year, we encourage you to join the mailing list 
to stay up to date on ARIN Election activity. Candidates have also been offered 
the opportunity to present themselves to the general membership in a brief, 
pre-recorded speech during ARIN 

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[John Curran]

On Oct 3, 2023, at 11:52 AM, Bryan Fields  wrote:

On 10/2/23 11:28 AM, Mel Beckman wrote:
I believe they got the contact information from ARIN

I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
email.  Makes it super simple to verify where it was sourced from.

(and yes I've got the same spam)

Bryan -

You are absolutely correct - it is wise to use a unique email address if at all 
possible, and please report misuse to 
complia...@arin.net.

It does make a difference, as abuse reports result in discussions with 
organizations regarding appropriate reeducation regimes for violating staff – 
and further implications if the pattern of abuse appears systemic as opposed to 
incidental.

(It is not a perfect system, as it often needs periodically refreshment in some 
orgs due to inevitable turnover and miscreant creativity, but it does tamp down 
the worst of the abuse that would occur otherwise…)

Thanks!
/John

John Curran
President and CEO
American Registry for Internet Numbers

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Collider]

Heh. (:

Le 4 octobre 2023 01:23:13 UTC, Owen DeLong via NANOG  a écrit 
:
>I was one of the main people behind their suspension from ARIN whois for 6 
>months.
>
>They have not spammed me since.
>
>They’re probably afraid of another cake.
>
>Owen
>
>
>> On Oct 3, 2023, at 18:18, Mike Lyon  wrote:
>> 
>> Give it time :)
>> 
>> -Mike
>> 
>>> On Oct 3, 2023, at 18:06, Owen DeLong via NANOG  wrote:
>>> 
>>> But I seem to have finally gotten Cogent trained not to spam this one, so 
>>> I think I’ll leave it as is.
>>> 
>>> YMMV
>>> 
>>> Owen
>>> 
>>> 
 On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
 
> On 10/2/23 11:28 AM, Mel Beckman wrote:
> I believe they got the contact information from ARIN
 
 I'd suggest everyone use an alias unique to ARIN for your POC and/or 
 public email.  Makes it super simple to verify where it was sourced from.
 
 (and yes I've got the same spam)
 -- 
 Bryan Fields
 
 727-409-1194 - Voice
 http://bryanfields.net
>>> 
>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: maximum ipv4 bgp prefix length of /24 ?

[Justin Wilson (Lists)]

I think it is going to have to happen.  We have several folks on the IX and 
various consulting clients who only need 3-6 Ips but have to burn a full /24 to 
participate in BGP. I wrote a blog post awhile back on this topic 
https://blog.j2sw.com/data-center/unpopular-opinion-bgp-should-accept-smaller-than-a-24/




Justin Wilson
j...@mtin.net

—
https://j2sw.com (AS399332)
https://blog.j2sw.com - Podcast and Blog

> On Sep 30, 2023, at 1:48 PM, Randy Bush  wrote:
> 
>> About 60% of the table is /24 routes.
>> Just going to /25 will probably double the table size.
> 
> or maybe just add 60%, not 100%.  and it would take time.
> 
> agree it would be quite painful.  would rather not go there.  sad to
> say, i suspect some degree of lengthening is inevitable.  we have
> ourselves to blame; but blame does not move packets.
> 
> randy, who was in the danvers cabal for the /19 agreement
> 

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Owen DeLong via NANOG]

But I seem to have finally gotten Cogent trained not to spam this one, so I 
think I’ll leave it as is.

YMMV

Owen


> On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
> 
> On 10/2/23 11:28 AM, Mel Beckman wrote:
>> I believe they got the contact information from ARIN
> 
> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
> email.  Makes it super simple to verify where it was sourced from.
> 
> (and yes I've got the same spam)
> -- 
> Bryan Fields
> 
> 727-409-1194 - Voice
> http://bryanfields.net

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Owen DeLong via NANOG]

I was one of the main people behind their suspension from ARIN whois for 6 
months.

They have not spammed me since.

They’re probably afraid of another cake.

Owen


> On Oct 3, 2023, at 18:18, Mike Lyon  wrote:
> 
> Give it time :)
> 
> -Mike
> 
>> On Oct 3, 2023, at 18:06, Owen DeLong via NANOG  wrote:
>> 
>> But I seem to have finally gotten Cogent trained not to spam this one, so I 
>> think I’ll leave it as is.
>> 
>> YMMV
>> 
>> Owen
>> 
>> 
>>> On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
>>> 
 On 10/2/23 11:28 AM, Mel Beckman wrote:
 I believe they got the contact information from ARIN
>>> 
>>> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
>>> email.  Makes it super simple to verify where it was sourced from.
>>> 
>>> (and yes I've got the same spam)
>>> -- 
>>> Bryan Fields
>>> 
>>> 727-409-1194 - Voice
>>> http://bryanfields.net
>> 

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[Mike Lyon]

Give it time :)

-Mike

> On Oct 3, 2023, at 18:06, Owen DeLong via NANOG  wrote:
> 
> But I seem to have finally gotten Cogent trained not to spam this one, so I 
> think I’ll leave it as is.
> 
> YMMV
> 
> Owen
> 
> 
>> On Oct 3, 2023, at 08:52, Bryan Fields  wrote:
>> 
>>> On 10/2/23 11:28 AM, Mel Beckman wrote:
>>> I believe they got the contact information from ARIN
>> 
>> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
>> email.  Makes it super simple to verify where it was sourced from.
>> 
>> (and yes I've got the same spam)
>> -- 
>> Bryan Fields
>> 
>> 727-409-1194 - Voice
>> http://bryanfields.net
> 

Re: ARIN email address (was cogent spamming directly from ARIN records?)

[John Curran]

> On Oct 3, 2023, at 11:52 AM, Bryan Fields  wrote:
> 
> On 10/2/23 11:28 AM, Mel Beckman wrote:
>> I believe they got the contact information from ARIN
> 
> I'd suggest everyone use an alias unique to ARIN for your POC and/or public 
> email.  Makes it super simple to verify where it was sourced from.
> 
> (and yes I've got the same spam)

Bryan - 

You are absolutely correct - it is wise to use a unique email address if at all 
possible, and please report misuse to complia...@arin.net 
. 

It does make a difference, as abuse reports result in discussions with 
organizations regarding appropriate reeducation regimes for violating staff – 
and further implications if the pattern of abuse appears systemic as opposed to 
incidental. 

(It is not a perfect system, as it often needs periodically refreshment in some 
orgs due to inevitable turnover and miscreant creativity, but it does tamp down 
the worst of the abuse that would occur otherwise…)

Thanks!
/John

John Curran
President and CEO
American Registry for Internet Numbers

Re: Legal system as a weapon (was Re: AFRINIC placed in receivership)

[Delron Troy]



On Sat, 30 Sep 2023, 06:03 Jay R. Ashworth,  wrote:

> Layer 8: People
>
> Layer 9: Money
>
> Layer 10: Lawyers.
>
> Cheers,
> -- jra
>
> - Original Message -
> > From: "David Conrad" 
> > To: nanog@nanog.org
> > Sent: Thursday, September 28, 2023 6:46:31 PM
> > Subject: Legal system as a weapon (was Re: AFRINIC placed in
> receivership)
>
> > Somewhat related (at least one of the principals is the same) and
> perhaps of
> > interest to some here. While I have strong opinions on the topic,
> provided
> > without comment:
> >
> > https://www.gofundme.com/f/supporting-and-protecting-internet-governance
> >
> > Regards,
> > -drc
> >
> >> On Sep 13, 2023, at 6:27 PM, Bryan Fields 
> wrote:
> >>
> >> I think this qualifies as potentially operational.
> >>
> >> Afrinic placed in receivership, board elections to be held in six
> months:
> >> https://archive.ph/jOFE4
> >> --
> >> Bryan Fields
> >>
> >> 727-409-1194 - Voice
> > > http://bryanfields.net
>
> --
> Jay R. Ashworth  Baylink
> j...@baylink.com
> Designer The Things I Think   RFC
> 2100
> Ashworth & Associates   http://www.bcp38.info  2000 Land
> Rover DII
> St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647
> 1274
>

Re: MX204 tunnel services BW

[Tom Beecher]

>
> AIUI, with Trio, you don’t have to disable a physical port, but that comes
> at the cost of “Tunnel gets whatever bandwidth is left after physical port
> packets are processed” and likely some additional overhead for managing the
> sharing.
>

This was pretty much my understanding as well, last time I dealt with this.
On MPC/Trio , you just enabled tunnel-services on a given PIC, and landed
your tunnel there. The tunnel capacity was just part of the PFE capacity.

Was only on pre-Trio that the bandwidth keyword was required, and that
actually reserved that much capacity strictly for the tunnel.

On Mon, Oct 2, 2023 at 6:48 PM Delong.com via NANOG  wrote:

> AIUI, with Trio, you don’t have to disable a physical port, but that comes
> at the cost of “Tunnel gets whatever bandwidth is left after physical port
> packets are processed” and likely some additional overhead for managing the
> sharing.
>
> Could that be what’s happening to you?
>
> Owen
>
>
> > On Oct 2, 2023, at 09:24, Jeff Behrns via NANOG  wrote:
> >
> > Encountered an issue with an MX204 using all 4x100G ports and a logical
> > tunnel to hairpin a VRF.  The tunnel started dropping packets around
> 8Gbps.
> > I bumped up tunnel-services BW from 10G to 100G which made the problem
> > worse; the tunnel was now limited to around 1.3Gbps.  To my knowledge
> with
> > Trio PFE you shouldn't have to disable a physical port to allocate
> bandwidth
> > for tunnel-services.  Any helpful info is appreciated.
>
>

Re: MX204 tunnel services BW

[Owen DeLong via NANOG]

> On Oct 2, 2023, at 20:18, behrnsj...@yahoo.com wrote:
> 
> 
> 
> -Original Message-
> From: Delong.com  
> Sent: Monday, October 2, 2023 5:47 PM
> To: behrnsj...@yahoo.com
> Cc: nanog@nanog.org
> Subject: Re: MX204 tunnel services BW
> 
>> “Tunnel gets whatever bandwidth is left after physical port packets are 
>> processed” and likely some additional overhead for managing the sharing.
> 
>> Could that be what’s happening to you?
> 
> Aggregate throughput for the box was less than 100Gbps while the tunnel was 
> being starved.
> 

Yeah, doesn’t quite work that way…

The tunnel is assigned to one particular PFE.

What was the aggregate throughput on that PFE (which spending on the card may 
well top out at 40Gbps or even 10Gbps, though not likely
on most Trio-based cards, that’s more of the DPC era cards, which did require 
you to sacrifice a port for tunnel bandwidth).

Owen

Re: MX204 tunnel services BW

[Owen DeLong via NANOG]

You can configure tunnel bandwidth everywhere, but you can’t configure
a given tunnel everywhere, you have to assign it to a particular FPC/PIC/0.

For example, with:
set chassis fps 2 pic 3 tunnel-services bandwidth 10g

You need to create gr-2/3/0 interfaces for tunnels to use that PFE.

You can create multiple tunnel-services bandwidth entries on multiple
PICs, but you can only put a given tunnel on one gr-x/y/0 interface.

Owen


> On Oct 2, 2023, at 23:21, Saku Ytti  wrote:
> 
> On Mon, 2 Oct 2023 at 20:21, Jeff Behrns via NANOG  wrote:
> 
>> Encountered an issue with an MX204 using all 4x100G ports and a logical
>> tunnel to hairpin a VRF.  The tunnel started dropping packets around 8Gbps.
>> I bumped up tunnel-services BW from 10G to 100G which made the problem
>> worse; the tunnel was now limited to around 1.3Gbps.  To my knowledge with
>> Trio PFE you shouldn't have to disable a physical port to allocate bandwidth
>> for tunnel-services.  Any helpful info is appreciated.
> 
> You might have more luck in j-nsp.
> 
> But yes you don't need any physical interface in trio to do tunneling.
> I can't explain your problem, and you probably need JTAC help. I would
> appreciate it if you'd circle back and tell what the problem was.
> 
> How it works is that when PPE decides it needs to tunnel the packet,
> you're going to send the packet back to MQ via SERDES (which will then
> send it again to some PPE, not the same). I think what that bandwidth
> command does is change the stream allocation, you should see it in
> 'show  <#> stream'.
> 
> In theory, because PPE can process packet forever (well, until
> watchdog kills the PPE for thinking it is stuck) you could very
> cheaply do outer+inner at the local PPE, but I think that would mean
> that certain features like QoS would not work on the inner interface,
> so I think all this expensive recirculation and SERDES consumption is
> to satisfy quite limited need, and it should be possible to implement
> some 'performance mode' for tunneling, where these MQ/XM provided
> features are not available, but performance cost in most cases is
> negligible.
> 
> In parallel to opening the JTAC case, you might want to try to
> experiment in which FPC/PIC you set the tunneling bandwidth to. I
> don't understand how the tunneling would work if the MQ/XM is remote,
> like would you then also steal fabric capacity every time you tunnel,
> not just  MQ>LU>MQ>LU SERDES, but MQ>LU>MQ>FAB>MQ>LU. So intuitively I
> would recommend ensuring you have the bandwidth configured at the
> local PFE, if you don't know what the local PFE is, just configure it
> everywhere?
> Also you could consult several counters to see if some stream or
> fabric is congested, and these tunneled packets are being sent over
> congested fabric every time with lower fabric qos.
> 
> I don't understand why the bandwidth command is a thing, and why you
> can configure where it is. To me it seems obvious they should always
> handle tunneling strictly locally, never over fabric, because you
> always end up stealing more capacity if you send it to remote MQ. That
> is, implicitly it should be on for every MQ, and every PPE tunnel via
> local MQ.
> 
> -- 
>  ++ytti