Re: [c-nsp] STP over Port-channel issue
On Mon, 6 May 2024 at 15:53, james list via cisco-nsp wrote: > The question: since the PO remains up, why we see this behaviour ? > are BDPU sent just over one link (ie the higher interfac e) ? Correct. > how can we solve this issue keeping this scenario ? > moving to RSTP could solve ? No. I understand you want topology to remain intact, as long as there is at least 1 member up, but I'm not sure we can guarantee that. I think if you set LACP to fast, it'll fail in max 3s, and you ensure STP fails slower (i.e. don't use rapid pvst), you probably will just see BPDU switch physical interface, instead of STP convergence. You'd need something similar to microBFD on LAGs, where BFD PDU is sent on each member, instead of an unspecified single member, but afaik this does not exist. So what you really need is L3/MPLS topology :/. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
On Mon, 12 Feb 2024 at 09:44, james list wrote: > I'd like to test with LACP slow, then can see if physical interface still > flaps... I don't think that's good idea, like what would we know? Would we have to wait 30 times longer, so month-3months, to hit what ever it is, before we have confidence? I would suggest - turn on debugging, to see cisco emitting LACP PDU, and juniper receiving LACP PDU - do packet capture, if at all reasonable, ideally tap, but in absence of tap mirror - turn off LACP distributed handling on junos - ping on the link, ideally 0.2-0.5s interval, to record how ping stops in relation to first syslog emitted about LACP going down - wait for 4days -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
On Sun, 11 Feb 2024 at 17:52, james list wrote: > - why physical interface flaps in DC1 if it is related to lacp ? 16:39:35.813 Juniper reports LACP timeout (so problem started at 16:39:32, (was traffic passing at 32, 33, 34 seconds?)) 16:39:36.xxx Cisco reports interface down, long after problem has already started Why Cisco reports physical interface down, I'm not sure. But clearly the problem was already happening before interface down, and first log entry is LACP timeout, which occurs 3s after the problem starts. Perhaps Juniper asserts for some reason RFI? Perhaps Cisco resets the physical interface once removed from LACP? > - why the same setup in DC2 do not report issues ? If this is is LACP related software issue, could be difference not identified. You need to gather more information, like how does ping look throughout this event, particularly before syslog entries. And if ping still works up-until syslog, you almost certainly have software issue with LACP inject at Cisco, or more likely LACP punt at Juniper. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
On Sun, 11 Feb 2024 at 15:24, james list wrote: > While on Juniper when the issue happens I always see: > > show log messages | last 440 | match LACPD_TIMEOUT > Jan 25 21:32:27.948 2024 MX1 lacpd[31632]: LACPD_TIMEOUT: et-0/1/5: lacp > current while timer expired current Receive State: CURRENT > Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACPD_TIMEOUT: et-0/1/5: lacp > current while timer expired current Receive State: CURRENT Ok so problem always starts by Juniper seeing 3seconds without LACP PDU, i.e. missing 3 consecutive LACP PDU. It would be good to ping while this problem is happening, to see if ping stops at 3s before the syslog lines, or at the same time as syslog lines. If ping stops 3s before, it's link problem from cisco to juniper. If ping stops at syslog time (my guess), it's software problem. There is unfortunately log of bug surface here, both on inject and on punt path. You could be hitting PR1541056 on the Juniper end. You could test for this by removing distributed LACP handling with 'set routing-options ppm no-delegate-processing' You could also do packet capture for LACP on both ends, to try to see if LACP was sent by Cisco and received by capture, but not by system. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
Hey James, You shared this off-list, I think it's sufficiently material to share. 2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel101 is down (No operational members) 2024 Feb 9 16:39:36 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel101: Ethernet1/44 is down Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACPD_TIMEOUT: et-0/1/5: lacp current while timer expired current Receive State: CURRENT Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACP_INTF_DOWN: ae49: Interface marked down due to lacp timeout on member et-0/1/5 We can't know the order of events here, due to no subsecond precision enabled on Cisco end. But if failure would start from interface down, it would take 3seconds for Juniper to realise LACP failure. However we can see that it happens in less than 1s, so we can determine the interface was not down first, the first problem was Juniper not receiving 3 consecutive LACP PDUs, 1s apart, prior to noticing any type of interface state related problems. Is this always the order of events? Does it always happen with Juniper noticing problems receiving LACP PDU first? On Sun, 11 Feb 2024 at 14:55, james list via juniper-nsp wrote: > > Hi > > 1) cable has been replaced with a brand new one, they said that to check an > MPO 100 Gbs cable is not that easy > > 3) no errors reported on both side > > 2) here the output of cisco and juniper > > NEXUS1# sh interface eth1/44 transceiver details > Ethernet1/44 > transceiver is present > type is QSFP-100G-SR4 > name is CISCO-INNOLIGHT > part number is TR-FC85S-NC3 > revision is 2C > serial number is INL27050TVT > nominal bitrate is 25500 MBit/sec > Link length supported for 50/125um OM3 fiber is 70 m > cisco id is 17 > cisco extended id number is 220 > cisco part number is 10-3142-03 > cisco product id is QSFP-100G-SR4-S > cisco version id is V03 > > Lane Number:1 Network Lane >SFP Detail Diagnostics Information (internal calibration) > > > Current Alarms Warnings > Measurement HighLow High Low > > > Temperature 30.51 C75.00 C -5.00 C 70.00 C0.00 C > Voltage3.28 V 3.63 V 2.97 V 3.46 V3.13 V > Current6.40 mA 12.45 mA 3.25 mA12.45 mA 3.25 > mA > Tx Power 0.98 dBm 5.39 dBm -12.44 dBm2.39 dBm -8.41 > dBm > Rx Power -1.60 dBm 5.39 dBm -14.31 dBm2.39 dBm-10.31 > dBm > Transmit Fault Count = 0 > > > Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning > > Lane Number:2 Network Lane >SFP Detail Diagnostics Information (internal calibration) > > > Current Alarms Warnings > Measurement HighLow High Low > > > Temperature 30.51 C75.00 C -5.00 C 70.00 C0.00 C > Voltage3.28 V 3.63 V 2.97 V 3.46 V3.13 V > Current6.40 mA 12.45 mA 3.25 mA12.45 mA 3.25 > mA > Tx Power 0.62 dBm 5.39 dBm -12.44 dBm2.39 dBm -8.41 > dBm > Rx Power -1.18 dBm 5.39 dBm -14.31 dBm2.39 dBm-10.31 > dBm > Transmit Fault Count = 0 > > > Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning > > Lane Number:3 Network Lane >SFP Detail Diagnostics Information (internal calibration) > > > Current Alarms Warnings > Measurement HighLow High Low > > > Temperature 30.51 C75.00 C -5.00 C 70.00 C0.00 C > Voltage3.28 V 3.63 V 2.97 V 3.46 V3.13 V > Current6.40 mA 12.45 mA 3.25 mA12.45 mA 3.25 > mA > Tx Power 0.87 dBm 5.39 dBm -12.44 dBm2.39 dBm -8.41 > dBm > Rx Power 0.01 dBm 5.39 dBm -14.31 dBm2.39 dBm-10.31 > dBm > Transmit Fault Count = 0 > > > Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning > > Lane Number:4 Network Lane >SFP Detail Diagnostics
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
I want to clarify, I meant this in the context of the original question.
That is, if you have a BGP specific problem, and no FCS errors, then
you can't have link problems.
But in this case, the problem is not BGP specific, in fact it has
nothing to do with BGP, since the problem begins on observing link
flap.
On Sun, 11 Feb 2024 at 14:14, Saku Ytti wrote:
>
> I don't think any of these matter. You'd see FCS failure on any
> link-related issue causing the BGP packet to drop.
>
> If you're not seeing FCS failures, you can ignore all link related
> problems in this case.
>
>
> On Sun, 11 Feb 2024 at 14:13, Havard Eidnes via juniper-nsp
> wrote:
> >
> > > DC technicians states cable are the same in both DCs and
> > > direct, no patch panel
> >
> > Things I would look at:
> >
> > * Has all the connectors been verified clean via microscope?
> >
> > * Optical levels relative to threshold values (may relate to the
> >first).
> >
> > * Any end seeing any input errors? (May relate to the above
> >two.) On the Juniper you can see some of this via PCS
> >("Physical Coding Sublayer") unexpected events independently
> >of whether you have payload traffic, not sure you can do the
> >same on the Nexus boxes.
> >
> > Regards,
> >
> > - Håvard
> > ___
> > juniper-nsp mailing list juniper-...@puck.nether.net
> > https://puck.nether.net/mailman/listinfo/juniper-nsp
>
>
>
> --
> ++ytti
--
++ytti
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
I don't think any of these matter. You'd see FCS failure on any
link-related issue causing the BGP packet to drop.
If you're not seeing FCS failures, you can ignore all link related
problems in this case.
On Sun, 11 Feb 2024 at 14:13, Havard Eidnes via juniper-nsp
wrote:
>
> > DC technicians states cable are the same in both DCs and
> > direct, no patch panel
>
> Things I would look at:
>
> * Has all the connectors been verified clean via microscope?
>
> * Optical levels relative to threshold values (may relate to the
>first).
>
> * Any end seeing any input errors? (May relate to the above
>two.) On the Juniper you can see some of this via PCS
>("Physical Coding Sublayer") unexpected events independently
>of whether you have payload traffic, not sure you can do the
>same on the Nexus boxes.
>
> Regards,
>
> - Håvard
> ___
> juniper-nsp mailing list juniper-...@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
--
++ytti
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
On Sun, 11 Feb 2024 at 13:51, james list via juniper-nsp wrote: > One think I've omit to say is that BGP is over a LACP with currently just > one interface 100 Gbs. > > I see that the issue is triggered on Cisco when eth interface seems to go > in Initializing state: Ok, so we can forget BGP entirely. And focus on why the LACP is going down. Is the LACP single port, eth1/44? When the LACP fails, does Juniper end emit any syslog? Does Juniper see the interface facing eth1/44 flapping? -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] [j-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
Open JTAC and CTAC cases. The amount of information provided is wildly insufficient. 'BGP flaps' what does that mean, is it always the same direction? If so, which direction thinks it's not seeing keepalives? Do you also observe loss in 'ping' between the links during the period? Purely stabbing in the dark, I'd say you always observe it in a single direction, because in that direction you are losing reliably every nTh keepalive, and statistically it takes 1-3 days to lose 3 in a row, with the probability you're seeing. Now why exactly is this, is one end not sending to wire or is one end not receiving from wire. Again stabbing in the dark, more likely that problem is in the punt path, rather than inject path, so I would focus my investigation on the party who is tearing down the session, due to lack of keepalive, on thesis this device has problem in punt path and is for some reason dropping at reliable probability BGP packets from the wire. On Sun, 11 Feb 2024 at 12:09, james list via juniper-nsp wrote: > > Dear experts > we have a couple of BGP peers over a 100 Gbs interconnection between > Juniper (MX10003) and Cisco (Nexus N9K-C9364C) in two different datacenters > like this: > > DC1 > MX1 -- bgp -- NEXUS1 > MX2 -- bgp -- NEXUS2 > > DC2 > MX3 -- bgp -- NEXUS3 > MX4 -- bgp -- NEXUS4 > > The issue we see is that sporadically (ie every 1 to 3 days) we notice BGP > flaps only in DC1 on both interconnections (not at the same time), there is > still no traffic since once noticed the flaps we have blocked deploy on > production. > > We've already changed SPF (we moved the ones from DC2 to DC1 and viceversa) > and cables on both the interconnetion at DC1 without any solution. > > SFP we use in both DCs: > > Juniper - QSFP-100G-SR4-T2 > Cisco - QSFP-100G-SR4 > > over MPO cable OM4. > > Distance is DC1 70 mt and DC2 80 mt, hence is less where we see the issue. > > Any idea or suggestion what to check or to do ? > > Thanks in advance > Cheers > James > ___ > juniper-nsp mailing list juniper-...@puck.nether.net > https://puck.nether.net/mailman/listinfo/juniper-nsp -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9000 QoS counters on LAG
On Jun, 21 Jan 2024 at 03:17, Ross Halliday wrote: > Moving to qos-group for egress classes got me the result I was looking for. > Thank you very much! I'm happy that you got the results you wanted, but that shouldn't have fixed it. The 'priority level 3' is the only thing that I can think of which might cause it to fail to program. Just to continue on the priority level, if you set it on ingress, it'll still carry the values on egress. But you receive a lot more protection, because you ensure that fabric isn't being congested by less important traffic, causing more important traffic to drop during fabric congestion. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9000 QoS counters on LAG
On Fri, 19 Jan 2024 at 05:10, Ross Halliday via cisco-nsp wrote: > We've inherited some older ASR9000 systems that we're trying to support > in-place. The software version on this one router is fairly old at 6.1.4. > Driving it are a pair of RSP440-SE. The line cards are A9K-MOD160-SE with > A9K-MPA-8X10GE in each. > > I haven't had any issues until trying to apply a policy map in the egress > direction on a LAG. The counters simply don't increase. I'm aware of the > complexities of policing, but right now I just want to see packets match a > class - any class - even class-default doesn't increment as expected. > Everything works as expected on a non-LAG port. Ingress works fine, as well - > this is just egress on a LAG. > > IOS-XR is not my strong point at all. I'm not sure if I'm missing something > very obvious, but this seems so weird that it feels like a display bug. > > The LAG members are split between the two linecards. > > Any suggestions would be greatly appreciated! Any syslog messages when you attach it? I don't think the device supports 'priority level 3', there is only default, 2 and 1 . Default being the worst and 1 the best (well in CLI, in NPU they are reversed to make debugging less boring). Practically all the utility of priority level has already been used, by the time egress policy is considered, so they don't much here, you should set them on ingress. Not related, but I can't help myself, you shouldn't classify and schedule on egress, you classify on ingress, and schedule/rewrite on egress. That is 'your match dscp/cos' should be on ingress, with 'set qos-group N', and on 'egress' you do 'match qos-group N'. Not only will this separation of concerns make things a lot easier to rationale and manage, but it's the only way you can do QoS on many other platforms, so you don't have re-invent policies for different platforms. Do remember that by default 'police X' in LAG in ASR9000 means X in each interface, for total LAG capacity of X*interfaces_up (variant). There is no way to configure shared policer in any platform at all, but there is a way to tell platform to divide X by active member count for each member, so aggregate cannot be more than X, but no single interface can burst more than X/member_count. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9902 fpd upgrade
On Thu, 21 Dec 2023 at 09:21, Hank Nussbacher via cisco-nsp wrote: > It used to be TAC was a main selling card of Cisco vs competitors. Not > any longer :-( Don't remember them ever being relatively or absolutely good. Having one support channel for all requests doesn't work, because >99% cases are useless cases from people who didn't do their homework and are just wasting resources, so everyone optimises the process around dealing with those, which makes the experience feel horrible to the legitimate support cases. But vendors sell at premium cost different experience, Cisco calls it HTTS, it's not necessarily that you get better people, but you get named people who will quickly learn that the previous optimisation point doesn't work with you, because your cases are legitimate, so they don't perform the useless volleys in hope that the customer realises their error and doesn't come back, which is good strategy as it works often. Unfortunately HTTS is quite expensive and it feels backwards, that the people who are reporting legitimate problems in your product also have to pay more to get support. It often feels like no one buys Ciscos or Junipers, but leases them, as the support contracts are so outrageous, and you can't go without the support contracts. I'm not blaming Cisco or any other vendor, I think this outcome is a market driven fact. If I try to imagine that someone releases a new NOS, which works as well as Windows or Linux, in that the OS almost never is the reason why basic functionality of your product is broken, then I can imagine lot of my customers would choose not to buy this lease-level support contract, and I'd be out of business. Market requires NOS' to be of really poor quality. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR 1000 series replacement
On Sat, 16 Dec 2023 at 18:38, Charles Sprickman via cisco-nsp wrote: > > There are hundreds of GRE tunnels. > > I have nothing to offer, and I'm mostly out of the ISP game, but I am so > curious what the use-case is here, especially the "BGP to each CPE". I > understand this might be private info, but I'm just so very curious. The BGP > part is where I'm stumped... Any reason why you'd need hub+spoke topology, so many use cases. I've use it for two things. Mobile backup termination OOB termination In both cases with BGP, because I had 2 hubs, for redundancy. But BGP would be needed in the first case anyhow, as the customer delivers IPs. And helps in the 2nd case, even without redundancy to simplify configuration and keep hub configuration free (no touching on hub, when adding or removing spokes, due to BGP listen/allow). I mean this is what got rebranded under SDN but has existed before, it's just pragmatic and specific. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Netflow vs SNMP
On Mon, 2 Oct 2023 at 13:22, Dobbins, Roland via cisco-nsp wrote: > cache timeout inactive 15 > Kentik recommends 15s: > > This is an old, out-of-date recommendation from Cisco should be retired. > > 5s is plenty of time for inactive flows. What is the basis for this recommendation? With 1:10k or 1:1k, either way you'll have 1 packet per cache item. So 15, 5, 1, 0 would allow an equal amount of cache row re-use, which is none. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Netflow vs SNMP
On Mon, 2 Oct 2023 at 09:14, Hank Nussbacher via cisco-nsp wrote: > Does this make sense to go 1:1 which will only increase the number of > Netflow record to export? Everyone that does 1:1000 or 1:1 > sampling, do you also seen a discrepancy between Netflow stats vs SNMP > stats? Both 1:1000 and 1:1 make netflow expensive sflow. You will see almost all records exported are exactly 1 packet of data. You are spending a lot of resources storing that data and later exporting that data out, when you only ever punch the flow exactly once. This is because people have run the same configuration for decades, while traffic has exponentially grown, so the probability of hitting packets in the same flow twice has exponentially gone down. As the amount of traffic grows, sampling needs to become more and more aggressive to retain the same resolution. It is basically becoming massively more expensive over time, and likely cache based in-line netflow is dead in the water, and will become specialised in-line tap devices for the few who actually can justify the cost. Juniper has realised this, and PTX no longer uses cache at all, but exports immediately after sampling. IPFIX has newer sampling entities, which allow you to communicate that every N packet you sample C packets. This would allow you to ensure that once you fire sampling/export, you can sample enough packets to fill the MTU on export, to have an ideal balance of resource use and data density. Again entirely without cache, as cache does nothing unless you have very very aggressive sampling. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Port-channel not working Juniper vs Cisco
of "show interface" counters 3d20h > 0 interface resets > Load-Interval #1: 30 seconds > 30 seconds input rate 0 bits/sec, 0 packets/sec > 30 seconds output rate 0 bits/sec, 0 packets/sec > input rate 0 bps, 0 pps; output rate 0 bps, 0 pps > Load-Interval #2: 5 minute (300 seconds) > 300 seconds input rate 0 bits/sec, 0 packets/sec > 300 seconds output rate 0 bits/sec, 0 packets/sec > input rate 0 bps, 0 pps; output rate 0 bps, 0 pps > RX > 0 unicast packets 0 multicast packets 0 broadcast packets > 0 input packets 0 bytes > 0 jumbo packets 0 storm suppression bytes > 0 runts 0 giants 0 CRC 0 no buffer > 0 input error 0 short frame 0 overrun 0 underrun 0 ignored > 0 watchdog 0 bad etype drop 0 bad proto drop 0 if down drop > 0 input with dribble 0 input discard > 0 Rx pause > TX > 0 unicast packets 0 multicast packets 0 broadcast packets > 0 output packets 0 bytes > 0 jumbo packets > 0 output error 0 collision 0 deferred 0 late collision > 0 lost carrier 0 no carrier 0 babble 0 output discard > 0 Tx pause > > > Il giorno dom 11 giu 2023 alle ore 09:59 Saku Ytti ha scritto: >> >> You've changed JNPR from 30s to 1s, but not CSCO. I'm not sure if this >> is the only problem, as insufficient data is shown about the state and >> LACP PDUs. >> >> I believe the command is 'lacp rate fast' or 'lacp period short', to >> reduce risk of operators getting bored, In your case, the former. >> >> On Sun, 11 Jun 2023 at 10:38, james list via cisco-nsp >> wrote: >> > >> > Dear expert >> > we've an issue in setting up a port-channel between a Juniper EX4400 and a >> > Cisco Nexus N9K-C93180YC-EX over an SX 1 Gbs link. >> > >> > We've implemented the following configuration but on Juniper side it is >> > interface flapping while on Cisco side it remains down. >> > Light levels seem ok. >> > >> > Has anyone ever experienced the same ? Any suggestions ? >> > >> > Thanks in advance for any hint >> > Kind regards >> > James >> > >> > JUNIPER * >> > >> > > show configuration interfaces ae10 | display set >> > set interfaces ae10 description "to Cisco leaf" >> > set interfaces ae10 aggregated-ether-options lacp active >> > set interfaces ae10 aggregated-ether-options lacp periodic fast >> > set interfaces ae10 unit 0 family ethernet-switching interface-mode trunk >> > set interfaces ae10 unit 0 family ethernet-switching vlan members 301 >> > >> > > show configuration interfaces ge-0/2/3 | display set >> > set interfaces ge-0/2/3 description "to Cisco leaf" >> > set interfaces ge-0/2/3 ether-options 802.3ad ae10 >> > >> > > show vlans VLAN_301 >> > >> > Routing instanceVLAN name Tag Interfaces >> > default-switch VLAN_301 301 >> >ae10.0 >> > >> > >> > >> > >> > CISCO *** >> > >> > interface Ethernet1/41 >> > description <[To EX4400]> >> > switchport >> > switchport mode trunk >> > switchport trunk allowed vlan 301 >> > channel-group 41 mode active >> > no shutdown >> > >> > interface port-channel41 >> > description <[To EX4400]> >> > switchport >> > switchport mode trunk >> > switchport trunk allowed vlan 301 >> > >> > >> > # sh vlan id 301 >> > >> > VLAN Name StatusPorts >> > - >> > --- >> > 301 P2P_xxx activePo1, Po41, Eth1/1, Eth1/41 >> > >> > VLAN Type Vlan-mode >> > --- >> > 301 enet CE >> > >> > Remote SPAN VLAN >> > >> > Disabled >> > >> > Primary Secondary Type Ports >> > --- - --- >> > --- >> > ___ >> > cisco-nsp mailing list cisco-nsp@puck.nether.net >> > https://puck.nether.net/mailman/listinfo/cisco-nsp >> > archive at http://puck.nether.net/pipermail/cisco-nsp/ >> >> >> >> -- >> ++ytti -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Port-channel not working Juniper vs Cisco
You've changed JNPR from 30s to 1s, but not CSCO. I'm not sure if this is the only problem, as insufficient data is shown about the state and LACP PDUs. I believe the command is 'lacp rate fast' or 'lacp period short', to reduce risk of operators getting bored, In your case, the former. On Sun, 11 Jun 2023 at 10:38, james list via cisco-nsp wrote: > > Dear expert > we've an issue in setting up a port-channel between a Juniper EX4400 and a > Cisco Nexus N9K-C93180YC-EX over an SX 1 Gbs link. > > We've implemented the following configuration but on Juniper side it is > interface flapping while on Cisco side it remains down. > Light levels seem ok. > > Has anyone ever experienced the same ? Any suggestions ? > > Thanks in advance for any hint > Kind regards > James > > JUNIPER * > > > show configuration interfaces ae10 | display set > set interfaces ae10 description "to Cisco leaf" > set interfaces ae10 aggregated-ether-options lacp active > set interfaces ae10 aggregated-ether-options lacp periodic fast > set interfaces ae10 unit 0 family ethernet-switching interface-mode trunk > set interfaces ae10 unit 0 family ethernet-switching vlan members 301 > > > show configuration interfaces ge-0/2/3 | display set > set interfaces ge-0/2/3 description "to Cisco leaf" > set interfaces ge-0/2/3 ether-options 802.3ad ae10 > > > show vlans VLAN_301 > > Routing instanceVLAN name Tag Interfaces > default-switch VLAN_301 301 >ae10.0 > > > > > CISCO *** > > interface Ethernet1/41 > description <[To EX4400]> > switchport > switchport mode trunk > switchport trunk allowed vlan 301 > channel-group 41 mode active > no shutdown > > interface port-channel41 > description <[To EX4400]> > switchport > switchport mode trunk > switchport trunk allowed vlan 301 > > > # sh vlan id 301 > > VLAN Name StatusPorts > - > --- > 301 P2P_xxx activePo1, Po41, Eth1/1, Eth1/41 > > VLAN Type Vlan-mode > --- > 301 enet CE > > Remote SPAN VLAN > > Disabled > > Primary Secondary Type Ports > --- - --- > --- > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BGP Routes
On Sun, 12 Mar 2023 at 20:50, Mark Tinka via cisco-nsp wrote: > ASR9K1 has more routes with better paths toward destinations via its > upstream than ASR9K2 does. Or at worst, race. You might want add-path or best-external for predictability and improved convergence time. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NCS IOS-XR rant (was:Re: Internet border router recommendations and experiences)
On Wed, 1 Mar 2023 at 02:41, Phil Bedard via cisco-nsp wrote: > With XR7 the idea was to mimic how things are done with Linux repos by having > a specific RPM repo for the routers and the patches which is managed similar > to Linux and that’s how all software is packaged now. Dependencies are > resolved automatically, etc. RPMs are installed as atomic operations, there > is no more install apply, etc. Most customers do not want to manage an RPM > repo for their routers, so they just use whole images. I believe this is why people prefer Linux containers to legacy time-shared mutable boxes, the mutable package management is actually anti-pattern today. I wonder why I can upgrade my IRC client while keeping state, but I can't upgrade my BGP. There are two paths that consumers would accept a) immutable NOS, you give it image, it boots up and converges in <5min b) mutable NOS, process restarts keep state, if upgrade is hitful, forwarding stoppage should be measured in low seconds I think a) is far easier to achieve. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco IOS switch SSH connections not working
On Tue, 14 Feb 2023 at 02:21, Lee Starnes via cisco-nsp wrote: > So attempted to just remove the ACL and try. Still nothing. Lastly, I > enabled telnet and tried to connect via telnet. Still nothing. I really > don't want to restart the switch if there is any other way to resolve this. > > Anyone have any suggestions? I assume you have connectivity to the box by some means, based on the content of your email. If packets are getting delivered to the device port, then it seems like they fail to enter from HW to the control-plane, a somewhat common problem and this would require deep dive into how to debug each step in the punt path. But as some starting points, by no means not a complete view into the punt path. You could try ELAM capture to see what PFC thinks of the packet, is it going to punt it to software. You could try pinnacle capture to see what the punt looks like. - show plat cap buffer asic pinnacle port 4 direction out priority lo. ## sup => rp path - show plat cap buffer filter data ipv4 IP_SA= - show plat cap buffer data filt - show plat cap buffer data sample You could try to look at input buffers on input interface, to see if buffers are full, very often there are bugs where control-plane packets get wedged, eventually filling the buffer precluding new packets from entering software. - show buffer input X dump, if so You could review TCP/TCB for stuck sessions you might need to clear manually - clear tcp tcb might help You could review TTY/VTY session box thinks it is holding - clear line might help You might not be able to fix the problem without boot, but you can absolutely find incriminating evidence of the anatomy of the problem, assuming you supply enough time on the keyboard. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] How can one escalate within Cisco TAC?
On Wed, 8 Feb 2023 at 09:48, Hank Nussbacher via cisco-nsp wrote: > So how does one escalate such an issue within TAC? Is there some secret > email like escalati...@cisco.com or vp-...@cisco.com that one can contact? You call your account team, express your grief and set expectations. Then you have someone in your corner internally, which is far more effective than externally trying to fix it. It saddens me greatly, because it shouldn't work in a world full of responsible adults, but having weekly case review calls works very well, because then the account team will be embarrassed to say 'ok this didn't move since last week', and they ensure things move even a little bit. It steals 30min-1h per week per vendor of your time, but pays dividends. Working would be much more pleasurable if half the world's white collar workers wouldn't be unemployed plat card holders and cruising without output, while looking down on people doing 3 jobs and not qualifying for a mortgage. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] MTU and PMTUD
On Thu, 8 Dec 2022 at 11:40, Marcin Kurek wrote: > > > https://www.rfc-editor.org/rfc/rfc8654.html > > Thanks! > > > But it was [8936, 1240].min - so it was 'negotiated' here to the > > smallest? If you change the 8936 end to 1239, then that will be used, > > regardless who starts it. > > Yes, but why would XR advertise 1240 if I'm using 'tcp mss 8936' for that > neighbor? > Initially I thought that the whole point of this command is to set MSS to a > fixed value of our choice. I may have misunderstood you. Ddi you have 8936 configured on both ends? I thought you had only 8936 on the CSR. How I understood it: *Dec 8 11:17:15.453: TCP0: Connection to 12.0.0.7:179, advertising MSS 8936 *Dec 8 11:17:15.456: TCP: tcb 7FFB9A6D64C0 connection to 12.0.0.7:179, peer MSS 1240, MSS is 1240 Local 12.0.0.13 CSR is advertising 8936 to remote 12.0.0.7 Remote 12.0.0.7 is advertising 1240 We settle to 1240 I guess you are saying the remote 12.0.0.7 is as well configured to 8936? Then I agree, I wouldn't expect that, and can't explain it. Almost sounds like a bug where the configuration command is only read when IOS-XR establishes the connection, but not when it receives it? -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] MTU and PMTUD
On Thu, 8 Dec 2022 at 11:25, Marcin Kurek wrote: > Interesting, but why would 'sh run' or 'write' raise an interrupt? > Isn't this a branch in code that handles the CLI? Maybe to access NVRAM? I don't know for sure, I didn't pursue it, I just know I could observe it. > I'm not sure if I'm reading it right - on the one hand, the interrupts are > disabled, but on the other hand, some CLI commands actually raise them? I don't know if IOS does polling or interrupt for NIC packets, but there are tons of reasons to raise interrupt, not just NIC. > Would you mind elaborating on why going above 4k would mean "newish features" > and what are they? https://www.rfc-editor.org/rfc/rfc8654.html > So here CSR1kv is initiating the connection to XR box advertising MSS 8936 > (as expected). > However, peer MSS is 1240, which is not quite expected, considering XR config: But it was [8936, 1240].min - so it was 'negotiated' here to the smallest? If you change the 8936 end to 1239, then that will be used, regardless who starts it. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] MTU and PMTUD
On Wed, 7 Dec 2022 at 22:20, Marcin Kurek wrote: > > I've seen Cisco presentations in the 90s and early 00s showing > > significant benefit from it. I have no idea how accurate it is > > today,nor why it would have made a difference in the past, like was > > the CPU interrupt rate constrained? > > I'm sorry, I didn't get that part about constrained CPU interrupt rate? > My simple way of looking into that is that if we bump up the MTU, we end > up with fewer packets on the wire, so less processing on both sides. To handle NIC received packets you can do two things a) CPU can get interrupt, and handle the interrupt b) Interrupts can be disabled, and CPU can poll to see if there are packets to process The mechanism a) is the norm and the mechanism b) is modernish. To improve PPS performance under heavy rate, at cost of increasing jitter and latency because it takes variable time to pick up packet. In software based routers, like VXR, if you had precise enough (thanks Creanord!) measurements of network performance, you could observe jitter during rancid (Thanks Heas!) collections, because 'show run' and 'write' raises interrupts, which stops packet forwarding. So less PPS, less interrupt, might be one contributing factor. I don't know what the overhead cost of processing packets is, but intuitively I don't expect much improvement with large MTU BGP packets. And at any rate, going above 4k would mean newish features you don't have. But I don't have high confidence in being right. > Testing using XR 7.5.2 and older IOS XE, resulting MSS depends on who is > passive/active. MSS is 'negotiated' to the smallest. Much like BGP timers are 'negotiated' to the smallest (so your customer controls your BGP timers, not you). Does this help to explain what you saw? -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] MTU and PMTUD
Hey Marcin, > XR without enabled PMTUD (default setting) means ~1240 bytes available > for TCP payload. > > That seems to be a bit small, did you perform any kind of performance > testing to see the difference between defaults and let's say 9000 for BGP? I am embarrassed to say, but I _just_ found out, like literally weeks ago, that Junos BGP TCP window is 16kB, I did also find hidden command (https://github.com/ytti/seeker-junos) to bump it up to 64kB. I am working with JNPR to have public documentation for the hidden command to improve supportability and optics. I am separately working on hopes of getting TCP window scaling. I know that we are limited by TCP window, as the BGP transfer speed is within 0.5% of theoretical max, and increasing 16kB to 64kB increases BGP transfer speed exactly 4 times, being still capped by TCP window. I think Cisco can go to 130k, but won't by default. Maybe after that issue is remedied I will review packet size. > I'm thinking about RRs in particular, higher MTU (9000 vs 1200) should > result in some performance benefit, at least from CPU point of view. I > haven't tested this though. I've seen Cisco presentations in the 90s and early 00s showing significant benefit from it. I have no idea how accurate it is today,nor why it would have made a difference in the past, like was the CPU interrupt rate constrained? > Agree. Thing is, PMTUD on XR is a global setting, so it does affect all > TCP based protocols. You can do 'tcp mss X' under neighbor stanza. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] MTU and PMTUD
> Assuming typical MPLS network running L2VPNs and L3VPNs, how do you > configure MTU values for core interfaces? Do you set interface (L2) MTU, > MPLS MTU and IP MTU separately? We set the 'physical' MTU (In IOS-XR+Junos L2 but no CRC is in this humber) as high as it went when the decision was made. Today you can do I think 10k in Cisco8k and 16k in Juniper. We do not seet MPLS or IP MTUs separately in Core. On the edge you should always set L3 MTU, because you want to have the ability to add subinterfaces with large MTU, so physical MTU must be large, as change will affect all subinterfaces. This way you can later add big MTU subint, without affecting other subints. > Do you enable PMTUD for TCP based control plane protocols like BGP or LDP? No, not at this time, our BGP transfer performance is limited by TCP window-size, so larger packets would not do anything for us. And LDP has a trivial amount of stable data so it doesn't matter. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Help with MAC control on redundant L2 links.
How would you imagine it works, in a world without any limitations? This seems like a day1 building ethernet LANs question, unless I'm missing something. Comcast learns the 1701 now from every site. Frame comes in with DMAC 1701 Where does Comcast send it? Should it balance between three sites? And re-receive then if it happened to send it to B or C? My imagination doesn't stretch how this could possibly work. And this is exactly why Radia invented STP, to automatically remove all redundancy until such time that it is needed. What I would do is run MPLS point-to-point A<->B, A<->C, B<->C (not use any comcast or zayo provided multipoint service). And then run EVPN on my edge ports. EVPN will allow you to even ECMP. If that is not an option, run standard MSTP with two topologies. One topology blocks on Zayo another on Comcast. Put half VLANs on Zayo-first half vlans on Comcast-first. Now because STP blocks redundant ports, you'll learn [BC] MAC from a single port guaranteed. You will lose RST convergence benefits because your 'core' port is facing >1 other 'core', basically you have a 'hub' between your core ports. So even this topology would be better if you don't buy multipoint service from a vendor, but point-to-points (w/o mac learning). I would very strongly encourage not to go the STP route, and look towards EVPN. On Sat, 12 Nov 2022 at 23:54, Howard Leadmon via cisco-nsp wrote: > > > I have an issue I am trying to sort out that I haven't run into > before, and figured someone might be able to give me a few pointers. > > I have three sites lets say A, B and C, and they all have redundant > connectivity via two Layer 2 fiber loops with two different carriers. > > We use Comcast and Zayo to reach the various sites, but realized that > I was having connectivity issues, but after talking with Comcast, they > are informing me the issue is the MAC being presented from different > locations at the same time. > > So say at Site-A I am presenting a mac ending in 1701, I of course > present this to both Comcast and Zayo, as expected. Now at Site-B, I > am being informed that when my switch receives that 1701 down the loop > from Zayo, it is of course presenting it back to Comcast as a valid > MAC. As such they say they are learning this MAC from multiple > locations at the same time, and they can only support learning it from > one point, so they drop the MAC. Of course Site-C has the same issue, > also presenting what it knows from the other points. > > I thought setting 'spanning-tree link-type shared' allowed it to handle > this, but I am guessing not well enough. Well it might let the Cisco > handle it, but apparently is playing havoc with the Ciena switches that > Comcast is using. > > I looked at setting a mac filter (maybe I am looking at this wrong) to > say if you saw this coming in, don't resend it back out to any other > place. The issue I saw, was it only allowed it to be an ingress filter, > which means I would discard the address completely which doesn't seem > good either. > > I am sure there is a right way to handle this, but honestly not > something I have encountered before. If anyone could give me any hints, > or point me to something that might help it would be appreciated.. > > > --- > Howard Leadmon - how...@leadmon.net > PBW Communications, LLC > http://www.pbwcomm.com > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] NTP network design considerations
On Fri, 14 Oct 2022 at 23:32, Gert Doering via cisco-nsp wrote: > For a true time geek, the time the rPIs provide is just not good > enough (fluctuates +/- 20 usec if the rPI has work to do and gets > warm) :-) Meinberg does not do HW timestamping, nor does NTP. Almost every NIC out there actually does support HW timestamping, but you'd need chrony to actually enable the support. When using Meinberg and clocking your host, ~all of the inaccuracy is due to SW timestamping, oscillator doesn't matter. Basically with NTP you are measuring the host context switch times in your jitter. This is because networks have become organically extremely low jitter (because storing packets is expensive). We see across our network single digit microsecond jitters (in my M1 computer I see loopback pings having >50us stddev in latency), and because the timing we use is SW timestamp, our one-way delay measurement precision is measuring the NTP host kernel/software context switch delays. The most expensive oscillator would do nothing for us, but HW timestamping and cheap 2eur OXCO would radically improve the quality of our one-way delay measurements. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] storm-control errdisable with no traffic or vlan
On Sat, 6 Aug 2022 at 05:27, Paul via cisco-nsp wrote: > Storm control pps is bugged on the 10g ports on the older 4900 > platforms, 4948E , 4900M, sup6 platforms. When you say 'bugged', what do you specifically mean? Do you mean the platform can do PPS accounting in HW and there is DDTS or do you mean the platform cannot do PPS accounting? It is not given at all that the platform does PPS accounting, for example much more modern JNPR paradise chip doesn't do it, while their next-generation triton does it. And I know most Cisco pipeline gear didn't do it either. My guess would be that PPS is not supported by the hardware and behaviour is undefined, and you would need to poke hardware to understand what was actually programmed when you asked it to PPS, i.e. it hasn't worked as desired in the 1GE either. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] storm-control errdisable with no traffic or vlan
Are you sure packet based storm-control is even supported in the platform? What does 'show storm-control' say? It is interesting that all packets are multicast packets in the Ten interfaces, but the packet count is so low, I don't think we can put a lot of weight into it. On Thu, 4 Aug 2022 at 13:52, Joe Maimon wrote: > > Thanks for responding. I was looking for a controller like command to > see maybe there were some malformed frames or something, but couldnt > find one on this platform. > > > > Saku Ytti wrote: > > On Thu, 4 Aug 2022 at 02:06, Joe Maimon via cisco-nsp > > wrote: > > > >> I have a vendor trying to turn up a 10gb link from their juniper mx to a > >> cisco 4900M, using typical X2 LR. > >> > >> The link was being upgraded from a functioning 1gb. Same traffic. > GigabitEthernet2/21 is up, line protocol is up (connected) >Hardware is Gigabit Ethernet Port, address is 588d..a8f8 (bia > 588d..a8f8) >Description: Cx 12.KFxxx >MTU 9198 bytes, BW 100 Kbit/sec, DLY 10 usec, > reliability 255/255, txload 1/255, rxload 1/255 >Encapsulation ARPA, loopback not set >Keepalive set (10 sec) >Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseLH >input flow-control is off, output flow-control is off >ARP type: ARPA, ARP Timeout 04:00:00 >Last input 00:00:00, output never, output hang never >Last clearing of "show interface" counters never >Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0 >Queueing strategy: fifo >Output queue: 0/40 (size/max) >30 second input rate 263000 bits/sec, 147 packets/sec >30 second output rate 663000 bits/sec, 201 packets/sec > 5900382712 packets input, 1369541098917 bytes, 0 no buffer > Received 100294521 broadcasts (100294050 multicasts) > 0 runts, 0 giants, 0 throttles > 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored > 0 input packets with dribble condition detected > 7240868168 packets output, 6306357851288 bytes, 0 underruns > 0 output errors, 0 collisions, 3 interface resets > 0 unknown protocol drops > 0 babbles, 0 late collision, 0 deferred > 0 lost carrier, 0 no carrier > 0 output buffer failures, 0 output buffers swapped out > > interface GigabitEthernet2/21 > description Cx 12.KFxxx > switchport trunk allowed vlan 1660-1679 > switchport trunk native vlan 1001 > switchport mode trunk > switchport nonegotiate > switchport port-security maximum 100 > switchport port-security aging time 3 > switchport port-security aging type inactivity > switchport port-security > mtu 9198 > load-interval 30 > no cdp enable > storm-control broadcast level pps 10k > storm-control action shutdown > spanning-tree portfast edge > > > > > >> Even with switchport mode trunk and switchport allowed vlan none, with > >> input counters in single digits, storm control immediately takes the > >> port down after link up. There was negligible traffic on the link before > >> or after the attempt. > > Broadcast, multicast or unicast storm-control? What rate? Have you > > tried increasing the rate? Can you provide the logs of storm-control > > taking the port down? > > Gobs of this > > Aug 2 22:18:22: %PM-4-ERR_DISABLE: storm-control error detected on > Te1/3, putting Te1/3 in err-disable state > Aug 2 22:18:22: %STORM_CONTROL-3-SHUTDOWN: A packet storm was detected > on Te1/3. The interface has been disabled. > Aug 2 22:19:07: %PM-4-ERR_RECOVER: Attempting to recover from > storm-control err-disable state on Te1/3 > Aug 2 22:19:10: %PM-4-ERR_DISABLE: storm-control error detected on > Te1/3, putting Te1/3 in err-disable state > > Tried another X2 > > Aug 2 22:31:33: %C4K_IOSINTF-5-TRANSCEIVERREMOVED: Slot=1 Port=3: > Transceiver has been removed > Aug 2 22:31:48: %C4K_IOSINTF-5-TRANSCEIVERINSERTED: Slot=1 Port=3: > Transceiver has been inserted > Aug 2 22:31:50: %SFF8472-5-THRESHOLD_VIOLATION: Te1/3: Tx power low > alarm; Operating value: -40.0 dBm, Threshold value: -12.2 dBm. > Aug 2 22:32:09: %SYS-5-CONFIG_I: Configured from console by joe on vty0 > (216.222.148.103) > Aug 2 22:32:14: %PM-4-ERR_RECOVER: Attempting to recover from > storm-control err-disable state on Te1/3 > Aug 2 22:32:17: %PM-4-ERR_DISABLE: storm-control error detected on > Te1/3, putting Te1/3 in err-disable state > Aug 2 22:32:17: %STORM_CONTROL-3-SHUTDOWN: A packet storm was detected > on Te1/3. The interface has been disabled. > > Tried another port > > Aug 2 22:5
Re: [c-nsp] storm-control errdisable with no traffic or vlan
On Thu, 4 Aug 2022 at 02:06, Joe Maimon via cisco-nsp wrote: > I have a vendor trying to turn up a 10gb link from their juniper mx to a > cisco 4900M, using typical X2 LR. > > The link was being upgraded from a functioning 1gb. Same traffic. 10G will serialise packets 10x faster. Even if the average packet rate is the same, you can have a 10x faster traffic rate on smaller sampling intervals, which may exceed configured protection rates. > Even with switchport mode trunk and switchport allowed vlan none, with > input counters in single digits, storm control immediately takes the > port down after link up. There was negligible traffic on the link before > or after the attempt. Broadcast, multicast or unicast storm-control? What rate? Have you tried increasing the rate? Can you provide the logs of storm-control taking the port down? -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
On Fri, 15 Jul 2022 at 03:09, Charles Sprickman wrote: > I’m doing much less work with Cisco these last few years, and you reminded me > I do have some folks with ASR-1000 series that are way, way, way overdue for > some work. I have literally no idea about how the current licensing scheme > works, nor the whole split/change to IOS. I think that’s all too basic for > this list, but if anyone here has some pointers to resources outside of > cisco’s own site that could get me up to speed a bit, I’d really appreciate > it. I would suggest to use this: https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/smart-licensing/qsg/b_Smart_Licensing_QuickStart/b_Smart_Licensing_QuickStart_chapter_01000.html With it you never need to phone home after initial install and you'll never expire your license. Technical license enforcement is an entirely non-workable idea, we've had HTTPS for almost 30 years and regularly serious, well resourced companies fail to re-up their licenses before they expire. In HTTPS we can probably justify the benefits of expiry outweigh the harm, but in licensing we cannot, and we must not accept technical enforcement from any vendor. Juniper is coming up with licensing but have strategically decided not to do technical enforcement. I am not against licensing wholesale, but I want it to be a commercial problem, not a technical one. I'm fine calling home and reporting non-compliance. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] v6 vrrp
Junos automatically assigns LL as 1st. IOS-XR can be made to do this auto-assign, and will use the same policy to generate it. SROS validates that the set of virtuals are identical, so having SROS in the network forces you to look a little bit deeper, if you want VRRP to actually work. It is easy to come up with a config which does not interoperate, and possible to find two implementations which won't, business as usually in IPv6, as no one uses it, edges are rough. On Sun, 10 Jul 2022 at 00:20, Doug McIntyre wrote: > > On Sat, Jul 09, 2022 at 01:44:28PM -0700, Randy Bush wrote: > > now to make the matching junos. for a junos facing an xr, i > > did not have to do this link local stuff. > > The standard states that the first address in VRRP v3 IPv6 needs to be > an IPv6 link-local address. > https://datatracker.ietf.org/doc/html/rfc5798 > > > In the IPv6 case (that is, IPvX is IPv6 everywhere in the figure), > > each router has a link-local IPv6 address on the LAN interface (Rtr1 > > is assigned IPv6 Link-Local A and Rtr2 is assigned IPv6 Link- > > Local B), and each host learns a default route from Router > > Advertisements through one of the routers (in this example, they all > > use Rtr1's IPv6 Link-Local A). > > Due to RA. > > Some vendors force or interpret the standard different than others. > > > > > > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Link down affecting BGP peer
On Thu, 19 May 2022 at 10:27, Hank Nussbacher wrote: > Others have explained this. Basically, a BGP peer gets locked onto one > of the LAG links and will migrate to another link in the event that the > specific link goes down. This is normal behavior. I'm not sure about normal behaviour and certainly objectively broken. Even though ultimately some physical interfaces serialise those BGP packets out, the fast external fallover should be tracking the aggregate interface, not some member. What should happen when a member comes down is that the hash=>interface table has one interface less, so the packet is now hashed out to some of the remaining interfaces. We can accept flaps if we don't know the physical interface is down, while it is down. Like if the carrier-delay down is higher than bgp keepalive or if the interface is blackholing for whatever reason. Other than that, no, it's broken. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] C8200/Spitfire/Pacific
Hey Chris, On Fri, 18 Mar 2022 at 11:03, Chris Welti wrote: > Can't report from production, but we have a 8201-32FH (Q200/Gibraltar) in the > lab > right now. Currently considering it as a successor for 400G deployments > where we had NCS55A1-24H for 100G before. > So far so good for our use case as a basic PE. (unicast/multicast v4/v6, > OSPFv2/v3, BGP, MPLS for L2VPN VPLS/EoMPLS only, access ACLs) > Our needed feature set is very limited, without QoS, VRFs, MPLS TE, SR or > SRv6, so can't comment on any of those features. > > Overall it seems it has more features and less limitations than the Jericho+ > in the NCS55A1-24H, e.g. v6 egress ACLs work, support for flowspec, uRPF > allow-default. > My hope is that due to Cisco not depending on Broadcom and their SDK in those > chips that there will be less limitations and quicker fixes than in their > Jericho products, but who knows. > Otherwise seems pretty similar to Jericho2 products, except its less power > hungry. Thank you, I appreciate this. Are you focusing on Q200 because it ships, or did you look at Q100 but decided against it? I also similarly view it as a direct J competitor, and of course a lot of the same people were involved designing both (J1 and Q100). -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
[c-nsp] C8200/Spitfire/Pacific
Yellow, The box has been out for quite some time now, but I've not heard much from the community. I don't even know anyone else but 1299 who operate it. I'd very much like to hear from anyone who is running the device in production about their experience with it, even if the experience is just 'i configured it, we run features xyz, seems to work'. Or if you specifically decided not to run it, why not? I know there is a Juniper commissioned test report comparing Pacific to Triton, but obviously we know that the commissioning party will always win the test. Thank you! -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS-XR and Netflow filtering?
On Tue, 28 Dec 2021 at 11:36, Hank Nussbacher wrote: > Using just IOS-XR, is one able to filter out Netflow records (example) > based solely on IP address, so flows are not recorded if any record > starts with 192.168.*.* ? If not, is there an external box one can buy > that can do that? I don't think it is possible in IOS-XR. This is a very typical difference in IOS and JunOS, where IOS makes very laser focused features that do exactly one thing, JunOS does expressive features that can be used to implement the specific one thing, which leaves sometimes customers doing something emergent that even Juniper didn't think of, but the expressive architecture allows for. In this specific case, in Juniper you can do netflow via filter terms, so you could first permit all SIP with 192.168/16, then 2nd term permit+sample rest. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS XR RPL Matching on neighbor IP/ASN
On Mon, 22 Nov 2021 at 11:14, Gert Doering wrote: > Haven't tried, but that would be extremely annoying. > > The use case I have in mind is using large communities to control > per-peer-AS exports, as in: > > :0: --> "do not announce to $yourasn" > :1: --> "prepend to $yourasn" We need to start rejecting complex DSLs for routing policies. And start asking for correct solution a) policy api (e.g. gRPC call, where reply gives actions) - could be your program running on the router itself, not necessarily centralised server b) mruby or lua instead of vendor DSL for policy evaluation - ideally something >1 vendor will implement So that the built-in DSL is for simple/naive cases, and operators who need to implement complex policies across multiple vendors have much simpler time doing that. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Sat, 13 Nov 2021 at 21:23, Mark Tinka wrote: > And that is all fine, provided YOU, as the operator, are deciding policy. I don't think IETF is making standards for specific implementation. The implementation agnostic solution is to keep all routes which we rejected due to consulting validation database, regardless of validation state. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Sat, 13 Nov 2021 at 13:48, Mark Tinka wrote: > > So some friends and I are working on an RFC draft to fix this: > > https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr > > Comments and contributions are most welcome. I chose my words carefully when I said 'RPKI rejects', instead of 'invalid'. The problem only cursorily relates to a specific RPKI validation state. We may reject RPKI 'unknown', we may even imagine policies which reject based on some criteria AND RPKI 'valid' (maybe I have my own motivations for how I use VRP and want to capitalise on all three states arbitrarily, maybe I'm rejecting valids, because I'm collecting invalids to some separate RIB for research purposes). That is: soft-reconfiguration inbound never # don't keep anything soft-reconfiguration inbound rpki ## default, keep if policy rejected route while using validation database state (may have used something else, but as long as reject policy used validation state, regardless of state, we need to keep it). -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Thu, 11 Nov 2021 at 10:19, Mark Tinka wrote: > Thanks for the clue, Saku. Hopefully someone here has the energy to ask > Cisco to update their documentation, to make this a recommendation. I > can't be asked :-). I think it should just be a config error. You're not just cucking yourself, but your peers and customers. So it shouldn't be a choice you can make. We can also imagine improvements 1) by default keep all RPKI rejects, and have 'soft-inbound never' optionally to turn that off 2) have 1 bit per neighbor indicating policy had rpki rejects and 2 bits for validation database update iindicating database become less/more permissive IFF database became more permissive and neighbor has rpki rejects and we have soft-inbound never, then refresh -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Thu, 11 Nov 2021 at 10:08, Mark Tinka wrote: > And Junos defaults to always maintaining a copy of Adj-RIB-In, which is > why we don't have that issue there, non? Correct. Add 'keep none' to junos, and you'll have the same issue. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Thu, 11 Nov 2021 at 09:50, Mark Tinka wrote: > What's the thought-process here? When you receive an RTR update, you change your ingress policy, and you don't know what is the correct result without re-evaluating. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Thu, 11 Nov 2021 at 09:38, Mark Tinka wrote: > We are currently running 6.7.1, mainly to fix some LDPv6 issues. > > However, we started seeing this "too many BGP refresh messages" issue as > far back as 6.4.2. Did you run RPKI? Did you have softinbound disabled? This would cause a refresh on every RTR update. Basically a misconfiguration, if you run RPKI you must keep policy rejected routes. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] FIB scale on ASR9001
On Tue, 9 Nov 2021 at 19:13, Gert Doering wrote:
> Cisco is a textbook example how to drive away a truly loyal user base,
> and then blaim it on stock market analysts ("they said that any company
> without a recurring revenue software model will be dead soon").
Ranting and raving follows.
All (smart) executives claim the upside is because of their leadership
and downside is because of the market. While no data supports that
replacing executive A with executive B improved or reduced company
performance, that is, we don't know what qualities make companies fail
or succeed.
But I admire the beauty of something like this: https://exainfra.net/about-us/
'Under Xs’ leadership, GTT bought over 40 companies and grew annual
revenues from $65 million to over $1.6 billion during his tenure'
You have <150 words to highlight your most important achievements in
your career. And you choose to focus on the time when not only
shareholders but many classes of debt holders got completely wiped due
to over-extending.
In most other cases you just can't do that. Crane operator can't brag
about that one time when his mistake caused the building to collapse,
in fact he'll struggle to get hired by anyone aware of it. But
management has no metrics, so you are as competent and valuable as you
confidently say you are (which is why being tall helps being a
successful manager, as it's a metric we are able to compare easily and
being tall means to us being more competent).
Having said that, 5y performance:
SP500: 110%
CSCO: 90%
NOK: 20%
JNPR: 10%
PANW: 300%
ANET: 450%
So Cisco is losing to the wide market only very little, and is
outperforming other SP vendors (Huawei excluded). So the market
doesn't entirely agree with your assertion of user base attrition.
--
++ytti
___
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS-XR Vs. NTP in a duel to the death.
On Tue, 2 Nov 2021 at 09:00, Drew Weaver wrote: > RP/0/RSP0/CPU0:Oct 31 15:18:37.422 EDT: ntpd[263]: %IP-IP_NTP-5-HP_CONN_LOST > : High priority NTP peer connection lost - Stratum 2->3. > RP/0/RSP0/CPU0:Oct 31 15:34:14.370 EDT: ntpd[263]: > %IP-IP_NTP-5-HP_CONN_RECOVERED : High priority NTP peer connection recovered > - Stratum 3->2. > RP/0/RSP0/CPU0:Oct 31 16:09:25.511 EDT: ntpd[263]: %IP-IP_NTP-5-SYNC_LOSS : > Synchronization lost : 192.168.123.6 : System clock selection failed This may be CSCsz22456 or CSCvr17937. You can probably ignore it, but it might go away if you upgrade. I do not think this communicates any problem in your specific NTP setup. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Third party optics
On Thu, 21 Oct 2021 at 09:22, wrote: > Risk? Not really any risk there ... in 20+ years of using third party > optics/SFPs, we've never had an issue with any ... only situation, as > others have stated, could be when opening a ticket with TAC and the > optics aren't reported as Cisco ... When optic MSA is young (100G still kind of qualifies, 400G definitely) there are a lot of interop issues, as different parties interpret MSA ambiguities differently. Copper link-down detection is a notorious source of issues as well. So certainly the 1st party has a higher probability of not having interop problems. Over time the market converges to specific interpretation of MSA and the probability of parts working together becomes so high you can't justify testing. Having said that, the only way to actually know what you are buying is to buy from a 3rd party. Vendor changes the optic without changing SKU, so if you're doing any type of testing, it's largely belief work with low utility, as you won't know if the next order is the same parts or not. It is possible to source from 3rd party in a way where you know what parts are used and they can guarantee to ship those parts or notify of changes. So if you are willing to test and occasionally work with your 3rd party provider to solve interop issues, I think 3rd party is much more preferable. If you just want to call a single number and say 'make it go' and never test, 1st party is better. The range of ways to source from 3rd parties is large, there are brokers who ship whatever they can find, there are resellers who regularly change suppliers, there are resellers who always source from X if they have the part and then you can buy directly from the manufacturer. So there is much more room to do 3rd party sourcing wrong compared to 1st party sourcing. But if you are able to do it right, it tends to be the superior way to do it. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On Fri, 10 Sept 2021 at 14:53, Mark Tinka wrote: > With Cisco putting a lot more effort into IOS XR, I really wonder if the > ASR1000 and other platforms based on IOS XE will be around in the > medium-to-long term. Didn't they just release next-gen catalyst switches and isr cpes (rebranded as catalyst?) with IOS-XE? -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
Aww. For competitive analysis this is supported in MX Trio (actually my RLI) and PTX Triton (but not Paradise, hw limit, unsure if SW for Triton exists yet, haven't tested.). Definitely no reason why ASR1k could not support it if you have leverage towards vendor. On Thu, 9 Sept 2021 at 20:02, james list wrote: > > Hi > just tested and police rate x pps is only applicable to control plane > > Cheers > > Il giorno mer 8 set 2021 alle ore 15:51 Lukasz Bromirski > ha scritto: >> >> Saku is always on point ;) >> >> > On 8 Sep 2021, at 15:31, Saku Ytti wrote: >> > >> > On Wed, 8 Sept 2021 at 16:30, Lukasz Bromirski >> > wrote: >> > >> >>> 3) is there any mode to limit pps and not only bandwidth >> >> >> >> I no longer remember this from top of my mind, but there’s bunch of good >> >> QoS/HQoS presentations about ASR 1000 in particular on ciscolive.com that >> >> you can use as reference. >> > >> > police rate x pps >> >> Just checked this on 17.x based release (3k = 3000 for this example): >> >> rtr-edge(config-pmap-c)#police rate 3k ? >> account Overhead Accounting >> bps Treat 'rate' value in bits-per-second >> burst Specify 'burst' parameter >> conform-action action when rate is less than conform burst >> cps Treat 'rate' value in cells-per-second >> peak-rate Specify peak rate or PCR for single-level ATM 4.0 policer >> policies >> pps Treat 'rate' value in packets-per-second >> >> >> -- >> ./ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] policer on ASR1001X
On Wed, 8 Sept 2021 at 16:30, Lukasz Bromirski wrote: > > 3) is there any mode to limit pps and not only bandwidth > > I no longer remember this from top of my mind, but there’s bunch of good > QoS/HQoS presentations about ASR 1000 in particular on ciscolive.com that you > can use as reference. police rate x pps -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9K using XR 7
On Fri, 30 Jul 2021 at 20:29, Nick Hilliard wrote: > there are several reconfig / deconfig operations that can't be handled > on XR using a single commit, e.g. changing ISIS NET address, some > netflow stuff, etc. Deapplying and removing QoS policy in a single commit :). Opening tickets about these tactically feels frustrating when the vendor doesn't understand there is a strategic problem under the hood causing these. Some other vendors who simply cannot produce config without having models first don't have these commit time problems or they are orders of magnitude rarer events. Cisco keeps being confused about what does 'model driven' mean, the moment you talk about coverage of models and being model driven, you're confused what model driven means. Cisco doesn't even have real config infra, if QoS policy doesn't commit, that is QoS people problem, if tunnel config doesn't commit, that's tunnel team problem and so-forth. Instead of them consuming some internal config API, and having all commit problems be a config team problem. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Nexus Architecture question
On Thu, 3 Jun 2021 at 22:46, Drew Weaver wrote: > IP access list custom-copp-system-p-acl-bgp-allow > 3 permit tcp 192.168.1.2/32 gt 1023 any eq bgp > 4 permit tcp 192.168.1.2/32 eq bgp any gt 1023 > > IP access list custom-copp-system-p-acl-bgp-deny > 1 permit tcp any any eq bgp > 10 permit tcp any gt 1023 any eq bgp > 20 permit tcp any eq bgp any gt 1023 a) there is no reason to limit far-end ephemeral range (added cost, complexity and it might break some broken implementation causing work on your end, while you don't actually care if your customer uses broken implementation). b) there is good reason to limit near-end ephemeral range to actual ephemeral range, as there can be local ports listening at >1024 XR appears to use an ephemeral range of 15000-57343, unfortunately as far as i can see it is not documented therefore not guaranteed across upgrades :( -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Hardening LPTS
On Fri, 4 Jun 2021 at 17:19, Mark Smith wrote: > Thanks for comments. This is very valuable info. What are your thoughts about: > flow udp default rate 0 > flow tcp default rate 0 > > Are they safe to use? Cisco did not recommend them but I dont understand why. > And they failed to explain. Maybe because they dont understand themselves > either As LPTS is never going to be particularly safe for attackers but mostly will work for accidental congestion I would personally not change anything, until you have realised risk, at least then I will have some confidence that the change improves availability. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Hardening LPTS
Hey Mark, > Any comments on this? @Saku Ytti you probably have good opinions and inside > knowledge? Sorry, not really. LPTS is quite a blockbox and there is not much you can do to improve if you have actual control-plane issues after LPTS. Unlike in Junos where you can have multiple levels of policers (flow => ifl => ifd => npu => aggregate) ASR9k has just npu level LPTS policer, this means if the LPTS policers are being violated there is collateral damage shared by all ports on NPU. We've had several outages due to this, the typical reason is the customer has maybe an L2 loop, and gives us a large rate of say ND/ARP, and all ports experience L2 cache expiring and total outage. You can protect yourself from this scenario to a degree with 'lpts punt excessive-flow-trap' but it is very poorly documented and understood by everyone, including Cisco. Infact whole LPTS is extremely poorly understood by Cisco. We recently had an problem on injection path which caused PE-CE BGP to time out, Cisco spent good month trying to review our MQC config, even though we kept telling them LPTS packets are not subject to MQC in either punt or inject path, but they wouldn't have any of it. But because LPTS is not subject to MQC you can't put MQC policy to your customer QoS in, where you police arp/nd/bgp to avoid collateral damage, as this MQC policy won't be consulted for LPTS punt. As far as I am aware JNPR Trio is the only networking platform which actually is possible to protect against motivated attackers, but it is far too hard to configure correctly. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR1009x and 10G thruput
I'm assuming the egress port te0/1/7 is not simply full (including microbursts), because I guess we wouldn't be here if it was. Have you walked through QFP packet drop decks? Can you review if you may be software switching, instead of QFP. show platform hardware qfp active infrastructure punt statistics interface name te0/1/4 show platform hardware qfp active infrastructure punt statistics interface name te0/1/7 show platform hardware qfp active infrastructure punt statistics type per-cause I would expect to see 'transmitted punt packets' as 0, Unfortunately I only have ASR1001-X so it doesn't translate 1:1 to another QFP platform. But my first guess would be you're SW switching. There is a good cisco live deck on troubleshooting drops on QFP platforms, which might help your research. On Fri, 7 May 2021 at 11:15, wrote: > > When we collected the packet traces on interface TenGigabitEthernet0/1/7 > and noticed drops with Taildrops with Feature QoS as can be seen below. > The > problem is there is no QoS on the interface. > > teg#show platform packet-trace summary > > Pkt Input OutputState Reason > > 0 Te0/1/4 Te0/1/7 DROP 23 > (TailDrop) > > 1 Te0/1/4 Te0/1/7 DROP 23 > (TailDrop) > > 2 Te0/1/4 Te0/1/7 DROP 23 > (TailDrop) > > 3 Te0/1/4 Te0/1/7 DROP 23 > (TailDrop) > > What feature in IOS-XE can possibly cause Taildrops? > > Thanks, > Hank > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR1009x and 10G thruput
Hey, > We recently upgraded our network from ASR1004s to ASR1009xs. > > We are encountering thruput limits on 10G interfaces and were wondering > whether others have seen that as well. > We have a Cisco TAC case open now for 2 weeks but nothing has progressed > so I was wondering whether anyone here has a clue how this can be > happening. No experience on ASR1009-X specifically, but you might receive better quality help if you included a bit more information. These might help: show platform hardware throughput level show platform hardware qfp active datapath utilization summary show licence all -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Redistribute interface address as a /32 or /128 into BGP
On Sat, 27 Mar 2021 at 17:11, Maximilian Wilhelm wrote: > I'm wondering if the default classfulness is biting you here. Have you > tried > > network 10.0.0.2 mask 255.255.255.255 His problem is that the connected network is less specific and he wants to (potentially incorrect solution) advertise some addresses of connected network as more-specific. So the problem is getting that /32 to RIB in the first place, the problem is not how to advertise after he gets it to the RIB, which is what you're solving. And solution to the question (probably not right solution) is to static route /32 to the interface: int eth42 192.0.2.1/24 ip route 192.0.2.2/32 eth42 Now you can advertise 192.0.2.2/32. This trick is particularly useful so limit attack surface of your infrastructure, so instead of having every PE-CE/31 address reachable, you can make it so that only CE/32 address is reachable, limiting DFZ wide surface. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Redistribute interface address as a /32 or /128 into BGP
You can create a static route pointing to the interface and redistribute that static route. But you're doing it wrong. I'm not sure what is right without understanding more accurately what you are doing, but some flavor of a) have different logical interface for edge and !edge, with different ACL b) have VRF On Wed, 10 Mar 2021 at 13:02, Johannes Erwerle wrote: > > Hello. > > I have a network, e.g. 10.0.0.0/24 with several routers that have > interfaces (lets say they are 10.0.0.2 and .3) that are connected in the > same L2 (because they are running VRRP). The routers are connected to a > backbone network. There is also a customer in this network. Of course I > want to put an ACL for anti spoofing (and some other things) on the > customer facing interfaces of the routers. > > Now some of my monitoring and management traffic, which is addressed to > the customer facing interface addresses takes the shortest path into > 10.0.0.0/24 and through this network and might then hit the interface of > the router. But there is a ACL that blocks that, because it looks like > the customer spoofed the source address of the monitoring system. > > I basically see 2 "solutions": > 1. open up the ACL to allow monitoring/management traffic from inside > the network. Not nice, because that allows the customer to spoof some of > our addresses... > 2. announce the interface addresses (10.0.0.2 and 10.0.0.3 in this case) > as a /32 into the backbone so that they are more specifics and take the > right way through the backbone and not through the 10.0.0.0/24 network. > > My problem is that I can not convince my router to announce the > interface addresses. I tried to simply add a > > network 10.0.0.2 > > to the BGP config, but apparently the local routes that the router > creates for it's interfaces are not announced. > Also there is no "redistribute local" to tell the router to do that. > Adding a null-route with 10.0.0.2/32 does not work because the local > route exists in the routing table and the null route is therefor not > considered for redistribution. > > Does anyone know of any hacks I could do to achieve this? > > Of course the same problem exists for IPv6 with the appropriate subnet > masks. > > Greetings > Jo > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] IOS-XE Smart licensing
https://www.mail-archive.com/cisco-nsp@puck.nether.net/msg68161.html On Wed, 24 Feb 2021 at 13:26, Hank Nussbacher wrote: > > So we bought a bunch of ASR1009x along with IOS-XE and are encountering > the joy of Smart licensing. > > Once we have our license established, do we need to leave the > "call-home" section? > > To me it screams "security violation" and something I'd like to > permanently disable after getting the license activated. > > Or does Cisco like to have their routers constantly ping the mothership > in regards to the licensing? > > > Regards, > > Hank > > ___ > cisco-nsp mailing list cisco-nsp@puck.nether.net > https://puck.nether.net/mailman/listinfo/cisco-nsp > archive at http://puck.nether.net/pipermail/cisco-nsp/ -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] ASR9K to ASR920 MPLS issue
On Sat, 30 Jan 2021 at 02:51, Jerry Bacon wrote: > Finally was able to get this all working and tested. As we surmised, it > works properly either with or without the "rewrite", as long as it's > symmetrical. So I guess it comes down to a personal or network > preference. I can see a slight advantage to always doing it, as it > uncouples the VLAN encapsulation on the two sides. There is VLAN rewrite, always (except some really old linecards), so you do not need to have the same VLAN-id on both ends. You do want to normalise your network to 0 or 1 SVLAN, so that A end provisioning is independent of B end provisioning, greatly reducing complexity and configuration permutations. I personally like 1 SVLAN normalisation, so that we can carry 802.1p. This also means, even in port mode, I'll impose additional SVLAN on the port-mode end, and force the type to VLAN, so that far-side VLAN mode is unaware that it is interoperating with port mode. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On Mon, 21 Dec 2020 at 18:07, wrote: > Good point, also all the potential attribute filtering (in XR) would it be > applied prior to accepting the route into soft-reconfig version of the > table? IOS-XR is only post-policy. So whatever you reject does not contribute towards the limit, allowing DRAM exhaustion attack. SROS is only pre-policy. So if someone leaks bad prefixes you reject in policy, it's still going to be flap, potentially BGP reset attack. JunOS supports pre and post. Both are needed as they protect from different issues. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On Sat, 19 Dec 2020 at 13:45, Lukas Tribus wrote: > soft-reconfig inbound always amounts to 100 MB of memory consumption > for a v4 + v6 full feed as of last week on 32-bit XR. I can live with > 100MB of memory consumption per full feed, so I'm doing soft-reconfig > inbound always everywhere. This also helps with troubleshooting. It is also DRAM exhaustion attack vector. But of course routers are extremely fragile and anyone motivated can easily bring them down by plethora of methods. Even with max-prefix it may be, as max-prefix may mean before or after policy count, depending on platform and configuration toggle. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] RPKI extended-community RFC8097
On Fri, 18 Dec 2020 at 22:07, Jakob Heitz (jheitz) via cisco-nsp wrote: > Testing the RPKI validity in route-map causes BGP REFRESH messages. > Lots of them. I think the community largely got blindsided by this, I suspect marketability of the whole solution would have been a lot poorer if this argument was thrown around at standardisation. However, that ship has sailed, we can implement new cheaper methods, but the damage is done and it will be there long after we've retired. I know I got blindsided, and it was so obvious, but not a problem I was aware until a customer complained about excessive refresh. It would be funny to analyse how much more wattage is drawn because of this globally. how many early control-plane upgrades. Is it immaterial or material? I don't know. But it does seem to put some customers control-planes over the edge. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Unussual bandwidth limit question :) (Cisco ASR1002-X)
On Wed, 16 Dec 2020 at 17:57, Sheremet Roman wrote: > Thank you for your time, i just can't understand how i can apply > received prefixes to my current ACL's. With QPPB, you don't, with QPPB while processing the BGP NLRI, based on community or whatever information you have in RIB you assign QoS class. This is then given to the FIB and will be part of the lookup process, when DADDR is looked up, it will get rewrite information and QoS class information. So your BGP community could be 65000:fuckup, 65000:fuckup5mbps and so forth (of course some number representing fuckup). Then when you originate those prefixes, you need to attach the right community to them. But you don't touch the QoS config on the far end, that would be done automatically based on the community. If you must push new ACL on the device then this is more question of automation. Your options would be screenscraping or netconf. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Unussual bandwidth limit question :) (Cisco ASR1002-X)
On Thu, 17 Dec 2020 at 13:56, Sheremet Roman wrote: > So, i should read more about QoS? There i can limit speed to X mb/s > based on BGP community ? Yes, you should read up on QPPB if that fits your bill: https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_pi/configuration/xe-3s/iri-xe-3s-book/iri-qos-policy-prop-via-bgp.html -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Unussual bandwidth limit question :) (Cisco ASR1002-X)
Hey, > So, this is my current configuration for cap bandwidth, when i add > IP like "x.x.x.x" into access list cisco cap this IP. I don't agree with any of this as a good product or good technical implementation of the product, but putting that aside. > How i can manage ACL's remotely, i need dynamicly add/remove ips from > class-map match-all fuckup > description "ClassMap for BW limit (0 mbps)" > match community AS:NN You do it 'other way around', you set packet QoS behaviour in QPPB per BGP community, as_path or whatever. So if a customer needs 5Mbps class, or 0Mbps class or whatnot, you originate the prefix differently. https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_pi/configuration/xe-3s/iri-xe-3s-book/iri-qos-policy-prop-via-bgp.html -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] vs isis delay propagation of loopback interfec
Hey, > Can someone help me out here? I'm trying to find a way to delay the > propagation of a loopback interface in isis. > > The problem is the border node in sd-access, which uses the loopback > interface for Lisp, and as soon the fabric sees the interface it sends > traffic to the address. > > But at this time bgp might not be ready out of the fabric. I assume this means you have multiple options in iBGP and you are redirecting it too early. Perhaps: set-overload-bit on-startup wait-for-bgp Or perhaps have another loopback for services which is iBGP only carried. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LSR platforms
On Fri, 11 Dec 2020 at 16:09, wrote: > With "Spitfire" do you mean the "Silicon One Q100" by Israeli Leaba > Semiconductor (bought by cisco in 2016) founders include Eyal Dagan and Ofer > Iny, founders of Dune Networks, which got sold to Broadcom in 2009 please? Yes. > And with "paradise" I guess you mean the PTX NPU? Not sure what you mean by > "BT" though. Yes. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LSR platforms
On Fri, 11 Dec 2020 at 11:02, wrote: > I'm hesitant to recommend Broadcom based platform (NCS5k/Arista/ACX) for > platform certification testing in a worry that it will bite us in a long run, > though less so for a P role (as opposed to PE role). You will struggle to use data to state why Broadcom has fundamentally different risk profile than Spitfire and BT/paradise. Now I understand for Cisco, clearly their focus will be Spitfire going forward. But that argument can't be used to discriminate against Arista. Jericho and Spitfire were literally designed under the same leadership. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Whats happens when TCAM is full on 7600/RSP720RSP-3CXL?
Hey, > So in most cases it will look that way: > #show mls cef exception status > Current IPv4 FIB exception state = TRUE > Current IPv6 FIB exception state = FALSE > Current MPLS FIB exception state = FALSE > > And yes, the box will drop down to a few MBit of Traffic. Not only that, but there are three possible configurable actions for exception state, freeze (default), reset and recover. CTAC didn't know what recovery does. Freeze means no updates are going to HW, so understanding that it just affects prefixes not fitting HW is incorrect, if label gets reprogrammed in software, HW retains old information and you break your VPN security promise. The correct configuration has 'reset' manually configured and box will reload in loop until recovered. I.e. don't let it happen. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] XR6 process conflicts
On Tue, 15 Sep 2020 at 11:22, t...@pelican.org wrote: > I'd usually want to err on the side of having more data and putting > appropriate filtering between the data and the person viewing, rather than > NOT having data it later turns out would be useful. Yes tons of (bad) input isn't a problem. Where we make mistakes is generating a lot of inactionable or redundant output for human consumption. It is much better to omit sending alerts about real problems to humans than to generate a lot of inactionable alerts and messages for human consumption. We will quickly learn to ignore input if it's rarely actionable and mistakes due to humans ignoring legit alerts will be far more common than legit alerts not being generated. Of course oftentimes this is a game of where the blame falls, if you generate a lot of useless alerts but never miss alerts, you did your job and the problem is on the consumption side for not reacting. So rather fix situations where you discover them where you suppress legit alerts than spew out trash you don't know for a fact to be actionable. You will have better overall results but of course you will have to carry the blame of managers asking 'why didn't we get alert'. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] BGP - advertising default route to Branch offices
On Thu, 13 Aug 2020 at 03:25, Yham wrote: > Can anyone please tell which is considered a best practice when it comes to > the advertising default route? If any vendor documentation addresses this, > please feel free to share. In my opinion there is never a need to carry default route in dynamic routing protocols, it's always inferior solution to static. In this particular case, two options 1) BGP a) Advertise some candidate route from BGP (like your PA aggregate, which you originate from stable backbone boxes) b) Recursive static default route in branch routers pointing to the PA aggregate as next-hop 2) IGP a) have loopback42 in every LSR/P box with same IP address, which you add to IGP b) Recursive static default route in branch routers pointing to the loop42 address as next-hop In both cases isolated edge won't blackhole the the branch by advertising default it doesn't have as well as you'll always choose closest working exit. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Netflow/Sflow for "irrelevant" traffic?
On Thu, 30 Jul 2020 at 19:12, wrote: Hey, > If I'm not mistaken, sflow/netflow does not pick up null0 routed > flows. Plz correct me if I am wrong. I don't think there is a single answer to this question. It depends on a platform, where netflow/sflow is done and in what order are functions executed. There will be a lot of complex corner cases particularly with QoS, PBR and so forth. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Netflow/Sflow for "irrelevant" traffic?
On Thu, 30 Jul 2020 at 15:37, Drew Weaver wrote: > So if a flow is less than the sampling rate it does not export anything, I > believe is what you are saying. If none of the 500th packets belong to flow of your interest, you won't see anything of the flow. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Netflow/Sflow for "irrelevant" traffic?
On Thu, 30 Jul 2020 at 15:26, Drew Weaver wrote: > So just for a refresher if you are sampling lets say at 1:500 and lets say 1 > byte goes through an interface that is not intended to produce an export? > The exporting only happens if the amount of data is over a certain threshold? > Does that threshold vary? You'd pick up every nTh packet for sampling. sample(packet) if packet_count % 500 == 0 -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Rehosting a perpetual CSR1000V license
On Fri, 24 Jul 2020 at 16:52, Nick Hilliard wrote: > yep, that works fine for electricity because the cost of generating > electricity is a significant percentage of the amount that the end user > pays. I.e. the marginal cost is significant, so it's worth billing per > kWh. If this model had been a better way of charging for residential ip > data delivery, it would have been deployed a long time ago, but the > marginal cost per bit isn't worth it in the majority of cases because > the cost of mass billing is so high. I've had few upgrades since LS1010+VXR network and there is a statistically relevant correlation to more bandwidth demand related to the upgrade cycles. If you remove the heavy users, you can skip entire cycles, putting your CAPEX in 50%, 33%, 25% of less of what it is. Also your wave and transit costs decrease rapidly over time, as they become cheaper faster than your user traffic increase, if you cherry-pick the lowest 70% users. Consumption is a significant cost driver. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Rehosting a perpetual CSR1000V license
On Fri, 24 Jul 2020 at 16:24, Nick Hilliard wrote: > in this specific case, you're confusing the total cost of customer > ownership with cost of service delivery. The main individual components > of residential ip service access are fixed business costs and whether > people avail of customer support; bandwidth consumption usually only > has a marginal impact on overall service costs, to the point that > creating the accounting and billing systems to handle the difference > usually isn't worth it. Yes. Transmission cost would be fixed and cover the cost of delivering the first bit, consumption cost would be variant and cover the cost of adding capacity, this is the model for electricity in some markets and I think it's a great model. In some markets transmission you can buy only from one player, depending on location, but consumption you can buy from anyone. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Rehosting a perpetual CSR1000V license
On Thu, 23 Jul 2020 at 21:08, Nick Hilliard wrote: > The whole idea of having your routing stack poll a remote server with a > query which essentially asks "should I continue to operate?" with a > default answer of "No" seems like a unusually stupid way to provision a > network. Regardless of the timeout parameters. I think it's well done and I can see applications where it adds real value to customers. For us the OPEX of dealing with licenses is too much, we want a one-time fire and forget solution, which they offer. But if I'd install and decom hundreds of CPEs yearly, with varying level of features and I'd immediately transfer feature costs to customers, this is really attractive. You buy the boxes without licenses and you buy licenses separately, and you ship just-in-time the license you actually need, and you return it to your pool once you're done. If pools run dry, you get alerts and you procure more. I also think licenses are a good idea, but often horrible execution. Not having licenses means you're subsiding people who use features heavily. Not having licenses also means the vendor doesn't know where money is pouring in, should they invest in multicast, 6VPE, LISP NRE or something else? Licenses mean you don't subsidize other players, you pay for features you use, vendor will understand where to invest NRE for better return. Similarly as a metered Internet is a great idea, with almost universally horrible executions. I am a heavy user, who is being subsidized by low income moms and pops, doesn't feel fair. For my electricity I pay separately for transmission and consumption, which is a great and fair model. Transmission is fixed cost, use or not, consumption is not. Uncongested Internet would be market driven fact for metered, because in flat rate Internet dropping packets increases margins, in metered Internet it reduces. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Rehosting a perpetual CSR1000V license
On Thu, 23 Jul 2020 at 11:02, Mark Tinka wrote: > The other option we'd looked at was the SSM (Smart Software Manager) > on-prem, as I'm not keen on having routers make arbitrary calls to some > cloud over at Cisco. You could also use local HTTP proxy, which may be less OPEX to maintain or may already exist. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Rehosting a perpetual CSR1000V license
On Thu, 23 Jul 2020 at 08:50, Mark Tinka wrote: > Is this part of their new Smart Licensing strategy? > > We are still running IOS XE 3.17S on our CSR1000v RR's, and that still > uses the trusty Permanent AX license. You can still have SLR https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/smart-licensing/qsg/b_Smart_Licensing_QuickStart/b_Smart_Licensing_QuickStart_chapter_01000.html and get persistent smart license you install on your device and it'll never phone home. You might want to argue an air gapped router to get itt, or if you have commercial leverage apply that liberally. For HW routers the process is like this a) give your implied HW license to license server b) allocate that from license server back, as SLR file c) install SLR file, never worry about it again -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Thu, 16 Jul 2020 at 16:31, Tim Durack wrote: > Not trying to be smart or pedantic: modern routers are built out of lots of > "ASICs". I imagine the forwarding element design is the differentiator: I don't think there is other option here :) > 1. Fixed pipeline: EARL family > 2. Progammable pipeline: UADP family > 3. Run-to-completion: "Silicon One" family > > Not an exhaustive list, lots of other examples etc... I mean what is pipeline? Silicon One is pipeline. Ingress pipe is parser + npu (terminate) + npu (lookup), egress pipe is npu (rewrite). Nokia FP is pipeline, but like Silicon one, it's pipeline of identical NPUs, just lot more identical NPUs in pipeline compared to Silicon One. Trio OTOH hits only one core in LU, one given PPE handles everything for given packet. So not a pipeline. I like Trio approach more, as the more NPUs you have in the pipeline, the more difficult it looks to program it right. Because if your NPU1 is parser, and you have big buggy code and your parsing of IPv6 extension headers is pathologically slow, now you're HOLB the whole line, rest of the cores are doing nothing. In Trio they don't need to be so careful, as you can think of it as single fat core in stead of many slim cores in pipe, so you get to use whole cycle pool, and if not every packet is pathological, you get away with lot worse ucode design. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Thu, 16 Jul 2020 at 11:25, James Bensley wrote: > You're right, and I should have been clearer that the ES20+ cards used > the NP3C NPU. But I wouldn't say that ES20 cards / PFC3 cards clearly > are not an NPU. I think they are actually in the interesting middle > ground between what I would call an ASIC powered device and an NPU > powered device. ES20 is Toaster/PXF, which can be said to be NPU. But if PFC3[ABC] is NPU, then I'd say there are no-non NPU forwarding chips. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Mon, 13 Jul 2020 at 20:39, James Bensley wrote: > Back in the 7600s it was NPU based, and what we call NPUs today are > sometimes a collection of ASICs that form a "complex of ASICs". That > is what powered the 7600, the NP3C NPU. 7600s used a group of ASICs > working together to perform forwarding lookups, buffering, backplane > sending/receiving etc. NP3C was on ES20+ (not ES20). The ASR9k Trident was the same EZchip NP3C. But of course the vast majority of 7600 linecards were PFC3, clearly not a NPU. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Mon, 13 Jul 2020 at 12:42, wrote: > On J2 you can pretty much enable all the bells and whistles you can and > still get the same pps rate out of it as with no features enabled. I don't think this is strictly true, maybe it's true for several cases. But even without recirculation, I think you can take variable time through J2 and if you wanted, you could make it perform pathologically poor, it is flexible enough for that. You can do a lot of stuff on the lookup pipeline for the first 144B in J2. Then there is the programmable elements matrix for future functions, can I put every frame there with no cost? You can use system headers to skip devices inside the pipeline. It's hard for me to imagine with all this flexibility we'd still always guarantee constant time. J2 is much more flexible than what J1 was. But of course it's still very much a pipeline of different types of silicons, some more some less programmable. So I don't know. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Mon, 13 Jul 2020 at 00:54, Mark Tinka wrote: > The general messaging, over the years, has been that ASIC is quick but > not flexible, while NPU is flexible but can get bogged down by added > flexibility in time. The classical view is that packet through ASIC takes constant, invariant time, and packet through NPU takes variant time, depending on how many instructions the NPU needs to perform for this packet. But if that is a strict definition, then we don't really have ASICs outside really cheap switches, as there is some programmability in all new stuff being released. So I'm not sure what the correct definition is. Equally when does a software router become a hardware router? Why is XEON not NPU but Trio is? Are there some objective facts which differentiate CPU from NPU and NPU from ASIC? # NPU vs CPU? - NPU tends to have more cores than CPU - NPU has application specific instruction set - NPU has application specific memory interface # NPU vs ASIC? - ASIC does parsing and lookup in silicon, not by running a set of instruction given by a program - ASIC is constant time, NPU is variable time - ASIC has many type of silicons for different function, NPU has many identical siicons running different set of instruction depending on packet/config -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Sun, 12 Jul 2020 at 21:30, Mark Tinka wrote: > NPU or TCAM, both provide hardware-based forwarding. This is a bit orthogonal. You can have an NPU with TCAM or NPU with DRAM. You could also have ASIC with TCAM or ASIC with DRAM. But if there is a clear line when a piece of hardware becomes an NPU instead of ASIC, I don't know it. Trio and QFP are unambiguously NPUs, is J2 still ASIC? In addition to J2 supporting NPL it has some unassigned ALUs to add new features with 0 cost. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Sun, 12 Jul 2020 at 12:45, Radu-Adrian FEURDEAN wrote: > > then also say Juniper MX, PTX are not hardware, Cisco 8k is > > not hardware, Jericho2 isn't hardware, modern stuff tends to run off > > of DRAM, not TCAM. > > Most of them can definitely do more than dumb packet forwarding, but not so > much as to do LNS or CGN in the main NPU (whatever variety the NPU is). A1k > OTOH is THE platform to be used if you want to do LNS and CGN with Cisco. I don't know where you base this claim, and I believe it's wrong. I know Trio could do any of this, I think Cisc8k could, and I'm fairly certain PTX1k could not. Both Trio and Ciscd8k are run-to-completion, you run ucode on the NPU and you're only limited by time. Perhaps your instruction set isn't conducive towards CGN or LNS, meaning you need too many cycles to do the feature, but certainly it could be done. > As for TCAM vs *RAM, lack of TCAM signals a FIB scale significantly larger > than a full table at the moment the box was designed. > Back to the question, NCS540 is merchant ASIC, while A1K is custom > network-oriented processor. I'd expect FIB scale to be a few (~4-5) times > higher on A1K than on NCS540. J2 is merchant and does DRAM, so that doesn't seem to be a signal either. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Cisco N540-ACC-SYS ipv4 routes
On Sat, 11 Jul 2020 at 21:09, Radu-Adrian FEURDEAN wrote: > Hi, wasn't ASR1k a "software-based" series, with route lookups NOT done from > TCAM, fib limit being the RAM, and forwarding done by homegrown QFP? I would say no, it was not software based, by HW. First gen QSFP or Popey was 400MHz (1.2GHz if you count multithreading) 40 core, 307M transistors, 20MB SRAM, 90nm lithography. It was tensilica platform (like npower) but Cisco IP. I think 2nd gen upped core count to 64 and frequency to 1.5GHz, changed to 40nm lithography and transistor count to 1.8B. Unsure what came after that. But what is software based and what is hardware based? To me ASR1k is HW based, it's an NPU box in my mind. Not having TCAM does not exclude box from being hardware, if not having TCAM means it's not hardware, then also say Juniper MX, PTX are not hardware, Cisco 8k is not hardware, Jericho2 isn't hardware, modern stuff tends to run off of DRAM, not TCAM. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 14:26, Mark Tinka wrote: > > ASR9k also has low and high scale cards, we buy the low scale, even > > for edge. But even low scale is pretty high scale in this context. > > You're probably referring to the TR vs. SE line cards, yes? I do, correct. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 14:23, Benny Lyne Amorsen via cisco-nsp wrote: > Per-packet overhead is hefty. Is that a problem today? For us it is in DDoS scenario, we have customers whose product is to ingest DDoS and send clean out, so we need to deliver the bad traffic to them. With large per-packet overhead attacker gets huge leverage, as they inject pure IP, then we add overhead, and we need that overhead capacity everywhere to transport it. I'd say the fundamental metrics are a) tunnel must be LEM only on a small on-chip database b) tunnel header must be narrow c) tunnel header must be transistor cheap to parse (wattage, yield) d) all traffic in core must be tunneled -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 14:04, Mark Tinka wrote: > Even Cisco sort of went down this path with the CRS-3 when they - very > briefly - sold the so-called CRS LSP (Label Switch Processor) forwarding > engine: ASR9k also has low and high scale cards, we buy the low scale, even for edge. But even low scale is pretty high scale in this context. I think there would be market for on-chip only LSR/core cards. Ultimately if you design for that day1, it won't add lot of costs to you. Yes the BOM difference may not be that great, because ultimately silicon is cheap and costs are in NRE. But the on-chip only cards would have bit more ports and they'd have better availability due to less component failures. I would fight very hard for us buy such cards in core, if vendor would offer them. Like the trio-in-pci full open, I think this is just marketing failure. Vendors are wearing their DC glasses and don't see anything else. While the big-name DC are lost cause, AMZN employs more chip designers than JNPR, matter of time before JNPR loses amzn edge too to internal amzn product. Someone with bit bolder vision on what the market could be, may have real opportunity here. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 13:29, Robert Raszuk wrote: Hey, > What you are saying is technically true but not realistically important. > > Why - the answer is history of PTX. I think this is interesting anecdote, but not much more. > It was originally designed and architected on the very basis of hardware cost > and performance when you would only need to switch at rates MPLS. > > Well real world showed that you can't sell such box and IP switching has been > added to data plane there. IP switching was there day 1, just not at DFZ scale in Broadway. But indeed, they got DFZ scale at Paradise, using off-chip HMC. Which is bad in numerous ways, not just because it costs a lot (there is either 2 or 3 HMC chip for every PE chip, so like double the front-plate without HMC) but it also makes the device fragile. There are 6 PE chips per LC, so 3*6 18 HMC chips, any of these HMC chips gets any problem and it's a full linecard reload of 15-20min. Even though 2/3 of the HMC chips are just delay buffer, and could be reloaded locally without impacting anything else. The remaiing 1/3 is lookup tables, and it can be argued it's cheaper to reload whole linecard than figure out how to resynchornise the FIB. Anyhow this design adds your cost, removes your ports and increases your outages. > Bottom line - I doubt you will find any vendor (from OEM to big ones) which > can afford to differentiate price wise boxes which would do line rate MPLS > and any thing less then line rate for IP. And as such IP clearly brings a lot > of value for simplification of control plane and route aggregation and IMHO > is a good (well best) universal transport today for all types of services > from WAN via Campus to DCs (or even MSDCs). Maybe I'm naive, but I believe we can learn. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 11:35, Mark Tinka wrote: > > So instead of making it easy for software to generate MPLS packets. We > > are making it easy for hardware teo generate complex IP packets. > > Bizarre, but somewhat rational if you start from compute looking out > > to networks, instead of starting from networks. > > Which I totally appreciate and, fundamentally, have nothing against. > > My concern is when we, service providers, start to get affected because > equipment manufacturers need to follow the data centre money hard, often > at our expense. This is not only in the IP world, but also in the > Transport world, where service providers are having to buy DWDM gear > formatted for DCI. Yes, it does work, but it's not without its > eccentricities. > > Cycling, over the past decade, between TRILL, OTV, SPB, FabricPath, > VXLAN, NV-GRE, ACI... and perhaps even EVPN, there is probably a lesson > to be learned. Maybe this is fundamental and unavoidable, maybe some systematic bias in human thinking drives us towards simple software and complex hardware. Is there an alternative future, where we went with Itanium? Where we have simple hardware and an increasingly complex compiler and increasingly complex runtime making sure the program runs fast on that simple hardware? Instead we have two guys in tel aviv waking up in night terror every night over confusion why does x86 run any code at all, how come it works. And I'm at home 'hehe new intc make program go fast:)))' Now that we have comparatively simple compilers and often no runtime at all, the hardware has to optimise the shitty program for us, but as we don't get to see how the sausage is made, we think it's probably something that is well done, robust and correct. If we'd do this in software, we'd all have to suffer how fragile the compiler and runtime are and how unapproachable they are. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Fri, 19 Jun 2020 at 11:03, Mark Tinka wrote: > MPLS has been around far too long, and if you post web site content > still talking about it or show up at conferences still talking about it, > you fear that you can't sell more boxes and line cards on the back of > "just" broadening carriage pipes. > > So we need to invent a new toy around which we can wrap a story about > "adding value to your network" to "drive new business" and "reduce > operating costs", to entice money to leave wallets, when all that's > really still happening is the selling of more boxes and line cards, so > that we can continue to broaden carriage pipes. I need to give a little bit of credit to DC people. If your world is compute and you are looking out to networks. MPLS _is hard_, it's _harder_ to generate MPLS packets in Linux than arbitrarily complex IP stack. Now instead of fixing that on the OS stack, to have a great ecosystem on software to deal with MPLS, which is easy to fix, we are fixing that in silicon, which is hard and expensive to fix. So instead of making it easy for software to generate MPLS packets. We are making it easy for hardware teo generate complex IP packets. Bizarre, but somewhat rational if you start from compute looking out to networks, instead of starting from networks. > > There are very few things that have been designed well from scratch, and > stand the test of time regardless of how much wind is thrown at them. > MPLS is one of those things, IMHO. Nearly 20 years to the day since > inception, and I still can't find another packet forwarding technology > that remains as relevant and versatile as it is simple. > > Mark. > -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Thu, 18 Jun 2020 at 22:25, Benny Lyne Amorsen via cisco-nsp wrote: > > I don't understand the point of SRv6. What equipment can support IPv6 > > routing, but can't support MPLS label switching? > This probably does not change anything for SRv6, as that too will likely > be an extra cost license. It makes non-MPLS tunnelling solutions very > attractive though, since you can get away with a very "cost-effective" > core and only require smarts in the edge. This is simply not fundamentally true, it may be true due to market perversion. But give student homework to design label switching chip and IPv6 switching chip, and you'll use less silicon for the label switching chip. And of course you spend less overhead on the tunnel. We need to decide if we are discussing a specific market situation or fundamentals. Ideally we'd drive the market to what is fundamentally most efficient, so that we pay the least amount of the kit that we use. If we drive SRv6, we drive cost up, if we drive MPLS, we drive cost down. Even today in many cases you can take a cheap L2 chip, and make it an MPLS switch, due to them supporting VLAN swap! Which has no clue of IPV6 or IPV4. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Thu, 18 Jun 2020 at 13:28, Robert Raszuk wrote: > To your IGP point let me observe that OSPF runs over IP and ISIS does not. > That is first fundamental difference. There are customers using both all over > the world and therefore any suggestion to just use OSPFv3 is IMHO quite > unrealistic. Keep in mind that OSPF hierarchy is 2 (or 3 with super area) > while in IETF there is ongoing work to extend ISIS to 8 levels. There is a > lot of fundamental differences between those two (or three) IGPs and I am > sure many folks on the lists know them. Last there is a lot of enterprise > networks happily using IPv4 RFC1918 all over their global WAN and DCs > infrastructure and have no reason to deploy IPv6 there any time soon. I view the 802.3 and CLNS as liability, not an asset. People who actually route CLNS are a dying breed, think just DCN of a legacy optical. Many platforms have no facilities to protect ISIS, any connected attacker can kill the box. Nokia handles generated packets classification by assigning DSCP value to application then DSCP to forwarding-class, which precludes from configuring ISIS qos. Very few people understand how ISIS works before ISIS PDU is handed to them, world from 802.3 to that is largely huge pile of hacks, instead of complete CLNS stack implementation. There is no standard way to send large frames over 802.3, so there is non-standard way to encap ISIS for those links. Also due to lack of LSP roll-over, ISIS is subject to a horrible attack vector which is very difficult to troubleshoot and solve. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Thu, 18 Jun 2020 at 10:13, Mark Tinka wrote: > Which is great for you, me, and a ton of other folk that run IS-IS on > Juniper. What about folk that don't have Juniper, or run OSPF? > > I know, not your or my problem, but the Internet isn't just a few networks. Yes work left to be done. Ultimately the root problem is, no one cares about IPv6. But perhaps work with vendors in parallel to LDPv6 to get them to fix OSPFv3 and/or ISIS. > I'm not saying it should be an SR vs. LDP debate like it was > BGP-signaling vs. LDP-signaling for VPLS 12+ years ago. All I'm saying FWIW I am definitely saying that, and it should be IGP+BGP. I do accept and realise a lot of platforms only did and do Martini not Kompella, so reality isn't quite there. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
On Thu, 18 Jun 2020 at 01:17, Mark Tinka wrote: > IOS XR does not appear to support SR-OSPFv3. > IOS XE does not appear to support SR-ISISv6. > IOS XE does not appear to support SR-OSPFv3. > Junos does not appear to support SR-OSPFv3. The IGP mess we are in is horrible, but I can't blame SR for it. It's really unacceptable we spend NRE hours developing 3 identical IGP (OSPFv2, OSPFv3, ISIS). We all pay a 300-400% premium for a single IGP. In a sane world, we'd retire all of them except OSPFv3 and put all NRE focus on there or move some of the NRE dollars to some other problems we have, perhaps we would have room to support some different non-djikstra IGP. In a half sane world, IGP code, 90% of your code would be identical, then you'd have adapter/ospfv2 adapter/ospfv3 adapter/isis which translates internal struct to wire and wire to internal struct. So any features you code, come for free to all of them. But no one is doing this, it's 300% effort, and we all pay a premium for that. In a quarter sane world we'd have some CIC, common-igp-container RFC and then new features like SR would be specified as CIC-format, instead of OSPFv2, OSPFv3, ISIS and BGP. Then each OSPFv2, OSPFv3, ISIS and BGP would have CIC-to-x RFC. So people introducing new IGP features do not need to write 4 drafts, one is enough. I would include IPv4+IPv6 my-igp-of-choice SR in my RFP. Luckily ISIS is supported on platforms I care about for IPV4+IPV6, so I'm already there. > MPLS/VPN service signaling in IPv6-only networks also has gaps in SR. I don't understand this. > So for networks that run OSPF and don't run Juniper, they'd need to move to > IS-IS in order to have SR forward IPv6 traffic in an MPLS encapsulation. > Seems like a bit of an ask. Yes, code needs to be written, which is fine by > me, as it also does for LDPv6. And it's really just adding TLV, if it already does IPv4 all the infra should be in place, only thing missing is transporting the information. Adding TLV to IGP is a lot less work than LDPv6. > I'd be curious to understand what bugs you've suffered with LDP in the last > 10 or so years, that likely still have open tickets. 3 within a year. - PR1436119 - PR1428081 - PR1416032 I don't have IOS-XR LDP bugs within a year, but we had a bunch back when going from 4 to 5. And none of these are cosmetic, these are blackholing. I'm not saying LDP is bad, it's just, of course more code lines you exercise more bugs you see. But yes, LDP has a lot of bug surface compared to SR, but in _your network_ lot of that bug surface and complexity is amortised complexity. So status quo bias is strong to keep running LDP, it is simpler _NOW_ as a lot of the tax has been paid and moving to an objectively simpler solution carries risk, as its complexity is not amortised yet. > Yes, we all love less state, I won't argue that. But it's the same question > that is being asked less and less with each passing year - what scales better > in 2020, OSPF or IS-IS. That is becoming less relevant as control planes keep > getting faster and cheaper. I don't think it ever was relevant. > I'm not saying that if you are dealing with 100,000 T-LDP sessions you should > not consider SR, but if you're not, and SR still requires a bit more > development (never mind deployment experience), what's wrong with having > LDPv6? If it makes near-as-no-difference to your control plane in 2020 or > 2030 as to whether your 10,000-node network is running LDP or SR, why not > have the choice? I can't add anything to the upside of going from LDP to SR that I've not already said. You get more by spending less, it's win:win. Only reason to stay in LDP is status quo bias which makes short term sense. > Routers, in 2020, still ship with RIPv2. If anyone wants to use it (as I am > sure there are some that do), who are we to stand in their way, if it makes > sense for them? RIP might make sense in some deployments, because it's essentially stateless (routes age out, no real 'session') so if you have 100k VM per router that you need to support and you want dynamic routing, RIP might be the least resistance solution with the highest scale. Timing wheels should help it scale and maintain great number of timers. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] Devil's Advocate - Segment Routing, Why?
Hey, > Why do we really need SR? Be it SR-MPLS or SRv6 or SRv6+? I don't like this, SR-MPLS and SRv6 are just utterly different things to me, and no answer meaningfully applies to both. I would ask, why do we need LDP, why not use IGP to carry labels? Less state, protocols, SLOC, cost, bug surface And we get more features to boot, with LDP if you want LFA, you need to form tLDP to every Q-space node, on top of your normal LDP, because you don't know label view from anyone else but yourself. With SR by nature you know the label view for everyone, thus you have full LFA coverage for free, by-design. Also by-design IGP/LDP Sync. So no need to justify it by any magic new things, it's just a lot simpler than LDP, you don't need to need new things to justify SR-MPLS, you need to want to do existing things while reducing complexity and state. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: [c-nsp] LDPv6 Census Check
On Mon, 15 Jun 2020 at 12:46, wrote: > Yes it can indeed, and that's moving towards the centre between the extreme > cases that David laid out. > It's about how granular one wants to be in identifying an end-to-end path > between a pair of edge nodes. > I agree with you that MPLS is still better than IP, > and I tried to illustrate that even enumerating every possible paths using > deep label stack is not a problem (and even that can be alleviated using > hierarchy of LSPs). The entirety of my point is, if we were rational, we'd move towards increasingly efficient solutions. And technically everything we do in MPLS tunnels, we can do in IP tunnels and converse. Should we imagine a future where all features and functions are supported in both, it's clear we should want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to IP 40B overhead should drive the point home, and ultimately, that's the only difference, rest is implementation. And I'm saddened we've been marketed snake-oil like SRv6 with fake promises of inherent advantages or simplicity 'just IP'. We can do better than MPLS, absolutely. But IP is worse. -- ++ytti ___ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
