Re: [c-nsp] OSPF equal cost load balancing

[CiscoNSP List]

Hmm - Well this is just not wanting to play nicely at all


Ive added another 2 links (Now 4 total), all equal cost - Egress load (From 
ASR920->ME3600) went from Gi0/0/22 doing 950M/sec,  Gi0/0/23 doing 5-10Mb/sec, 
to Gi0/0/20 now taking all the load...


So, we have gi0/0/20,21,22,23 connected to the "corresponding" ports on the 
ME3600 (gi0/20,21,22,23)


Now gi0/0/20 is doing 970Mb/s a sec.Ive tried every combination of 
load-sharing in global conf, and they initially make a bit of a difference 
(i.e. the other ports will do 5-10Mb/sec each), but then revert back to 
Gi0/0/20 being maxed out.


 #show int gigabitEthernet 0/0/20 | inc 30 sec
  30 second input rate 9019 bits/sec, 14882 packets/sec
  30 second output rate 969898000 bits/sec, 144872 packets/sec
 #show int gigabitEthernet 0/0/21 | inc 30 sec
  30 second input rate 74069000 bits/sec, 13780 packets/sec
  30 second output rate 1778000 bits/sec, 312 packets/sec
 #show int gigabitEthernet 0/0/22 | inc 30 sec
  30 second input rate 9676 bits/sec, 15992 packets/sec
  30 second output rate 3067000 bits/sec, 444 packets/sec
 #show int gigabitEthernet 0/0/23 | inc 30 sec
  30 second input rate 103174000 bits/sec, 16690 packets/sec
  30 second output rate 395000 bits/sec, 101 packets/sec


Help?  




From: CBL 
Sent: Thursday, 31 August 2017 1:13 PM
To: CiscoNSP List
Cc: James Bensley; cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] OSPF equal cost load balancing

What if you were to setup four BDIs running OSPF/MPLS across these two physical 
interfaces. Two BDIs per physical interface. Would that make ECMP work 
correctly using an ASR920?

We're going to be in the same boat soon too.. ASR920's on both sides with OSPF 
across two physical paths and worried about load sharing. Most of our traffic 
is MPLS xconnects traversing these links (licensed backhauls).


On Wed, Aug 30, 2017 at 6:35 PM, CiscoNSP List 
> wrote:
AAh - Thank you James!  So the ASR920 will not ECMP over 2 links, it requires 
4...that would explain the difference between egress/ingress (and why the 920 
is not working particularly well!)


Yes, this is ECMP, not LAG - So changing the load sharing algorithm can only be 
done globally (As I tried to do it under the individual interfaces, and was 
only presented with per dst as an option)


(config-if)#ip load-sharing ?
  per-destination  Deterministic distribution


So, changing globally will potentially cause a service disruption? (May need to 
do this in maintenance window) - Do you suggest "include-ports" as a possible 
candidate?


#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original   Original algorithm
  tunnel Algorithm for use in tunnel only environments
  universal  Algorithm for use in most environments

And yes, we are running MPLS over these links (But not a LAG as mentioned) - So 
does your comment re MPLS hasting still apply to our setup, or only to a LAG?


Thanks again for your response - Extremely helpful!



From: cisco-nsp 
> 
on behalf of James Bensley >
Sent: Thursday, 31 August 2017 6:43 AM
To: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] OSPF equal cost load balancing

I think two layer ECMP links are being used here, both of which are in
the IGP. Are you running MPLS over these links too?

The ME3600 is able to ECMP over any number of links as far as I know
(up to the max, which is 8 or 16) however I think the ASR920 will only
ECMP over 4 or 8 links (so not 2 as in your case). This could be the
problem here.

Could you also try to change the CEF load balancing algorithm
(assuming this is ECMP and not LAG, this won't affect a LAG):

ASR920(config)#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original   Original algorithm
  tunnel Algorithm for use in tunnel only environments
  universal  Algorithm for use in most environments

If it is a LAG then on the ASR920 try to adjust these options:

ASR920(config)#port-channel load-balance-hash-algo ?
  dst-ip Destination IP
  dst-macDestination MAC
  src-dst-ip Source XOR Destination IP Addr
  src-dst-macSource XOR Destination MAC
  src-dst-mixed-ip-port  Source XOR Destination Port, IP addr
  src-ip Source IP
  src-macSource MAC

If you're running MPLS over the LAG the ASR920 can hash MPLS over the
LAG and the ASR920 should hash over 2 links just fine.

Cheers,
James.
___
cisco-nsp mailing list  
cisco-nsp@puck.nether.net

Re: [c-nsp] OSPF equal cost load balancing

[CBL]

What if you were to setup four BDIs running OSPF/MPLS across these two
physical interfaces. Two BDIs per physical interface. Would that make ECMP
work correctly using an ASR920?

We're going to be in the same boat soon too.. ASR920's on both sides with
OSPF across two physical paths and worried about load sharing. Most of our
traffic is MPLS xconnects traversing these links (licensed backhauls).


On Wed, Aug 30, 2017 at 6:35 PM, CiscoNSP List 
wrote:

> AAh - Thank you James!  So the ASR920 will not ECMP over 2 links, it
> requires 4...that would explain the difference between egress/ingress (and
> why the 920 is not working particularly well!)
>
>
> Yes, this is ECMP, not LAG - So changing the load sharing algorithm can
> only be done globally (As I tried to do it under the individual interfaces,
> and was only presented with per dst as an option)
>
>
> (config-if)#ip load-sharing ?
>   per-destination  Deterministic distribution
>
>
> So, changing globally will potentially cause a service disruption? (May
> need to do this in maintenance window) - Do you suggest "include-ports" as
> a possible candidate?
>
>
> #ip cef load-sharing algorithm ?
>   include-ports  Algorithm that includes layer 4 ports
>   original   Original algorithm
>   tunnel Algorithm for use in tunnel only environments
>   universal  Algorithm for use in most environments
>
> And yes, we are running MPLS over these links (But not a LAG as mentioned)
> - So does your comment re MPLS hasting still apply to our setup, or only to
> a LAG?
>
>
> Thanks again for your response - Extremely helpful!
>
>
> 
> From: cisco-nsp  on behalf of James
> Bensley 
> Sent: Thursday, 31 August 2017 6:43 AM
> To: cisco-nsp@puck.nether.net
> Subject: Re: [c-nsp] OSPF equal cost load balancing
>
> I think two layer ECMP links are being used here, both of which are in
> the IGP. Are you running MPLS over these links too?
>
> The ME3600 is able to ECMP over any number of links as far as I know
> (up to the max, which is 8 or 16) however I think the ASR920 will only
> ECMP over 4 or 8 links (so not 2 as in your case). This could be the
> problem here.
>
> Could you also try to change the CEF load balancing algorithm
> (assuming this is ECMP and not LAG, this won't affect a LAG):
>
> ASR920(config)#ip cef load-sharing algorithm ?
>   include-ports  Algorithm that includes layer 4 ports
>   original   Original algorithm
>   tunnel Algorithm for use in tunnel only environments
>   universal  Algorithm for use in most environments
>
> If it is a LAG then on the ASR920 try to adjust these options:
>
> ASR920(config)#port-channel load-balance-hash-algo ?
>   dst-ip Destination IP
>   dst-macDestination MAC
>   src-dst-ip Source XOR Destination IP Addr
>   src-dst-macSource XOR Destination MAC
>   src-dst-mixed-ip-port  Source XOR Destination Port, IP addr
>   src-ip Source IP
>   src-macSource MAC
>
> If you're running MPLS over the LAG the ASR920 can hash MPLS over the
> LAG and the ASR920 should hash over 2 links just fine.
>
> Cheers,
> James.
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> cisco-nsp Info Page - puck.nether.net nether.net/mailman/listinfo/cisco-nsp>
> puck.nether.net
> To see the collection of prior postings to the list, visit the cisco-nsp
> Archives. Using cisco-nsp: To post a message to all the list members, send
> ...
>
>
>
> archive at http://puck.nether.net/pipermail/cisco-nsp/
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] OSPF equal cost load balancing

[CiscoNSP List]

Hi Pshem - No, only L3VPN and "standard" Inet links


cheers



From: Pshem Kowalczyk 
Sent: Thursday, 31 August 2017 6:51 AM
To: CiscoNSP List; cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] OSPF equal cost load balancing

Are you running L2VPN traffic across those ECMP links?

kind regards
Pshem


On Wed, 30 Aug 2017 at 16:59 CiscoNSP List 
> wrote:
Hi Everyone,


Have an ASR920 connected to an ME3600 with 2 x 1Gb links with same ospf cost 
(It was a single 1Gb, but secondary 1Gb was added as utilization was getting 
close to 1Gb) - Was hoping for at least a partial balance of traffic across the 
2 links, but egress from ASR920 to the ME3600(Ingress to customers), we are 
seeing one of the 1Gb links basically maxing out, and the other doing virtually 
nothing (10-15Mb/sec)...other direction we are seeing pretty much 50:50 balance 
across the 2 x 1Gb linksI know per-dest algorithm is used, and know that 
there are a only few big bandwidth users on the ME3600, but I cant understand 
why basically "all" of the traffic is going down one link?


Is there anyway to "tweak" the load-sharing of the equal cost paths (I can only 
see per-dst as an option)


Is a L3 etherchannel going to be any "better" with load-balancing than the 
current ospf equal cost?  (we have voip running over these links, so want to 
avoid packet delivery order issues)


Is TE a potential solution in this case?


We cant go 10G unfortunately, as the ME3600's dont have 10G ports unlocked, and 
they are earmarked for retirement - so stuck with multiple 1G links for a 
short-term fix 


Appreciate any feedback/suggestions.


Thanks
___
cisco-nsp mailing list  
cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] OSPF equal cost load balancing

[CiscoNSP List]

AAh - Thank you James!  So the ASR920 will not ECMP over 2 links, it requires 
4...that would explain the difference between egress/ingress (and why the 920 
is not working particularly well!)


Yes, this is ECMP, not LAG - So changing the load sharing algorithm can only be 
done globally (As I tried to do it under the individual interfaces, and was 
only presented with per dst as an option)


(config-if)#ip load-sharing ?
  per-destination  Deterministic distribution


So, changing globally will potentially cause a service disruption? (May need to 
do this in maintenance window) - Do you suggest "include-ports" as a possible 
candidate?


#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original   Original algorithm
  tunnel Algorithm for use in tunnel only environments
  universal  Algorithm for use in most environments

And yes, we are running MPLS over these links (But not a LAG as mentioned) - So 
does your comment re MPLS hasting still apply to our setup, or only to a LAG?


Thanks again for your response - Extremely helpful!



From: cisco-nsp  on behalf of James Bensley 

Sent: Thursday, 31 August 2017 6:43 AM
To: cisco-nsp@puck.nether.net
Subject: Re: [c-nsp] OSPF equal cost load balancing

I think two layer ECMP links are being used here, both of which are in
the IGP. Are you running MPLS over these links too?

The ME3600 is able to ECMP over any number of links as far as I know
(up to the max, which is 8 or 16) however I think the ASR920 will only
ECMP over 4 or 8 links (so not 2 as in your case). This could be the
problem here.

Could you also try to change the CEF load balancing algorithm
(assuming this is ECMP and not LAG, this won't affect a LAG):

ASR920(config)#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original   Original algorithm
  tunnel Algorithm for use in tunnel only environments
  universal  Algorithm for use in most environments

If it is a LAG then on the ASR920 try to adjust these options:

ASR920(config)#port-channel load-balance-hash-algo ?
  dst-ip Destination IP
  dst-macDestination MAC
  src-dst-ip Source XOR Destination IP Addr
  src-dst-macSource XOR Destination MAC
  src-dst-mixed-ip-port  Source XOR Destination Port, IP addr
  src-ip Source IP
  src-macSource MAC

If you're running MPLS over the LAG the ASR920 can hash MPLS over the
LAG and the ASR920 should hash over 2 links just fine.

Cheers,
James.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
cisco-nsp Info Page - 
puck.nether.net
puck.nether.net
To see the collection of prior postings to the list, visit the cisco-nsp 
Archives. Using cisco-nsp: To post a message to all the list members, send ...



archive at http://puck.nether.net/pipermail/cisco-nsp/
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] OSPF equal cost load balancing

[CiscoNSP List]

Hi Aaron - Apologies, yes, we currently have 2 separate L3 ints of equal cost 
between the asr920 and me3600 Just relying on ospf to do the load balancing) - 
This is not working particularly well 


sho ip route x.x.x.x will only provide loop of next hop (As we use RRs), so Ive 
included sh ip cef exact route instead:


From ASR920 :

#sho ip cef exact-route XXX.XXX.66.210 YYY.YYY.229.193
XXX.XXX.66.210 -> YYY.YYY.229.193 => label [explicit-null|explicit-null]TAG adj 
out of GigabitEthernet0/0/22, addr XXX.XXX.67.152

#sho ip cef exact-route XXX.XXX.66.210 YYY.YYY.229.194
XXX.XXX.66.210 -> YYY.YYY.229.194 => label [explicit-null|explicit-null]TAG adj 
out of GigabitEthernet0/0/23, addr YYY.YYY.230.102

#sh ip arp XXX.XXX.67.152
Protocol  Address  Age (min)  Hardware Addr   Type   Interface
Internet  XXX.XXX.67.152 97   3462.882a.49d7  ARPA   
GigabitEthernet0/0/22

#sh ip arp YYY.YYY.230.102
Protocol  Address  Age (min)  Hardware Addr   Type   Interface
Internet  YYY.YYY.230.102   3   3462.882a.49d8  ARPA   
GigabitEthernet0/0/23



...and some more cef output also:


#sh ip cef YYY.YYY.229.193
YYY.YYY.229.192/30
  nexthop YYY.YYY.230.102 GigabitEthernet0/0/23 label 
[explicit-null|explicit-null]
repair: attached-nexthop XXX.XXX.67.152 GigabitEthernet0/0/22
  nexthop XXX.XXX.67.152 GigabitEthernet0/0/22 label 
[explicit-null|explicit-null]
repair: attached-nexthop YYY.YYY.230.102 GigabitEthernet0/0/23


#sh ip cef YYY.YYY.229.193 internal
YYY.YYY.229.192/30, epoch 2, flags [rnolbl, rlbls], RIB[B], refcnt 6, 
per-destination sharing
  sources: RIB
  feature space:
IPRM: 0x00018000
Broker: linked, distributed at 4th priority
  ifnums:
GigabitEthernet0/0/22(29): XXX.XXX.67.152
GigabitEthernet0/0/23(30): YYY.YYY.230.102
  path list 3C293988, 35 locks, per-destination, flags 0x26D [shble, hvsh, rif, 
rcrsv, hwcn, bgp]
path 3C292714, share 1/1, type recursive, for IPv4
  recursive via XXX.XXX.76.211[IPv4:Default], fib 3C9AE64C, 1 terminal fib, 
v4:Default:XXX.XXX.76.211/32
  path list 3D583FF0, 13 locks, per-destination, flags 0x49 [shble, rif, 
hwcn]
  path 3D4A221C, share 0/1, type attached nexthop, for IPv4, flags 
[has-rpr]
MPLS short path extensions: MOI flags = 0x21 label explicit-null
nexthop YYY.YYY.230.102 GigabitEthernet0/0/23 label 
[explicit-null|explicit-null], IP adj out of GigabitEthernet0/0/23, addr 
YYY.YYY.230.102 3C287540
  repair: attached-nexthop XXX.XXX.67.152 GigabitEthernet0/0/22 
(3D4A44A4)
  path 3D4A44A4, share 1/1, type attached nexthop, for IPv4, flags 
[has-rpr]
MPLS short path extensions: MOI flags = 0x21 label explicit-null
nexthop XXX.XXX.67.152 GigabitEthernet0/0/22 label 
[explicit-null|explicit-null], IP adj out of GigabitEthernet0/0/22, addr 
XXX.XXX.67.152 3CC74980
  repair: attached-nexthop YYY.YYY.230.102 GigabitEthernet0/0/23 
(3D4A221C)
  output chain:
loadinfo 3D43D410, per-session, 2 choices, flags 0103, 21 locks
  flags [Per-session, for-rx-IPv4, indirection]
  16 hash buckets
< 0 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


< 1 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


< 2 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


< 3 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


< 4 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


< 5 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


< 6 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


< 7 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


< 8 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


< 9 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


<10 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


<11 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


<12 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


<13 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51BA40)


<14 > label [explicit-null|explicit-null]
  FRR Primary (0x3D51B980)


<15 > 

Re: [c-nsp] OSPF equal cost load balancing

[Pshem Kowalczyk]

Are you running L2VPN traffic across those ECMP links?

kind regards
Pshem


On Wed, 30 Aug 2017 at 16:59 CiscoNSP List 
wrote:

> Hi Everyone,
>
>
> Have an ASR920 connected to an ME3600 with 2 x 1Gb links with same ospf
> cost (It was a single 1Gb, but secondary 1Gb was added as utilization was
> getting close to 1Gb) - Was hoping for at least a partial balance of
> traffic across the 2 links, but egress from ASR920 to the ME3600(Ingress to
> customers), we are seeing one of the 1Gb links basically maxing out, and
> the other doing virtually nothing (10-15Mb/sec)...other direction we are
> seeing pretty much 50:50 balance across the 2 x 1Gb linksI know
> per-dest algorithm is used, and know that there are a only few big
> bandwidth users on the ME3600, but I cant understand why basically "all" of
> the traffic is going down one link?
>
>
> Is there anyway to "tweak" the load-sharing of the equal cost paths (I can
> only see per-dst as an option)
>
>
> Is a L3 etherchannel going to be any "better" with load-balancing than the
> current ospf equal cost?  (we have voip running over these links, so want
> to avoid packet delivery order issues)
>
>
> Is TE a potential solution in this case?
>
>
> We cant go 10G unfortunately, as the ME3600's dont have 10G ports
> unlocked, and they are earmarked for retirement - so stuck with multiple 1G
> links for a short-term fix 
>
>
> Appreciate any feedback/suggestions.
>
>
> Thanks
> ___
> cisco-nsp mailing list  cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] OSPF equal cost load balancing

[James Bensley]

I think two layer ECMP links are being used here, both of which are in
the IGP. Are you running MPLS over these links too?

The ME3600 is able to ECMP over any number of links as far as I know
(up to the max, which is 8 or 16) however I think the ASR920 will only
ECMP over 4 or 8 links (so not 2 as in your case). This could be the
problem here.

Could you also try to change the CEF load balancing algorithm
(assuming this is ECMP and not LAG, this won't affect a LAG):

ASR920(config)#ip cef load-sharing algorithm ?
  include-ports  Algorithm that includes layer 4 ports
  original   Original algorithm
  tunnel Algorithm for use in tunnel only environments
  universal  Algorithm for use in most environments

If it is a LAG then on the ASR920 try to adjust these options:

ASR920(config)#port-channel load-balance-hash-algo ?
  dst-ip Destination IP
  dst-macDestination MAC
  src-dst-ip Source XOR Destination IP Addr
  src-dst-macSource XOR Destination MAC
  src-dst-mixed-ip-port  Source XOR Destination Port, IP addr
  src-ip Source IP
  src-macSource MAC

If you're running MPLS over the LAG the ASR920 can hash MPLS over the
LAG and the ASR920 should hash over 2 links just fine.

Cheers,
James.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] OSPF equal cost load balancing

[Aaron Gould]

Are you doing a 2-port etherchannel between the 920 and 3600 ?  Asking since 
you seem to be asking question about etherchannel load balancing and hashing

...or...

Are you doing 2 separate layer 3 subnets between the 920 and 3600 ?  asking 
since your subject heading implies so. (ospf equal cost LB)

...you might be confusing/mixing 2 different subjects and how-to's in the same 
explanation.

I think you mentioned the 920 is network side and 3600 is closer to customer... 
if so, please go to 920 and show a customer route on the 3600 that you wish you 
would load balance please... sanitize your output to protect the innocent...

Show ip route a.b.c.d

Show ip arp of next hop

If it goes via L2

Show mac-address-table address ..


-Aaron


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: [c-nsp] nfSen / nfDump

[Nick Hilliard]

Nick Cutting wrote:
> The main SFlow collection point(s) are 36 port 100g nexus 9236c, so I
> think it is based on different chipsets – ASE2

Right, I missed it was a different asic.  You should reach out to your
SE and ask her/him if there is any way of poking this in hardware, in
the same way that you can do it on broadcom chipsets.  It would be
extraordinary if the hardware didn't support this.

Nick


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/