Hello,
We just started using the ME3400's (ME-3400G-12CS-A, 12.2(58)SE2,
ME340x-METROIPACCESSK9-M ) in our network and are noticing a fairly high CPU
load compared what we were used to on the older 3550's.
The config is pretty much the same on the new M#3400's then what we had on the
3550's. The 3550's never really ran higher than 3%/0% CPU load.
So, I'm not sure why the ME3400's are running so much higher in CPU...
Here are some outputs:
switch # sho proc cpu sorted 5sec
PU utilization for five seconds: 30%/20%; one minute: 34%; five minutes: 32%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
85 186523 11652 16007 0.79% 0.15% 0.12% 0 HULC Tcam
Memory
12 402028 1335248 301 0.63% 0.55% 0.57% 0 ARP Input
96 91391 1707572 53 0.47% 0.13% 0.11% 0 hpm main
process
62 63815 2667 23927 0.47% 0.05% 0.00% 0 Per-minute
Jobs
200 59364 425347 139 0.31% 0.07% 0.05% 0 Spanning Tree
68 100263 6213238 16 0.31% 0.15% 0.10% 0 Fifo Error
Detec
192 139880 328694 425 0.31% 0.20% 0.16% 0 IP Input
133 92952 206477 450 0.31% 0.19% 0.15% 0 HL3U bkgrd
proce
316 61245 174270 351 0.15% 0.04% 0.05% 0 IP SNMP
113 6308 579307 10 0.15% 0.01% 0.00% 0 Hulc Storm
Contr
83 17874 3460721 5 0.15% 0.05% 0.01% 0 HLFM address
ret
142 22760 46634 488 0.15% 0.03% 0.00% 0 HRPC qos
request
198 110177 244449 450 0.15% 0.19% 0.16% 0 ADJ resolve
proc
318 103809 87120 1191 0.15% 0.10% 0.09% 0 SNMP ENGINE
14 0 1 0 0.00% 0.00% 0.00% 0 CEF MIB API
13 17713 121776 145 0.00% 0.03% 0.00% 0 ARP Background
5 82 106 773 0.00% 0.00% 0.00% 0 Pool Manager
[...]
It looks like we are punting packets like crazy...
switch# show platform port-asic stats drop
Port-asic Port Drop Statistics - Summary
========================================
RxQueue 0 Drop Stats: 0
RxQueue 1 Drop Stats: 0
RxQueue 2 Drop Stats: 0
RxQueue 3 Drop Stats: 0
Port 0 TxQueue Drop Stats: 0
Port 1 TxQueue Drop Stats: 0
Port 2 TxQueue Drop Stats: 0
Port 3 TxQueue Drop Stats: 0
Supervisor TxQueue Drop Statistics
Queue 0: 0
Queue 1: 0
Queue 2: 0
Queue 3: 0
Queue 4: 0
Queue 5: 0
Queue 6: 17931
Queue 7: 0
Queue 8: 82
Queue 9: 0
Queue 10: 0
Queue 11: 46734696
Queue 12: 0
Queue 13: 0
Queue 14: 0
Queue 15: 0
Port-asic Port Drop Statistics - Details
========================================
RxQueue Drop Statistics
Queue 0
Weight 0 Frames: 0
Weight 1 Frames: 0
Weight 2 Frames: 0
Queue 1
Weight 0 Frames: 0
Weight 1 Frames: 0
Weight 2 Frames: 0
Queue 2
Weight 0 Frames: 0
Weight 1 Frames: 0
Weight 2 Frames: 0
Queue 3
Weight 0 Frames: 0
Weight 1 Frames: 0
Weight 2 Frames: 0
Port 0 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Port 1 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Port 2 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Port 3 TxQueue Drop Statistics
Queue 0
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 1
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 2
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Queue 3
Weight 0 Frames 0
Weight 1 Frames 0
Weight 2 Frames 0
Supervisor TxQueue Drop Statistics
Queue 0: 0
Queue 1: 0
Queue 2: 0
Queue 3: 0
Queue 4: 0
Queue 5: 0
Queue 6: 17931
Queue 7: 0
Queue 8: 82
Queue 9: 0
Queue 10: 0
Queue 11: 46734702
Queue 12: 0
Queue 13: 0
Queue 14: 0
Queue 15: 0
switch# show controllers cpu-interface
ASIC Rxbiterr Rxunder Fwdctfix Txbuflos Rxbufloc Rxbufdrain
-------------------------------------------------------------------------
ASIC0 0 0 0 0 0 0
ASIC1 0 0 0 0 0 0
ASIC2 0 0 0 0 0 0
ASIC3 0 0 0 0 0 0
HOL Fix Counts
--------------
No Fixes: 0 Added: 4 In Use: 0 Both: 0
CPU Heartbeat Statistics
Tx Success Tx Fail 1st Thr 2nd Thr Unthr RetryCtMax
---------- ---------- ---------- ---------- ---------- ----------
2320747 0 0 0 0 1
Rx Delay
0 1 2 3 4
---------- ---------- ---------- ---------- ----------
2320747 0 0 0 0
AddlDelay AdvanceCnt
---------- ----------
0 0
Rx Retries by RetryCount
0 1 2 3 4 5 6
---------- ---------- ---------- ---------- ---------- ---------- ----------
2320747 0 0 0 0 0 0
7 8 9
---------- ---------- ----------
0 0 0
AddlRetry
----------
0
cpu-queue-frames retrieved dropped invalid hol-block stray
----------------- ---------- ---------- ---------- ---------- ----------
rpc 0 0 0 0 0
stp 234154 0 0 0 0
ipc 0 0 0 0 0
routing protocol 1871818 0 0 0 0
L2 protocol 208159 0 0 0 0
remote console 0 0 0 0 0
sw forwarding 14959846 0 0 0 0
host 305132 0 0 0 0
broadcast 1754197 0 0 0 0
cbt-to-spt 0 0 0 0 0
igmp snooping 369150 0 0 0 0
icmp 731133339 0 0 4 0
logging 0 0 0 0 0
rpf-fail 0 0 0 0 0
dstats 0 0 0 0 0
cpu heartbeat 2320747 0 0 0 0
cpu-queue static inuse static added
----------------- ------------ ------------
rpc 0 0
stp 0 0
ipc 0 0
routing protocol 0 0
L2 protocol 0 0
remote console 0 0
sw forwarding 0 0
host 0 0
broadcast 0 0
cbt-to-spt 0 0
igmp snooping 0 0
icmp 0 4
logging 0 0
rpf-fail 0 0
dstats 0 0
cpu heartbeat 0 0
Supervisor ASIC receive-queue parameters
----------------------------------------
queue 0 maxrecevsize 7E0 pakhead 503C76C paktail 5E93634
queue 1 maxrecevsize 7E0 pakhead 5133F00 paktail 5133A7C
queue 2 maxrecevsize 7E0 pakhead 50F79B4 paktail 5F1231C
queue 3 maxrecevsize 7E0 pakhead 525CA38 paktail 5261278
queue 4 maxrecevsize 7E0 pakhead 513E7A0 paktail 5F4CF30
queue 5 maxrecevsize 7E0 pakhead 522EF20 paktail 606BA9C
queue 6 maxrecevsize 7E0 pakhead 60857C8 paktail 6085344
queue 7 maxrecevsize 7E0 pakhead 5FF3E9C paktail 5211B7C
queue 8 maxrecevsize 7E0 pakhead 602838C paktail 602B538
queue 9 maxrecevsize 7E0 pakhead 5FC16C8 paktail 5FC16C8
queue A maxrecevsize 7E0 pakhead 5171904 paktail 5F824F4
queue B maxrecevsize 7E0 pakhead 526896C paktail 5268DF0
queue C maxrecevsize 7E0 pakhead 5178C54 paktail 5F97AC0
queue D maxrecevsize 7E0 pakhead 4E04E74 paktail 51B556C
queue E maxrecevsize 0 pakhead 0 paktail 0
queue F maxrecevsize 7E0 pakhead 516D9CC paktail 5F7E8B4
Supervisor ASIC exception status
--------------------------------
Receive overrun 00000000 Transmit overrun 00000000
FrameSignatureErr 00000000 MicInitialize 00000000
BadFrameErr 00000000 LenExceededErr 00000000
BadJumboSegments 00000000
Supervisor ASIC Mic Registers
------------------------------
MicDirectPollInfo 80000200
MicIndicationsReceived 00000000
MicInterruptsReceived 00000000
MicPcsInfo 0000001F
MicPlbMasterConfiguration 00000000
MicRxFifosAvailable 00000000
MicRxFifosReady 0000B7FF
MicTimeOutPeriod: FrameTOPeriod: 00000EA6 DirectTOPeriod: 00004000
MicTransmFramesCopied 00000003
MicTxFifosAvailable 0000000F
MicConfiguration: Conf flag: 00000110 Interrupt Flag: 0000000A
MicReceiveFifoAssignmen Queue 0 - 7: 00000000 Queue 8 - 15:00000000
MicReceiveFramesReady: FrameAvailable: 00000880 frameAvaiMask: 00000000
MicException:
Exception_flag 00000000
Message-1 00000000
Message-2 00000000
Message-3 00000000
MicIntRxFifo:
ReadPtr 000000D8 WritePtr 000000D8
WHeadPtr 000000D8 TxFifoDepth C0000800
MicIntTxFifo:
ReadPtr 000008C8 WritePtr 000008C8
WHeadPtr 000008C8 TxFifoDepth C0000800
MicDecodeInfo:
Fifo0: address: 03FF4000 asic_num: 00000100
Fifo1: address: 03FF4400 asic_num: 00000101
MicTransmitFifoInfo:
Fifo0: StartPtrs: 066CE800 ReadPtr: 066CEDE8
WritePtrs: 066CEDE8 Fifo_Flag: 8A800800
Weights: 001E001E
Fifo1: StartPtrs: 0642D000 ReadPtr: 0642D018
WritePtrs: 0642D018 Fifo_Flag: 89800400
Weights: 000A000A
MicReceiveFifoInfo:
Fifo0: StartPtr: 066E2000 ReadPtr: 066E2FA0
WritePtrs: 066E2000 Fifo_Flag: 8B000060
writeHeaderPtr: 066E2000
Fifo1: StartPtr: 069BF000 ReadPtr: 069BF158
WritePtrs: 069BF158 Fifo_Flag: 89800400
writeHeaderPtr: 069BF158
Fifo2: StartPtr: 06977800 ReadPtr: 06977800
WritePtrs: 06977800 Fifo_Flag: 89800400
writeHeaderPtr: 06977800
Fifo3: StartPtr: 06C80000 ReadPtr: 06C80250
WritePtrs: 06C80250 Fifo_Flag: 89800400
writeHeaderPtr: 06C80250
Fifo4: StartPtr: 06A06800 ReadPtr: 06A068F8
WritePtrs: 06A068F8 Fifo_Flag: 89800400
writeHeaderPtr: 06A068F8
Fifo5: StartPtr: 06BEF800 ReadPtr: 06BEF800
WritePtrs: 06BEF800 Fifo_Flag: 88800200
writeHeaderPtr: 06BEF800
Fifo6: StartPtr: 06C36400 ReadPtr: 06C36748
WritePtrs: 06C36748 Fifo_Flag: 89800400
writeHeaderPtr: 06C36748
Fifo7: StartPtr: 06AB3000 ReadPtr: 06AB39B0
WritePtrs: 06AB3F60 Fifo_Flag: 8B000A50
writeHeaderPtr: 06AB3F60
Fifo8: StartPtr: 06BC9800 ReadPtr: 06BC98A8
WritePtrs: 06BC98A8 Fifo_Flag: 88800200
writeHeaderPtr: 06BC98A8
Fifo9: StartPtr: 066E0CD8 ReadPtr: 066E0CD8
WritePtrs: 066E0CD8 Fifo_Flag: 82800008
writeHeaderPtr: 066E0CD8
Fifo10: StartPtr: 06A50000 ReadPtr: 06A501F0
WritePtrs: 06A501F0 Fifo_Flag: 88800200
writeHeaderPtr: 06A501F0
Fifo11: StartPtr: 06C36300 ReadPtr: 06C36378
WritePtrs: 06C36378 Fifo_Flag: 86400000
writeHeaderPtr: 06C36378
Fifo12: StartPtr: 06A73C00 ReadPtr: 06A73F00
WritePtrs: 06A73C00 Fifo_Flag: 89000100
writeHeaderPtr: 06A73C00
Fifo13: StartPtr: 0642CF00 ReadPtr: 0642CF00
WritePtrs: 0642CF00 Fifo_Flag: 86800080
writeHeaderPtr: 0642CF00
Fifo14: StartPtr: 00000000 ReadPtr: 00000000
WritePtrs: 00000000 Fifo_Flag: 00800000
writeHeaderPtr: 00000000
Fifo15: StartPtr: 0642D480 ReadPtr: 0642D498
WritePtrs: 0642D498 Fifo_Flag: 84800020
writeHeaderPtr: 0642D498
===========================================================
Complete Board Id:0x0060
===========================================================
We are using the default template...
switch# (sho sdm prefer
The current template is "default" template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1024 VLANs.
number of unicast mac addresses: 5K
number of IPv4 IGMP groups + multicast routes: 1K
number of IPv4 unicast routes: 9K
number of directly-connected IPv4 hosts: 5K
number of indirect IPv4 routes: 4K
number of IPv4 policy based routing aces: 0.5K
number of IPv4/MAC qos aces: 0.5K
number of IPv4/MAC security aces: 1K
switch# sho ip cef sum
IPv4 CEF is enabled for distributed and running
VRF Default
457 prefixes (457/0 fwd/non-fwd)
Table id 0x0
Database epoch: 2 (457 entries at this epoch)
switch# sho ip route sum
IP routing table name is default (0x0)
IP routing table maximum-paths is 32
Route Source Networks Subnets Replicates Overhead Memory (bytes)
connected 0 10 0 600 1720
static 4 22 0 1560 4472
ospf 1 1 148 0 8940 26224
Intra-area: 79 Inter-area: 0 External-1: 43 External-2: 27
NSSA External-1: 0 NSSA External-2: 0
internal 5 9288
Total 10 180 0 11100 41704
Do you see anything obvious that may cause the high CPU load?
Thanks for your help
JB
_______________________________________________
cisco-nsp mailing list [email protected]
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
