Hello All, Have you encountered a similar situation?
Thank you, On Tue, Mar 31, 2009 at 3:43 PM, Ioan Branet <[email protected]> wrote: > Hello, > > We have many Cisco 3750 switches in our network which have high CPU > utilization.It seems that the process that cause this high load is:HL3U > bkgrd process. > > The problem is solved after a reload but appears again after 3-4 months. > > We changed also the IOS but with no results. > > It seems that it is a bug but I am not very sure. > > > > sh processes cpu sorted | ex 0.00 > CPU utilization for five seconds: 99%/28%; one minute: 85%; five minutes: > 81% > PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process > 108 389775804 4389443 88799 57.57% 40.01% 39.39% 0 HL3U bkgrd > proce > 58 11854779 72185839 164 3.50% 2.77% 2.31% 0 HLFM address > lea > 292 689 192 3588 1.91% 0.33% 0.07% 1 Virtual > Exec 47 12845296 2142151 5996 1.11% 1.00% 1.04% 0 FE > free chunk 245 17376827 532655 32623 0.63% 0.51% 0.52% 0 > MFI LFD Stats Pr > 107 5476276 58476944 93 0.63% 0.62% 0.58% 0 Hulc LED > Process > 74 768210 21312879 36 0.31% 0.09% 0.08% 0 hpm main > process > 135 6540410 20282165 322 0.15% 0.18% 0.22% 0 IP > Input 143 3566619 27781902 128 0.15% 0.24% 0.20% 0 > Spanning Tree 45 1004640 128285520 7 0.15% 0.15% 0.13% > 0 Fifo Error Detec > 138 1152329 2735155 421 0.15% 0.13% 0.12% 0 PI MATM Aging > Pr > > > sh version > Cisco IOS Software, C3750ME Software (C3750ME-I5-M), Version 12.2(37)SE1, > RELEASE SOFTWARE (fc1) > Copyright (c) 1986-2007 by Cisco Systems, Inc. > Compiled Thu 05-Jul-07 20:06 by antonino > Image text-base: 0x00003000, data-base: 0x0163F400 > > ROM: Bootstrap program is C3750 boot loader > BOOTLDR: C3750ME Boot Loader (C3750ME-HBOOT-M) Version 12.1(14r)AX, RELEASE > SOFTWARE (fc1) > > vic102 uptime is 4 weeks, 4 days, 10 hours, 9 minutes > System returned to ROM by power-on > System restarted at 03:01:54 GMT Fri Feb 27 2009 > System image file is "flash:c3750me-i5-mz.122-37.SE1.bin" > > cisco ME-C3750-24TE (PowerPC405) processor (revision F0) with > 118784K/12280K bytes of memory. > Processor board ID CAT1043NM05 > Last reset from power-on > 8 Virtual Ethernet interfaces > 24 FastEthernet interfaces > 4 Gigabit Ethernet interfaces > The password-recovery mechanism is enabled. > > 1024K bytes of flash-simulated non-volatile configuration memory. > Base ethernet MAC Address : 00:19:E8:87:23:00 > Motherboard assembly number : 73-9938-04 > Motherboard serial number : CAT104356B7 > Model revision number : F0 > Motherboard revision number : A0 > Model number : ME-C3750-24TE-M > Daughterboard assembly number : 73-9939-02 > Daughterboard serial number : CAT104355CQ > System serial number : CAT1043NM05 > Top Assembly Part Number : 800-25952-04 > Top Assembly Revision Number : C0 > Version ID : V05 > CLEI Code Number : COM1510ARA > Daughterboard revision number : A0 > Hardware Board Revision Number : 0x09 > > > Switch Ports Model SW Version SW > Image ------ ----- ----- ---------- > ---------- * 1 28 ME-C3750-24TE > 12.2(37)SE1 C3750ME-I5-M > Configuration register is 0xF > > #sh memory | i HL > 030C8118 0000005000 030C804C 030C94CC 001 -------- -------- 005CF8E4 HLFM > MAC > 030C94CC 0000000808 030C8118 030C9820 001 -------- -------- 005CF93C HLFM > IP > 0320A434 0000000808 0320A008 0320A788 001 -------- -------- 00CB2F74 > HL3U_IPV4_TABLE_CHUNK > 0320A788 0000020000 0320A434 0320F5D4 001 -------- -------- 00CB2F9C > HL3U_FIB_TYPE_CHUNK > 0320F5D4 0000032768 0320A788 03217600 001 -------- -------- 00CB2FC4 > HL3U_MPATH_ADJ_TYPE_CHUNK > 03217600 0000000808 0320F5D4 03217954 001 -------- -------- 00CB2FEC > HL3U_FIB_WITH_ADJ_OR_TCAM_FAIL_CHUNK > 03217954 0000002000 03217600 03218150 001 -------- -------- 00CB3014 > HL3U_COVERING_FIB_CHUNK > 03218150 0000000808 03217954 032184A4 001 -------- -------- 00CB303C > HL3U_ARP_HRPC_THROTTLE_CHUNKS > 032184A4 0000000432 03218150 03218680 001 -------- -------- 00CB3064 > HL3U_HSRP_RETRY_CHUNKS > 03218680 0000000808 032184A4 032189D4 001 -------- -------- 00CB308C > HL3U_PROXY_ARP_CHUNKS > 032189D4 0000000432 03218680 03218BB0 001 -------- -------- 00CB30B4 > HL3U_QUERIER_INFO_CHUNKS > 03218BB0 0000003620 032189D4 03219A00 001 -------- -------- 00CB30DC > HL3U_ICMP_REDIRECT_Q_CHUNK > 03219A00 0000000296 03218BB0 03219B54 001 -------- -------- 00CB3104 > HL3U_OUT_ACL_FULL_CHUNKS > 032F2BCC 0000000960 032F252C 032F2FB8 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 0330BD38 0000000176 0330B698 0330BE14 001 -------- -------- 00B18A5C > HL2MCM > 0330C174 0000000160 0330C0C0 0330C240 001 -------- -------- 01622A8C > HL2MCM > 036F9C6C 0000000972 036F9A8C 036FA064 001 -------- -------- 00CB7108 > HL3U_FIB_WITH_ > 036FA064 0000000872 036F9C6C 036FA3F8 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 0390629C 0000000024 03906258 039062E0 001 -------- -------- 00B1AFAC > HL2MCM > 039906D8 0000001292 0399008C 03990C10 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03993CC4 0000000808 03993690 03994018 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 0399C258 0000001476 0399BC14 0399C848 001 -------- -------- 00CB7108 > HL3U_FIB_WITH_ > 0399C964 0000005000 0399C848 0399DD18 001 -------- -------- 005CB1E0 HLFM > MAC > 03A0A610 0000001096 03A0A3EC 03A0AA84 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03A2B8E4 0000000808 03A2B858 03A2BC38 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03A2BC38 0000000872 03A2B8E4 03A2BFCC 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03A2E864 0000007768 03A2E624 03A306E8 001 -------- -------- 005CB1E0 HLFM > MAC > 03A30D28 0000000808 03A30CBC 03A3107C 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03A92BD4 0000000808 03A92590 03A92F28 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03EE36A8 0000005000 03EE3634 03EE4A5C 001 -------- -------- 005CB1E0 HLFM > MAC > 03F24DCC 0000001008 03F24D54 03F251E8 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03F43C34 0000008200 03F43BE8 03F45C68 001 -------- -------- 005CB1E0 HLFM > MAC > 03F4C260 0000000808 03F4C1CC 03F4C5B4 001 -------- -------- 005CB5B8 HLFM > IP > 03F85410 0000000808 03F84DBC 03F85764 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03F85EF0 0000001008 03F85764 03F8630C 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03F872A8 0000001288 03F86F08 03F877DC 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03FEBD34 0000000808 03FEB5A8 03FEC088 001 -------- -------- 00CB7108 > HL3U_FIB_WITH_ > 03FEC088 0000001076 03FEBD34 03FEC4E8 001 -------- -------- 00CB7108 > HL3U_FIB_WITH_ > 03FEEF64 0000000808 03FEE7D8 03FEF2B8 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03FEF2B8 0000000832 03FEEF64 03FEF624 001 -------- -------- 00CBE090 > HL3U_FIB_WITH_ > 03FF05F0 0000005944 03FF04FC 03FF1D54 001 -------- -------- 005CB1E0 HLFM > MAC > > > #sh sdm prefer > The current template is "desktop routing" template. > The selected template optimizes the resources in > the switch to support this level of features for > 8 routed interfaces and 1024 VLANs. > > number of unicast mac addresses: 3K > number of IPv4 IGMP groups + multicast routes: 1K > number of IPv4 unicast routes: 11K > number of directly-connected IPv4 hosts: 3K > number of indirect IPv4 routes: 8K > number of IPv4 policy based routing aces: 0.5K > number of IPv4/MAC qos aces: 0.5K > number of IPv4/MAC security aces: 1K > > Mar 31 12:48:11: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:48:17: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:48:28: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:48:46: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:49:00: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:49:47: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:50:39: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:51:02: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:51:34: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:51:55: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:53:11: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:53:25: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:53:37: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:54:36: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:56:44: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 12:57:48: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 13:02:22: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 13:02:48: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 13:03:56: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 13:07:31: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > Mar 31 13:30:37: %OCE-3-MISSING_HANDLER_FOR_SW_OBJ_TYPE: Missing handler > for 'non choice oce get next' function for type Midchain > > > On other 3750 from our network: > > sh processes cpu sorted | ex 0.00 > CPU utilization for five seconds: 43%/2%; one minute: 58%; five minutes: > 59% > PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process > 108 2635783702 47442165 55558 30.55% 48.37% 49.28% 0 HL3U bkgrd > proce > 294 500 2692 185 3.19% 0.55% 0.13% 1 Virtual > Exec 47 296813504 43714587 6789 1.43% 1.70% 1.74% 0 FE > free chunk 107 58156203 265066803 219 0.63% 0.42% 0.37% 0 > Hulc LED Process > 247 88726248 2800468 31682 0.47% 0.45% 0.47% 0 MFI LFD Stats > Pr > 117 3310849 6664259 496 0.15% 0.04% 0.03% 0 HRPC qos > request > 135 8065160 45776840 176 0.15% 0.07% 0.04% 0 IP > Input 243 28572622 56335656 507 0.15% 0.17% 0.18% 0 > ISIS Upd > #sh ver > Cisco IOS Software, C3750ME Software (C3750ME-I5-M), Version 12.2(37)SE1, > RELEASE SOFTWARE (fc1) > Copyright (c) 1986-2007 by Cisco Systems, Inc. > Compiled Thu 05-Jul-07 20:06 by antonino > Image text-base: 0x00003000, data-base: 0x0163F400 > > ROM: Bootstrap program is C3750 boot loader > BOOTLDR: C3750ME Boot Loader (C3750ME-HBOOT-M) Version 12.1(14r)AX, RELEASE > SOFTWARE (fc1) > > rom101 uptime is 27 weeks, 5 days, 18 hours, 13 minutes > System returned to ROM by power-on > System restarted at 20:03:10 CETDST Wed Sep 17 2008 > System image file is "flash:c3750me-i5-mz.122-37.SE1.bin" > > cisco ME-C3750-24TE (PowerPC405) processor (revision F0) with > 118784K/12280K bytes of memory. > Processor board ID CAT1111NLH3 > Last reset from power-on > 3 Virtual Ethernet interfaces > 24 FastEthernet interfaces > 4 Gigabit Ethernet interfaces > The password-recovery mechanism is enabled. > > 1024K bytes of flash-simulated non-volatile configuration memory. > Base ethernet MAC Address : 00:1B:2B:E6:4B:00 > Motherboard assembly number : 73-9938-04 > Motherboard serial number : CAT11115HNX > Model revision number : F0 > Motherboard revision number : B0 > Model number : ME-C3750-24TE-M > Daughterboard assembly number : 73-9939-02 > Daughterboard serial number : CAT11115KVD > System serial number : CAT1111NLH3 > Top Assembly Part Number : 800-25952-04 > Top Assembly Revision Number : C0 > Version ID : V05 > CLEI Code Number : COM1510ARA > Daughterboard revision number : A0 > Hardware Board Revision Number : 0x09 > > > Switch Ports Model SW Version SW > Image ------ ----- ----- ---------- > ---------- * 1 28 ME-C3750-24TE > 12.2(37)SE1 C3750ME-I5-M > Configuration register is 0xF > Do you have any idea what is the root cause of this issue? > > > > > Thnak you, > > -- > Ioan Branet > > -- Ioan Branet CCIE #23474 R&S _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
