The router has been upgraded to 12.4 (16). The router CPU is now back to normal.
The problem was the result of an issue on Ether-Channel process. We briefly recall the configuration of Ether-Channel few days ago, and Ether-Channel interrupts were getting sent to the RSP for some reason. I think there is a bug in Ether-Channel process. If we configure the Ether-Channel at the interface, and then revert it to a previous level. it consume some amount of CPU and it's process stay active till the next reboot. Regards, Masood Ahmad Shah -----Original Message----- From: Rodney Dunn [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 31, 2007 5:32 PM To: Masood Ahmad Shah Cc: 'Rodney Dunn'; [email protected] Subject: Re: [c-nsp] 7507 GEIP Controller On Thu, Jul 26, 2007 at 03:04:30AM +0500, Masood Ahmad Shah wrote: > > Well, the CPU usage is high even without cable. I started generating > CPU graph and it shows the CPU usage is pretty high after inserting > GEIP controller in. I tried to get "sh stack" picture and according to > stats "Network Interrupt" is being called higher than others. Traffic > is being passed from this interface and there are no CRCs or input errors. Huh? That seems whacked. What did 'sh int' snapshots say for a couple outputs? Did you reload it after you inserted the GEIP? > > The question is, how can I check the GEIP controller is faulty or GEIP card? > How can one check that faulty card is causing high cpu load while the > card is working good if we talk about traffic passing out/in from that interface. > > Is there any workaround for such issues? What do you suggest upgarding > from > 12.4 (3a) to 12.4 (16) will have positive impact :) It's a good move to be proactive given 12.4 is the bugfix train for the rest of the support life of the 75xx. > > Anyways once again thanks for your reply. > > Regards, > Masood Ahmad Shah > > > -----Original Message----- > From: Rodney Dunn [mailto:[EMAIL PROTECTED] > Sent: Thursday, July 26, 2007 2:26 AM > To: Masood Ahmad Shah > Cc: 'Rodney Dunn'; [email protected] > Subject: Re: [c-nsp] 7507 GEIP Controller > > On Thu, Jul 26, 2007 at 01:26:46AM +0500, Masood Ahmad Shah wrote: > > Thanks for your reply..I have not tried without cable, I will check it. > > > > Does cable effects even if the interface is shutdown? > > For packet switching it shouldn't but if the driver is still sending > an interrupt it may. > > > Why I can't see the process taking this high cpu usage while I do > > show proc cpu etcc.? > > Because it's interrupt level work the CPU is doing. We could try > profiling the CPU and see what it says. > > It may or may not help. > > > > > Here is the output for show diag, show stacks and show align > > > > Router#show diag 5 ( this is GEIP controller slot ) Slot 5: > > Physical slot 5, ~physical slot 0xA, logical slot 5, CBus 1 > > Microcode Status 0x4 > > Master Enable, LED, WCS Loaded > > Board is analyzed > > Pending I/O Status: None > > EEPROM format version 1 > > GEIP controller, HW rev 2.02, board revision A0 > > Serial number: 27588119 Part number: 73-2167-05 > > Test history: 0x00 RMA number: 00-00-00 > > Flags: cisco 7000 board; 7500 compatible > > > > EEPROM contents (hex): > > 0x20: 01 21 02 02 01 A4 F6 17 49 08 77 05 00 00 00 00 > > 0x30: 50 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 > > > > Slot database information: > > Flags: 0x4 Insertion time: 0x33C0 (4d00h ago) > > > > Controller Memory Size: 128 MBytes DRAM, 4096 KBytes SRAM > > > > PA Bay 0 Information: > > Gigabit-Ethernet PA, 1 ports > > EEPROM format version 1 > > HW rev 1.01, Board revision A0 > > Serial number: 29339093 Part number: 73-3144-05 > > > > --Boot log begin-- > > > > Cisco IOS Software, VIP Software (SVIP-DW-M), Version 12.4(3a), > > RELEASE SOFTWARE (fc2) Technical Support: > > http://www.cisco.com/techsupport Copyright (c) 1986-2005 by Cisco > > Systems, Inc. > > Compiled Fri 30-Sep-05 07:50 by hqluong Image text-base: 0x6001104C, > > data-base: 0x60720000 > > > > Port Statistics for unclassified packets is not turned on. > > > > --Boot log end-- > > > > > > > > Router# #show stacks > > > > Minimum process stacks: > > Free/Size Name > > 2340/3000 RSP memory size check > > 11280/12000 DHCPD Receive > > 5340/6000 Inspect Init Msg > > 59136/60000 script background loader > > 5224/6000 CDP Protocol > > 5028/6000 Clock Server > > 11004/12000 Router Init > > 3220/12000 Init > > 4184/6000 Update prst > > 4976/6000 DIB error message > > 5152/6000 RADIUS INITCONFIG > > 5188/6000 DRMI Master Reg. Slave Process > > 5136/6000 CEF Reloader > > 5312/6000 MDFS Reload > > 34844/36000 TCP Command > > 5332/6000 BGP Accepter > > 4892/6000 BGP Open > > 2276/3000 Rom Random Update Process > > 9236/12000 IPS SDF Loader > > 8220/12000 Virtual Exec > > 8220/12000 Exec > > 10884/12000 SSH Process > > > > Interrupt level stacks: > > Level Called Unused/Size Name > > 1 2213677378 6892/9000 Network Interrupt > > 2 5585183 7676/9000 Network Status Interrupt > > 3 0 8692/9000 OIR interrupt > > 4 0 9000/9000 PCMCIA Interrupt > > 5 171491 8596/9000 Console Uart > > 6 0 9000/9000 Error Interrupt > > 7 86692017 8604/9000 NMI Interrupt Handler > > > Get a couple of sh stacks and look at the interrupt level calls and > see which one is going up the most. > > It may be a bad card. > > > > > Router# #show align > > > > No alignment data has been recorded. > > > > No spurious memory references have been recorded. > > > > > > Regards, > > Masood Ahmad Shah > > > > > > -----Original Message----- > > From: Rodney Dunn [mailto:[EMAIL PROTECTED] > > Sent: Thursday, July 26, 2007 1:14 AM > > To: Masood Ahmad Shah > > Cc: [email protected] > > Subject: Re: [c-nsp] 7507 GEIP Controller > > > > What does 'sh diag' say and sh stack and show align. > > > > It's as if interrupts are getting sent to the RSP for some reason. > > > > Does the same thing happen if no cables are attached. > > > > 'sh diag'? > > > > On Thu, Jul 26, 2007 at 12:47:43AM +0500, Masood Ahmad Shah wrote: > > > I have strange issue with 7507. I have added a new GEIP > > > controller into > > it. > > > By adding this GEIP controller the CPU usage has been increased up > > > to > 20%. > > > The strange thing is that CPU usage is always normal + 20% and > > > there is no traffic on this interface ( even if interface is > > > shutdown the usage remains same ). And if there is traffic it's > > > same J > > > > > > > > > > > > I can't see the process taking this high CPU utilization if I look > > > into CPU usage stats. > > > > > > > > > > > > What the heck has happened to this? J > > > > > > > > > > > > Here are CPU stats: > > > > > > > > > > > > Router# #show proc cpu | excl 0.00%__0.00%__0.00% > > > > > > CPU utilization for five seconds: 15%/13%; one minute: 15%; five > minutes: > > > 13% > > > > > > PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process > > > > > > 5 518320 41974 12348 0.00% 0.11% 0.11% 0 Check > heaps > > > > > > > > > 20 66004 924869 71 0.08% 0.01% 0.00% 0 IPC Seat > > > Manager > > > > > > 22 204088 1466063 139 0.00% 0.03% 0.02% 0 ARP Input > > > > > > > > > 31 7080 1609 4400 0.98% 1.44% 0.59% 2 Virtual > > Exec > > > > > > > > > 52 117672 5843 20138 0.00% 0.02% 0.00% 0 > Per-minute > > > Jobs > > > > > > 73 3586004 17990635 199 0.82% 0.60% 0.49% 0 IP Input > > > > > > > > > 189 38964 213398 182 0.00% 0.01% 0.00% 0 IP SNMP > > > > > > > > > 191 56184 104952 535 0.00% 0.01% 0.00% 0 SNMP > ENGINE > > > > > > > > > 196 160560 1397067 114 0.08% 0.01% 0.00% 0 BGP > Router > > > > > > > > > 197 49200 250208 196 0.00% 0.01% 0.00% 0 BGP I/O > > > > > > > > > > > > IOS version: > > > > > > > > > > > > IOS Version 12.4(3a) > > > > > > > > > > > > Controller > > > > > > Router# sh ver | inc GEIP > > > > > > > > > > > > 1 GEIP controller (1 GigabitEthernet). > > > > > > 1 GEIP controller (1 GigabitEthernet) > > > > > > > > > > > > > > > > > > Regards, > > > > > > Masood Ahmad Shah > > > > > > > > > > > > _______________________________________________ > > > cisco-nsp mailing list [email protected] > > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > > archive at http://puck.nether.net/pipermail/cisco-nsp/ _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
