Do you have traffic graphs during this timeframe? Maybe a DDOS at or through these boxes tied up the available memory. Especially since 'I/O' was the pool it was trying to grab from at the time?
Ken Matlock Network Analyst Exempla Healthcare (303) 467-4671 [email protected] -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Jon Lewis Sent: Monday, January 04, 2010 2:01 PM To: David Freedman Cc: [email protected] Subject: Re: [c-nsp] 3550 IO memory fragmentation Most of the 3550s, including the one that crashed, are running 121-22.EA10b. CSCdz51522 seems unlikely as there was nobody logged in making changes and nobody should have been making (and there are no logged signs of) physical changes to the network at the time of the event. On Mon, 4 Jan 2010, David Freedman wrote: > What release are you running? could it be CSCdz51522? > > Dave. > > > Jon Lewis wrote: >> We had a recent network event during which all of our 3550 access layer >> switches started logging things like: >> >> %SYS-2-MALLOCFAIL: Memory allocation of 1680 bytes failed from 0x17FC48, >> alignment 0 >> Pool: I/O Free: 5936 Cause: Memory fragmentation >> Alternate Pool: None Free: 0 Cause: No Alternate pool >> >> -Process= "Pool Manager", ipl= 0, pid= 5 >> -Traceback= 1C919C 1CA760 17FC4C 1DB43C 1DB658 1EBACC 1EF0FC >> >> A few minutes later, all was back to normal, though one 3550 did a >> software forced crash / reload. >> >> Under normal circumstances, these switches have 4.5-5mb of free IO memory. >> >> This looks very similar to what was posted several years ago at >> >> http://www.velocityreviews.com/forums/t31947-catalysts-trouble.html >> >> Searching bug toolkit, I didn't find anything that looked relevant. Has >> anyone else run into this sort of thing with 12.1EA software or have an >> idea what the cause/solutions might be? >> >> ---------------------------------------------------------------------- >> Jon Lewis | I route >> Senior Network Engineer | therefore you are >> Atlantic Net | >> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ >> _______________________________________________ >> cisco-nsp mailing list [email protected] >> https://puck.nether.net/mailman/listinfo/cisco-nsp >> archive at http://puck.nether.net/pipermail/cisco-nsp/ >> > ---------------------------------------------------------------------- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________ _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
