What if you interconnect the two switches with 1G rather than 10? Frank
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Matthew Huff Sent: Friday, December 09, 2011 2:18 PM To: 'Chuck Church'; 'cisco-nsp' Subject: Re: [c-nsp] Weird Multicast microburst amplification issue Yes, only the correct stream. I've opened a case with Cisco. I'm suspecting that the multicast replication engine is doing something that causes it to amplify the bursty nature of the traffic causing the microburst overruns. ---- Matthew Huff | 1 Manhattanville Rd Director of Operations | Purchase, NY 10577 OTA Management LLC | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 > -----Original Message----- > From: Chuck Church [mailto:[email protected]] > Sent: Friday, December 09, 2011 3:01 PM > To: Matthew Huff; 'cisco-nsp' > Subject: RE: [c-nsp] Weird Multicast microburst amplification issue > > Hmmm. If it's not spanning tree, I'd have to say it's something not > working right with IGMP, and the server's port is getting more streams > than it should. Have you checked the IGMP port association to see what > it's subscribed to? > > Chuck > > > -----Original Message----- > From: Matthew Huff [mailto:[email protected]] > Sent: Friday, December 09, 2011 2:48 PM > To: 'Chuck Church'; 'cisco-nsp' > Subject: RE: [c-nsp] Weird Multicast microburst amplification issue > > Yes. The problem only occurs when connected to any other switch than > the source switch. We have over 300 servers, it isn't anything with a > specific server, but rather the nature of not being on the same switch > as the source. > The data is a high packet count, bursty traffic. However, the drops > only occur on the 6748 module via the layer 2 output on a server > subscribing to that multicast traffic. This occurs if we turn layer 2 > flowcontrol on or off. No pause packets are generated from the server, > rather this is 100% related to the output ASIC queues on the 6748 > module. > > > > ---- > Matthew Huff | 1 Manhattanville Rd > Director of Operations | Purchase, NY 10577 > OTA Management LLC | Phone: 914-460-4039 > aim: matthewbhuff | Fax: 914-460-4139 > > > > -----Original Message----- > > From: Chuck Church [mailto:[email protected]] > > Sent: Friday, December 09, 2011 2:43 PM > > To: Matthew Huff; 'cisco-nsp' > > Subject: RE: [c-nsp] Weird Multicast microburst amplification issue > > > > Can you move the source server over to switch B to see if the problem > > still exists on switch B then, or moves to switch A? Anything > showing > > up in the logs? > > > > Chuck > > > > > > -----Original Message----- > > From: Matthew Huff [mailto:[email protected]] > > Sent: Friday, December 09, 2011 2:25 PM > > To: 'Chuck Church'; 'cisco-nsp' > > Subject: RE: [c-nsp] Weird Multicast microburst amplification issue > > > > Unfortunately, it isn't something simple like that. The output drops > > are continuously happening. The network is very stable. There are not > > other issues during this time. It's something amplifying the burst of > > the stream, either the multicast replication or passing through the > > layer 3 interface. > > > > IF we run a test with a server on switch a, a client on switch a, and > > a client on switch b, only the client on switch b is seeing the > problem. > > The problem isn't passing the data from switch-a to b, but rather > > something during the transmission that changes the shape of the data > > to be a heavier burst. > > > > > > > > ---- > > Matthew Huff | 1 Manhattanville Rd > > Director of Operations | Purchase, NY 10577 > > OTA Management LLC | Phone: 914-460-4039 > > aim: matthewbhuff | Fax: 914-460-4139 > > > > > > > -----Original Message----- > > > From: Chuck Church [mailto:[email protected]] > > > Sent: Friday, December 09, 2011 2:11 PM > > > To: Matthew Huff; 'cisco-nsp' > > > Subject: RE: [c-nsp] Weird Multicast microburst amplification issue > > > > > > Are there multiple streams passing through the switch? A spanning > > > tree recalculation will cause IGMP to flush associations, and flood > > > all streams out all ports until they're relearned. Portfast will > > > fix it, as will a multicast-specific interface command, would need > > > to > > look > > > it up. > > > > > > Chuck > > > > > > -----Original Message----- > > > From: [email protected] > > > [mailto:[email protected]] On Behalf Of Matthew > Huff > > > Sent: Friday, December 09, 2011 1:49 PM > > > To: 'cisco-nsp ([email protected])' > > > Subject: [c-nsp] Weird Multicast microburst amplification issue > > > > > > We have a multicast data stream (real-time ticker data) that by its > > > nature is very bursty. > > > > > > When we connect a source server via gigabit Ethernet to our > > > 6500/sup720 switch via a 6748 module and a destination server via > > > gigabit to the same or different module in the same switch, > > > everything works fine. If the destination server is on a different > > > switch connected by a layer3 10GB connection then we have > > > significant output drops on the Ethernet connected to the > destination server. > > > > > > All switches are 6509/sup720 with 6748 line cards. QoS is disabled > > > globally. > > > The servers are identical. The output drops only occur on the > > Ethernet > > > drop connected to the server. > > > > > > The only thing I can think is happening is that by routing the > > traffic > > > via the 10gb L3 interface, something is causing the traffic burst > to > > > amplify, overrunning the output port. Has anyone seen this, and > does > > > anyone know how to mitigate this? > > > > > > ---- > > > Matthew Huff | 1 Manhattanville Rd > > > Director of Operations | Purchase, NY 10577 > > > OTA Management LLC | Phone: 914-460-4039 > > > aim: matthewbhuff | Fax: 914-460-4139 > > > > > > _______________________________________________ > > > cisco-nsp mailing list [email protected] > > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > > archive at http://puck.nether.net/pipermail/cisco-nsp/ _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/ _______________________________________________ cisco-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
