After doing further investigation, I found that in-fact my Cisco-vxr Npe-g2 and g1 in the path (between M7i and customer router) suffered the Dos and due to cpu saturation the bgp flapped. Earlier I did not noticed because the cpu utilization graph of Cisco showed only 50% in npe-g2 and 80% in npe-g1 and straightened perhaps it was not responding mrtg polling, however "show proc cpu history" showed the different story.
M7i was not affected...bravo Juniper..! Thanks everyone. Regards, Samit Nilesh Khambal wrote: > I don't see any drops in the sofware or hardware queues towards RE. So > it does not look like it was this router that was affected by DOS attack > and caused BGP flap. As Stefan mentioned, check the logs for the BGP > notification reason and to find out if we sent or received the > Notification. > > For your M7i, this traffic is the transit traffic and should be handled > in the PFE itself. It is possible that this router may get affected by > DoS, if the BGP peer that flapped during DOS and the destination of DOS > are both reachable via same egress interface and the egress interface > does not have enough bandwidth to handle the traffic. As you mentioned, > the DOS traffic was seen at the rate of 90 Mbps. It can saturate egress > interface's queues if it is an FE interfaces, with other production > traffic in the background. Check the egress interface queues and > evidence of drops in either best-effort or network-control queue. > > Did you have a filter on ingress or egress interface to drop such > traffic with filter action "reject"? > > Thanks, > Nilesh. > > > > > > > On Feb 15, 2009, at 2:50 AM, "Samit" <[email protected]> wrote: > >> I do have filter in placed to protect the RE. But the attack is not >> targeted or directed to any interfaces of my router. My customer network >> as under DoS attacked , tcpdump snapshot attached below "x" is source >> and "y" is target. >> >> 04:16:18.225986 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226063 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226072 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226091 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226095 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226112 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226115 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> length 36 >> 04:16:18.226131 IP x.x.x.x.12372 > y.y.y.y.18990: UDP, >> >> I don't have pfe stat during Dos but this is how it the output look like >> now. >> >> Packet Forwarding Engine traffic statistics: >> Input packets: 40918149601 102324 pps >> Output packets: 40903880367 102281 pps >> Packet Forwarding Engine local traffic statistics: >> Local packets input : 4603616 >> Local packets output : 5077330 >> Software input control plane drops : 0 >> Software input high drops : 0 >> Software input medium drops : 0 >> Software input low drops : 0 >> Software output drops : 0 >> Hardware input drops : 0 >> Packet Forwarding Engine local protocol statistics: >> HDLC keepalives : 143360 >> ATM OAM : 0 >> Frame Relay LMI : 0 >> PPP LCP/NCP : 0 >> OSPF hello : 0 >> OSPF3 hello : 0 >> RSVP hello : 0 >> LDP hello : 0 >> BFD : 0 >> IS-IS IIH : 0 >> Packet Forwarding Engine hardware discard statistics: >> Timeout : 0 >> Truncated key : 0 >> Bits to test : 0 >> Data error : 0 >> Stack underflow : 0 >> Stack overflow : 0 >> Normal discard : 14002963 >> Extended discard : 41297 >> Invalid interface : 0 >> Info cell drops : 0 >> Fabric drops : 0 >> Packet Forwarding Engine Input IPv4 Header Checksum Error and Output MTU >> Error statistics: >> Input Checksum : 196 >> Output MTU : 0 >> >> >> I don't have JTAC support access.. :) >> >> Regards, >> Samit >> >> >> >> >> Nilesh Khambal wrote: >>> Hi Samit, >>> >>> Do you have the output of "show pfe statistics traffic" from this >>> router? >>> >>> What was the type of DoS attack traffic? Was it directed to any of the >>> interfaces on the router? Did you have any filter applied to loopback >>> interface to drop such traffic? If yes, did any of the filters that were >>> applied to the interface matching DoS traffic had reject action in them? >>> Is any syslogging enabled in any of the filter terms that were matching >>> the attack traffic? >>> >>> Also, I would recommend involving JTAC during such incidents in future. >>> They can help you figure out the problem. >>> >>> Thanks, >>> Nilesh >>> >>> >>> On Feb 14, 2009, at 11:19 PM, "Samit" <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> Today early in the morning around 4am we had a udp based DoS from the >>>> Internet destinate to one of my customer network for about over 1.5hr. >>>> The pps rate was from 165k to 245k peak and at the rate of around >>>> 90Mbps >>>> as per the mrtg graphs. I don't have any Qos running, but I noticed >>>> later that all Bgp peer sessions flapped during that period though I >>>> have plenty of capacity in my upstream as well as in downstream links, >>>> therefore I don't call it M7i fully survived and handled it. M7i is >>>> capable of forwarding 16million pps and additionally I have plenty of >>>> free bandwidth available, so there should not be any interface buffer >>>> exhaustion or link saturation. Therefore, I failed to understood the >>>> reason of the BGP flaps. Can anyone help me explain to understand? >>>> >>>> >>>> Regards, >>>> Samit >>>> >>>> _______________________________________________ >>>> juniper-nsp mailing list [email protected] >>>> https://puck.nether.net/mailman/listinfo/juniper-nsp >>> >>> > > _______________________________________________ juniper-nsp mailing list [email protected] https://puck.nether.net/mailman/listinfo/juniper-nsp
