Re: Looking for Fiber Plant Management software
CableProject USA offers a free trial and a YT demo video. I can't vouch for it, never having witnessed it in operation personally, but it looks interesting. [1]http://www.cableprojectusa.com/ Cable Management Software runs the full gamut, from simplistic to near-ERP in scope, while others (e.g., VPI) also perform autoconfiguration and coordinated, parametric link designs for specific types of hardware (WDM, Switches/Routers, ROADMs, etc.). One can spend anywhere from free to $29.99 to tens of thousands of dollars, so decide carefully what you need, know specifically what you're looking for, and most of all, caveat emptor. --- franc...@menards.ca wrote: From: Francois Menard franc...@menards.ca To: Jim Devane jdev...@switchnap.com Cc: Jason Lixfeld ja...@lixfeld.ca, nanog@nanog.org nanog@nanog.org Subject: Re: Looking for Fiber Plant Management software Date: Sat, 28 Aug 2010 00:53:04 -0400 We use Fiberworks from Enghouse. Its built atop ArcObjects and all data is stored in an ARCGIS geodatabase, providing good flexibility to get the data brought up on ArcGIS Server (Web) for web-based editing. The good thing about this system is that it can also be used for design of FTTH as well, and makes it possible to produce for-construction as well as as-built drawings (with cut sheets etc.) http://www.enghouse.com/amd/products/fiberworks.html Our sister company (Xit telecom) which does OSP engineering/consulting/GIS can help implement this system. Regards, -=Francois=- -- Francois D. Menard Director of technology Xittel telecommunications inc. 1350 Royale #800 Trois-Rivieres, QC, G9A 4J4 Canada Tel: +1 819 601-6633 Fax: +1 819 374-0395 fmen...@xittel.net On 2010-08-27, at 5:31 PM, Jim Devane wrote: OSP Insight. Pricey but an excellent tool for OSP documentation. -Original Message- From: khatfi...@socllc.net [mailto:khatfi...@socllc.net] Sent: Friday, August 27, 2010 2:24 PM To: Jason Lixfeld; Jeff Saxe Cc: nanog@nanog.org Subject: Re: Looking for Fiber Plant Management software Most of the ones I have seen (2 out of 3) were inhouse/home-grown solutions. I believe the other was provided by SA (Scientific Atlanta). I tried to do a quick search on it and it appears that product may now be provided by Cisco in partnership with SA. Best of luck -Original Message- From: Jason Lixfeld ja...@lixfeld.ca Date: Fri, 27 Aug 2010 12:13:35 To: Jeff Saxejs...@briworks.com Cc: nanog@nanog.org Subject: Re: Looking for Fiber Plant Management software I've got a client who uses AutoCAD. They use it exclusively and have a pretty big fibre network for someone who's not an ILEC, so I guess it works fairly well. On 2010-08-27, at 11:39 AM, Jeff Saxe wrote: Good morning, NANOGers. My colleague at work wonders if anyone has suggestions for software to database all our fiber plant that we're constructing. We started out with paper, then Excel spreadsheets in a folder and on paper in a book, but clearly as our plant grows and we do more splicing this is not going to scale. We have started a MySQL database with a few tables, but wonder if someone has already invented this wheel. What do the big boys use? Homegrown solutions developed in-house and jealously guarded? Something standard? Expensive or cheap? Free open-source? He'd like to see... outside plan facilities: cables, fibers, splice points, poles; copper and fiber, preferably, but fiber is more important circuit or DLR that knows what elements are involved in a circuit GIS integration so that cables can be drawn on a map automagically low cost, of course Thanks in advance, everyone. -- Jeff Saxe, Network Engineer Blue Ridge InternetWorks, Charlottesville, VA 434-817-0707 ext. 2024 / js...@briworks.com References 1. http://www.cableprojectusa.com/
Re: Did your BGP crash today?
I'm assuming that they weren't really expecting this to cause issues... Where does one draw the line? I'm planning on announcing x.y.z.0/20 later in the week -- x, y and z are all prime and the sum of all 3 is also a prime. There is a non-zero chance that something somewhere will go flooie, shall I send mail now or later? In this case the researchers sent an new packet that would never have been generated by any operational router ever before to their peer. They knew their packet would cause the router to run less/un tested and code path in BGP. To their defence, the risk was low. That said when I wrote my own BGP injector I accidentally sent badly formed known messages (like UPDATE,etc.) with bad attributes (like transitive when the RFC it MUST not be, and vice versa) to my routers. Juniper would kill the session at the validation stage and be quite verbose in the log but Cisco - at least on the 7301 I tested last year with a then recent IOS - would accept the packet as it. Yep, IOS do accept INVALID packets. I have no idea what happens after but if a Cisco router is passing the packet to a Juniper router it could have the same effect that what we saw, again, and tear down a session which is not the one which initiated the badly formed packet. That said I suspect that the message may not have been fully parsed, for performance reasons, with the outgoing packet partially generated following the RFC. Quagga is even worse that Cisco when it comes to packet validation but it should not surprise anyone :p Now, Should I research the described BGP behaviour (for a white hat conference for example) and send my possibly risking packets to my peer without telling them ? Hell no ! I am pretty sure that if I did I would loose quite a few session afterwards. People trust me not to absuse my BGP connections but for sending safe known message about my network and not some research stuff. That said vendor SHOULD research (and hopefully did) this kind of behaviour, but as yesterday shown, causing packet corruption through a router is bad for its stability :p Also, I would prefer that this gets discovered and dealt with (in this case by stopping the announcement :-)) than having folk not willing to try things and ending up with a weaponized version... No argument here. Thomas
Re: Did your BGP crash today?
imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. and, considering its placement in the net (big core), i consider ios xr to be a major speaker. i suspect that these folk will test better next time. i sure hope so. randy
Re: Did your BGP crash today?
On 28 Aug 2010, at 08:56, Randy Bush wrote: imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. and, considering its placement in the net (big core), i consider ios xr to be a major speaker. i suspect that these folk will test better next time. i sure hope so. Not sure the researcher can afford to buy a ios xr and may not have access to one ! Thomas
Re: Did your BGP crash today?
i suspect that these folk will test better next time. i sure hope so. Not sure the researcher can afford to buy a ios xr and may not have access to one ! then ask on *nog for someone against whom they can test. randy
Re: Did your BGP crash today?
On (2010-08-28 09:22 +0100), Thomas Mangin wrote: i suspect that these folk will test better next time. i sure hope so. Not sure the researcher can afford to buy a ios xr and may not have access to one ! Indeed. Also testing is hard, especially so, when you essentially need to reinvent the wheel every time, which might not even fit your time schedule. Maybe we as community could build 'BGPSpec' testing suite, simply python (or ruby yay!) script which has been thought at least to puke out UPDATEs that have known to break implementations before. Test cases being unique files for easy contribution. This BGPSpec could then be ran by vendors, researchers and operators, and we could be sure that at least same mistake is not done twice. With this suite in place, it would be easier for researcher to write new test case for the suite and then ask people to run it against their gear. From global network security/reliability point-of-view BGP is pretty much only important protocol and as such maybe should enjoy special status in collaborative quality assurance. Considering this issue, late junos 32b ASN, mikrotik long AS path this http://www.cisco.com/en/US/products/products_security_advisory09186a0080094a58.shtml and probably many others, it seems we've been exceptionally lucky, that someone hasn't been fuzzing Internet BGP with target of breaking as much of it as possible, as it wouldn't really been that hard. -- ++ytti
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 09:22:34AM +0100, Thomas Mangin wrote: On 28 Aug 2010, at 08:56, Randy Bush wrote: imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. and, considering its placement in the net (big core), i consider ios xr to be a major speaker. i suspect that these folk will test better next time. i sure hope so. Not sure the researcher can afford to buy a ios xr and may not have access to one ! Thomas while this is undoubtedly true for hobbiest researchers, there are pretty good relationships between vendors and some research facilities with a strong interst in ensuring there is external review of the code base(es). (I am personally aware of at least five such facilities...:) hence I am going to have to echo Randys sentiments. This was just sloppy. --bill
Re: Did your BGP crash today?
i suspect that these folk will test better next time. i sure hope so. Not sure the researcher can afford to buy a ios xr and may not have access to one ! Also testing is hard so is cleaning up the mess when you screw up enough of the internet to make the international press. Maybe we as community could build 'BGPSpec' testing suite, simply python (or ruby yay!) script which has been thought at least to puke out UPDATEs that have known to break implementations before. Test cases being unique files for easy contribution. a bgp regression suite would not have caught this as it was not a repeat. but it sure would be useful to implementors. randy
Re: Did your BGP crash today?
while this is undoubtedly true for hobbiest researchers, there are pretty good relationships between vendors and some research facilities with a strong interst in ensuring there is external review of the code base(es). (I am personally aware of at least five such facilities...:) hence I am going to have to echo Randys sentiments. This was just sloppy. I am really surprised by these attitudes. Guys (and gals), these incidents simply go to reinforce that the software we depend on, has not received sufficient testing and that we all have gigantic exposures due to things outside of our direct control (eg: cisco, juniper or other router software quality control). You can't just demand that people don't do things that wind up being destructive to you on your production network, thats asking the world to be responsible for you. There are lots of bugs in this stuff and the sooner that we find out about them, the sooner we can get updates to address them and hopefully, shorten the window in which those issues could be painful to us and cause us grief.
Re: Did your BGP crash today?
On (2010-08-28 18:20 +0900), Randy Bush wrote: a bgp regression suite would not have caught this as it was not a repeat. but it sure would be useful to implementors. Naturally 'proving' that non-trivial software works is practically impossible. But stating what non-existing test-suite would or would not have covered is not a topic I'm particularly interested to engage. -- ++ytti
Re: Did your BGP crash today?
I am really surprised by these attitudes. Guys (and gals), these incidents simply go to reinforce that the software we depend on, has not received sufficient testing and that we all have gigantic exposures due to things outside of our direct control nice anti-vendor rant. but over the last decades we the ops have asked for a jillion features which creates massive code, and there is no hope of testing all the code paths rigorously. the vendors have large test labs, and do what they can. sure, they could do better. so could we all. but it is also coders' responsibility, whether vendors or researchers or hackers, to also test what they send. in this case, clearly that was not done sufficiently. if i am sloppy in my receiving code, the pain is mine. if you are sloppy in your sending code, the pain is not yours. randy
Re: Did your BGP crash today?
Quagga is even worse that Cisco when it comes to packet validation but it should not surprise anyone :p To substantiate my claim, my mercurial log tells me that for MPRNLRI and MPURNLRI having the flag set as Transitive instead of Optional did not cause Quagga to complain. It just took the IPv4/IPv6 route . Now it may have been fixed. I should really check and if not pass this to the quagga dev list. I am lazy. Thomas
Re: Did your BGP crash today?
* Christopher Morrow: (you are asking your vendors to run full bit sweeps of each protocol in a regimented manner checking for all possible edge cases and properly handling them, right?) The real issue is that both spec and current practice say you need to drop the session as soon as you encounter any unexpected data. That's just wrong---you can't really be sure that it's a temporary glitch caused by your peer. If it's not, you are unnecessarily taking yourself off the net. Of course, there is little you can do when the outer framing at the internal BGP layer is wrong (resyncing is way too risky). A tear-down might be in order if you receive an unrecognized message type, too. But a BGP update message which is malformed internally should just be ignored. From a theoretical point of view, it's no worse than the operator configuring a prefix-list that filters out all the NLRI entries.
Re: Did your BGP crash today?
On 08/28/2010 11:39 AM, Saku Ytti wrote: On (2010-08-28 18:20 +0900), Randy Bush wrote: a bgp regression suite would not have caught this as it was not a repeat. but it sure would be useful to implementors. Naturally 'proving' that non-trivial software works is practically impossible. But stating what non-existing test-suite would or would not have covered is not a topic I'm particularly interested to engage. I suggest the test-tool has 2 bgp-sessions and tests if what it put in did or did not come out on the otherside and in what shape or form. There are already atleast 2 projects which have BGP-code which could probably be adapted: http://code.google.com/p/exabgp/ http://code.google.com/p/bgpsimple/ Can I suggest a fuzzer as wel ?
Re: Did your BGP crash today?
Those tools are not suitable for regression testing ( I know I wrote exabgp ) not saying they could not be adapted though. Fizzing may return crashes or issues with the daemon but it is unlikely. You need predictable input for regression testing and in our particular case how do you detect a corruption without knowing what the behaviour of the router should be on that particular input. If it was that simple vendors would have done it --- from my iPhone On 28 Aug 2010, at 13:09, Leen Besselink l...@consolejunkie.net wrote: On 08/28/2010 11:39 AM, Saku Ytti wrote: On (2010-08-28 18:20 +0900), Randy Bush wrote: a bgp regression suite would not have caught this as it was not a repeat. but it sure would be useful to implementors. Naturally 'proving' that non-trivial software works is practically impossible. But stating what non-existing test-suite would or would not have covered is not a topic I'm particularly interested to engage. I suggest the test-tool has 2 bgp-sessions and tests if what it put in did or did not come out on the otherside and in what shape or form. There are already atleast 2 projects which have BGP-code which could probably be adapted: http://code.google.com/p/exabgp/ http://code.google.com/p/bgpsimple/ Can I suggest a fuzzer as wel ?
Re: Did your BGP crash today?
* Randy Bush: imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. Practically, this boils down to don't do that, which is certainly fine by me. To carry out such experiments responsibly, you have to conduct so much testing beforehand that the live test on the actual Internet will not yield new insights (assuming you did your pre-experiment testing properly).
Re: Did your BGP crash today?
* Randy Bush: a bgp regression suite would not have caught this as it was not a repeat. Eh, it was just another corrupt-and-propagate issue combined with the broken (but RFC-required) session reset policy on malformed updates.
Re: Did your BGP crash today?
On (2010-08-28 13:23 +0200), Thomas Mangin wrote: Those tools are not suitable for regression testing ( I know I wrote exabgp ) not saying they could not be adapted though. Fizzing may return crashes or issues with the daemon but it is unlikely. You need predictable input for regression testing and in our particular case how do you detect a corruption without knowing what the behaviour of the router should be on that particular input. It doesn't actually matter how likely or unlikely one expect such tool to be finding new issues. There is already value, that researchers like RIPE in this case, could simply write new test case, instead of needing to build whole infrastructure. -- ++ytti
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 04:56:05PM +0900, Randy Bush wrote: imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. and, considering its placement in the net (big core), i consider ios xr to be a major speaker. i suspect that these folk will test better next time. i sure hope so. I think you blame the wrong people. The vendor should make sure that their implementation does not violate the very basics of the BGP protocol. This bug in the IOS XR BGP implementation was a ticking time bomb and it was just a matter of when it would blow up. I suspect that Cisco will test better next time when they release a new version of their software. I sure hope so. -- :wq Claudio
Re: Did your BGP crash today?
I think that focusing on researchers (who we assume are good-intentioned) misses the point. Any connected BGP speaker can inject any form of ugliness. The routers that mishandled these updates were bounded by routers that were able to 'properly' handle corrupted updates. The question of aggressive teardown of BGP sessions after a speaker receives garbage has been well considered for a long time. Stop the problem at the edges. The only difference here is that the edge moved one hop closer to the core. /c Sent from my iPhone On Aug 28, 2010, at 7:31 AM, Florian Weimer f...@deneb.enyo.de wrote: * Randy Bush: imiho, researchers injecting data into the control plane are responsible to have tested it at least against major bgp speakers. Practically, this boils down to don't do that, which is certainly fine by me. To carry out such experiments responsibly, you have to conduct so much testing beforehand that the live test on the actual Internet will not yield new insights (assuming you did your pre-experiment testing properly).
Re: Did your BGP crash today?
My point was not about crafted bgp message to test border cases - this is what one would expect in a regression suite. It is about the use of a fuzzer to corrupt packet when you then do not know if the router is then behaving correctly or not. --- from my iPhone On 28 Aug 2010, at 13:36, Saku Ytti s...@ytti.fi wrote: On (2010-08-28 13:23 +0200), Thomas Mangin wrote: Those tools are not suitable for regression testing ( I know I wrote exabgp ) not saying they could not be adapted though. Fizzing may return crashes or issues with the daemon but it is unlikely. You need predictable input for regression testing and in our particular case how do you detect a corruption without knowing what the behaviour of the router should be on that particular input. It doesn't actually matter how likely or unlikely one expect such tool to be finding new issues. There is already value, that researchers like RIPE in this case, could simply write new test case, instead of needing to build whole infrastructure. -- ++ytti
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 01:09:47PM +0200, Leen Besselink wrote: On 08/28/2010 11:39 AM, Saku Ytti wrote: On (2010-08-28 18:20 +0900), Randy Bush wrote: a bgp regression suite would not have caught this as it was not a repeat. but it sure would be useful to implementors. Naturally 'proving' that non-trivial software works is practically impossible. But stating what non-existing test-suite would or would not have covered is not a topic I'm particularly interested to engage. I suggest the test-tool has 2 bgp-sessions and tests if what it put in did or did not come out on the otherside and in what shape or form. There are already atleast 2 projects which have BGP-code which could probably be adapted: http://code.google.com/p/exabgp/ http://code.google.com/p/bgpsimple/ Can I suggest a fuzzer as wel ? There was once cert-bgp-testcases-28may03-final.tar.gz which did some testing (including expected responses). I use it from time to time. From the README: For more information see the NANOG 28 (http://www.nanog.org) presentation ... BGP Vulnerability Testing: Separating Fact from FUD by Sean Convery (s...@cisco.com) and Matthew Franz (mfr...@cisco.com) But my quick googeling failed to locate a link to it. -- :wq Claudio
Re: Did your BGP crash today?
To carry out such experiments responsibly, you have to conduct so much testing beforehand that the live test on the actual Internet will not yield new insights (assuming you did your pre-experiment testing properly). you seem to assume the purpose of the test was to see if routers crashed. i certainly think mor ehighly of ripe lans folk than that. randy
Re: Did your BGP crash today?
On 08/28/2010 01:52 PM, Thomas Mangin wrote: My point was not about crafted bgp message to test border cases - this is what one would expect in a regression suite. It is about the use of a fuzzer to corrupt packet when you then do not know if the router is then behaving correctly or not. I wasn't saying you should use both at the same time, but I thought it might be a good idea to add a fuzzer so that it could be run seperately. Any bugs we can find before it is in production causing problems is useful. Although most code I've seen which deals with the BGP-protocol directly seemed to be pretty simple/smart about it. --- from my iPhone On 28 Aug 2010, at 13:36, Saku Ytti s...@ytti.fi wrote: On (2010-08-28 13:23 +0200), Thomas Mangin wrote: Those tools are not suitable for regression testing ( I know I wrote exabgp ) not saying they could not be adapted though. Fizzing may return crashes or issues with the daemon but it is unlikely. You need predictable input for regression testing and in our particular case how do you detect a corruption without knowing what the behaviour of the router should be on that particular input. It doesn't actually matter how likely or unlikely one expect such tool to be finding new issues. There is already value, that researchers like RIPE in this case, could simply write new test case, instead of needing to build whole infrastructure. -- ++ytti
Re: Did your BGP crash today?
* Claudio Jeker: I think you blame the wrong people. The vendor should make sure that their implementation does not violate the very basics of the BGP protocol. The curious thing here is that the peer that resets the session, as required by the spec, causes the actual damage (the session reset), and not the peer producing the wrong update. This whole thread is quite schizophrenic because the consensus appears to be that (a) a *researcher is not to blame* for sending out a BGP message which eventually leads to session resets, and (b) an *implementor is to blame* for sending out a BGP messages which eventually leads to session resets. You really can't have it both ways. I'm fed up with this situation, and we will fix it this time. My take is that if you reset the session, you're part of the problem, and consequently deserve part of the blame. So if you receive a properly-framed BGP update message you cannot parse, you should just log it, but not take down the session.
Re: Did your BGP crash today?
Am I the only one on the list which saw the sentence in Cisco's Advisory Before sending the the unknown attribute to peers, the IOS XR corrupted it which clearly states this was a bug?!
Re: Did your BGP crash today?
Hi! I think you blame the wrong people. The vendor should make sure that their implementation does not violate the very basics of the BGP protocol. The curious thing here is that the peer that resets the session, as required by the spec, causes the actual damage (the session reset), and not the peer producing the wrong update. This whole thread is quite schizophrenic because the consensus appears to be that (a) a *researcher is not to blame* for sending out a BGP message which eventually leads to session resets, and (b) an *implementor is to blame* for sending out a BGP messages which eventually leads to session resets. You really can't have it both ways. I'm fed up with this situation, and we will fix it this time. My take is that if you reset the session, you're part of the problem, and consequently deserve part of the blame. So if you receive a properly-framed BGP update message you cannot parse, you should just log it, but not take down the session. Not sure if the link was posted allready ... http://www.cisco.com/en/US/products/products_security_advisory09186a0080b4411f.shtml 'The vulnerability manifests itself when a BGP peer announces a prefix with a specific, valid but unrecognized transitive attribute. On receipt of this prefix, the Cisco IOS XR device will corrupt the attribute before sending it to the neighboring devices. Neighboring devices that receive this corrupted update may reset the BGP peering session.' Bye, Raymond.
Re: Did your BGP crash today?
* Raymond Dijkxhoorn: Not sure if the link was posted allready ... http://www.cisco.com/en/US/products/products_security_advisory09186a0080b4411f.shtml Cisco posts their advisories to the NANOG list. 'The vulnerability manifests itself when a BGP peer announces a prefix with a specific, valid but unrecognized transitive attribute. On receipt of this prefix, the Cisco IOS XR device will corrupt the attribute before sending it to the neighboring devices. Neighboring devices that receive this corrupted update may reset the BGP peering session.' I'm not sure what you intend to say by quoting this part of the advisory. If you think that it's an IOS XR bug which only needs fixing in IOS XR, you're showing the very attitude which has stopped us from making the network more resilient to these types of events.
Re: Did your BGP crash today?
* Randy Bush: To carry out such experiments responsibly, you have to conduct so much testing beforehand that the live test on the actual Internet will not yield new insights (assuming you did your pre-experiment testing properly). you seem to assume the purpose of the test was to see if routers crashed. i certainly think mor ehighly of ripe lans folk than that. We don't yet precisely what was the point of the experiment. But it is very likely that it intended to study propagation of such updates. Not propagating them is a protocol violation, so in order to observe anything beyond propagation times, they would have to intend to cause protocol violations, which is, in fact, awfully close to session resets (thanks to the BGP protocol design).
Re: Did your BGP crash today?
Hi! Cisco posts their advisories to the NANOG list. 'The vulnerability manifests itself when a BGP peer announces a prefix with a specific, valid but unrecognized transitive attribute. On receipt of this prefix, the Cisco IOS XR device will corrupt the attribute before sending it to the neighboring devices. Neighboring devices that receive this corrupted update may reset the BGP peering session.' I'm not sure what you intend to say by quoting this part of the advisory. If you think that it's an IOS XR bug which only needs fixing in IOS XR, you're showing the very attitude which has stopped us from making the network more resilient to these types of events. Its more a workaround then a bugfix ... Dont try to write down what I might think. I am perfectly capable of explaining this myselve. The narrow minded response you just did tells more about you then about me. So far for the rant. I think i am around long enough that you would not even consider thinking that i would say 'hey this is a IOS XR BUG. Its not.' I didnt say this at all. Did I? If it affects a large part of traffic on the internet and it obviously did. It took down a couple of the larger networks. http://www.ams-ix.net/cgi-bin/stats/16all?log=totalall;png=daily You can clearly see the drop there also. I think a 'fix' 'bugfix' 'workaround' whatever you want to call it, i still think its good they released it and fast. A more structural approach is nice but wont help a lot of networks right now. I am sorry i tried to add something to the thread. Think about this Florian. We are not the bad guys. Bye, Raymond.
Re: Did your BGP crash today?
We had ASN4, AS-PATH and this one. More or less we hit this session reset problem once a year but nothing was done yet to change the RFC. So I am to blame as much as every network engineer to not have pushed for a change or at least a comprehensive explanation on the session teardown behaviour is like it is and should not be changed. It is only our fault for not having dealt with the problem the first time correctly, and will be next time if nothing is changed once more. I agree correctly framed invalid packet should be discarded without tearing the session down. --- from my iPhone On 28 Aug 2010, at 14:27, Florian Weimer f...@deneb.enyo.de wrote: * Raymond Dijkxhoorn: Not sure if the link was posted allready ... http://www.cisco.com/en/US/products/products_security_advisory09186a0080b4411f.shtml Cisco posts their advisories to the NANOG list. 'The vulnerability manifests itself when a BGP peer announces a prefix with a specific, valid but unrecognized transitive attribute. On receipt of this prefix, the Cisco IOS XR device will corrupt the attribute before sending it to the neighboring devices. Neighboring devices that receive this corrupted update may reset the BGP peering session.' I'm not sure what you intend to say by quoting this part of the advisory. If you think that it's an IOS XR bug which only needs fixing in IOS XR, you're showing the very attitude which has stopped us from making the network more resilient to these types of events.
Hyperchip (was Re: PacketShader)
http://docs.google.com/viewer?url=http://www.hyperchip.com/H40GPresentation.pdf --- ober...@es.net wrote: From: Kevin Oberman ober...@es.net To: na...@shankland.org Cc: nanog@nanog.org Subject: Re: PacketShader Date: Mon, 23 Aug 2010 11:56:29 -0700 Date: Mon, 23 Aug 2010 06:27:00 -0700 From: Jim Shankland na...@shankland.org Mark Smith wrote: On Mon, 23 Aug 2010 05:59:43 -0400 valdis.kletni...@vt.edu wrote: I missed that, and that answers the was it a GigaBytes verses Gigabits error question. Nothing new here by the looks of it - people in this thread were getting those sorts of speeds a year ago out of PC hardware under Linux - http://lkml.org/lkml/2009/7/15/234 I have achieved a collective throughput of 66.25 Gbit/s. We've achieved 70 Gbps aggregate unidirectional TCP performance from one P6T6 based system to another. Very nice, but doing this with 1514-byte packets is the low-hanging fruit. (9K packets? That's the fruit that falls off the tree and into your basket while you're napping :-).) The more interesting limit: how many 40-byte packets per second can you shovel into this system and still have all of them come out the other end? Seems reasonable, but in our testing of 100G Ethernet capable routers we found one that handled 8000 bytes just fine, but could only run 9000 byte packets at about 90G. Just a bit unexpected. Really, in this day and age, a chassis throughput of 100G is pretty trivial. When you start getting up to the Tbps range on a system using standard components, then I'll be really interested. We do have a network of many end systems attached with 10Gbps Ethernet cards. I'm sure that we are not unique, though probably unusual. We are achieving stable disk to disk transfer rates of well over 3G between the US and Australia. I don't think that PacketShader would handle the load too well. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 _ Get your own *free* email address like this one from www.OwnEmail.com
Re: Did your BGP crash today?
I agree correctly framed invalid packet should be discarded without tearing the session down. This statement is way to simplistic. I would be interested if anyone has pointers toward any work which was done to sort this issue. Thanks. Thomas
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 02:51:17PM +0200, Thomas Mangin wrote: We had ASN4, AS-PATH and this one. More or less we hit this session reset problem once a year but nothing was done yet to change the RFC. You are mixing up three totaly different problems. Sure the result was the same (session drops). This time a IOS XR device was corrupting an attribute before sending it out. The corruption had to be in the header section of the attribute or the other side would not have detected it (since the neighbor did not know about this attribute either). Now if a system sends out corrupted BGP messages there is no way out, you need to close the session because not doing so may result in much bigger mayhem. It was not mentioned what the corruption was actually, was the lenght wrong or was the optional flag missing (makeing the attribute well known)? Unlike in the ASN4 issue this time the session to the faulty system was dropped and by doing so stopped further issues. So I am to blame as much as every network engineer to not have pushed for a change or at least a comprehensive explanation on the session teardown behaviour is like it is and should not be changed. It is only our fault for not having dealt with the problem the first time correctly, and will be next time if nothing is changed once more. I agree correctly framed invalid packet should be discarded without tearing the session down. Great, corrupting your RIB and FIB and every of your peers RIB. Thanks a lot for routing loops and wrong announcements. The only thing you can drop without causing troubles are (tranistive) optional attributes. This is covered by draft-ietf-idr-optional-transitive and hopefully it will be adopted as RFC and implemented by vendors. If a well known attribute like AS-PATH is corrupted then there is no choice, the session needs to be reset. Which is bad when the AS-PATH validation code has a bug. -- :wq Claudio
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 02:19:28PM +0200, Florian Weimer wrote: * Claudio Jeker: I think you blame the wrong people. The vendor should make sure that their implementation does not violate the very basics of the BGP protocol. The curious thing here is that the peer that resets the session, as required by the spec, causes the actual damage (the session reset), and not the peer producing the wrong update. This whole thread is quite schizophrenic because the consensus appears to be that (a) a *researcher is not to blame* for sending out a BGP message which eventually leads to session resets, and (b) an *implementor is to blame* for sending out a BGP messages which eventually leads to session resets. You really can't have it both ways. The researcher is not to blame because all the BGP messages he sent out were properly formed. The implementor is to blame becuase the code he wrote send out BGP messages which were not properly formed. I'm fed up with this situation, and we will fix it this time. My take is that if you reset the session, you're part of the problem, and consequently deserve part of the blame. So if you receive a properly-framed BGP update message you cannot parse, you should just log it, but not take down the session. If you get your wish, and that gets implemented, in some numer of years trree will be a NANOG posting (perhaps from you, perhaps not) arguing that any malformed BGP message should result in the session being torn down. This will be after a router develops a failure that causes it to send many incorrect messages, but only some of them malformed. So the malformed ones will be discarded, the remainder will be propogated throughout the Internet. If the ones that are incorrect but not malformed are, say, filled with more specifics for large portions of the Internet, someone will be asking: How could all the other routers accept these advertisement from a router known to be broken ... it was sending malformed advertisements, but instead of tearning down the sessions, you decided to trust all the validly formed messages from this known-to-be-broken router. My point is: we can't always look at the most recent failure to decide what the correct policy is. We have good data on the cases where NOTIFY on any malformed packet has caused significantly outages in the Internet. We don't have nearly as good data on the cases where NOTIFY-on-any-malformed-packet saved the Internet from a significant outage. I don't claim to know which is the bigger problem. But any serious argument to change the behavior needs to consider the risk from propogating information received from a router known to be broken, on the theory that the brokenness only causes malformed messages (which can be discarded) and does not also cause incorrect but correctly formed messages to be sent. -- Brett
Re: Did your BGP crash today?
On Fri, Aug 27, 2010 at 2:33 PM, Dave Israel da...@otd.com wrote: On 8/27/2010 3:22 PM, Jared Mauch wrote: [snip] an MD5 hash that can be added to the packet. If the TCP hash checks Hello, layering violation.If the TCP MD5 option was used, the MD5 checksum was probably correct. Malformed BGP Protocol messages, not malformed TCP messages. The BGP protocol that lives on top of TCP is a non-packetized stream. Dropping the IP packets, would just mean that the IP packets containing the malformed BGP data need to get resent (still containing malformed BGP application protocol data, when resent). out, then you know the packet wasn't garbled, and just contained information you didn't grok. That seems like enough evidence to be able to shrug and toss the packet without dropping the session. If the attribute is malformed, and in particular, if the _length_ value is malformed, because more bits have been transmitted as part of an update than indicated in the length, how do you reliably determine exactly where the junk ends, and the next attribute starts, and resume the stream without loss of other critical data? Without a valid length of the attribute, you don't know which bit the next attribute starts at, or which bit the next update starts at. If the apparently length of the update is wrong, the rest of your session appears to be malformed. If your guess is wrong, you could wind up interpreting part of the attribute DATA portion as another route update, allowing an adversary to exploit the malformed bug to possibly inject new routes. A recovery mechanism could lead to worse problems, or lead to problems not being discovered. -Dave -- -J
Re: Comcast enables 6to4 relays
These are good news. However, if Comcast provides native IPv6 to their customers, then the IPv6 native customers don't need these 6to4 relays? Airport Extreme, Linksys and other user equipment, enable IPv6 by doing 6to4 tunnels, so what this press release says, is that there are many users who are already on IPv6 via Comcast network but not native? Providing relays close to them, is a good transition move. Alternatively, the measurement of this 6to4 bandwidth on IPv4 may give you an idea of the demand for IPv6 from your customers? May be you detected a non null number here? I'm just trying to understand more IPv6 by the examples. I'm personally using 6to4 at home, and experiencing some MTU issues, which seems related to some PTB packets suppressed on the way between some end points, and that can depend on which 6to4 relay I'm using. Still trying to debug this (I'm not too fanatic about it, but work on it when I have a bit of time). I thought I would mention that. The WAND people have done some good studies: http://www.ripe.net/ripe/meetings/ripe-60/presentations/Stasiewicz-Measurements_of_IPv6_Path_MTU_Discovery_Behaviour.pdf At the office, I have a more classical tunnel with he.net and do not have any issue there. - Original Message - From: John Jason Brzozowski john_brzozow...@cable.comcast.com To: NANOG nanog@nanog.org Sent: Sunday, 29 August, 2010 5:49:30 AM Subject: Comcast enables 6to4 relays FYI - thought this would be of interest to some of you, there will be more news on this front shortly. http://www.comcast6.net/ 6to4 Relays Activated Tuesday, August 17, 2010 As we started our IPv6 trials, we began to observe an increase in 6to4 relay traffic. 6to4 is a transition mechanism built into some operating systems and home gateways. While it is not a transition technology that Comcast planned to invest in due to limitations related to performance, we did observe poor performance when 6to4 was used by our customers. In many cases, these customers were not even aware that 6to4 was enabled by default or that their device or operating system was attempting to use 6to4 to communicate with IPv6 resources on the Internet.
Re: Did your BGP crash today?
On Sat, Aug 28, 2010 at 6:14 AM, Florian Weimer f...@deneb.enyo.de wrote: * Christopher Morrow: (you are asking your vendors to run full bit sweeps of each protocol in a regimented manner checking for all possible edge cases and properly handling them, right?) The real issue is that both spec and current practice say you need to drop the session as soon as you encounter any unexpected data. That's sorry, I conflated two things... or didn't mean to but did anyway. 1) users of gear that does BGP really need to ask loudly and longly (and then go test for themselves) that their BGP speakers do the 'right thing' when faced with oddball scenarios. If someone sends you a previously unknown attribute... don't corrupt it and pass it on, pass if transitive, drop if not. 2) some thought and writing and code-changes need to go into how the bgp-speakers of the world deal with bad-behaving bgp speakers. Is 'send notify and reset' the right answer? is there one 'right answer' ? Should some classes of fugly exchange end with a 'dropped that update, moved along' and some end with 'pull eject handle!' ? it's doubtful that 2 can get solved here (nanog, though certainly some operational thought on the right thing would be great as guidance). i would hope that 1 can get some traction here (via folks going back to their vendors and asking: Did you run the Mu-security/Oolu-univ/etc fuzzing test suites against this code? can I see the results? I hope they match the results I'm going to be getting from my folks in ~2wks... or we'll be having a much more structured/loud conversation... another poster had a great point about 'all the world can screw with you, you have no protections other than trust that the next guy won't screw you over (inadvertently)'. There are no protections available to you if someone sets (example) bit 77 in an ipv4 update message to 1 when it should by all accounts be 0. Or (apparently) if they send a previously unknown attribute on a route :( You can put in max-prefix limits, as-path limits (length and content), prefix-filters.. but internal-message-content you are stuck hoping the vendors all followed the same playbook. With everyone saying together: Please appropriately test your implementation for all boundary cases maybe we can get to where these happen less often (or nearly never) - every 3 months is a little tedious. -chris
Re: Did your BGP crash today?
On BB, so top posting. Apologies. It seems that creating a worst case BGP test suite for all kinds of nastiness (in light of the recent RIPE thing) might not be a bad idea - so that we can all test the implementation ourselves before we deploy new code. Like all funky attributes, all funky AS SETs... With knobs for 1 to mem exhaust (for long data sets, etc). Unless BGP is massively more complicated than I remember, its not a very advanced CS grad project. I'm thinking a quagga or perl BGP talker would be a good place to start. Deepak - Original Message - From: Christopher Morrow morrowc.li...@gmail.com To: Florian Weimer f...@deneb.enyo.de Cc: nanog@nanog.org nanog@nanog.org Sent: Sun Aug 29 01:12:00 2010 Subject: Re: Did your BGP crash today? On Sat, Aug 28, 2010 at 6:14 AM, Florian Weimer f...@deneb.enyo.de wrote: * Christopher Morrow: (you are asking your vendors to run full bit sweeps of each protocol in a regimented manner checking for all possible edge cases and properly handling them, right?) The real issue is that both spec and current practice say you need to drop the session as soon as you encounter any unexpected data. That's sorry, I conflated two things... or didn't mean to but did anyway. 1) users of gear that does BGP really need to ask loudly and longly (and then go test for themselves) that their BGP speakers do the 'right thing' when faced with oddball scenarios. If someone sends you a previously unknown attribute... don't corrupt it and pass it on, pass if transitive, drop if not. 2) some thought and writing and code-changes need to go into how the bgp-speakers of the world deal with bad-behaving bgp speakers. Is 'send notify and reset' the right answer? is there one 'right answer' ? Should some classes of fugly exchange end with a 'dropped that update, moved along' and some end with 'pull eject handle!' ? it's doubtful that 2 can get solved here (nanog, though certainly some operational thought on the right thing would be great as guidance). i would hope that 1 can get some traction here (via folks going back to their vendors and asking: Did you run the Mu-security/Oolu-univ/etc fuzzing test suites against this code? can I see the results? I hope they match the results I'm going to be getting from my folks in ~2wks... or we'll be having a much more structured/loud conversation... another poster had a great point about 'all the world can screw with you, you have no protections other than trust that the next guy won't screw you over (inadvertently)'. There are no protections available to you if someone sets (example) bit 77 in an ipv4 update message to 1 when it should by all accounts be 0. Or (apparently) if they send a previously unknown attribute on a route :( You can put in max-prefix limits, as-path limits (length and content), prefix-filters.. but internal-message-content you are stuck hoping the vendors all followed the same playbook. With everyone saying together: Please appropriately test your implementation for all boundary cases maybe we can get to where these happen less often (or nearly never) - every 3 months is a little tedious. -chris
