On Thu, Mar 26, 2015 at 12:11 PM, Lorenzo Colitti <[email protected]> wrote: > On Thu, Mar 26, 2015 at 2:00 PM, Mikael Abrahamsson <[email protected]> > wrote: >>> >>> Just to be pedantic, my comment was completely serious and not a "hint". >>> New implementations almost always suffer from lack of testing of corner >>> cases. Same thing with protocol specs. If something exists to exercise those >>> cases, that would be good both from an implementation standpoint, and to >>> make certain that those cases are sufficiently clear from the spec text. >> >> >> I have a PDF with hundreds of test cases with red and green boxes for test >> cases the Auto-ISIS implementation has passed or failed. It's currently on >> par with Cisco IOS when it comes to compliance.
A set of web pages, a dedicated effort, and continuous testing, with good tools, would be nice to have for *all* routing protocols and *all* routing protocol daemons. I pointed to the on-going FIB table re-work going on in linux in a previous email. I would be seriously comforted if this level of testing was going on continuously as other people swap out the engines in the airplane in mid-air. For those that are not keeping up with all the innovations going on in linux throughout the stack, a good summary can be had via dave miller's keynote talk at the recent netdev01 conference: talks slides and videos here: https://www.netdev01.org/sessions >> So making the ISIS code "work" in the FOSS sense of "working" was easy, >> making it work according to the standard for all kinds of cases took a lot >> more work. Agreed corner cases are a pita. To give one example of a corner case, recently solved, I hope, in babeld, is the case where an interface comes up or goes down in a matter of us. It first occurred with dnsmasq, and while the fix was trivial, testing for this corner case is a PITA. Some details on that bug are in the two messages on that thread here: http://lists.alioth.debian.org/pipermail/babel-users/2014-October/001775.html I imagine many a network facing daemon has a similar bug, possibly even quagga-isis. >> These are the kinds of tools babel is going to need if we're going to see >> multiple implementations of it from vendors and that they're going to >> interoperate. > > > No, it is *not* going to need that. IS-IS needs that sort of testing because > it is widely used on networks that carry substantial parts of the entire > Internet's traffic, in extremely complex configurations like multi-topology, > traffic engineering, etc. etc. The reliability required of implementations > that carry such a burden is orders of magnitude greater than what is needed > in a home network that in most cases will consist of a few devices that can > easily be rebooted by the operator. You never know where your code will end up and I will always be happier the more testing gets done on every mission critical portion of a system. If you had asked me 20 years ago if the entire future of the world's space program would depend on open source code, I would have been dubious... but as it now *is* - anything that can add another sigma to reliability or capability in our software makes me happier. In addition to spacex's operations, and now the ISS, there are multiple satellite and spacecraft projects that leverage core bits of linux and the networking stack. As one example, on april 8th, this linux based spacecraft flies (I think) as a secondary payload on spacex crs-6: http://space.skyrocket.de/doc_sdat/arkyd-3.htm and I've had many a discussion about what routing protocols to use in a solar system filled with such spacecraft, (prefer dv to link state, obviously) and... ... while eternally grateful that the open source process has led in part to a vast growth in space-capable hardware, it is always possible to do better. So by all means! Please find ways to make all things less buggy and more reliable! I appreciate very much all the work done here, and in hardware, and software, to make our expanding out into the solar system a little bit easier. > Home gateway routers routinely have broken implementations and crash all the > time, and have close to zero testing. What's the point of making the routing > protocol they run be 100 times more reliable? Every little bit towards making systems rock solid reliable counts. My own metric for shipping a stable cerowrt "Cero - spanish word for zero" "Wrt - Warts" - Zero Warts - was a minimum of 3 months of uptime. We (finally) achieved that. http://www.bufferbloat.net/projects/cerowrt/news I am sad that so few others in the home router and cpe market have the same standards for reliability... ...as you have no idea where your code will end up, one day. It pays to try for the lowest bug count and the highest reliability, possible, whenever you can. So bring on better tests and testbeds! > > _______________________________________________ > homenet mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/homenet > -- Dave Täht Let's make wifi fast, less jittery and reliable again! https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb _______________________________________________ homenet mailing list [email protected] https://www.ietf.org/mailman/listinfo/homenet
