>>>>> On Sun, 26 Dec 2010 08:42:14 -0500, Bill Fenner <fen...@gmail.com> said:
BF> One proposal for an addition could be regressions from the previous BF> major release. I upgraded our system from 5.5 to 5.6 and found at BF> least 3 regressions (tcpConnTable, replies on multihomed hosts (i.e., BF> routers) not using the routing table, now this.) If regressions are BF> not treated seriously, it makes it hard for those of us who ship BF> net-snmp in our products to want to try new versions. Well... this has a been a long standing subject of debate. There are some tricky issues involved with regression tests when it comes to both the suite: 1) We've only recently introduced a better test suite that can be used for testing application specific code. Though this exists, there aren't a huge number of tests written for it yet. This is slowly improving, but isn't complete. 2) Writing tests for MIB object support is difficult, though technically possible. Checking that a particular object, for example, reports the right value is often difficult or impossible to code. IE, the way you'd test a value is to retrieve the value using the exact same method that the code does, which obviously will produce a match even if it's wrong. 3) What makes this all truly close to impossible is that testing things on one platform (say, linux, which is what many developers use) would actually produce useful results but we'd still fail to catch breakage on other systems. We try and support a huge number of platforms, and each platform has different support and would require different tests and ... it gets messy. 4) To try and combat #3, we make lots of pre-releases and release candidates in an effort to attract people toward testing it and reporting critical problems. However, many years of experience have shown that most people don't generally test pre-releases. Some do, and we've caught some great errors that way but they're rarely at the deep MIB level. Now, having sounded pessimistic, I do think we can improve upon what we have now. It would most likely involve: 1) writing more tests [being done] 2) writing a report-a-bug script, which we've talked about too (but haven't done). 3) getting more people to help test things prior to release 4) fix the sourceforge bug tracker so it was more usable (and fast) :-( 5) Find a bank of test machines to use. We actually had one for a while when sourceforge had their "compile farm", but unfortunately they turned that off a few years ago. I think we'd all love to hear other options as well. -- Wes Hardaker Please mail all replies to net-snmp-coders@lists.sourceforge.net ------------------------------------------------------------------------------ Learn how Oracle Real Application Clusters (RAC) One Node allows customers to consolidate database storage, standardize their database environment, and, should the need arise, upgrade to a full multi-node Oracle RAC database without downtime or disruption http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ Net-snmp-coders mailing list Net-snmp-coders@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
