> But what metric is used on the space of cost/risk/benefit? Of course, > ultimate responsibility of the corporation to its owners means that > cost/risk/benefit are all judged in terms of fiscal income. The argument > that this is bad or good is irrelevant, but this is the standard against > which those terms are judged. The point of the interview I posted was that > bug-fixing is low benefit, high cost using that metric. For me, the high > cost is mistakes controlling equipment which for one of the long standing > bugs (which is high cost, low return) is that valves get modified on actual > equipment at times the operator did not intend to actuate things. This > involved a rather kludgy chunk of code to get around, and then it is high > cost (to me) to maintain, since explaining this obscure code takes > time/effort/cost. This end user cost needs to be translated to cost to NI. >
This information about causing a user to unintentionally modify a valve is exactly the type of information that I was talking about at the bottom of my post. The metric isn't a numerically accurate technique, but the bug reports are classified as product liability, crash, internal error, incorrect behavior, cosmetic, and suggestion. This is often something of a guess until the bug can be reproduced, and even then, it is a fuzzy classification since under the right circumstances, anything could be elevated to a product liability. So, this is where the human element comes into play. You look at the typical result of this bug on the Average user. If many users will be affected, you may round up. If the user has to stand on their head to even find the feature that the bug is in, then it is rounded down. It is common for the values to wiggle around as a bug is investigated, and one possibility is that a bug gets deferred to be fixed in a later release. Before release, the deferred bug reports get reviewed by some pretty picky members of R&D management, and again, a balance is struck between delaying the schedule and shipping with known bug reports. I understand that nobody likes being a squeaky wheel, especially you, but giving a good explanation so that the right triage choices are made is once again the best advice I can give. To try and draw an analogy, designing defect-free software is hard and expensive, sort of like designing rockets or planes that never fail. And even with big budgets and big brains, NASA and the other space agencies around the world aren't even able to achieve that goal. What we can do is set up a system that makes sure that bug fixing time is as effective as possible, then set up a cost system that encourages the fixing of bugs and innovative products. That system includes forums like this, web sites that list pesky bugs that continue to get deferred, etc. Like I said, this is a difficult conversation to have over email. If you have specific questions, I'll try to answer them, and if you make it to NIWeek again, we can have a better medium for philosophical discussions. Greg McKaskle
