Hi there, On Thu, 29 Oct 2009 aCaB wrote:
> On our side we do a lot of QA > ... > I really believe something needs to happen here so that these type of > bugs can be caught quickly before they affect a number of users. > > Thoughts? I suspect that rather than QA, what you do is just a lot of hap-hazard testing. That's why, whenever I see a new release of ClamAV, first I will suppress a groan and then, before I risk it on any of my servers, I'll wait a while and watch the users' list to see how much trouble it causes. This approach serves me well, although I can't say I'm proud of the fact that I'm letting a lot of poor innocents do my acceptance testing for me. To be brutally frank, if I upgraded ClamAV on the basis of the release announcements, ClamAV would in my installations cause a lot more trouble than the things which it's designed to combat. You need to develop a proper QA system. Briefly, that means you need to document exactly how you're going to avoid this sort of chaos, and then you have to do what it says in the documentation that you're going to do. There's a fairly famous transcript of some evidence to a House of Commons committee around here somewhere... ah, here it is: [excerpt] I would like to tell you something that you will not believe but which I think it is important that you hear, and that is that almost every IT supplier in the world today is incompetent. I have worked in the IT industry almost all my working life for large and small organisations, and I know what I speak. For example, the typical rate of delivered faults after full user acceptance testing from the maker suppliers in the industry over many years has been steady at around 20 faults per thousand lines of code. We know how to deliver software with a fault rate that is down around 0.1 faults per thousand lines of code and the industry does not adopt these techniques. We are as an industry very much in the early stages. The industry is only 50 years old. If you compare that with civil engineering, which is several thousand years old, we are tackling some of the most complex engineering designs and building some of the most complex engineering systems that the world has ever seen, essentially using craft technology. If you looked at the methods that are employed in most companies you would come to the conclusion that actually IT system development is a fashion business, not an engineering business, because they jump from one methodology to another year after year so long as it has a whizzy name, "Agile this" or "Intensive that". The underlying engineering disciplines that every mature engineering discipline has learnt it needs to use in order to be able to show that the system it is building has the required properties have not yet been employed in software and systems engineering, and that is at the heart of why these things do not work. [/excerpt] Professor Martyn Thomas, Minutes of Evidence taken before Home Affairs Committee Inquiry into Identity Cards, Tuesday 24 February 2004. http://www.parliament.the-stationery-office.co.uk/pa/cm200304/cmselect/cmhaff/uc130-iv/uc13002.htm -- 73, Ged. _______________________________________________ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
