Derek, I think I agree with you more than my last message may seem to indicate. This topic has, in the past, been a somewhat sore subject for me, for several reasons. I can't speak for the software that you have written or support. However, I have had issues with a couple of COTS packages (one in particular) that shall remain nameless, and I am sure it is not the one you support. :-) And, I can only speak for the auditors that I have had experience with. Some are better than others. Yes, they do want evidence that the measurements are accurate, regardless of COTS or in-house. But, in my experience with software I have written, "in-house" waves all sorts of red flags, WHICH IT SHOULD, but my point was that COTS software does not seem to draw the attention from the auditors that it should, at least that is my opinion from the experiences that I have had. This problem does not exist, to this extent, in the FDA world, everything must be validated to the nth degree. I went to work for a lab that was using COTS software that was installed, setup and tested by the software vendor, and the lab assumed all was well. When I started working there, I found multiple problems with the setup files and instrument drivers. Now, I know that this experience is limited to just one software package and is probably worse-case scenario, but it is this worst case that I base my argument on. (This was an internal, mostly pre-compliance lab). I agree 100% with your point about some standards being vague. The MIL standards and automotive standards are (mostly) better in this respect. Having written software for both, I can say it is much better to write software to clear specifications. Vague specs allows a programmer to take shortcuts. The labs can also take shortcuts when it comes to test procedures, but that is another topic. Your points 4 - 7 I can agree with, with regards to your own experience and practices. Again, I speak from my experience with vendors, one in particular. I apologize for sounding like I was making a blanket statement about all vendors. The point about letting engineers be engineers instead of programmers is very well taken. I always say "beware of an engineer with a compiler". Would I recommend a lab write their own software? No. If I were running a lab, I would buy COTS software, but I would also be extremely picky about what I purchase, and I would thoroughly test it for flaws. I will say, however, that I learned a LOT about EMC (standards, test methods, instrumentation) by writing software for it. And, it was all flexible enough to change instruments without having to recompile. (Some things we learn the hard way!!!) :-) Great discussion! Cheers, Bob Richards, NCT.
[email protected] wrote: Ahh, Bob, seems we disagree on this one.. Point 1 Both the software I install, and our biggest competitor are installed by chaps with many, many years of practical experience. In my case 26 years, and I'm sure my competitor is just as much... Point 2 As a Lab Assessor myself, I take issue also wit that point also. We ( the group of us to chat you know... ), require users of software to show that the results are as expected. What we don't ask for is a line by line validation of the code. With 100+ installations of our SW, each one validated to the extreme by very very picky clients ( and trust me, they find errors!! ) means the code is much more reliable than any assessor could ever hope to check during an assessment, or any individual could achieve. Why make a lab pay for this pointless act. What we check is that they know how to use it. Now, if I were assessing a lab with non-COTS software, I would expect to see evidence of every aspect of the program. Point 3 As I've stressed, the standards are so poorly written that there are interpretations of how the test should be run. A good software package has the flexibility to enable a user to have it do what THEY want, so they can stand behind their approach. Goes back to point 2, the lab has ultimate responsibility, BUT the software does what it's supposed to which is what the lab want. Point 4 Instrument drivers are key: but all COTS SW drivers are different, and are certainly not interchangeable. And I beg to differ that drivers are written with Rented equipment: you should talk to our insurers that pay out for equipment damaged in transit! Further, in the case of my software, the drivers are tied to the model and SERIAL number of the instrument. So unless it is a deliberate action, instruments can't be swapped out. On top of this, with a team of 6 people world wide ( collectively 100 years of instrument control ), with experience on almost every instrument made, we know a few tricks.... An internal test lab ( with few exceptions ) just cannot call on that wealth of knowledge. Point 5 Good software should work with ANY instrument. After all, if it's written for dedicated equipment that suffers prolonged damage, the lab would be off line until the WHOLE process is done again..... What a serious financial and chronological hit to a laboratory! Point 6 While experience is beneficial, after all I boast about it above, past experience IS NO indicator of future performance. After all, we all ( well, most of us ) rush out and upgrade to the newest version of windows or Office when it comes out, because the new features are typically over come the resentment we hold cos' the darn software crashed on us many times. Point 7 Having written my own EMC test software 20 years ago, and having COTS now, it's a no brainer. As a lab manager, I'd rather my EMC techs and engineers be working in their field rather than trying to be programmers... Yes, I'm biased, but I'm also battered and bruised from MANY MANY years of testing. Right, back to the possible reasons that Manual and Automated testing may differ. Cheers, Derek Walton L F Research ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. Website: http://www.ieee-pses.org/ To post a message to the list, send your e-mail to [email protected] Instructions: http://listserv.ieee.org/listserv/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas [email protected] Mike Cantwell [email protected] For policy questions, send mail to: Richard Nute: [email protected] Jim Bacher: [email protected] All emc-pstc postings are archived and searchable on the web at: http://www.ieeecommunities.org/emc-pstc ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. Website: http://www.ieee-pses.org/ To post a message to the list, send your e-mail to [email protected] Instructions: http://listserv.ieee.org/listserv/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas [email protected] Mike Cantwell [email protected] For policy questions, send mail to: Richard Nute: [email protected] Jim Bacher: [email protected] All emc-pstc postings are archived and searchable on the web at: http://www.ieeecommunities.org/emc-pstc
