Derek,
ISO/IEC 17025:2005 is a great document. However, some of the statements you made below are not entirely correct, based on what I’ve seen at a few labs (and not all that far in the past). Much of the problem can be traced to auditors who know 17025 forward and backward, but don’t know the technical standards. We had a problem in the past where CISPR 24 (Edition 1) called out (dated reference) an older version of IEC 61000-4-4 than the latest version. No problem except that the test setup for table top equipment was different. I audited a lab to put them on the Intel approved EMC lab list and caught the error. And they had IEC 61000-4-4 on their Scope of Accreditation. Got that problem (now not a problem with CISPR 24 Edition 2 or CISPR 35) fixed. My first visit to a number of labs in Taiwan (all accredited by NIST NVLAP) showed none of them had masts that could do what is colloquially called “bore sighting”, and yet ANSI C63.4 was on their Scope of Accreditation. When someone said that was a new requirement I agreed, it was new, in 1991! I’ve run into labs in the US that use the wrong version of the basic standards routinely, and when challenged get very defensive about it. And another that was quite creative (and not correct) in their interpretation of what was a proper vertical ground plane for power line conducted emissions testing. All were accredited. I could go on about a few other problems I’ve encountered in accredited labs. My point is that, as you are well aware, accreditation is not a cure all. The assessors are not all EMC experts (as I recall, the first assessor we had at Intel years ago wasn’t even an EE). You, the customer, must know a significant amount about the test standards, especially as they apply to your product, when choosing a lab to test your product. Not all companies are fortunate enough to have such people, but if they don’t they should hire a consultant who does. There are many EMC standards that I don’t claim any expertise in, but those that apply to ITE I’m quite familiar with (being the Chair of CISPR SC I does help 😊). Ghery S. Pettit, iNCE 43 years (and counting) of EMC experience From: DEREK WALTON <[email protected]> Sent: Thursday, April 18, 2019 7:14 AM To: [email protected] Subject: Re: [PSES] Question re: Measuring a signal in a noisy environment Hi Manny, Et al. It’s not my intent to create division amongst followers, I simply want to correct many misstatements. For instance, sentences that begin “Years ago”… Well, years ago we were wire-wrapping boards not using the engineered products we see for the most part nowadays. Upon reflection, the arrival of the CE mark lead to many AdHoc labs that indeed may have been fraught with errors and bad practice: but 25 years on, most of those have “gone away”. Some of the comments made by the authors imply that todays labs are still in those days. That is simply not true. For those of us raised in the MIL STD 461 world, we have been used to set-up checks for some years. And, since the outcome of these are often required in a report, are easy to check. Speaking of checking: A lab that operates to ISO17025 ( as does mine though I’m not accredited ), specifically counter some of the statements made earlier. For example ( and I use the ISO170225:2005 here because the wording is clearer ): Clause 5.4.5.1 Validation is the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled. SO, if a test is on the scope: it means that there is data showing that the way the lab runs the test has been evaluated and documented. The on-site assessment checks those records exist. Further, Clause 5.5.2 Equipment and its software used for testing, calibration and sampling shall be capable of achieving the accuracy required and shall comply with specifications relevant to the tests and/or calibrations concerned. Calibration programmes shall be established for key quantities or values of the instruments where these properties have a significant effect on the results. Before being placed into service, equipment (including that used for sampling) shall be calibrated or checked to establish that it meets the laboratory's specification requirements and complies with the relevant standard specifications. It shall be checked and/or calibrated before use (see 5.6). SO, what this says is that the equipment ( and that includes software ) the lab uses has to be evaluated to ensure its suitable for testing, before its employed for testing. The on-site assessment checks those records exist. Further, Clause 5 .5.3 Equipment shall be operated by authorized personnel. Up-to-date instructions on the use and maintenance of equipment (including any relevant manuals provided by the manufacturer of the equipment) shall be readily available for use by the appropriate laboratory personnel. So, what this means is that the people on the sharp end are trained on their equipment ( not the EUT ) and that records exist for each and every person. The on-site assessment checks those records exist. Further, Clause 5.5.10 When intermediate checks are needed to maintain confidence in the calibration status of the equipment, these checks shall be carried out according to a defined procedure. So what this means is that the labs are directed to evaluate if checks are needed and must have a procedure to follow and keep a record. The on-site assessment checks those records exist. Not only that, but they must document what happens if a problem comes up. Further a whole section is particularly intrusive, but essential, Clause 5.9 Assuring the quality of test and calibration results 5.9.1 The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following: a) regular use of certified reference materials and/or internal quality control using secondary reference materials; b) participation in interlaboratory comparison or proficiency-testing programmes; c) replicate tests or calibrations using the same or different methods; d) retesting or recalibration of retained items; e) correlation of results for different characteristics of an item. NOTE The selected methods should be appropriate for the type and volume of the work undertaken. 5.9.2 Quality control data shall be analysed and, where they are found to be outside pre-defined criteria, planned action shall be taken to correct the problem and to prevent incorrect results from being reported. SO, what this means is that labs already have significant overhead assuring the their data is good. And yes, the on-site assessment checks those records exist. As if this wasn’t enough, the assessing bodies add another layer by having program specific requirements that are a direct result of the field experience of their assessors. By now it’s clear that I have been a lab assessor, for about 22 years to be exact. It’s been a wonderful experience and I haven’t been shot, yet. What I have learned is that there are many very smart people working in labs and they have been kind enough on occasion to point out I’ve been wrong without recrimination: OK, with one exception, who still argues about EN61000-4-6 to this day. What I have also experienced is quite a few so called experts that accompany products, or witness tests, that believe they have knowledge, but have never stepped outside their limited domain. The “general" trait is to talk in Generalisms and recall stories from way back: no need to elaborate on that. Now, I’m not saying that labs are perfect. What I am saying is that some of the opinions expressed here are describing unique situations ( a lot are from “Long ago” and when I hear that, I hear “Once upon a time” ), and do not hold for the current laboratory community: It is a fallacy to think that the statements apply to most modern labs. For instance, I have a line item charge that is specifically for “being audited” by a client or 3rd party: it’s double my normal billing rate because of the annoyance factor: by the same token, if something is uncovered that improves my lab, then I wave all charges. Another for instance, while assessing an OEM, when I asked how to you assure no bias, the told me this: XYZ company has deep pockets, if it could be proven that they shipped non-compliant product the regulators would have a field day with fines and recalls. That particular company performed hundreds of hours of just emissions testing on each variant of its product including testing cold and hot products. OEMs are among the most diligent test bodies. Anyways, work calls, but I stress once again. I didn’t write all this to disagree with folks, or to cause division. I took the time to challenge the notion that the EMC test industry is rife with issues: IT IS NOT! Have a great day Y’all. Derek. On Apr 18, 2019, at 3:50 AM, Manny Barron <[email protected] <mailto:[email protected]> > wrote: Hi Derek, I read Ken Wyatt's e-mail plus Ghery Pettit's Blog link and find them very informative with insightful tidbits of knowledge that could be helpful to the reader who uses 3rd party EMC test labs or even in-house original equipment manufacturer (OEM) test labs. Years ago when I worked for a major OEM, an engineering director requested that I audit some of the EMC test labs being used by 3rd party hardware firms from which we procured hardware that was integrated into a product system that the OEM marketed for sale. The audits were initiated due to the quality of some of the test reports we reviewed. The hardware supplier remained on the list of approved hardware suppliers if the 3rd party lab they used agreed to a requested audit and passed (or deleted from the list if they declined or failed). So Ghery's blog is very useful for the things to be aware of when choosing a test lab or if you're a major OEM with clout, even requesting a lab audit. And the statements in Ken's e-mail are very important points to be aware of for anyone using any EMC test lab or looking into choosing an EMC test lab. Matter of fact, I personally have witnessed all of the deficiencies Ken points out in his e-mail. I like what he wrote and agree with it because I have seen it in the various labs I've worked in or have visited over the years. And if I had any authority or influence over the lab quality system of these labs, the deficiencies I saw were eventually resolved. I believe most labs do a pretty good job, but statistically there will always be deficiencies until an auditor or a knowledgeable person speaks up. You'll probably put me on the same side as Ken and Ghery. But that's okay, I'll be glad to be there. Manny Barron From: DEREK WALTON [ <mailto:[email protected]> mailto:[email protected]] Sent: Wednesday, April 17, 2019 8:47 PM To: <mailto:[email protected]> [email protected] Subject: Re: [PSES] Question re: Measuring a signal in a noisy environment Whoah Ed, you just landed along side Ken and Ghery! I have to strongly oppose some of your views on what the test lab should or shouldn’t Know/do. Wow, have you any idea of the variety of equipment that roles in the lab door, and the complexity of it? NO, it is NOT the labs job to understand the EUT, other than how to interpret the standard so it can be tested. It is totally unreasonable to expect a test person to muse the subtleties of sub micron silicon one day then the ramifications of 10,000 psi hydraulic pumps the next. I doubt few could listening in. Having had the luxury of “visiting" well over 350 labs around the globe over the last 23 years I’ve gathered quite a bit of insight on lab operations and their clients. I happen to have worked in one for 40 years and owned one or two for 30+ years. So much of the email thread is huff and puff. IN the sales world it would be called FAD: fear and disillusionment. Lets set some records straight: The majority of labs do a superb job with all the constraints they are under. An ISO 17025 assessment cannot prevent mistakes, but it does make provisions for just about all foreseeable, and quite a few unforeseeable problems. Everyone can have a bad day: that includes test engineers/technicians. Technology evolves, and what’s true at one point may not be true sometime later. Unless everything is checked to the nth degree, EVERY time, things will escape notice. For all those calling out check after check, I bet under the same breath they are complaining about the cost of testing! And for the record, those critical of overseas test labs should go and pay a visit: most times they are careful to incredible levels running tests. If a manufacture brings a device in for testing, no-one, that’s NO-ONE knows it better than he does. He has a responsibility to understand the testing his device will be subjected to and its behavior, heck, he’s supposed to have designed the device to pass the test! That includes support equipment too! A lab should help someone who’s a novice in the EMI field to avoid pitfalls, that’s professional curtesy. It’s also in the labs interest as they don’t want someone packing up and leaving a few hours into the day and leaving the rest of the day not generating revenue. This topic is huge, and I hear so called experts spout contentious opinions that are ill founded. It’s particularly distasteful especially in a professional form such as this as it forms false opinions in peoples minds, especially as some of these experts have a high platform from which to pontificate. There isn’t a rampant problem in the testing community, and many hardworking technicians, engineers, managers, QA people, Assessors, assessing bodies and even test witnesses would be very aggrieved to hear some of these statements. Testing is a team mentality, not a glass wall mentality. And valid data is the responsibility of BOTH parties. It requires capable people who have a systematic work approach. So unless ALL the full details are known spreading scuttlebutt or opinions of what could have/should have, is NOT helpful. My 10 cents as I have to go back and tend to a test I’ve been running since 6am. Shalom. Derek. On Apr 17, 2019, at 6:21 PM, Edward Price < <mailto:[email protected]> [email protected]> wrote: Derek: When a test lab is engaged to perform testing on a product, the test lab owes the client, at the minimum, accurate test data. However, it is not the “chamber” that delivers that assurance of quality workmanship, rather it is the EMC Engineer in charge of that facility. When the client walks in the door, he expects a knowledgeable testing operation, not an environment that seems to be learning as they go. Not recognizing Ambient noise polluting the Operational emission data is a failure to deliver the expected professional level of competence. To me, it seems pretty clear that the testing lab is at fault. If the problem was rapidly recognized by them, and a new set of data was taken, then the problem is mainly one of embarrassment of the testing lab’s people. However, if it goes on without recognition, if the lab lets the client take his equipment home with the assumption that the equipment has failed, then the longer it goes on, the greater the testing lab’s error and financial responsibility becomes. Perhaps the testing lab never noticed the problem on their own? That would have been a pretty serious indication that the testing lab just wasn’t ready to deliver professional quality testing services. Another interesting comment was that there was difficulty in “setting up” the test specimen and its support equipment. A good test lab will know what a customer needs to supply long before the test date. Time should be budgeted for the setup and trouble-shooting of ancillary devices, and the customer should be warned that home-brew support equipment may cause EMI issues all on their own. A good test lab will make some suggestions about best practices, but alert the customer that it may be necessary to ameliorate support equipment problems before EUT testing can begin, and this may entail additional time and materials needed to make the support equipment quiet or immune enough to allow testing. Further, a good test lab should always expect some set-up problems, and be ready to surmount these with a plentiful junkbox of rolls of aluminum foil, conductive tape, bond straps, shielded boxes, knitted wire mesh, capacitors, inductors, ferrite beads, isolation transformers, sheet metal and a decent assortment of common hand tools. In short, if your customer says his gadget needs an external water chiller, the test lab should have already thought about ground loops, how to get water in and out of the chamber, and have considered what problems a water chiller might induce in the lab’s electrical environment. The test lab’s customer should not feel that the test lab was unprepared to receive him or that the test lab was anything less than expert in integrating the test specimen and support equipment into the test chamber. It would appear that the selected test lab was just not ready for prime time. Ed Price WB6WSN Chula Vista, CA USA From: DEREK WALTON [ <mailto:[email protected]> mailto:[email protected]] Sent: Wednesday, April 17, 2019 12:08 PM To: <mailto:[email protected]> [email protected] Subject: Re: [PSES] Question re: Measuring a signal in a noisy environment Interesting discussion, not surprising I have a little empathy, and a whole slew of disagreement with both Ken and Ghery ( both Chaps I have a lot of respect for ) on this one. How best to respond is the question? Cheers, Derek Walton. On Apr 17, 2019, at 1:12 PM, Bill Stumpf < <mailto:[email protected]> [email protected]> wrote: Absolutely agree with Ken. Bill Stumpf Lab/Technical Manager D.L.S. Elecronic Systems, Inc. From: Kenneth Wyatt [ <mailto:[email protected]> mailto:[email protected]] Sent: Wednesday, April 17, 2019 12:21 PM To: <mailto:[email protected]> [email protected] Subject: Re: [PSES] Question re: Measuring a signal in a noisy environment Boy, I have heard all sorts of horror stories from clients with knowledgeable EMC backgrounds who have witnessed sloppy, or just plain wrong, procedures used in commercial EMC test labs. In a lot of cases, the technicians operating these tests have limited background in EM theory and poor knowledge of the actual tests they are running and standards the tests are based on. Just because a test lab is assessed per IEC 17025, doesn’t mean much unless they show evidence the documented procedures are actually being followed. Many test labs fail to perform frequent verification tests to confirm the measurement system is accurate and is repeatable from one day to the next. When I worked for HP, we did a daily verification test using an RF generator connected to the antenna cable to ensure the back-end system measured the same as the day before. We also ran comb generator tests frequently. I always suggest to my clients to make their preferred test lab measure a client-owned comb generator prior to any testing in order to ensure the chamber continues to be reasonably consistent before real measurements are taken. It’s also very important to understand the test standards and EUT configurations well enough to ensure the test technician is setting up things correctly. I know of one case where the EUT cabling was configured wrong and their client repeatedly had emissions failures over weeks of retesting until the correct configuration was pointed out in the standard. Taking photos of the test setup is very important for day to day test consistency. A difference in one cable position can completely throw off repeatability and thus, mislead any troubleshooting efforts. What about ESD simulators? When was it verified last? Does the test lab even have the means to verify the correct tip voltage and pulse characteristics? Is all the measurement equipment calibrated and cal tags current? I could go on… My colleague, Ghery Pettit wrote a recent blog on the subject for Interference Technology: <https://interferencetechnology.com/emc-laboratory-selection-audit-items/> https://interferencetechnology.com/emc-laboratory-selection-audit-items/ Cheers, Ken _______________________ I'm here to help you succeed! Feel free to call or email with any questions related to EMC or EMI troubleshooting - at no obligation. I'm always happy to help! Kenneth Wyatt Wyatt Technical Services LLC 56 Aspen Dr. Woodland Park, CO 80863 Phone: (719) 310-5418 <http://www.emc-seminars.com/> Web Site | <https://design-4-emc.com/> Blog <https://www.edn.com/electronics-blogs/4376432/The-EMC-Blog> The EMC Blog (EDN) <http://www.emc-seminars.com/Newsletter/Newsletter.html> Subscribe to Newsletter <https://www.linkedin.com/in/kennethwyatt/> Connect with me on LinkedIn On Apr 17, 2019, at 10:46 AM, Grasso, Charles < <mailto:[email protected]> [email protected]> wrote: To those wondering what the background and conclusion to that question was: Background: Our testing was performed in a newly minted chamber so proper EMC installation of our product was challenging. Effect: Ingress of high levels of external bb noise. Result: Proper install eliminated the external noise and now the system passed. Concern: While I accept that proper installation and operation of our system is our responsibility, I had expected that the tester would point to the excessive ambient and indicate that our data may not be valid. An inexperienced customer would have left thinking that their product had failed. Am I wrong? Thanks! Charles Grasso W: 303-706-5467 - - ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to < <mailto:[email protected]> [email protected]> All emc-pstc postings are archived and searchable on the web at: <http://www.ieee-pses.org/emc-pstc.html> http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at <http://product-compliance.oc.ieee.org/> http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: <http://www.ieee-pses.org/> http://www.ieee-pses.org/ Instructions: <http://www.ieee-pses.org/list.html> http://www.ieee-pses.org/list.html (including how to unsubscribe) List rules: <http://www.ieee-pses.org/listrules.html> http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas < <mailto:[email protected]> [email protected]> Mike Cantwell < <mailto:[email protected]> [email protected]> For policy questions, send mail to: Jim Bacher < <mailto:[email protected]> [email protected]> David Heald < <mailto:[email protected]> [email protected]> - ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to < <mailto:[email protected]> [email protected]> All emc-pstc postings are archived and searchable on the web at: <http://www.ieee-pses.org/emc-pstc.html> http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at <http://product-compliance.oc.ieee.org/> http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: <http://www.ieee-pses.org/> http://www.ieee-pses.org/ Instructions: <http://www.ieee-pses.org/list.html> http://www.ieee-pses.org/list.html (including how to unsubscribe) List rules: <http://www.ieee-pses.org/listrules.html> http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas < <mailto:[email protected]> [email protected]> Mike Cantwell < <mailto:[email protected]> [email protected]> For policy questions, send mail to: Jim Bacher < <mailto:[email protected]> [email protected]> David Heald < <mailto:[email protected]> [email protected]> - ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to <[email protected] <mailto:[email protected]> > All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://www.ieee-pses.org/list.html (including how to unsubscribe) <http://www.ieee-pses.org/list.html> List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas <[email protected] <mailto:[email protected]> > Mike Cantwell <[email protected] <mailto:[email protected]> > For policy questions, send mail to: Jim Bacher <[email protected] <mailto:[email protected]> > David Heald <[email protected] <mailto:[email protected]> > - ---------------------------------------------------------------- This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to <[email protected]> All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://www.ieee-pses.org/list.html (including how to unsubscribe) List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas <[email protected]> Mike Cantwell <[email protected]> For policy questions, send mail to: Jim Bacher: <[email protected]> David Heald: <[email protected]>
