Re: [PSES] 3rd party labs due-diligence
To add to previous comments, reviews on various labs could serve as voice of customer feedback mechanism to manufacturers seeking to know more about labs - we've started to do that recently. Having done testing at multiple labs, the experience isn't the same, and being accredited doesn't mean testing will be handled in the same way. There will be good, better and best experiences when one considers the test engineer's proficiency, documentation process or conditions of the lab. Bayo From: Dward dw...@pctestlab.com To: EMC-PSTC@LISTSERV.IEEE.ORG Sent: Friday, August 30, 2013 10:34 PM Subject: Re: [PSES] 3rd party labs due-diligence Well, unfortunately it is too often that some labs just know how to pass an audit, but when it comes to testing, they fall very short. Far too many ‘so called’ good labs rely on instrumentation and simply do not question results. This leads to bad measurements, even though the lab is supposedly accredited. Dennis Ward Senior Certification Engineer PCTEST This communication and its attachments contain information from PCTEST Engineering Laboratory, Inc., and is intended for the exclusive use of the recipient (s) named above. It may contain information that is confidential and/or legally privileged. Any unauthorized use that may compromise that confidentiality via distribution or disclosure is prohibited. Please notify the sender immediately if you receive this communication in error, and delete it from your computer system. Usage of PCTEST email addresses for non-business related activities is strictly prohibited. No warranty is made that the e-mail or attachment(s) are free from computer virus or other defect. Thank you. From:Kevin Robinson [mailto:kevinrobinso...@gmail.com] Sent: Friday, August 30, 2013 6:19 PM To: EMC-PSTC@LISTSERV.IEEE.ORG Subject: Re: [PSES] 3rd party labs due-diligence Full disclosure, I used to work at a test lab, and I now work for a government regulator that audits test labs. In most cases, as Richard said, test labs, except for possibly your small local labs have accreditations from multiple accreditation bodies, all auditing to some variant of 17025/17065/Guide 65. This means that the labs are subject to audits 12 or more times a year. Now each audit may not cover the full scope that the laboratory has, but their core QA systems are generally being reviewed at each audit, and individual sectors (EMC, Safety, Performance etc.) are typically audited several times a year . Their systems are generally pretty tight, and staff have been trained and audited enough to make sure that they are following their own top level procedures for the most part. If you do audit, you might find a deficiency in corporate level procedures, but most will be relatively minor, and are attributed to human error/laziness. I tell people that ask me to visit your lab(s), talk to the engineers and your account reps. Ask them about the standards, perhaps pick a few sections that are confusing to you and ask them to explain them to you. It will become very obvious very quickly how knowledgeable they are in the standards, and if you are new to the world of testing, how well they are able to take a complex standard and break it down into something that you can understand. Take a tour of their facilities. Try to look past all of the glitz and focus on the condition of their storage areas and equipment. If things are neat and orderly, chances are, they will put the same effort into clearly and properly recording your test results. Also consider their corporate culture. Is the lab focused on customer service, clearly explaining things to you every step of the way, or are they extremely efficient and would prefer to just give you a report with little communication throughout the testing process. Find a lab that is a good fit for your personality and your corporate culture. As far as conducting an audit, when you have most labs being audited by ANSI, A2LA, IAS, NIST/NVLAP, FCC, FDA, OSHA/NRTL, Standards Council of Canada, Industry Canada, Verizon, ATT, EPA, IECEE/CB Scheme and many others, chances are, you will not find any significant issues, especially if the lab has been with the same accreditors for many years. Kevin Robinson On Fri, Aug 30, 2013 at 4:39 PM, John Woodgate j...@jmwa.demon.co.uk wrote: In message 63e38a5b081437478c77651f3d56c64f57f4f...@orsmsx102.amr.corp.intel.com, dated Fri, 30 Aug 2013, Pettit, Ghery ghery.pet...@intel.com writes: I've found cases where the documentation was in good shape, indicating that the assessor understood ISO/IEC 17025, but the test setups were wrong, indicating that either the assessor didn't understand the EMC standards Quite often, I believe. In some cases, they defy understanding! or the lab went back to the way they were used to doing the test after the assessor left. Also quite often. In some cases, they may
Re: [PSES] 3rd party labs due-diligence
The 2012 PSES Symposium had a session on the assessment of CABs and labs. But the presenter was influenced by space aliens, so information could have been compromised. Do not think that any of my PPT files made it to the disc, but may be online. Brian -Original Message- From: emc-p...@ieee.org [mailto:emc-p...@ieee.org] On Behalf Of Gartman, W. Richard Sent: Friday, August 30, 2013 11:43 AM To: emc-p...@ieee.org Subject: 3rd party labs due-diligence PSES network, I am looking at do due-diligence assessment of 3rd party labs that we use for EMC, Wireless, and product safety testing. There are the standard lab certifications: IEC17025, A4LA, NVLAP, IECEE CB Scheme, etc. I get many 3rd party reports from labs, that have the proper certification and reference standards in them. What I am looking for is, what beyond these certifications do you look for in a lab to consider it acceptable? Check list, on site/facility reviews? Do you do the due-diligence assessment or is it paper exercise. At what point do you feel sufficient due diligence has been done (acceptable risk tolerance level)? Any comments or perspective are welcome. Thank you. W. Richard Gartman, MS, CSP Product Stewardship Manager Texas Instruments, Education Technology Office: 214-567-7927Email: rgart...@ti.com www.education.ti.com/us/productstewardship www.ti.com/ccr - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com
Re: [PSES] 3rd party labs due-diligence
We require an on-site audit of the lab. 3rd party accreditation bodies aren't perfect. I've found cases where the documentation was in good shape, indicating that the assessor understood ISO/IEC 17025, but the test setups were wrong, indicating that either the assessor didn't understand the EMC standards or the lab went back to the way they were used to doing the test after the assessor left. I have a checklist of common areas of concern in the lab that I use. I can take a 5 minute walk-through and generate enough check-marks to keep me busy at the computer for an hour or two writing the report. Know the standards that apply to your product. Know them in detail. You'll be amazed at what you find in the labs. Ghery S. Pettit -Original Message- From: emc-p...@ieee.org [mailto:emc-p...@ieee.org] On Behalf Of Gartman, W. Richard Sent: Friday, August 30, 2013 11:43 AM To: emc-p...@ieee.org Subject: 3rd party labs due-diligence PSES network, I am looking at do due-diligence assessment of 3rd party labs that we use for EMC, Wireless, and product safety testing. There are the standard lab certifications: IEC17025, A4LA, NVLAP, IECEE CB Scheme, etc. I get many 3rd party reports from labs, that have the proper certification and reference standards in them. What I am looking for is, what beyond these certifications do you look for in a lab to consider it acceptable? Check list, on site/facility reviews? Do you do the due-diligence assessment or is it paper exercise. At what point do you feel sufficient due diligence has been done (acceptable risk tolerance level)? Any comments or perspective are welcome. Thank you. W. Richard Gartman, MS, CSP Product Stewardship Manager Texas Instruments, Education Technology Office: 214-567-7927Email: rgart...@ti.com www.education.ti.com/us/productstewardship www.ti.com/ccr - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com
Re: [PSES] 3rd party labs due-diligence
In message fe37b7a5275ae24195833ce152b33e432e6ed...@dlee08.ent.ti.com, dated Fri, 30 Aug 2013, Gartman, W. Richard rgart...@ti.com writes: What I am looking for is, what beyond these certifications do you look for in a lab to consider it acceptable? I suspect that after seeing all the paperwork and listening to the spiel, you have to ask 'Would I buy a used car from these guys?'. But you can't put that in your due-diligence report, of course. -- OOO - Own Opinions Only. With best wishes. See www.jmwa.demon.co.uk Why is the stapler always empty just when you want it? John Woodgate, J M Woodgate and Associates, Rayleigh, Essex UK - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com
Re: [PSES] 3rd party labs due-diligence
In message 63e38a5b081437478c77651f3d56c64f57f4f...@orsmsx102.amr.corp.intel.com, dated Fri, 30 Aug 2013, Pettit, Ghery ghery.pet...@intel.com writes: I've found cases where the documentation was in good shape, indicating that the assessor understood ISO/IEC 17025, but the test setups were wrong, indicating that either the assessor didn't understand the EMC standards Quite often, I believe. In some cases, they defy understanding! or the lab went back to the way they were used to doing the test after the assessor left. Also quite often. In some cases, they may be right (as well as wrong). But I suppose the OP has enough experience to know about those issues. What I meant by my apparently facetious response is: 'Unless you have full confidence, don't go there'. -- OOO - Own Opinions Only. With best wishes. See www.jmwa.demon.co.uk Why is the stapler always empty just when you want it? John Woodgate, J M Woodgate and Associates, Rayleigh, Essex UK - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.ieee.org/ can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com
Re: [PSES] 3rd party labs due-diligence
Full disclosure, I used to work at a test lab, and I now work for a government regulator that audits test labs. In most cases, as Richard said, test labs, except for possibly your small local labs have accreditations from multiple accreditation bodies, all auditing to some variant of 17025/17065/Guide 65. This means that the labs are subject to audits 12 or more times a year. Now each audit may not cover the full scope that the laboratory has, but their core QA systems are generally being reviewed at each audit, and individual sectors (EMC, Safety, Performance etc.) are typically audited several times a year . Their systems are generally pretty tight, and staff have been trained and audited enough to make sure that they are following their own top level procedures for the most part. If you do audit, you might find a deficiency in corporate level procedures, but most will be relatively minor, and are attributed to human error/laziness. I tell people that ask me to visit your lab(s), talk to the engineers and your account reps. Ask them about the standards, perhaps pick a few sections that are confusing to you and ask them to explain them to you. It will become very obvious very quickly how knowledgeable they are in the standards, and if you are new to the world of testing, how well they are able to take a complex standard and break it down into something that you can understand. Take a tour of their facilities. Try to look past all of the glitz and focus on the condition of their storage areas and equipment. If things are neat and orderly, chances are, they will put the same effort into clearly and properly recording your test results. Also consider their corporate culture. Is the lab focused on customer service, clearly explaining things to you every step of the way, or are they extremely efficient and would prefer to just give you a report with little communication throughout the testing process. Find a lab that is a good fit for your personality and your corporate culture. As far as conducting an audit, when you have most labs being audited by ANSI, A2LA, IAS, NIST/NVLAP, FCC, FDA, OSHA/NRTL, Standards Council of Canada, Industry Canada, Verizon, ATT, EPA, IECEE/CB Scheme and many others, chances are, you will not find any significant issues, especially if the lab has been with the same accreditors for many years. Kevin Robinson On Fri, Aug 30, 2013 at 4:39 PM, John Woodgate j...@jmwa.demon.co.uk wrote: In message 63E38A5B081437478C77651F3D56C**64f57f4f...@orsmsx102.amr.** corp.intel.com63e38a5b081437478c77651f3d56c64f57f4f...@orsmsx102.amr.corp.intel.com, dated Fri, 30 Aug 2013, Pettit, Ghery ghery.pet...@intel.com writes: I've found cases where the documentation was in good shape, indicating that the assessor understood ISO/IEC 17025, but the test setups were wrong, indicating that either the assessor didn't understand the EMC standards Quite often, I believe. In some cases, they defy understanding! or the lab went back to the way they were used to doing the test after the assessor left. Also quite often. In some cases, they may be right (as well as wrong). But I suppose the OP has enough experience to know about those issues. What I meant by my apparently facetious response is: 'Unless you have full confidence, don't go there'. -- OOO - Own Opinions Only. With best wishes. See www.jmwa.demon.co.uk Why is the stapler always empty just when you want it? John Woodgate, J M Woodgate and Associates, Rayleigh, Essex UK - --**--** This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-**pstc.htmlhttp://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE PSES Online Communities site at http://product-compliance.oc.**ieee.org/http://product-compliance.oc.ieee.org/can be used for graphics (in well-used formats), large files, etc. Website: http://www.ieee-pses.org/ Instructions: http://listserv.ieee.org/**request/user-guide.htmlhttp://listserv.ieee.org/request/user-guide.html List rules: http://www.ieee-pses.org/**listrules.htmlhttp://www.ieee-pses.org/listrules.html For help, send mail to the list administrators: Scott Douglas emcp...@radiusnorth.net Mike Cantwell mcantw...@ieee.org For policy questions, send mail to: Jim Bacher: j.bac...@ieee.org David Heald: dhe...@gmail.com - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee-pses.org/emc-pstc.html Attachments are not permitted but the IEEE
Re: [PSES] 3rd party labs due-diligence
Well, unfortunately it is too often that some labs just know how to pass an audit, but when it comes to testing, they fall very short. Far too many 'so called' good labs rely on instrumentation and simply do not question results. This leads to bad measurements, even though the lab is supposedly accredited. Dennis Ward Senior Certification Engineer PCTEST This communication and its attachments contain information from PCTEST Engineering Laboratory, Inc., and is intended for the exclusive use of the recipient (s) named above. It may contain information that is confidential and/or legally privileged. Any unauthorized use that may compromise that confidentiality via distribution or disclosure is prohibited. Please notify the sender immediately if you receive this communication in error, and delete it from your computer system. Usage of PCTEST email addresses for non-business related activities is strictly prohibited. No warranty is made that the e-mail or attachment(s) are free from computer virus or other defect. Thank you. From: Kevin Robinson [mailto:kevinrobinso...@gmail.com] Sent: Friday, August 30, 2013 6:19 PM To: EMC-PSTC@LISTSERV.IEEE.ORG Subject: Re: [PSES] 3rd party labs due-diligence Full disclosure, I used to work at a test lab, and I now work for a government regulator that audits test labs. In most cases, as Richard said, test labs, except for possibly your small local labs have accreditations from multiple accreditation bodies, all auditing to some variant of 17025/17065/Guide 65. This means that the labs are subject to audits 12 or more times a year. Now each audit may not cover the full scope that the laboratory has, but their core QA systems are generally being reviewed at each audit, and individual sectors (EMC, Safety, Performance etc.) are typically audited several times a year . Their systems are generally pretty tight, and staff have been trained and audited enough to make sure that they are following their own top level procedures for the most part. If you do audit, you might find a deficiency in corporate level procedures, but most will be relatively minor, and are attributed to human error/laziness. I tell people that ask me to visit your lab(s), talk to the engineers and your account reps. Ask them about the standards, perhaps pick a few sections that are confusing to you and ask them to explain them to you. It will become very obvious very quickly how knowledgeable they are in the standards, and if you are new to the world of testing, how well they are able to take a complex standard and break it down into something that you can understand. Take a tour of their facilities. Try to look past all of the glitz and focus on the condition of their storage areas and equipment. If things are neat and orderly, chances are, they will put the same effort into clearly and properly recording your test results. Also consider their corporate culture. Is the lab focused on customer service, clearly explaining things to you every step of the way, or are they extremely efficient and would prefer to just give you a report with little communication throughout the testing process. Find a lab that is a good fit for your personality and your corporate culture. As far as conducting an audit, when you have most labs being audited by ANSI, A2LA, IAS, NIST/NVLAP, FCC, FDA, OSHA/NRTL, Standards Council of Canada, Industry Canada, Verizon, ATT, EPA, IECEE/CB Scheme and many others, chances are, you will not find any significant issues, especially if the lab has been with the same accreditors for many years. Kevin Robinson On Fri, Aug 30, 2013 at 4:39 PM, John Woodgate j...@jmwa.demon.co.uk wrote: In message 63e38a5b081437478c77651f3d56c64f57f4f...@orsmsx102.amr.corp.intel.com, dated Fri, 30 Aug 2013, Pettit, Ghery ghery.pet...@intel.com writes: I've found cases where the documentation was in good shape, indicating that the assessor understood ISO/IEC 17025, but the test setups were wrong, indicating that either the assessor didn't understand the EMC standards Quite often, I believe. In some cases, they defy understanding! or the lab went back to the way they were used to doing the test after the assessor left. Also quite often. In some cases, they may be right (as well as wrong). But I suppose the OP has enough experience to know about those issues. What I meant by my apparently facetious response is: 'Unless you have full confidence, don't go there'. -- OOO - Own Opinions Only. With best wishes. See www.jmwa.demon.co.uk Why is the stapler always empty just when you want it? John Woodgate, J M Woodgate and Associates, Rayleigh, Essex UK - This message is from the IEEE Product Safety Engineering Society emc-pstc discussion list. To post a message to the list, send your e-mail to emc-p...@ieee.org All emc-pstc postings are archived and searchable on the web at: http://www.ieee
