Kepa, "CERTIFICATION" Service is great.As I understand it makes sure that "each and every type of scenario" file is submitted for testing.After the user has got 0 errors in all lines of business in that file then it is certified.
Testing as defined by other vendors is a process ( manual / semi auomated) and you have packaged/automated that "entire process" as certification. Certification uses testing. But as far as "Testing" is concerned >That is >just fine, as there are many different approaches to putting a trading >partner into production. We believe out test tools are better than our >competitors', but I am sure they believe theirs are better. But testing is >only part of the picture. Why does the industry not get this part of the picture clear and have an unambiguous definition of a standard and then next Testing ? A vendor consortium may help to lay down all the rules of Testing a standard. Common interpretation of IGs is must. Given a hipaa file (A single business unit : a claim) all vendors must report common /same test result for that file. > So as far as testing is concerned, Patrice showed that the three main > contenders can test the files with some differences. However, had we tested > a different file, a more moderately sized file for example, we would had seen > a number of significant differences in the quality and completeness of the > results. If these reports differ at reporting "the HIPAA Errors, with the standard 7 types defined in the SNIP white paper" then This is alarming ! Thank You, Mishra. ----- Original Message ----- From: "Kepa Zubeldia" <[EMAIL PROTECTED]> To: "WEDI SNIP Testing Subworkgroup List" <[EMAIL PROTECTED]> Sent: Sunday, November 24, 2002 3:01 PM Subject: Re: VALIDATION or Certification > Rama, > > I agree with you that the work of comparing results between the three largest > testing vendors was very much needed. Patrice and Mike are to be commended > for all the work they put into this, and I hope there will be more extensive > studies in the future. > > You bring up some great points. And since you invited me to address them, > here it goes... > > First, in spite of the title of the session being "Validation or > Certification", the specific instructions from Patrice to all three vendors > were that we were ONLY to discuss testing. We were not supposed to discuss > certification, only testing. I specifically asked Patrice about this because > it seemed odd, given the title of the session and she specifically forbade me > to discuss certification or HIPAAmetrics. My presentation was to be ONLY > about testing outbound transactions using Velocedi. > > Since I was the first presenter, I stuck to the script. The ONLY thing we > were supposed to do in our 10 minutes each was to test/analyze/validate the > one file that Patrice had provided us with, and no other. No other > discussion or presentation was to take place, per Patrice's instructions. > Much to my dismay, the other two presenters chose to present their entire > product lines rather than sticking to the rules. Patrice, given the > situation, passed me a note inviting me to take a second turn of 5 minutes to > make a "sales pitch", but I declined. > > Had I know that I could present not only about outbound testing but also about > certification and inbound testing and inbound certification, I would have > used my 10 minutes very differently. > > So, given that the only topic was testing/analysis/validation, you can > probably understand why when you brought the topic of the difference between > testing and certification from the audience, Patrice diverted your question > and never addressed the certification part of it. She was not interested in > discussing certification in this meeting, only testing. I hope this email > addresses your question. > > Comparing the offering of these three vendors by only looking at the results > of testing outbound EDI files is equivalent to comparing automobiles by only > looking at their tires. Yes, they do all have four tires on the road. Yes, > all tires are made out of rubber, steel, plastic, cotton and other fibers. > Yet, some tires will separate the thread under certain conditions and other > tires will work better in the snow or may have studs for ice. But, would you > buy a car based on the tires? There are many other aspects that are much > more important than the tires. The Minnesota case study only looked at the > single aspect of outbound transactions test results, with the explicit > exclusion of other important aspects, such as certification or inbound > testing. > > So as far as testing is concerned, Patrice showed that the three main > contenders can test the files with some differences. However, had we tested > a different file, a more moderately sized file for example, we would had seen > a number of significant differences in the quality and completeness of the > results. This is left for a future exercise. For example, I tested a > medium size file with 3,500 professional claims and it took 133 seconds > (2m13s) using the Foresight software but only 3 seconds using Claredi's > Velocedi, using exactly the same hardware. This sort of differences are not > visible when the test files are as small as the file Mike and Patrice gave > us. I will stop here, as I don't want to start benchmarking discussions. > > As for your request for clarification about the disclaimer in the test > report... The test report is exactly that, a TEST report. It is NOT > certification. For almost two years now I have been making a big distinction > between testing and certification. Some people understand the difference, > others don't understand the difference. I think it is clearly expressed in > the white paper, but obviously it is not as clear as I think it is, or this > topic would not come up so often. I will try to explain it in this email, > but I suspect it is going to need a more detailed description, so I am going > to start working on a Claredi white paper to make sure people understand what > WE mean by certification. I am aware that most of our competitors do NOT > understand the difference and therefore try to equate the two concepts as if > they were one and the same. In their eyes, if you pass a test without > errors, that equates to being certified HIPAA compliant. I believe that is > not the case. Please keep reading. > > When Patrice approached Claredi about the case study, she was very explicit > about wanting to show only outbound test results. She did NOT want to show > inbound testing (with Claredi's Wibbler) and she did not want to show > certification in either direction. So we gave her a temporary test account > of the same kind we offer in our "trial run" accounts. Those accounts are > established with only a limited set of capabilities and do NOT have access to > Claredi's certification, they have access only to testing tools. In a > "normal" (that includes certification) account, the report summary that > contains the disclaimer you are citing also contains a few more sections > between the "Action" and the "Support" areas of the summary. These other > sections are labeled "Certification", "HIPAAmetrics", and "File Specific > Info". Again, these sections are only available to Claredi customers, not to > test accounts. Also, test accounts are restricted in other ways. I can give > you the restriction details off-line. > > Since I don't think it would be appropriate to turn this email into a sales > pitch, I will only describe the outbound "Certification" process. We also > have a completely different process for inbound certification, and I could > describe it another day if there is enough interest. > > Once a file has passed the Claredi testing/verification/validation (all > synonyms) process without HIPAA errors, it becomes eligible for certification > by Claredi. The "Certification" section of the summary page then shows a > button labeled "Submit this file for certification". If the file has HIPAA > errors the button does not appear. Unless the user clicks this button, the > file is not taken through Claredi's certification process. It is only tested, > not certified. This is why the summary report MUST NOT be construed as a > certification report, even if it shows that a file does not have "HIPAA > errors". And thus the disclaimer. > > As you can tell, the summary report shows three types of "errors": the "HIPAA > Errors", with the standard 7 types defined in the SNIP white paper, the > "Business Errors" section, and the "Warnings" section. Only the "HIPAA > Errors" section must be clean for the file to be a candidate for > certification as HIPAA compliant. The file could have "Business Errors" and > "Warnings" and still be certified as HIPAA compliant, since the certification > of compliance measures against the requirements of the HIPAA implementation > guides. > > In the next few weeks we will be relaxing the cleanliness requirement even > further. For a file to qualify as a candidate for certification, it will > need to have at least one "business unit" (e.g. claim) without "HIPAA Errors" > rather than the entire file being error free. The HIPAAmetrics certification > will give a detail of the contents of the file, including the percentage of > compliant and non-compliant claims. This is currently in "beta" with a > restricted number of accounts and will appear in all the Claredi accounts in > a few weeks, as well as in the certification details through the directory. > > But, back to the certification process... Once the file passes the testing > clean enough to be "certifiable", and the user clicks on the "Submit for > certification" button, Claredi takes the file through a process that looks at > the data content. Based on the data content of the file we can identify > patterns, such as (I will use the claim as an example) the presence of > Billing Provider and/or Pay-To provider, or the presence of both Subscriber > and Patient or only Subscriber, or the presence of secondary coverage, or the > presence of a prior adjudication (COB claims), or the presence of a referring > physician, or the maximum number of service lines submitted by this entity, > or the presence of a "cluster of data" that indicates this is an > anesthesiology claim, or a DME claim, or an ambulance claim, etc. > > We identify these data clusters through a process we call HIPAAmetrics. This > is a unique Claredi pattern recognition process that has never before been > applied to EDI data, and we have applied for a patent to cover it. The > result of the process gives us the "metrics" of this particular file. I am > sure you have heard me say many times that the fact that a file does not > contain errors is not enough. You have to know what was actually contained > in the file. > > Using HIPAAmetrics we can determine exactly what is in the file, from the > business perspective. The testing phase looks at the EDI perspective and > determines that the file (or the individual claim within the file) does not > contain errors. The HIPAAmetrics looks at the healthcare contents and > "characterizes" it. At the completion of the HIPAAmetrics phase Claredi can > say: > We have seen a file that contained the following data > - Office visits, 234 without errors, 3 with errors > - Primary claims, 180 without errors, 20 with errors > - Primary claims with a second payer, 0 without errors, 20 with errors > - Secondary claims, 0 without errors, 97 with errors > - Consultations, 5 without errors, 40 with errors > - etc. > Claredi is making a statement of what we have seen. Because there are some > claims that are "compliant", the provider could claim to be compliant. > However, the statement from Claredi clearly shows that although this provider > has the ability to generate HIPAA compliant office visits for "single payer" > situations, the provider does not have the capability to create correct > claims when two payers are involved. > > The HIPAAmetrics describe the profile that we have seen reflected in the EDI > data itself. The HIPAAmetrics certification gives the detailed data profile > necessary to make a determination of compliance. Without this sort of > detail, all you would know is that the provider has generated something that > did not have errors, but you don't know what that "something" was. Certainly > none of the other testing entities (neither our competitors nor the > translator validators) provide these details, and I suspect that they do not > yet understand the difference between passing a "clean test" and being > certified as HIPAA compliant. Both EDIFECS and Foresight show a pass/fail > approach in their reports. Of course, by the end of this email their eyes > will be wide open. > > So, there is a significant difference between a testing or validation service > and Claredi's certification. Big difference. The problem is that if you > don't understand the difference and assume that any entity that can produce > an error free file is automatically "certified" you will certainly find a > real problem when you look at the quality of the data that entity sends. And > thus the bad rap that the "pseudo-certification" provided by the pass/fail > approach of our competitors is giving to the word "certification". > > And this is also the reason why a vendor cannot really be certified unless all > of its clients are individually certified. It is just too easy for the > vendor to produce certain clean transactions, and for the clients to not be > able to produce the transactions THEY need in a compliant form. Ditto for > clearinghouses. > > From the usability side, a clean test that does not have the HIPAAmetrics type > of data analysis, still needs to be reviewed by an expert to determine > whether the test is sufficient in volume and quality, and to determine what > is it that was actually tested. Did the test include sufficient variety of > business scenarios to make it relevant? Were those scenarios complete? Is > the volume of the sample for each scenario big enough? These are some of the > sort of things the EDI folks at the payer's end have been looking in > provider's tests. Using an EDI test tool may tell you that the EDI data does > not contain errors, but that is only a small part of the answer. Most > translators can do that too, so, if that is all the EDI test tool does, it is > not much better than the translator rules. > > The Claredi HIPAAmetrics answer these questions. The payer can then define > what are the required parameters. Things as simple as saying "we need to see > 95% clean claims in a test containing at least 200 claims" can be answered by > HIPAAmetrics and the Claredi certification, and cannot be answered by any of > the other test tools in the market today, without human intervention. > > I hope this email sheds some more light on these concepts. I have been > talking about this for almost two years. It is (washed out) in the SNIP > white paper. And yet, the concept is still a mystery to most, even a lot of > experts. The reality is that it will take a while to grasp the concept. > Those of us that have done extensive testing of healthcare EDI before HIPAA > (NSF, UB92, etc.) understand the concept easily. For some reason the > traditional non-healthcare EDI community (X12) has more difficulty > understanding the concept. > > Now let me add another important difference between the "test results" and the > HIPAAmetrics. The test results, by their nature, will contain PHI. If there > are any errors, the PHI will most likely be part of the error message or > description. Claredi's test results also contain PHI. So we have a bold > statement on the test results report saying that they must be handled > according to the appropriate privacy and security procedures. And this in > spite or requiring a Business Associate Agreement with all our customers. > The HIPAAmetrics reports, on the other hand, do NOT contain PHI and thus are > shareable with the world. In fact, in a few weeks we will be releasing the > HIPAAmetrics results as part of the Claredi directory describing the details > of each certification obtained through Claredi. > > There is a concept of "community" testing that is being touted lately. The > problem that I see is that if the "community" is comprised by more than one > payer (e.g. several payers sharing the results of all the providers in the > community) then all the payers in the "community" are seeing the PHI from all > the testers. Unless the provider creates a separate test account for each > one of the payers in the "community" and then sends the appropriate test > files to each one of those accounts, so each of the payers will only see the > test files that corresponds to that payer, there is a problem... All the > payers in the "community" will be seeing (through the "community" testing > center) the PHI that belongs to the other payers. As far as I can tell this > is not kosher for healthcare privacy reasons. Even if the provider has a > "business associate agreement" with the community test center, and a BAA with > each one of the payers, and each payer has a BAA with each one of the payers > in the "community", I still think that the "minimum necessary" rules would > preclude this sort of bulleting board sharing of PHI within the testing > "community". > > Yes, you guessed right. If instead of sharing the test results, like these > "communities" do today, they were to share the HIPAAmetrics certification, > the PHI would never be shared. > > Some people have seen the vision. We have offered it to other testing vendors > and translator vendors. Some have expressed interest, others are not > interested. Clearly we need to do a lot more education on this topic. The > American Hospital Association understood the concept and is now endorsing > Claredi as the exclusive HIPAA transactions testing and certification service > to their members. > > So, going back to Patrice's case study, you cannot compare testing and > certification if the only thing you look at is testing. Testing your own > transactions is important. Some people will focus only on testing. That is > just fine, as there are many different approaches to putting a trading > partner into production. We believe out test tools are better than our > competitors', but I am sure they believe theirs are better. But testing is > only part of the picture. To establish a business relationship for HIPAA > transactions, you must go beyond compliance testing. Knowing that the data > you receive is clean is important, but you must also know what is contained > (from the business perspective) in that clean data stream. Only Claredi > gives you an automated way to determine that. We call it HIPAAmetrics > Certification. For the last 20 years we have done without it, and we have > been happy with just test tools. Some people are just happy with the current > tools, other people want to explore a more automated way to put a trading > partner into production. This country was built on freedom of choice. No > need to put someone else down just because you don't like what they have to > offer. > > If you want to see it in action, give me a call and I can show it to you. It > is really simple... > > Kepa > > > > On Saturday 23 November 2002 12:54 pm, [EMAIL PROTECTED] wrote: > > For the benefit of those who could not make it to the WEDI SNIP HIPAA > IMPLEMENTATION SUMMIT held in Phoenix from Nov 18 - 21, 2002. > > > > There was a session on Nov 20th between 9.45 AM - 11.45 AM on the topic > "Validation or Certification", moderated by Patrice Thaler of Allina Health > System and Mike Ubl of Blue Cross of Minnesota and the main participants > being three vendors of testing tools represented by their big chiefs... > Namely Sunny Singh from EDIFECS, ED Hafner from Foresight Corp, Kepa Zubeldia > from Claredi, presenting a case study of three vendors. > > > > First of my sincere thanks to Patrice, Mike and whoever helped in this case > study, besides the vendors. > > > > Being the author of the vendor-neutral statement in the Testing and > Certification White Paper (Hey John, why is my name missing in the > acknowledgements section? ;-)), and being the first (one of the?) to suggest > the formation of Vendor (of testing tools AND TRANSLATORS) Consortium in one > of the teleconference calls in May/June of 2002, I am really glad that this > case study was done. It is the first step in that direction of forming the > vendor consortium to help the CEs be able to pick and choose any one of them > vendors, for they ALL have to offer the same basic VALIDATION (not > certification) capability by coming up with the same ERROR messages and > similar warning messages for the same test files. This case study proved > just that point and am truly delighted and am happy to share it with one and > all. > > > > One more interesting thing and I would like Kepa Z to clarify for all of > us... > > > > On page 3(of the handout we were given there), on a "Claredi - Passed > Claim", there is a heavy disclaimer in BOLD, which reads "This is not a > certification or verification of HIPAA compliance Summary of test report for > confidential use by Claredi customer only". I know and agree that it is NOT > CERTIFICATION. I can also understand why the disclaimer is there. But Kepa > should explain to us why he touts Certification with an evangelical pitch and > then this disclaimer... saying it is NOT CERTIFICATION! > > > > Another important point, as suggested by Larry Watkins, as a member of the > X12N committee (correct me Larry if I am not saying this right), that the > discrepancies in the standards are not that many and there is a very good > process to address them. I can't agree more. > > > > The bottom line is, it is Testing, Verification and Validation and NOT > CERTIFICATION. End of discussion. I have been saying this all along and one > more time won't hurt ;-). That said, I really do believe all the vendors, > have a lot to offer in the validation part of transaction testing. I have no > bias or preference towards one vendor. The CEs (or their testing people) > should make the choice. Don't let anyone fool you that it is soooooooo big > and only I (my org) can lead you(r org) to salvation ;-). > > > > Accreditation, creating entities to certify and the process of certification > is a long one and can be done, but vendor consortium will eliminate the need > for certification all together, IMHO. > > > > This message is not intended to be against any individual or a flame and > rake up any unnecessary things but to share the joy of possible formation of > vendor consortium and also request the translator vendors to join in the > vendor consortium. This will really help in reducing the chaos and eliminate > the obfuscation and confusion that is there and being created... We don't > have time and anybody who can chip in to make things simple to whatever > extent should be commended... like Patrice and Mike and all those who > participated in the case study. In Patrice's words, paraphrased, "It is easy > and you can do it, and I have done it". > > > > Thanks for reading up to this point. > > > > Best Regards, --Rama. > > > > --- > > The WEDI SNIP listserv to which you are subscribed is not moderated. The > discussions on this listserv therefore represent the views of the individual > participants, and do not necessarily represent the views of the WEDI Board of > Directors nor WEDI SNIP. If you wish to receive an official opinion, post > your question to the WEDI SNIP Issues Database at > http://snip.wedi.org/tracking/. These listservs should not be used for > commercial marketing purposes or discussion of specific vendor products and > services. They also are not intended to be used as a forum for personal > disagreements or unprofessional communication at any time. > > > > You are currently subscribed to wedi-testing as: [EMAIL PROTECTED] > > To unsubscribe from this list, go to the Subscribe/Unsubscribe form at > http://subscribe.wedi.org or send a blank email to > [EMAIL PROTECTED] > > If you need to unsubscribe but your current email address is not the same as > the address subscribed to the list, please use the Subscribe/Unsubscribe form > at http://subscribe.wedi.org > > > > > > -- > This email contains confidential information intended only for the named > addressee(s). Any use, distribution, copying or disclosure by any other > person is strictly prohibited. > > > --- > The WEDI SNIP listserv to which you are subscribed is not moderated. The discussions on this listserv therefore represent the views of the individual participants, and do not necessarily represent the views of the WEDI Board of Directors nor WEDI SNIP. If you wish to receive an official opinion, post your question to the WEDI SNIP Issues Database at http://snip.wedi.org/tracking/. These listservs should not be used for commercial marketing purposes or discussion of specific vendor products and services. They also are not intended to be used as a forum for personal disagreements or unprofessional communication at any time. > > You are currently subscribed to wedi-testing as: [EMAIL PROTECTED] > To unsubscribe from this list, go to the Subscribe/Unsubscribe form at http://subscribe.wedi.org or send a blank email to [EMAIL PROTECTED] > If you need to unsubscribe but your current email address is not the same as the address subscribed to the list, please use the Subscribe/Unsubscribe form at http://subscribe.wedi.org > --- The WEDI SNIP listserv to which you are subscribed is not moderated. The discussions on this listserv therefore represent the views of the individual participants, and do not necessarily represent the views of the WEDI Board of Directors nor WEDI SNIP. If you wish to receive an official opinion, post your question to the WEDI SNIP Issues Database at http://snip.wedi.org/tracking/. These listservs should not be used for commercial marketing purposes or discussion of specific vendor products and services. They also are not intended to be used as a forum for personal disagreements or unprofessional communication at any time. You are currently subscribed to wedi-testing as: [email protected] To unsubscribe from this list, go to the Subscribe/Unsubscribe form at http://subscribe.wedi.org or send a blank email to [EMAIL PROTECTED] If you need to unsubscribe but your current email address is not the same as the address subscribed to the list, please use the Subscribe/Unsubscribe form at http://subscribe.wedi.org
