Hi Gerard, I am talking about metrics like the one you had suggested previously: # of interfaces to be implemented to achieve interoperability with no message standard, some message standard and two-level systems. It is clear and easily computable and very objective. Perhaps it is worth studying qualitative part of the story too apart from just # of interfaces.
Sam also suggested the possibility of assessing archetype reuse (I don't know how to measure that though) And Rong has suggested to explore how EHR systems work together with other EHR and surrounding systems - that is hard to assess but I think the only way to test it. In Sam and Rong's case we need some metrics which is applicable to both single level and two level apps and then measure accordingly. Now after quite a literature search and reading, considering both maintainability and interoperability are software quality characteristics there is vast amount of material out there; mainly under Software Product Quality Measures or specific on those attributes. Here is an example on maintainability: ? Fix backlog and backlog management index ? Fix response time and fix responsiveness ? Percent delinquent fixes Fix quality * backlog management index (BMI)= I think an archetype based two-level app will beat with this index * Fix Response Time and Fix Responsiveness: this will be the killer metric I assume. Reference: Software Quality Metrics Overview, Book Chapter (4) By Stephen H. Kan., Dec 20, 2002 There are many many more of those; and I think we need to identify relevant ones, especially the metrics which forecast on the quality of product based on design, before actual implementation. Sorry to bother with all this on discussion list and if there is more interest we can continue on the wiki. -koray Gerard Freriks wrote: > Koray, > > What metrics do you want to define? > > Gerard > > > On Feb 11, 2008, at 10:40 AM, Koray Atalag wrote: > >> Dear All, >> >> I started this thread to get some feedback for finding methods/metrics >> to test & validate maintainability and interoperability (of Archetype >> based two-level apps). And I got very nice ones; however for >> interoperability, apart from Gerard's interface numbers I did not get >> any and interestingly from a quick literature survey I got very little. >> I mean there are some indirect approaches but not straightforward. My >> case is a little more easier: >> >> 1) There is an up and running clinical IS developed with single level >> methodology based on an internationally agreed terminology including >> relationships and structure (domain knowledge let's say) >> 2) There is a complete Archetype model of this terminology using openEHR >> RM which can comfortably be considered as a domain ontology (it has more >> than what is given in terminology; i.e. existences, cardinalities) >> 3) These two can be said to have the same domain knowledge; ie. one >> hardcoded and one two-level modelled. >> >> Now can you think about a method to evaluate the interoperability >> levels/score of two systems? >> >> Do we need a remote system for benchmarking (i.e. connect and see how >> they interoperate)? >> >> Sorry to bother....but if we can get this straight perhaps we can >> express comfortably that a two-level app beats a single level app 7x in >> maintainability and 5x interoperability. Or beats 2x HL7 system in >> maintenance but is beaten 2x in interoperablity. Perhaps I am being too >> naive but it is worth trying. > > > > -- <private> -- > Gerard Freriks, MD > Huigsloterdijk 378 > 2158 LR Buitenkaag > The Netherlands > > T: +31 252544896 > M: +31 620347088 > E: gfrer at luna.nl <mailto:gfrer at luna.nl> > > > Those who would give up essential Liberty, to purchase a little temporary > Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755 > > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > openEHR-technical mailing list > openEHR-technical at openehr.org > http://lists.chime.ucl.ac.uk/mailman/listinfo/openehr-technical > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20080212/90c34890/attachment.html>
