To be clear "duplicate" mappings we ran into were cases where same source and source code were used to map more than one concept. It is not wrong, but we missed to notice that while writing the migration script. There is no need for you Andy to fix them.
Just for the record there are 44737 such mappings in MVP. I determined that with the following query: select sum(c.sameCount) from (select count(*) as sameCount from concept_map group by source_code, source having count(*) > 1) as c; We'll convert all MVP comments to map types as follows and then we'll drop the comments. | Comment | Map type ------------------------------------------------------------ | NULL | *NARROWER-THAN* | From Excel | *SAME-AS* | From UMLS RxNORM Map | *SAME-AS* | Map Type: 1 | *SAME-AS* | Map Type: 10 | *Has specimen* | Map Type: 17 | *Direct device ??? * | Map Type: 19 | *Direct substance ???* | Map Type: 2 | *NARROWER-THAN* | Map Type: 24 | *Finding method ???* | Map Type: 3 | *BROADER-THAN* | Map Type: 4 | *Associated finding* | Map Type: 5 | *Associated morphology* | Map Type: 6 | *Associated procedure* | Map Type: 7 | *Associated with* * * Andy, there are 3 map types left with "???". Map Type 17, 19 an 24. Can you confirm them? I'm still wondering if we should move all other comments to the description field in concept_reference_term. I think it's better to leave the comment column as is in concept_reference_map and only deprecate it and mark to be removed in 1.10 since we can't really migrate it correctly. -Rafał On 7 May 2012 21:38, Andrew Kanter <andy_kan...@yahoo.com> wrote: > Rafal, > > I am confused... the mapping below is correct. For the plan, I would do 1, > not 2 (just ignore the comment) I would make these probably SAME AS for > "from excel and from UMLS". For 3, I presume we will keep one of the > duplicate maps...and if I had to choose a map type (if there is no map id:) > then I would pick NARROWER-THAN, but perhaps I should review the list of > dupes first. > > *-------------------- > Andrew S. Kanter, MD MPH > > - Director of Health Information Systems/Medical Informatics* > *Millennium Villages Project, Earth Institute, Columbia University* > *- Asst. Prof. of Clinical Biomedical Informatics and Clinical > Epidemiology* > *Columbia University* > > Email: andrew.kan...@dbmi.columbia.edu > Mobile: +1 (646) 469-2421 > Office: +1 (212) 305-4842 > Skype: akanter-ippnw > Yahoo: andy_kanter > > ------------------------------ > *From:* Rafal Korytkowski <ra...@openmrs.org> > *To:* openmrs-deve...@listserv.iupui.edu > *Sent:* Monday, May 7, 2012 11:04 AM > *Subject:* Re: [OPENMRS-DEV] Migrating concept mappings to 1.9 > > Thanks Andy! When do you think you'll have time to look into that comments > and see if they can be matched with proper mapping types from 1.9? Below is > a full list of predefined mapping types in 1.9: > > +---------------------+------------------------------- > | concept_map_type_id | name > +---------------------+------------------------------- > | 1 | SAME-AS = 1 > | 2 | NARROWER-THAN = 2 > | 3 | BROADER-THAN = 3 > | 4 | Associated finding = 4 > | 5 | Associated morphology = 5 > | 6 | Associated procedure = 6 > | 7 | Associated with = 7 > | 8 | Causative agent > | 9 | Finding site > | 10 | Has specimen = 10 > | 11 | Laterality > | 12 | Severity > | 13 | Access > | 14 | After > | 15 | Clinical course > | 16 | Component > | 17 | Direct device > | 18 | Direct morphology > | 19 | Direct substance > | 20 | Due to > | 21 | Episodicity > | 22 | Finding context > | 23 | Finding informer > | 24 | Finding method > | 25 | Has active ingredient > | 26 | Has definitional manifestation > | 27 | Has dose form > | 28 | Has focus > | 29 | Has intent > | 30 | Has interpretation > | 31 | Indirect device > | 32 | Indirect morphology > | 33 | Interprets > | 34 | Measurement method > | 35 | Method > | 36 | Occurrence > | 37 | Part of > | 38 | Pathological process > | 39 | Priority > | 40 | Procedure context > | 41 | Procedure device > | 42 | Procedure morphology > | 43 | Procedure site > | 44 | Procedure site - Direct > | 45 | Procedure site - Indirect > | 46 | Property > | 47 | Recipient category > | 48 | Revision status > | 49 | Route of administration > | 50 | Scale type > | 51 | Specimen procedure > | 52 | Specimen source identity > | 53 | Specimen source morphology > | 54 | Specimen source topography > | 55 | Specimen substance > | 56 | Subject of information > | 57 | Subject relationship context > | 58 | Surgical approach > | 59 | Temporal context > | 60 | Time aspect > | 61 | Using access device > | 62 | Using device > | 63 | Using energy > | 64 | Using substance > | 65 | IS A > | 66 | MAY BE A > | 67 | MOVED FROM > | 68 | MOVED TO > | 69 | REPLACED BY > | 70 | WAS A > +---------------------+------------------------------- > > -Rafał > > > On 4 May 2012 23:35, Andrew Kanter <andy_kan...@yahoo.com> wrote: > > Yes, we did that from IMO and I included in that in some... however, it is > not at all consistent. Where we have it, we should use it. There shouldn't > be dupes with the same map type. I will look through this... > > Thanks! > Andy > > P.S. Great news about MDS... now just need to fix the concepts :) > > *-------------------- > Andrew S. Kanter, MD MPH > > - Director of Health Information Systems/Medical Informatics* > *Millennium Villages Project, Earth Institute, Columbia University* > *- Asst. Prof. of Clinical Biomedical Informatics and Clinical > Epidemiology* > *Columbia University* > > Email: andrew.kan...@dbmi.columbia.edu > Mobile: +1 (646) 469-2421 > Office: +1 (212) 305-4842 > Skype: akanter-ippnw > Yahoo: andy_kanter > > ------------------------------ > *From:* Rafal Korytkowski <ra...@openmrs.org> > *To:* openmrs-deve...@listserv.iupui.edu > *Sent:* Friday, May 4, 2012 11:08 AM > *Subject:* [OPENMRS-DEV] Migrating concept mappings to 1.9 > > Hi Andy, > > We have noticed that MVP uses the comment field in the concept_map table. > We are considering using that to determine the right map type in 1.9. > > I ran the following query select comment, count(*) from concept_map group > by comment; The results are below. I have also added corresponding map > types from 1.9, but I am not sure if they match right now. We could correct > them if needed. > > +----------------------+----------+ > | comment | count(*) | > +----------------------+----------+ > | NULL | 15516 | > | From Excel | 2381 | > | From UMLS RxNORM Map | 3010 | > | Map Type: 1 | 46897 | *(SAME-AS)* > | Map Type: 10 | 1 | *(Has specimen)* > | Map Type: 17 | 5 | *(Direct device)* > | Map Type: 19 | 3 | *(Direct substance)* > | Map Type: 2 | 1880 | *(NARROWER-THAN)* > | Map Type: 24 | 18 | *(Finding method)* > | Map Type: 3 | 30841 | *(BROADER-THAN)* > | Map Type: 4 | 126 | *(Associated finding)* > | Map Type: 5 | 81 | *(Associated morphology)* > | Map Type: 6 | 19 | *(Associated procedure)* > | Map Type: 7 | 2 | *(Associated with)* > +----------------------+----------+ > 14 rows in set (2.12 sec) > > Here's the proposed migration algorithm: > > (1) if the comment matches "Map Type: (\d+)" then use that to determine > the map type, and drop it > > (2) otherwise move the comment to concept_reference_term.description (even > though it doesn't really belong there) > > (3) delete duplicate concept_reference_terms (having same source and > source_code), though this means we may lose some concept_map.comment data > > The reason for these changes is: TRUNK-3296: Found multiple reference terms > https://tickets.openmrs.org/browse/TRUNK-3296 > > -Rafał > ------------------------------ > Click here to > unsubscribe<lists...@listserv.iupui.edu?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > > > ------------------------------ > Click here to > unsubscribe<lists...@listserv.iupui.edu?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > > > ------------------------------ > Click here to > unsubscribe<lists...@listserv.iupui.edu?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > > > ------------------------------ > Click here to > unsubscribe<lists...@listserv.iupui.edu?body=SIGNOFF%20openmrs-devel-l>from > OpenMRS Developers' mailing list > _________________________________________ To unsubscribe from OpenMRS Developers' mailing list, send an e-mail to lists...@listserv.iupui.edu with "SIGNOFF openmrs-devel-l" in the body (not the subject) of your e-mail. [mailto:lists...@listserv.iupui.edu?body=SIGNOFF%20openmrs-devel-l]
