Hello, I just also wanted to flag one Observation that I also faced the same problem of extremely slow Application and when I try to compute the Similarity between set of worlds in two lists (e.g of size 100 and 200) I never gets the results and the application always crashes in the Mid way and at this point I left UMLS and used WordNet which is quiet fast but results are not much satisfactory as I am working with Healthcare and Life Sciences Terminologies :(
Just an observation- Will come back to UMLS in future when I want to do more experiments. Best Regards, Ali On Fri, May 4, 2012 at 8:11 PM, Ted Pedersen <[email protected]> wrote: > ** > > > Oh dear. That's pretty slow. Are you using mysql? There may be some > database tuning that is required. Below is some NLM documentation about > settings for mysql parameters to improve performance. This strikes me as > one possible explanation. > > > http://www.nlm.nih.gov/research/umls/implementation_resources/scripts/README_RRF_MySQL_Output_Stream.html > > If you are using something other than mysql then I think similar issues > can exist as well. > > But, I know that I've set up mysql once or twice without setting these > parameters appropriately, and it really does make a big difference. So, I > think this might be worth checking into, if you haven't already... > > Let us know how this all goes... > > Good luck, > Ted > > > On Fri, May 4, 2012 at 1:59 PM, helen_geb <[email protected]> wrote: > >> ** >> >> >> Ted, >> >> I am using index( It took 5-6 hours to create the index) not --realtime. >> >> I don't know what is making mine this slow. >> >> I have my.ini modified per the documentation. And I create an index >> whenever the umls-similarity requests(usually when i change the config >> files REL/SAB values). >> >> Thanks, >> >> Helen >> >> --- In [email protected], Ted Pedersen <duluthted@...> >> wrote: >> > >> > Hi Helen, >> > >> > Your hardware is more than adequate, so that isn't the problem. >> > >> > I can run 10 pairs using SNOMEDCT with PAR/CHD on a comparable machine >> in a >> > relatively few seconds. >> > >> > Are you using the --realtime option? That could take quite a while, so >> if >> > you are the easiest way to speed things up would be to go ahead and let >> the >> > system build an index. That might take a few hours, but then you'd be >> able >> > to run queries very fast. >> > >> > If you are running against an index....then I'm not sure. Let us know >> > either way and we can see what makes sense... >> > >> > Good luck, >> > Ted >> > >> > On Fri, May 4, 2012 at 1:22 PM, helen_geb <helen_geb@...> wrote: >> > >> > > ** >> >> > > >> > > >> > > Hi Ted, >> > > >> > > Thanks again for taking time to respond. >> > > >> > > Here are the details of the computer I am using >> > > Memory: 12 GB >> > > Processor: Intel(R) CPU X5650 @2.67 2.67 2 processors >> > > System type: 64 bit >> > > >> > > Running 10 pairs of CUI's using SNMOEDCT and PAR/CHD and measure wup >> takes >> > > 30-40 minutes. >> > > >> > > Is there any options to specify to umls-similarity.pl to make it >> faster? >> > > >> > > Thanks >> > > Helen >> > > >> > > >> > > --- In [email protected], Ted Pedersen <duluthted@> >> > > wrote: >> > > > >> > > > Hi Helen, >> > > > >> > > > Performance issues are always tricky. One variable is the machines >> you >> > > are >> > > > using. What are the general parameters of the computer you are >> using for >> > > > the command line interface (console)? >> > > > >> > > > Also, your mysql configuration settings can make a big difference. >> The >> > > UMLS >> > > > is a rather large data base, and the default mysql settings are >> generally >> > > > speaking *not* adequate for decent performance. So, make sure you >> have >> > > > adjusted those as suggested in the UMLS documentation. If you are >> using >> > > > something other than mysql then the same applies, and I think the >> UMLS >> > > > folks have guidelines for other data bases too. >> > > > >> > > > As to your server troubles, I think that's probably an apache >> > > configuration >> > > > issue, and you might just need to talk to your local web/net admin >> to get >> > > > some guidance on configuring (if you have someone like that >> available). >> > > > >> > > > So, can you give us some specific times? Maybe run 10 pairs of cuis >> using >> > > > the --infile option and report how long that takes? We can give you >> an >> > > idea >> > > > if that is within the realm of our experience. In general once the >> > > indices >> > > > are built things go pretty quickly, but index building (especially >> for >> > > > SNOMED) can really take some time (a few hours as I recall, >> although I >> > > > haven't done that lately). >> > > > >> > > > Sorry to be a little vague, performance stuff tends to be pretty >> > > variable, >> > > > but it would indeed be interesting to hear from other Windows users >> as to >> > > > any issues they might have faced. We just don't use Windows so we >> don't >> > > > quite know. >> > > > >> > > > Hope this is at least partially helpful, i know it's probably not >> > > directly >> > > > on point. >> > > > >> > > > Thanks, >> > > > Ted >> > > > >> > > > On Thu, May 3, 2012 at 11:05 AM, helen_geb <helen_geb@> wrote: >> > > > >> > > > > ** >> > > >> > > > > >> > > > > >> > > > > 1. The web application is running in my local machine. >> > > > > >> > > > > I have deployed umls_similarity on apache and >> > > > > >> > > > > I started the server using the following command >> > > > > C:\Users\Administrator>umls_similarity_server >> > > --logfile=c:\tmp\serverlog1 >> > > > > >> > > > > But I never got it working it times out with(Gateway Time-out) >> > > > > >> > > > > 2. The command line interface runs but very slow and I need to >> run 1000 >> > > > > cui's using matrix option so I am wondering how I can speed it up >> ? >> > > > > >> > > > > How is it running for other users? Whats the average time it >> takes to >> > > get >> > > > > the similarity using MSH /SNMOED? What is the best and worst case? >> > > > > >> > > > > I am wondering if the problem is with me only or happening for >> other >> > > > > windows users. >> > > > > >> > > > > >> > > > > --- In [email protected], Bridget McInnes >> <btmcinnes@> >> > > > > wrote: >> > > > > > >> > > > > > Hi Helen, >> > > > > > >> > > > > > For the console application: >> > > > > > >> > > > > > The SNOMEDCT index will take longer than the MSH index. >> SNOMEDCT is >> > > the >> > > > > > largest source in the UMLS with the greatest number of PAR/CHD >> > > relations. >> > > > > > The index traverses these relations collecting all the paths >> from >> > > the CUI >> > > > > > to the root for each CUI. Once this index is created -- it >> should be >> > > > > > quicker to compute the similarity between any two concepts in >> that >> > > > > source. >> > > > > > So the second run with SNOMEDCT should be quicker because the >> index >> > > is >> > > > > > already stored in the database. I hope this makes sense -- if >> not >> > > let me >> > > > > > know :-) >> > > > > > >> > > > > > For the web interface application: >> > > > > > >> > > > > > Is this running locally on your machine, or the one that we have >> > > > > available >> > > > > > on atlas though query-umls-similarity-webinterface ? Would you >> send >> > > us >> > > > > the >> > > > > > command that you are using? >> > > > > > >> > > > > > Thanks! >> > > > > > >> > > > > > Bridget >> > > > > > >> > > > > > On Tue, May 1, 2012 at 12:59 PM, helen_geb <helen_geb@> wrote: >> > > > > > >> > > > > > > ** >> > > > > >> > > > > > > >> > > > > > > >> > > > > > > Hi Ted, >> > > > > > > >> > > > > > > >> > > > > > > 1. Console application: >> > > > > > > >> > > > > > > >> > > > > > > - When I use SNOMEDCT SAB with any of the REL's(CHD/PAR or >> RB/RN) >> > > and >> > > > > >> > > > > > > Measure is wup it asks me to create an index and that >> creation goes >> > > > > forever. >> > > > > > > - When I use MSH SAB with any of the REL's it asked me for >> index >> > > > > >> > > > > > > creation (only on the first time) and gives the result in 1-3 >> > > minutes, >> > > > > > > which is not bad. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > 2. Web Interface: >> > > > > > > What ever I use here gives me time out with the following >> error. I >> > > > > don't >> > > > > > > see the results in the browser. >> > > > > > > >> > > > > > > *Gateway Time-out >> > > > > >> > > > > > > The gateway did not receive a timely response from the >> upstream >> > > server >> > > > > or >> > > > > > > application.* >> > > > > >> > > > > > > Note:The similarity_server is running successfully here it >> the log >> > > > > when I >> > > > > > > computer similarity of cat and dog using the web interface >> > > > > > > >> > > > > > > *In setInterface(MSH, PAR/CHD, path)* >> > > > > > > *FILENAME: C:\Users\ADMINI~1\AppData\Local\Temp\2\0teMudc7uG* >> > > > > > > >> > > > > > > *CuiFinder User Options: * >> > > > > > > * --config option set* >> > > > > > > >> > > > > > > >> > > > > > > *UMLS-Interface Configuration Information* >> > > > > > > * Sources (SAB):* >> > > > > > > * MSH* >> > > > > > > * Relations (REL):* >> > > > > > > * CHD* >> > > > > > > * PAR* >> > > > > > > * Database: * >> > > > > > > * umls (MMSYS-2011AA-20110426)* >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > *PathFinder User Options:* >> > > > > > > * --forcerun option set* >> > > > > > > >> > > > > > > >> > > > > > > *Setting measures* >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > *TYPE: r* >> > > > > > > *cat dog Compute Similarity path MSH PAR/CHD* >> > > > > > > *In setInterface(MSH, PAR/CHD, path)* >> > > > > > > *FILENAME: C:\Users\ADMINI~1\AppData\Local\Temp\2\lYumkWJOay* >> > > > > > > >> > > > > > > *CuiFinder User Options: * >> > > > > > > * --config option set* >> > > > > > > >> > > > > > > >> > > > > > > *UMLS-Interface Configuration Information* >> > > > > > > * Sources (SAB):* >> > > > > > > * MSH* >> > > > > > > * Relations (REL):* >> > > > > > > * CHD* >> > > > > > > * PAR* >> > > > > > > * Database: * >> > > > > > > * umls (MMSYS-2011AA-20110426)* >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > *PathFinder User Options:* >> > > > > > > * --forcerun option set* >> > > > > > > >> > > > > > > >> > > > > > > *Setting measures* >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > >> > > > > > > *ICFinder User Options:* >> > > > > > > * --icpropagation >> > > > > > > C:/WebPages/umls_similarity/icpropagation/icprop.msh.par.chd* >> > > > > > > >> > > > > > > *In getAllForms (cat)* >> > > > > > > *Forms (cat) : C0007450* >> > > > > > > *Returning Forms (cat) : C0007450* >> > > > > > > *In getAllForms (dog)* >> > > > > > > *Forms (dog) : C0012984* >> > > > > > > *Returning Forms (dog) : C0012984* >> > > > > > > *TYPE: p* >> > > > > > > *In getAllForms (cat)* >> > > > > > > *Forms (cat) : C0007450* >> > > > > > > *Returning Forms (cat) : C0007450* >> > > > > > > *In getAllForms (dog)* >> > > > > > > *Forms (dog) : C0012984* >> > > > > > > *Returning Forms (dog) : C0012984* >> > > > > > > *HERE: C0007450 Felis catus C0012984 familiaris, Canis* >> > > > > > > *p C0007450 (Felis catus)|C0012984 (familiaris, >> Canis)|C0007450 >> > > (Felis >> > > > > >> > > > > > > catus) => C0524517 (Genus Felis) => C0325089 (Family Felidae) >> => >> > > > > C0007266 >> > > > > > > (Carnivora) => C0524515 (Family canidae) => C0012984 >> (familiaris, >> > > > > Canis)* >> > > > > > > *TYPE: g* >> > > > > > > *HERE (Compute Similarity) g (cat)* >> > > > > > > *In getAllForms (cat)* >> > > > > > > *Forms (cat) : C0007450* >> > > > > > > *Returning Forms (cat) : C0007450* >> > > > > > > *CUIS: C0007450* >> > > > > > > *g C0007450 MSH : The domestic cat, Felis catus, of the >> carnivore >> > > > > family >> > > > > >> > > > > > > FELIDAE, comprising over 30 different breeds. The domestic >> cat is >> > > > > descended >> > > > > > > primarily from the wild cat of Africa and extreme southwestern >> > > Asia. >> > > > > Though >> > > > > > > probably present in towns in Palestine as long ago as 7000 >> years, >> > > > > actual >> > > > > > > domestication occurred in Egypt about 4000 years ago. (From >> > > Walker's >> > > > > > > Mammals of the World, 6th ed, p801)|NCI : The domestic cat, >> Felis >> > > > > catus.|* >> > > > > > > *TYPE: g* >> > > > > > > *HERE (Compute Similarity) g (dog)* >> > > > > > > *In getAllForms (dog)* >> > > > > > > *Forms (dog) : C0012984* >> > > > > > > *Returning Forms (dog) : C0012984* >> > > > > > > *CUIS: C0012984* >> > > > > > > *g C0012984 NCI : The domestic dog, Canis familiaris.|MSH : >> The >> > > > > domestic >> > > > > >> > > > > > > dog, Canis familiaris, comprising about 400 breeds, of the >> > > carnivore >> > > > > family >> > > > > > > CANIDAE. They are worldwide in distribution and live in >> association >> > > > > with >> > > > > > > people. (Walker's Mammals of the World, 5th ed, p1065)|* >> > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > --- In [email protected], Ted Pedersen >> <duluthted@> >> > > > > > > wrote: >> > > > > > > > >> > > > > > > > Hi Helen, >> > > > > > > > >> > > > > > > > What sources (SAB) and relations (REL) are you using? Also, >> what >> > > > > measure >> > > > > > > > are you running? If you could send a few examples of what >> you are >> > > > > trying >> > > > > > > > then we could figure out if that is expected (some >> combinations >> > > are >> > > > > > > indeed >> > > > > > > > slow) or if there might be an issue on your particular >> system. >> > > > > > > > >> > > > > > > > Thanks! >> > > > > > > > Ted >> > > > > > > > >> > > > > > > > On Mon, Apr 30, 2012 at 4:42 PM, helen_geb helen_geb@wrote: >> > > > > > > > >> > > > > > > > > ** >> > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > I have installed umls similarity following the >> instructions. >> > > > > > > > > >> > > > > > > > > But it is too slow. It takes 2-3 minutes to compute >> similarity >> > > > > between >> > > > > > > two >> > > > > > > > > terms. >> > > > > > > > > >> > > > > > > > > Let me know what I need to do speed it up. >> > > > > > > > > >> > > > > > > > > It gets worst when I use UMLS_ALL for SAB and REF config >> files. >> > > > > > > > > >> > > > > > > > > Regards, >> > > > > > > > > Helen. >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > -- >> > > > > > > > Ted Pedersen >> > > > > > > > http://www.d.umn.edu/~tpederse >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Ted Pedersen >> > > > http://www.d.umn.edu/~tpederse >> > > > >> > > >> > > >> > > >> > >> > >> > >> > -- >> > Ted Pedersen >> > http://www.d.umn.edu/~tpederse >> > >> >> > > > -- > Ted Pedersen > http://www.d.umn.edu/~tpederse > > >
