Okay, I believe I have ctakes dictionary fast working now. Something I'm curious about, though, is how you extract the data in order to conduct analysis.
I've, in the past, been using the SparseDataExporterImpl from ytex in order to create a .arff file for use in weka, but the ctakes pipeline I'm using doesn't seem to be compatible with this ytex exporting as I'm not getting any cuis in my arff file. I'm using the aggregate plain text umls processor analysis engine from ctakes and then using the dbconsumer analysis engine from ytex (for storing into the database with regard to analysis batch). Any tips for exporting or some simple issue I'm missing? Thanks, Clayton On Mon, Aug 11, 2014 at 2:09 PM, Harpreet Khanduja <hsk5...@rit.edu> wrote: > Yes, absolutely and > no problem at all. > > Regards, > Harpreet > > > On Mon, Aug 11, 2014 at 1:16 PM, Finan, Sean < > sean.fi...@childrens.harvard.edu> wrote: > > > Thanks Harpreet, > > That is definitely necessary to build! > > > > Those lines should already be in the pom, but commented out. I think > that > > some version/branching issues may have arisen at some point wrt this > module > > ... > > > > If somebody beats me to it then cheers, otherwise I will try to check out > > tonight and get all the bits in place. > > > > Sean > > > > > -----Original Message----- > > > From: Harpreet Khanduja [mailto:hsk5...@rit.edu] > > > Sent: Monday, August 11, 2014 1:12 PM > > > To: dev@ctakes.apache.org > > > Subject: Re: v_snomed_fword_lookup view > > > > > > Hello Clayton, > > > I do not know about ytex, but I did switch from dictionary-lookup to > > dictionary- > > > lookup-fast. > > > I update my ctakes-dictionary-lookup-fast project using maven. > > > I think I used Team- Update and switched to the latest revision > > available and > > > then > > > I downloaded new 3.2 resources from the for umls. and then I added > > these > > > resources to my > > > ctakes-dictionary-lookup-fast resources folder and also the classpath > > in ctakes- > > > clinical-pipeline. > > > > > > Then I changed the pom.xml file which belongs to the whole ctakes > > project and > > > added <dependency> <groupId>org.apache.ctakes</groupId> > > > <artifactId>ctakes-dictionary-lookup-res</artifactId> > > > <version>${ctakes.version}</version> > > > </dependency> > > > <dependency> > > > <groupId>org.apache.ctakes</groupId> > > > <artifactId>ctakes-dictionary-lookup-fast</artifactId> > > > <version>${ctakes.version}</version> > > > </dependency> > > > > > > > > > these two dependencies to the file. > > > > > > > > > After this, I also added the dependency > > > <dependency> > > > <groupId>org.apache.ctakes</groupId> > > > <artifactId>ctakes-dictionary-lookup-fast</artifactId> > > > </dependency> > > > > > > to the pom.xml of ctakes-clinical-pipeline. > > > > > > And then add the resources folder in ctakes-clinical-pipeline using > > build path > > > configuration under "add class" option. > > > > > > After this it should work. > > > > > > > > > Regards, > > > Harpreet > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 12:44 PM, Clayton Turner <caturn...@g.cofc.edu > > > > > wrote: > > > > > > > I still get the same error with the ctakes3.2 branch. Any > suggestions? > > > > > > > > > > > > On Mon, Aug 11, 2014 at 12:06 PM, Clayton Turner > > > > <caturn...@g.cofc.edu> > > > > wrote: > > > > > > > > > I'm going to do a clean install through the repo rather than the > > > > > binaries and see if that fixes my issue because I think I just read > > > > > a past post saying the lookup2 folders exist there. > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 11:52 AM, Clayton Turner > > > > > <caturn...@g.cofc.edu> > > > > > wrote: > > > > > > > > > >> When navigating to > > > > >> ctakes-dictionary-lookup-fast\desc\analysis_engine > > > > >> there are 2 files, assumedly analysis engines. > > > > >> > > > > >> SnomedLookupAnnotator.xml and SnomedOvLookupAnnotator.xml > > > > >> > > > > >> If I pick either, I put in my UMLS information but receive an > error > > > > >> when trying to run the CPE: > > > > >> > > > > >> Initialization of CAS Processor with name > "SnomedOvLookupAnnotator" > > > > >> failed. > > > > >> CausedBy: org.apache.uima.resource.ResourceConfigurationException: > > > > >> Initialization of CAS processor with name > "SnomedOvLookupAnnotator" > > > > >> failed. > > > > >> CausedBy: > org.apache.uima.resource.ResourceInitializationException: > > > > Error > > > > >> initializing "org.apache.uima.resource.impl.DataResource_impl" > from > > > > >> descriptor file:..............SnomedLookupAnnotator.xml > > > > >> CausedBy: > org.apache.uima.resource.ResourceInitializationException: > > > > Could > > > > >> not > > > > >> access the resource data at > > > > >> > > > > >> > > > > > file:org\apache\ctakes\dictionary\lookup2\Snomed2011ab_ctakesTui\cTake > > > > sSnomed.xml > > > > >> > > > > >> Now, I don't even have a "lookup2" folder and, subsequently the > Tui > > > > >> folder and cTakesSnomed.xml file. This seems to be the problem, > but > > > > >> I'm > > > > not > > > > >> sure where these files are supposed to be grabbed from. > > > > >> > > > > >> > > > > >> On Mon, Aug 11, 2014 at 11:47 AM, Clayton Turner > > > > >> <caturn...@g.cofc.edu> > > > > >> wrote: > > > > >> > > > > >>> Hi again: > > > > >>> > > > > >>> How exactly do you switch to using the cTakes > > dictionary-lookup-fast. > > > > Do > > > > >>> I need to go in and alter xml files or is it as simple as adding > a > > > > certain > > > > >>> item to the list of analysis engines? > > > > >>> > > > > >>> > > > > >>> On Fri, Aug 8, 2014 at 3:48 PM, Finan, Sean < > > > > >>> sean.fi...@childrens.harvard.edu> wrote: > > > > >>> > > > > >>>> Hi Clayton, > > > > >>>> > > > > >>>> I don't know how the ytex dictionary lookup works, so I'm afraid > > > > >>>> that > > > > I > > > > >>>> can't help you with an answer. Maybe Vijay is the best person > to > > > > >>>> do > > > > this. > > > > >>>> If you aren't tied to ytex you could try the new cTakes > > > > >>>> dictionary-lookup-fast. I tested "Patient came in with a malar > > rash" > > > > and > > > > >>>> it found "malar" and "malar rash". > > > > >>>> > > > > >>>> Vijay, > > > > >>>> > > > > >>>> At some point the lookup-fast module will be the default for the > > > > cTakes > > > > >>>> clinical pipeline. In order to synchronize the ytex lookup with > > > > cTakes, > > > > >>>> would you like to eventually work together on reusing the same > > > > >>>> code > > > > for > > > > >>>> ytex? I have no idea what ytex does, but I know the ins and > outs > > > > >>>> of > > > > the > > > > >>>> cdl-fast module. > > > > >>>> > > > > >>>> Sean > > > > >>>> > > > > >>>> > -----Original Message----- > > > > >>>> > From: clayclay...@gmail.com [mailto:clayclay...@gmail.com] On > > > > Behalf > > > > >>>> Of > > > > >>>> > Clayton Turner > > > > >>>> > Sent: Friday, August 08, 2014 2:08 PM > > > > >>>> > To: dev@ctakes.apache.org > > > > >>>> > Subject: v_snomed_fword_lookup view > > > > >>>> > > > > > >>>> > Hi Everyone: > > > > >>>> > > > > > >>>> > I have a question about how the v_snomed_fword_lookup view > > > > >>>> > works > > > > when > > > > >>>> > running the CPE. > > > > >>>> > > > > > >>>> > So my understanding of the view is that it is a view comprised > > > > >>>> > of > > > > the > > > > >>>> > ytex.umls_aui_fword table, the umls.mrconso table and > > > > >>>> > bits/pieces > > > > from > > > > >>>> > other umls tables. > > > > >>>> > > > > > >>>> > I feel like this is not completely correct or my idea of how > > > > >>>> > the > > > > join > > > > >>>> to > > > > >>>> > create the view works is off. For example, let's say I want > the > > > > >>>> > CPE > > > > >>>> to find > > > > >>>> > "malar ____" (e.g. malar rash) as a concept in the > annotations. > > > > >>>> > It > > > > >>>> never > > > > >>>> > happens after running my CPE descriptor and I cannot find it > in > > > > >>>> > my v_snomed_fword_lookup view. > > > > >>>> > > > > > >>>> > select count(*) from umls_aui_fword where fword='malar'; > yields > > > > >>>> > 34 > > > > >>>> results > > > > >>>> > > > > > >>>> > select count(*) from umls.mrconso where str='malar'; yields 3 > > > > results. > > > > >>>> > > > > > >>>> > So clearly these two tables know what the cui and context(s) > > > > >>>> > are for > > > > >>>> malar > > > > >>>> > ____. Yet, whenever I run a gold standard set of notes through > > > > >>>> > the > > > > >>>> CPE, > > > > >>>> > malar is constantly flagged as just a word token and the > > > > >>>> > concept is > > > > >>>> never > > > > >>>> > grabbed. This is recurrent for lots of other concepts, as > well, > > > > >>>> > I > > > > just > > > > >>>> > wanted to use an example to illustrate my issue. > > > > >>>> > > > > > >>>> > Some troubleshooting I already went through: > > > > >>>> > 1) Reinstalled ytex and umls database objects > > > > >>>> > 2) Reinstalled a second time after redownloading umls through > > > > >>>> > metamorphosys, ensuring that snomed vocabularies were included > > > > >>>> > (also checked file sizes and noticed a big difference so I > know > > > > >>>> > those vocabularies ARE included > > > > >>>> > > > > > >>>> > Anyone got any ideas as to what the issue could be? > > > > >>>> > > > > > >>>> > Thank you, > > > > >>>> > Clayton Turner > > > > >>>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> -- > > > > >>> -- > > > > >>> Clayton Turner > > > > >>> email: caturn...@g.cofc.edu > > > > >>> phone: (843)-424-3784 > > > > >>> web: claytonturner.blogspot.com > > > > >>> > > > > >>> > > > > > ---------------------------------------------------------------------- > > > > --------------------------- > > > > >>> “When scientifically investigating the natural world, the only > > > > >>> thing worse than a blind believer is a seeing denier.” > > > > >>> - Neil deGrasse Tyson > > > > >>> > > > > >> > > > > >> > > > > >> > > > > >> -- > > > > >> -- > > > > >> Clayton Turner > > > > >> email: caturn...@g.cofc.edu > > > > >> phone: (843)-424-3784 > > > > >> web: claytonturner.blogspot.com > > > > >> > > > > >> > > > > > ---------------------------------------------------------------------- > > > > --------------------------- > > > > >> “When scientifically investigating the natural world, the only > > > > >> thing worse than a blind believer is a seeing denier.” > > > > >> - Neil deGrasse Tyson > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > -- > > > > > Clayton Turner > > > > > email: caturn...@g.cofc.edu > > > > > phone: (843)-424-3784 > > > > > web: claytonturner.blogspot.com > > > > > > > > > > > > > > > ---------------------------------------------------------------------- > > > > --------------------------- > > > > > “When scientifically investigating the natural world, the only > thing > > > > worse > > > > > than a blind believer is a seeing denier.” > > > > > - Neil deGrasse Tyson > > > > > > > > > > > > > > > > > > > > > -- > > > > -- > > > > Clayton Turner > > > > email: caturn...@g.cofc.edu > > > > phone: (843)-424-3784 > > > > web: claytonturner.blogspot.com > > > > > > > > > ---------------------------------------------------------------------- > > > > --------------------------- “When scientifically investigating the > > > > natural world, the only thing worse than a blind believer is a seeing > > > > denier.” > > > > - Neil deGrasse Tyson > > > > > > > -- -- Clayton Turner email: caturn...@g.cofc.edu phone: (843)-424-3784 web: claytonturner.blogspot.com ------------------------------------------------------------------------------------------------- “When scientifically investigating the natural world, the only thing worse than a blind believer is a seeing denier.” - Neil deGrasse Tyson
