It works! Thanks Gandhi and everyone else who helped, let me once again say the experience as a newbie in this group is amazing. FWIW, and future readers, I found the relevant information in the UIMA references: https://uima.apache.org/downloads/releaseDocs/2.2.2-incubating/docs/html/references/references.html#ugr.ref.xml.cpe_descriptor, linked from the tutorials mentioned above. Ory
On Tue, Aug 28, 2018 at 9:11 AM, Gandhi Rajan Natarajan < [email protected]> wrote: > Hi Ory, > > > > Piper files are equivalent of the XML descriptor files which helps in > adding required analysis engines. > > > > You can manipulate analysis engines for CPE using CPE configurator. > > > > Check this link for more info - http://uima.apache.org/ > downloads/releaseDocs/2.3.0-incubating/docs/html/ > tutorials_and_users_guides/tutorials_and_users_guides. > html#ugr.tug.cpe.configurator_and_viewer > > > > Regards, > > Gandhi > > > > *From:* Ory Henn <[email protected]> > *Sent:* Sunday, August 26, 2018 5:25 PM > > *To:* [email protected] > *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup? > [EXTERNAL] > > > > Thanks, Gandhi. > > I took a look at the module, but am still a bit confused, as the > connection between the CPE tool and the piper files is not clear to me. > > I've succeeded in using my own dictionary with the Default Clinical > Pipeline, both through -l argument and through adding a LookupXml parameter > in the DefaultFastPipeline.piper script (as in ctakes-rest-service above). > > But when I run the CPE tool, it still references the default dictionary. > > My question is - what is the link betwenn CVD/CPE tools and the piper > files? Specifically, can I configure the CPE tool to use my piper file, or > configure it to use a different lookup? > > Many thanks, > > Ory > > > > > > On Thu, Aug 23, 2018 at 8:04 AM, Gandhi Rajan Natarajan < > [email protected]> wrote: > > Hi Ory, > > > > I suggest you to have a look at this ctakes rest module - > https://github.com/GoTeamEpsilon/ctakes-rest-service/tree/master/ctakes- > web-rest > > > > The custom dictionary XML needs to be placed in > ‘resources/org/apache/ctakes/dictionary/lookup/fast’ folder and > ‘LookupXml’ attribute needs to be added in DictionarySubPipe.piper as > follows: > > > > add DefaultJCasTermAnnotator LookupXml=org/apache/ctakes/ > dictionary/lookup/fast/customDictionary.xml > > > > *From:* Ory Henn <[email protected]> > *Sent:* Wednesday, August 22, 2018 8:42 PM > > > *To:* [email protected] > *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup? > [EXTERNAL] > > > > Thanks, Timothy. > > I fixed the permissions issue and was able to progress further. > > I am now stuck at the very last step (it would seem), the Dictionary > Creator GUI page says this: > > --- > > Once a new dictionary has been built, point to it in one of 2 ways: > > Set the fast dictionary parameter LookupXml to > org/apache/ctakes/dictionary/lookup/fast/DictionaryName.xml > > --- > > > > Where is this LookupXml parameter to be found/added? > > > > > > > > On Wed, Aug 22, 2018 at 4:50 PM, Miller, Timothy < > [email protected]> wrote: > > Looks like maybe you don't have permissions to write to that directory > (/usr/local/). Can you write it somewhere else? > > > ------------------------------ > > *From:* Ory Henn <[email protected]> > *Sent:* Wednesday, August 22, 2018 9:15 AM > *To:* [email protected] > *Subject:* Re: How do I add a dictionary (like NCI) to cTakes lookup? > [EXTERNAL] > > > > Thanks a lot Ghandi, Michael, for quick and helpful replies. > > The video especially was very useful, but I'm encountering an error at the > last part (building the actual dictionary). > > It looks like it has a problem with an HsqlDB table (log output below). > > Can anyone help with this? Do I need to install HsqlDB, or does it come as > part of UMLS/Metamorphosys? > > Thanks, > > Ory > > > > > > 22 Aug 2018 16:06:40 INFO MrConsoParser - File Lines: 14089874 Concepts: > 450236 Texts: 1380174 > > 22 Aug 2018 16:06:40 INFO DictionaryBuilder - Performing Poor man's WSD > ... > > 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Concepts: 446510 Texts: > 1162474 > > 22 Aug 2018 16:06:42 INFO DictionaryBuilder - Done Parsing Concepts > > 22 Aug 2018 16:06:42 INFO JdbcUtil - Connecting to > jdbc:hsqldb:file:/usr/local/apache-ctakes-4.0.0/resources/ > org/apache/ctakes/dictionary/lookup/fast/umls_test_nci/umls_test_nci as SA > > 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint start > > 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose start > > 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose synched > > 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose script done > > 22 Aug 2018 16:06:42 INFO ENGINE - checkpointClose end > > 22 Aug 2018 16:06:42 INFO ENGINE - Checkpoint end - txts: 1 > > 22 Aug 2018 16:06:43 ERROR RareWordDbWriter - user lacks privilege or > object not found: NCI_NCI in statement [insert into nci_nci_hl7 > (CUI,NCI_NCI-HL7) values (?,?)] > > 22 Aug 2018 16:06:43 ERROR MainPanel - Dictionary umls_test_nci could not > be built in /usr/local/apache-ctakes-4.0.0 > > > > > > On Mon, Aug 20, 2018 at 10:13 PM, Michael Trepanier <[email protected]> > wrote: > > Ory, > > > > In response to Gandhi's comments, the video below outlines custom > dictionary creation in detail: > > > > https://www.youtube.com/watch?v=4aOnafv-NQs > <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_watch-3Fv-3D4aOnafv-2DNQs&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=A8Md_8ifzCAHha2tRJ_d-kGLk26gNMdOgfpXd3_qKHI&e=> > > > > Best, > > > > Mike > > > > > > > > On Mon, Aug 20, 2018 at 2:09 AM, Gandhi Rajan Natarajan < > [email protected]> wrote: > > Hi Ory, > > I guess RxNORM and SNOMED_CT dictionaries are loaded by default. If you > want to lookup from other dictionaries like MEDDRA etc. , you may have to > create your custom dictionary using cTAKES dictionary generator GUI. That’s > what I did to include MEDDRA dictionary terms. > > > -----Original Message----- > From: Ory Henn <[email protected]> > Sent: Monday, August 20, 2018 1:52 PM > To: [email protected] > Cc: Guy Gildor <[email protected]> > Subject: How do I add a dictionary (like NCI) to cTakes lookup? > > Hello, > New user here (-; > I've downloaded and installed cTakes (+UMLS + all resources), and am > trying to parse a single document. > I see that cTakes (CVD/CPE) identifies CUIs only from RxNORM and SNOMED_CT. > > 1. What is the way to make cTakes look in more UMLS dictionaries? I > specifically need NCI as well. > 2. Is there an easy way to make cTakes look in all UMLS dictionaries? > Thanks, > Ory > > -- > > <https://trialmatch.me > <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=> > >www.trialjectory.com > <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=ooC-oaVMc9CCPBbbTUyU23H7BlBi1TB3foqR0TCnHr8&e=> > <https://trialjectory.com > <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=> > > > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you are not the named addressee you should not disseminate, distribute > or copy this e-mail. Please notify the sender or system manager by email > immediately if you have received this e-mail by mistake and delete this > e-mail from your system. If you are not the intended recipient you are > notified that disclosing, copying, distributing or taking any action in > reliance on the contents of this information is strictly prohibited and > against the law. > > > > > > -- > > [image: MetiStream Logo - 500] > > Mike Trepanier| Big Data Engineer | MetiStream, Inc. | > [email protected] | 845 - 270 - 3129 (m) | www.metistream.com > <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.metistream.com_&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=_-eXxPXJ9Ltpd4KTdRwj-pDyQ42Xbpt0_9ydetCzIR0&e=> > > > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialmatch.me&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=lQh_YM3Iht4AIa0E39KpnFIY7XbD4TaYkEkMESEQoEA&e=> > www.trialjectory.com > <https://urldefense.proofpoint.com/v2/url?u=https-3A__trialjectory.com&d=DwMFaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h&m=JrSEaNyFWIA-oT8ihZMLVdfYZvdudMkgfc2e13UOvFc&s=GUqLl8XdNiDmvZStF8lT2GP9uRvxuncPAyxBrSk-uvU&e=> > > > > > > <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com> > > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you are not the named addressee you should not disseminate, distribute > or copy this e-mail. Please notify the sender or system manager by email > immediately if you have received this e-mail by mistake and delete this > e-mail from your system. If you are not the intended recipient you are > notified that disclosing, copying, distributing or taking any action in > reliance on the contents of this information is strictly prohibited and > against the law. > > > > > > <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com> > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you are not the named addressee you should not disseminate, distribute > or copy this e-mail. Please notify the sender or system manager by email > immediately if you have received this e-mail by mistake and delete this > e-mail from your system. If you are not the intended recipient you are > notified that disclosing, copying, distributing or taking any action in > reliance on the contents of this information is strictly prohibited and > against the law. > -- <https://trialmatch.me>www.trialjectory.com <https://trialjectory.com>
