Hi Abhishek, You have some interesting timing ... I can give you the xml specifications that you require if you send me the format of your dictionary.
Since you are new to the current dictionary module setup, I might also have a simpler solution for you ... A couple of days ago I checked a new module into Sandbox called ctakes-dictionary-lookup2 (how novel a name). It is a complete replacement of the current dictionary lookup module, but both can sit side-by-side in your local trunk sandbox or build. It has an example descriptor that tells it to read a bar-separated value file (BSV) as a dictionary, storing it (indexed) in memory for fast lookup. There is an example dictionary and xml descriptor for that dictionary. It accepts 2 or 3 column files in the format CUI|Text or CUI|TUI|Text. It automatically detects the number of columns, but they must be in that order. It also does not need the text fields to be tokenized, allowing it to accept "Tumor, malignant" as well as "tumor , malignant" as it will perform the tokenization upon reading the file. As the dictionary will be stored in-memory it should not be huge. If you do have a very large number of terms (>50k) then I recommend an hsql db. The new module will take an hsql db with the fixed field names CUI, TUI, RINDEX, TCOUNT, TEXT, RWORD. I will explain what those mean in some documentation that I plan to check into sandbox later today, but I can help you build an hsql dictionary db ... Yesterday I checked into sandbox a project named "dictionarytool". It is source-only, but I can give you a jar if you want one. Out-of-the-box it will build various dictionaries from a UMLS download. It can build BSV, Hsql (new format) and Hsql (current format) to be used by the new or current dictionary lookup modules. This devlist announcement is a little premature on my part. I will not get usage documentation into sandbox for a day or two, but I can send you copies as I go if you are in a hurry, or just give you xml snippets for the current module descriptors. If you send the format of your dictionary then that can be done quickly. I just wanted to let you know that there is another option wrt dictionary lookup. Sean -----Original Message----- From: Abhishek De [mailto:abhishek...@alumnux.com] Sent: Friday, February 28, 2014 6:58 AM To: dev@ctakes.apache.org Subject: How to add a new dictionary database to cTAKES Hi, How do I add a new database to the cTAKES pipeline to perform lookup from? How do I specify what columns to look up and how to annotate the text with the returned hits? I have gone through the DictionaryLookupAnnotatorDB.xml and LookupDesc_Db.xml files. However, I could not understand the meanings of the terms like "lookupField", "metaField", "maxPermutationLevel" and "exclusionTags". If I add a new database, I need to configure this xml file properly. Please guide me regarding these problems. Thanks and Regards, Abhishek De
