Hi Shyam,

I'm not sure what the [4] is doing in your nextLine String processing.

That aside, are you seeing the pipeline being initiated multiple times?  This 
could be the problem.

Your file reader looks nice, but as I advised in my last email, give 
LinesFromFileCollectionReader a try.  Instead of creating a new cas object and 
initializing the pipeline once per line, this will allow ctakes to reuse a 
single cas object and initialize the pipeline only once.

Sean

-----Original Message-----
From: Ks Sunder [mailto:shyam...@gmail.com] 
Sent: Friday, January 13, 2017 1:11 AM
To: dev@ctakes.apache.org
Subject: Re: Allergy Annotator

Thanq Sean,

   I have done coding for this  read the csv file purpose im using java, but 
cTakes UML Dictionary purpose I am using below fuction.


 public  AnalysisEngineDescription getUMLPipeline() throws 
ResourceInitializationException, URISyntaxException{
   AggregateBuilder builder = new AggregateBuilder();
   builder.add(SimpleSegmentAnnotator.createAnnotatorDescription());
   builder.add(SentenceDetector.createAnnotatorDescription());
   builder.add(TokenizerAnnotatorPTB.createAnnotatorDescription());
   builder.add(POSTagger.createAnnotatorDescription());
   builder.add(ClinicalPipelineFactory.getNpChunkerPipeline());
   builder.add(LvgAnnotator.createAnnotatorDescription());

     try {
         builder.add( AnalysisEngineFactory.createEngineDescription(
DefaultJCasTermAnnotator.class,
              AbstractJCasTermAnnotator.PARAM_WINDOW_ANNOT_PRP,
              "org.apache.ctakes.typesystem.type.textspan.Sentence",
              JCasTermAnnotator.DICTIONARY_DESCRIPTOR_KEY,
              ExternalResourceFactory.createExternalResourceDescription(
                    FileResourceImpl.class,
                    FileLocator.locateFile( 
"org/apache/ctakes/dictionary/lookup/fast/cTakesHsql.xml" )
                    )
        ) );
     } catch ( FileNotFoundException e ) {
        e.printStackTrace();
        throw new ResourceInitializationException( e );
     }

   return builder.createAggregateDescription();
 }


and next I am calling this fuction from here......



 reader = new CSVReader(new FileReader(ExelReadJava.NarrativeFile));
 String [] nextLine;
 int lineNumber = 0;


 while ((nextLine = reader.readNext()) != null) {
   lineNumber++;
   System.out.println("Line # " + lineNumber);

    //UML code start
      try {
if(nextLine[4].length()>1 ){

final JCas jcas = JCasFactory.createJCas(); jcas.setDocumentText( nextLine[4] 
); SimplePipeline.runPipeline(jcas, pipelineTesting.getUMLPipeline());

for ( IdentifiedAnnotation entity : JCasUtil.select( jcas, 
IdentifiedAnnotation.class ) ) {
     if(entity.getOntologyConceptArr() != null){

    add.append(entity.getCoveredText()+ ",");
     }
}


this function working properly , but processing time one line per 40sec, how 
can decrease the processing time .

i have 1lakh records(lines) in a csv file.

please give me a solution and example......





regards,
shyam k.

On Thu, Jan 12, 2017 at 8:48 PM, Finan, Sean < 
sean.fi...@childrens.harvard.edu> wrote:

> Hi Shyam,
>
> Have a look at the LinesFromFileCollectionReader class in ctakes-core.  
> It doesn't use csv files, but instead treats every newline character 
> as a separator.
>
> Sean
>
> -----Original Message-----
> From: Ks Sunder [mailto:shyam...@gmail.com]
> Sent: Wednesday, January 11, 2017 1:29 AM
> To: dev@ctakes.apache.org
> Subject: Re: Allergy Annotator
>
> Hi All,
>
> my scenario is, read the string content from csv file, and find out 
> medical terms from that content using cTakes UML.
>
> as per your suggestion i try to find CollectionReader in ctakes-core, 
> but i didnt get clear solution, please give valuable solution, and one 
> example.
>
>
> regards,
> shyam k.
>
> On Thu, Dec 22, 2016 at 9:16 PM, Finan, Sean < 
> sean.fi...@childrens.harvard.edu> wrote:
>
> > Hi Shyam,
> >
> > I think that the key to your first question
> > >   how can execute the single function to run all this jobs in 
> > > short
> > time...
> > Is in your code here:
> >
> > 1       final JCas jcas = JCasFactory.createJCas();
> > 2       jcas.setDocumentText( nextLine[0] );
> > 3       SimplePipeline.runPipeline(jcas, getUMLPipeline());
> >
> > What you probably want to do is replace lines #1 and #2 with a 
> > CollectionReader, and then in #3 use a different SimplePipeline call 
> > that runs the pipeline using the CollectionReader instead of a 
> > static
> cas.
> >
> > There are commonly used CollectionReaders in ctakes-core.  The most 
> > widely applicable is probably the FileTreeReader*, which reads a 
> > tree of ascii files.  If you have some other source of text data 
> > then look around the code for something that might fit and let the 
> > devlist know if you can't find anything that fits your needs.
> >
> > I don't understand your second question:
> > > how can i find sentence vised Dictionary words from string, give 
> > > me a
> > solution for this..
> > Can you rephrase it and post to the devlist again?
> >
> > * one advantage that the FileTreeReader has is that it stores 
> > metadata on the input file tree placement, which can then be 
> > reproduced by output file writers like the html writer.
> >
> > Sean
> >
> >
> > -----Original Message-----
> > From: Ks Sunder [mailto:shyam...@gmail.com]
> > Sent: Thursday, December 22, 2016 2:33 AM
> > To: dev@ctakes.apache.org
> > Subject: Re: Allergy Annotator
> >
> > Hi All,
> >
> > I have done the below code for finding medical terms from String 
> > information.
> >
> > step 1 :
> > public static AnalysisEngineDescription getUMLPipeline() throws 
> > ResourceInitializationException, URISyntaxException{
> >    AggregateBuilder builder = new AggregateBuilder();
> >    builder.add(SimpleSegmentAnnotator.createAnnotatorDescription());
> >    builder.add(SentenceDetector.createAnnotatorDescription());
> >    builder.add(TokenizerAnnotatorPTB.createAnnotatorDescription());
> >    builder.add(POSTagger.createAnnotatorDescription());
> >    builder.add(ClinicalPipelineFactory.getNpChunkerPipeline());
> >    builder.add(LvgAnnotator.createAnnotatorDescription());
> >
> >      try {
> >          builder.add( AnalysisEngineFactory.createEngineDescription(
> > DefaultJCasTermAnnotator.class,
> >               AbstractJCasTermAnnotator.PARAM_WINDOW_ANNOT_PRP,
> >               "org.apache.ctakes.typesystem.type.textspan.Sentence",
> >               JCasTermAnnotator.DICTIONARY_DESCRIPTOR_KEY,
> >               ExternalResourceFactory.createExternalResourceDescription(
> >                     FileResourceImpl.class,
> >                     FileLocator.locateFile(
> "org/apache/ctakes/dictionary/lookup/fast/cTakesHsql.xml"
> > ) )
> >         ) );
> >      } catch ( FileNotFoundException e ) {
> >         e.printStackTrace();
> >         throw new ResourceInitializationException( e );
> >      }
> >
> >    return builder.createAggregateDescription();
> >  }
> > step 2:
> >
> > final JCas jcas = JCasFactory.createJCas(); jcas.setDocumentText( 
> > nextLine[0] ); SimplePipeline.runPipeline(jcas, getUMLPipeline());
> >
> > for ( IdentifiedAnnotation entity : JCasUtil.select( jcas, 
> > IdentifiedAnnotation.class ) ) {
> >
> >          if(entity.getOntologyConceptArr() != null){
> >
> >         add.append(entity.getCoveredText()+ ",");
> >
> >          }
> > }
> >
> >
> >
> >
> >
> > its working Fine..
> >
> > But i have two quires..
> >
> > 1. step1 , i am using Annotator step by step ... that time its 
> > taking more time load the all fuctions
> >    how can execute the single function to run all this jobs in short 
> > time...
> >
> > 2. how can i find sentence vised Dictionary words from string, give 
> > me a solution for this..
> >
> >
> > ...please give me a solutions for this issues....
> >
> >
> >
> > regards,
> > shyam k.
> >
> > On Thu, Dec 8, 2016 at 1:59 AM, Mullane, Sean *HS < 
> > sp...@hscmail.mcc.virginia.edu> wrote:
> >
> > > I'm reviving this thread with reference to negation detection. I 
> > > previously posted about this to the User list but this is probably 
> > > a more appropriate venue.
> > >
> > > The way the sentences are split on ":" makes the negation 
> > > annotator miss negation in lists of this form:
> > >
> > > Hyperlipidemia:  Yes
> > > Hypercholesterolemia:  No
> > > Chronic Renal Insufficiency:  N/A
> > >
> > > I tried reversing order and removing ":"s and found that the 
> > > negation for Hypercholesterolemia is detected when in this form:
> > >
> > > Yes Hyperlipidemia
> > > No Hypercholesterolemia
> > > N/A Chronic Renal Insufficiency
> > >
> > > Our notes have quite a few places with this sort of list where 
> > > good negation detection is important but I haven't very good 
> > > results. The sentence segmentator sees this as 12 separate 
> > > sentences, but I would think proper behavior would be to consider 
> > > this as 6 sentences (breaking sentences on line break but not on 
> > > colons). I see previous discussion on the list about the sentence 
> > > segmentator breaking on newlines but little regarding colons. I 
> > > would think in most cases it would be more useful not to break on 
> > > ":". Or is there an overriding
> > reason for the current behavior?
> > > If changing the sentence segmentator isn't an option is there a 
> > > different way to configure the negation detection annotator that 
> > > would avoid this issue?
> > >
> > > Thanks,
> > > Sean
> > >
> > >
> > >
> > > Hi,
> > >
> > > I am interested in the design decision of the sentence detector.
> > >
> > > Why does it split a sentence of the form "WORD1: WORD2 WORD3." 
> > > into two sentences "WORD1:" and "WORD2 WORD3."? Do other 
> > > components of cTAKES require such a sentence splitting?
> > >
> > > It would seem to me that it should remain one sentence. For 
> > > example, the smoking status detector has its own SentenceAdjuster 
> > > that merges some of such sentences back into one, because of this design.
> > >
> > > Thanks, Tomasz
> > >
> > > ________________________________________ From: Finan, Sean [ 
> > > sean...@childrens.harvard.edu] Sent: Friday, July 10, 2015 3:20 PM To:
> > > de...@ctakes.apache.org Subject: RE: Allergy Annotator
> > >
> > > Hi Tom,
> > >
> > > It is exactly because the sentence detector splits "KEY:" from "VALUE"
> > > that I
> > > didn't suggest using sentences. Instead, I would just iterate over 
> > > the whole cas collection of medication events and attempt to match 
> > > allergy phrases ("allergic to medication") with text the note 
> > > spanning from
> > > event.begin-15 to
> > > event.end+15 or whatever window size you prefer.
> > >
> > > Sean
> > >
> > > -----Original Message----- From: Tom Devel 
> > > [mailto:deve...@gmail.com]
> > > Sent: Friday, July 10, 2015 4:12 PM To: de...@ctakes.apache.org
> Subject:
> > > Re: Allergy Annotator
> > >
> > > Sean and Dima, these are great suggestions, thanks so far.
> > >
> > > Sean, when looping over medication events as you say, I can see 
> > > how it is possible to take the textspan.Sentence of this 
> > > MedicationMention, and then do a regex check for the phrase 
> > > structure
> as Dima said.
> > >
> > > But instead of textspan.Sentence, you mention "see any is included 
> > > in a phrase".
> > > What cTAKES/UIMA class is related to this?
> > >
> > > Because if I would use textspan.Sentence, it would work for "The 
> > > patient is allergic to penicillin.", but cTAKES splits "ALLERGIES:
> > PENICILLIN, WHEAT"
> > > into two sentences, so that the MedicationMentions here would not 
> > > be in the same sentence as the word "ALLERGIES".
> > >
> > > Thanks again, Tom
> > >
> > > On Fri, Jul 10, 2015 at 2:12 PM, Finan, Sean < 
> > > sean...@childrens.harvard.edu>
> > > wrote:
> > >
> > > Hi Dima, Tom,
> > >
> > > I was thinking the same as Dima's first solution. Iterate through 
> > > the medication events and see any is included in a phrase as 
> > > mentioned in Tom's original email. Each phrase structure would 
> > > have to be specified beforehand. However, assigning appropriate 
> > > CUIs would require having a lookup table for each medication 
> > > allergy. I think that would be the simplest solution.
> > >
> > > Sean
> > >
> > > -----Original Message----- From: Dligach, Dmitriy [mailto:
> > > dmit...@childrens.harvard.edu] Sent: Friday, July 10, 2015 2:50 PM To:
> > > cTAKES Developer list Subject: Re: Allergy Annotator
> > >
> > > Hi Tom,
> > >
> > > If the patters are pretty simple, you could just add a few rules 
> > > on top of the cTAKES dictionary lookup output. Something of the 
> > > kind "allergic to <medication>" or "allergies: <medication1>, 
> > > <medication2>, <substance1>, ...".
> > >
> > > If these patterns are hard to express as rules, you should 
> > > consider a machine learning based sequence labeling route (e.g. 
> > > something similar to the cTAKES chunker).
> > >
> > > Dima
> > >
> > > -- Dmitriy (Dima) Dligach, Ph.D. Boston Children's Hospital and 
> > > Harvard Medical School (617) 651-0397
> > >
> > > On Jul 10, 2015, at 13:40, Tom Devel <deve...@gmail.com<mailto:
> > > deve...@gmail.com>> wrote:
> > >
> > > Sean,
> > >
> > > It would be a wider net, such that if an allergy is mentioned in 
> > > the clinical note, this is captured in the corresponding 
> > > IdentifiedAnnotation (or alternatively, if the 
> > > IdentifiedAnnotation class should not be changed with a new 
> > > attribute, in a separate allergy annotation).
> > >
> > > This annotator would then have to of course run after the clinical 
> > > pipeline has run and discovered all IdentifiedAnnotations.
> > >
> > > I am familiar with writing UIMA/cTAKES annotators, but not sure 
> > > how a new ML method could be integrated here for detecting 
> > > allergies. Do you have any thoughts about how to approach this in general?
> > >
> > > Thanks, Tom
> > >
> > > On Fri, Jul 10, 2015 at 11:54 AM, Finan, Sean < 
> > > sean...@childrens.harvard.edu<mailto:Sean.Finan@childrens.harvard.
> > > e
> > > du>>
> > > wrote:
> > >
> > > Hi Tom,
> > >
> > > Are you interested in catching all allergies or just a few 
> > > specific allergies for a study? If you are only concerned with a 
> > > few then there is a
> > > (possibly) simple solution. If you are interested in throwing a 
> > > wider net then I think that a new module would need to be created; 
> > > does anybody reading this have an ML or regex style module?
> > >
> > > Sean
> > >
> > > -----Original Message----- From: Tom Devel 
> > > [mailto:deve...@gmail.com]
> > > Sent: Friday, July 10, 2015 12:42 PM To: de...@ctakes.apache.org<
> mailto:
> > > de...@ctakes.apache.org> Subject: Allergy Annotator
> > >
> > > Hi,
> > >
> > > I would like to use/extend cTAKES to detect allergies.
> > >
> > > In the cTAKES publication (2010)
> > >
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ncbi.nlm.n
> > > ih
> > > .g
> > > ov_pmc_articles_PMC2995668_&d=BQIFaQ&c=qS4goWBT7poplM69zy_3xhKwEW1
> > > 4J
> > > ZM
> > > SdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=ZApJm
> > > GK
> > > jz
> > > vFfNco5rRFVwSIyxmg4MRsxakfuXHbMZME&s=mGWu0XBCJqG2MI5qPlwIpGbQL5IYe
> > > 7t 5E WcvhPYW7Lo&e= there is the mention that: "Allergies to a 
> > > given medication are handled by setting the negation attribute of 
> > > that medication to 'is negated'."
> > >
> > > However, in a post here in 2014 (RE: Allergy Indication) it is 
> > > said that cTAKES does not have a module for allergy discovery.
> > >
> > > 1. What is the current status of allergy detection in cTAKES?
> > >
> > > 2. I did some testing, while cTAKES discovers concepts about 
> > > allegies ("wheat allergy" is found as C0949570), using "ALLERGIES:
> > > PENICILLIN, WHEAT" or "The patient is allergic to penicillin." 
> > > does not give penicillin or wheat annotations allergy status.
> > >
> > > How would I go about detecting these allergy mentions?
> > >
> > > Thanks, Tom
> > >
> > >
> >
>

Reply via email to