RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Thanks a lot for the response Chen. Appreciate your time.
Regards,
Gandhi
-Original Message-
From: Lin, Chen [mailto:chen@childrens.harvard.edu]
Sent: Wednesday, March 14, 2018 9:20 PM
To: dev@ctakes.apache.org
Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
[SUSPICIOUS]
Thank you for testing the temporal model in detail. As I can recall, the
changes made between the two versions include added features and tuned
parameters. We do see significant improvement in our test corpus so that we
switch to the new model. The improvement is gauged statistically, not for every
single instance. I am sure there would be some instances that the demo version
cannot pick up but our new model can.
For your listed case, it's an event-event temporal relation. Event-event
relations generally have a lower annotation quality (lower Inter Annotator
Agreement). Through development we do observe the annotation quality decreases
as the distance increases. We did try enforcing to reason event-event arguments
that are within a certain distance like six-word.
But for your case, it's still considered as a good distance. So I would say
it's a system error, assuming you can identify both events("HBA1c" and
"good") correctly using the current system.
Best,
Chen
On 3/14/18, 11:20 AM, "Gandhi Rajan Natarajan"
<gandhi.natara...@arisglobal.com> wrote:
>Hi Chen/Sean,
>
>As per our earlier discussion, I was under the impression that tlinks
>will not be able to pick only the long distance relationship in the
>latest version of cTAKES. But even when I try smaller text like the one
>below, the tlinks is not getting picked up:
>
>"HBA1c is normal which is good"
>
>But when I try out the same text in demo URL -
>https://urldefense.proofpoint.com/v2/url?u=http-3A__54.68.117.30-3A8080
>_in
>dex.jsp=DwIFAw=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=PZ241C
>wYZ
>3AszaTEBtM2wl3EcIjNNNeKX8q7N_mt-aI=SEMrHyMpFRLj8NCb6iYM5ST92ybV8iqKkY
>2b2 iu7brc=iC5PBgT3lI7HbZ0xGDGBRF_ADnEnyWlgTTGOHbTCjMo= , I get the
>following tlink:
>
>" HBA1c CONTAINS good"
>
>Any change in the latest version impacted this functionality or is it
>the expected behavior? Could you please provide some more info on this?
>
>Regards,
>Gandhi
>
>-Original Message-
>From: Gandhi Rajan Natarajan [mailto:gandhi.natara...@arisglobal.com]
>Sent: Monday, February 12, 2018 11:00 PM
>To: dev@ctakes.apache.org
>Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS]
>[SUSPICIOUS] [SUSPICIOUS]
>
>Thanks Sean and Chen for your response. I really appreciate your time.
>
>Regards,
>Gandhi
>
>
>-----Original Message-
>From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu]
>Sent: Monday, February 12, 2018 9:18 PM
>To: dev@ctakes.apache.org
>Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS]
>[SUSPICIOUS] [SUSPICIOUS]
>
>Hi Chen,
>
>No worries! I often throw out answers before checking code myself. We
>are all busy with other items. Besides, If I recall (without checking
>code), it is because of an improvement in the Event annotator code that
>the Copy*** was no longer needed ... and you wrote that improvement!
>hurrah!
>
>I thank you for taking the time to answer questions on temporal
>matters, as well as being one of the developers involved working on the module!
>
>Cheers,
>Sean
>
>-Original Message-
>From: Lin, Chen [mailto:chen@childrens.harvard.edu]
>Sent: Monday, February 12, 2018 10:43 AM
>To: dev@ctakes.apache.org
>Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS]
>[SUSPICIOUS] [SUSPICIOUS]
>
>Thanks to Sean. My bad, I should check the code first before any answer
>attempt.
>
>Best,
>Chen
>
>On 2/12/18, 10:40 AM, "Finan, Sean" <sean.fi...@childrens.harvard.edu>
>wrote:
>
>>It should have nothing to do with the tlink extraction. It copied
>>properties from events to eventmentions and was in the pipeline after
>>the tlink annotation, not before.
>>
>>-Original Message-
>>From: Lin, Chen [mailto:chen@childrens.harvard.edu]
>>Sent: Monday, February 12, 2018 10:33 AM
>>To: dev@ctakes.apache.org
>>Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS]
>>
>>Mmm. Frankly, I don't know the rational behind commenting out
>>"FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator"
>>in that piper file. The TLINK model does take some of the event
>>properties (polarity, uncertainty, contextual modality, semantic
>>types,
>>etc.) for classification features. Without those features the
>>classifier may lose its grip. Have you tried commenting back the lin
Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Thank you for testing the temporal model in detail. As I can recall, the changes made between the two versions include added features and tuned parameters. We do see significant improvement in our test corpus so that we switch to the new model. The improvement is gauged statistically, not for every single instance. I am sure there would be some instances that the demo version cannot pick up but our new model can. For your listed case, it’s an event-event temporal relation. Event-event relations generally have a lower annotation quality (lower Inter Annotator Agreement). Through development we do observe the annotation quality decreases as the distance increases. We did try enforcing to reason event-event arguments that are within a certain distance like six-word. But for your case, it’s still considered as a good distance. So I would say it’s a system error, assuming you can identify both events(“HBA1c" and "good") correctly using the current system. Best, Chen On 3/14/18, 11:20 AM, "Gandhi Rajan Natarajan" <gandhi.natara...@arisglobal.com> wrote: >Hi Chen/Sean, > >As per our earlier discussion, I was under the impression that tlinks >will not be able to pick only the long distance relationship in the >latest version of cTAKES. But even when I try smaller text like the one >below, the tlinks is not getting picked up: > >"HBA1c is normal which is good" > >But when I try out the same text in demo URL - >https://urldefense.proofpoint.com/v2/url?u=http-3A__54.68.117.30-3A8080_in >dex.jsp=DwIFAw=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=PZ241CwYZ >3AszaTEBtM2wl3EcIjNNNeKX8q7N_mt-aI=SEMrHyMpFRLj8NCb6iYM5ST92ybV8iqKkY2b2 >iu7brc=iC5PBgT3lI7HbZ0xGDGBRF_ADnEnyWlgTTGOHbTCjMo= , I get the >following tlink: > >" HBA1c CONTAINS good" > >Any change in the latest version impacted this functionality or is it the >expected behavior? Could you please provide some more info on this? > >Regards, >Gandhi > >-Original Message- >From: Gandhi Rajan Natarajan [mailto:gandhi.natara...@arisglobal.com] >Sent: Monday, February 12, 2018 11:00 PM >To: dev@ctakes.apache.org >Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] >[SUSPICIOUS] > >Thanks Sean and Chen for your response. I really appreciate your time. > >Regards, >Gandhi > > >-----Original Message- >From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu] >Sent: Monday, February 12, 2018 9:18 PM >To: dev@ctakes.apache.org >Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] >[SUSPICIOUS] > >Hi Chen, > >No worries! I often throw out answers before checking code myself. We >are all busy with other items. Besides, If I recall (without checking >code), it is because of an improvement in the Event annotator code that >the Copy*** was no longer needed ... and you wrote that improvement! >hurrah! > >I thank you for taking the time to answer questions on temporal matters, >as well as being one of the developers involved working on the module! > >Cheers, >Sean > >-----Original Message- >From: Lin, Chen [mailto:chen@childrens.harvard.edu] >Sent: Monday, February 12, 2018 10:43 AM >To: dev@ctakes.apache.org >Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] >[SUSPICIOUS] > >Thanks to Sean. My bad, I should check the code first before any answer >attempt. > >Best, >Chen > >On 2/12/18, 10:40 AM, "Finan, Sean" <sean.fi...@childrens.harvard.edu> >wrote: > >>It should have nothing to do with the tlink extraction. It copied >>properties from events to eventmentions and was in the pipeline after >>the tlink annotation, not before. >> >>-Original Message- >>From: Lin, Chen [mailto:chen@childrens.harvard.edu] >>Sent: Monday, February 12, 2018 10:33 AM >>To: dev@ctakes.apache.org >>Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] >> >>Mmm. Frankly, I don't know the rational behind commenting out >>"FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator" >>in that piper file. The TLINK model does take some of the event >>properties (polarity, uncertainty, contextual modality, semantic types, >>etc.) for classification features. Without those features the >>classifier may lose its grip. Have you tried commenting back the line? >> >>Best, >>Chen >> >>On 2/12/18, 10:23 AM, "Gandhi Rajan Natarajan" >><gandhi.natara...@arisglobal.com> wrote: >> >>>Hi Chen, >>> >>>Thanks a lot for your response. >>> >>>So I guess this behavior is nothing to do with commented out >>>
RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Hi Chen/Sean, As per our earlier discussion, I was under the impression that tlinks will not be able to pick only the long distance relationship in the latest version of cTAKES. But even when I try smaller text like the one below, the tlinks is not getting picked up: "HBA1c is normal which is good" But when I try out the same text in demo URL - http://54.68.117.30:8080/index.jsp , I get the following tlink: " HBA1c CONTAINS good" Any change in the latest version impacted this functionality or is it the expected behavior? Could you please provide some more info on this? Regards, Gandhi -Original Message- From: Gandhi Rajan Natarajan [mailto:gandhi.natara...@arisglobal.com] Sent: Monday, February 12, 2018 11:00 PM To: dev@ctakes.apache.org Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] Thanks Sean and Chen for your response. I really appreciate your time. Regards, Gandhi -Original Message- From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu] Sent: Monday, February 12, 2018 9:18 PM To: dev@ctakes.apache.org Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] Hi Chen, No worries! I often throw out answers before checking code myself. We are all busy with other items. Besides, If I recall (without checking code), it is because of an improvement in the Event annotator code that the Copy*** was no longer needed ... and you wrote that improvement! hurrah! I thank you for taking the time to answer questions on temporal matters, as well as being one of the developers involved working on the module! Cheers, Sean -Original Message- From: Lin, Chen [mailto:chen@childrens.harvard.edu] Sent: Monday, February 12, 2018 10:43 AM To: dev@ctakes.apache.org Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] Thanks to Sean. My bad, I should check the code first before any answer attempt. Best, Chen On 2/12/18, 10:40 AM, "Finan, Sean" <sean.fi...@childrens.harvard.edu> wrote: >It should have nothing to do with the tlink extraction. It copied >properties from events to eventmentions and was in the pipeline after >the tlink annotation, not before. > >-Original Message- >From: Lin, Chen [mailto:chen@childrens.harvard.edu] >Sent: Monday, February 12, 2018 10:33 AM >To: dev@ctakes.apache.org >Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] > >Mmm. Frankly, I don't know the rational behind commenting out >"FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator" >in that piper file. The TLINK model does take some of the event >properties (polarity, uncertainty, contextual modality, semantic types, >etc.) for classification features. Without those features the >classifier may lose its grip. Have you tried commenting back the line? > >Best, >Chen > >On 2/12/18, 10:23 AM, "Gandhi Rajan Natarajan" ><gandhi.natara...@arisglobal.com> wrote: > >>Hi Chen, >> >>Thanks a lot for your response. >> >>So I guess this behavior is nothing to do with commented out >>'FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator' >>in ' TemporalSubPipe.piper'? >> >>Thanks again. >> >>Regards, >>Gandhi >> >>-Original Message- >>From: Lin, Chen [mailto:chen@childrens.harvard.edu] >>Sent: Monday, February 12, 2018 8:47 PM >>To: dev@ctakes.apache.org >>Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] >> >>Hi Gandhi, >> >>It looks like the newest version is picking up the temporal expression >>complete, ³12th June 2018². The previous version only partially pick >>up ³June 2018². It is due to the the improvement of the timex annotator. >>As for the relations(tlinks), the current distributed TLINK model is a >>conventional SVM-based model which is tuned to favor close relation >>for high precision, like "12th June 2018 CONTAINS paracetamol². It can >>pick up some longer-distance relationship as well, but obliviously it >>fails for your case for long distance relations. We are currently >>researching a LSTM-based model, which can take more consideration of >>sentence structure and long-distance relationship. Will update more >>details in the future. >> >>Best, >>Chen >> >>On 2/12/18, 7:50 AM, "Gandhi Rajan Natarajan" >><gandhi.natara...@arisglobal.com> wrote: >> >>>Hi, >>> >>>I have a question related to cTAKES Tlinks. >>> >>>Is there any particular reason why >>>FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator >>>is commented in TemporalSubPipe.piper? >&
RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Thanks Sean and Chen for your response. I really appreciate your time. Regards, Gandhi -Original Message- From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu] Sent: Monday, February 12, 2018 9:18 PM To: dev@ctakes.apache.org Subject: RE: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] Hi Chen, No worries! I often throw out answers before checking code myself. We are all busy with other items. Besides, If I recall (without checking code), it is because of an improvement in the Event annotator code that the Copy*** was no longer needed ... and you wrote that improvement! hurrah! I thank you for taking the time to answer questions on temporal matters, as well as being one of the developers involved working on the module! Cheers, Sean -Original Message- From: Lin, Chen [mailto:chen@childrens.harvard.edu] Sent: Monday, February 12, 2018 10:43 AM To: dev@ctakes.apache.org Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] Thanks to Sean. My bad, I should check the code first before any answer attempt. Best, Chen On 2/12/18, 10:40 AM, "Finan, Sean" <sean.fi...@childrens.harvard.edu> wrote: >It should have nothing to do with the tlink extraction. It copied >properties from events to eventmentions and was in the pipeline after >the tlink annotation, not before. > >-Original Message- >From: Lin, Chen [mailto:chen@childrens.harvard.edu] >Sent: Monday, February 12, 2018 10:33 AM >To: dev@ctakes.apache.org >Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] [SUSPICIOUS] > >Mmm. Frankly, I don't know the rational behind commenting out >"FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator" >in that piper file. The TLINK model does take some of the event >properties (polarity, uncertainty, contextual modality, semantic types, >etc.) for classification features. Without those features the >classifier may lose its grip. Have you tried commenting back the line? > >Best, >Chen > >On 2/12/18, 10:23 AM, "Gandhi Rajan Natarajan" ><gandhi.natara...@arisglobal.com> wrote: > >>Hi Chen, >> >>Thanks a lot for your response. >> >>So I guess this behavior is nothing to do with commented out >>'FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator' >>in ' TemporalSubPipe.piper'? >> >>Thanks again. >> >>Regards, >>Gandhi >> >>-Original Message- >>From: Lin, Chen [mailto:chen@childrens.harvard.edu] >>Sent: Monday, February 12, 2018 8:47 PM >>To: dev@ctakes.apache.org >>Subject: Re: Regarding cTAKES Tlinks [EXTERNAL] >> >>Hi Gandhi, >> >>It looks like the newest version is picking up the temporal expression >>complete, ³12th June 2018². The previous version only partially pick >>up ³June 2018². It is due to the the improvement of the timex annotator. >>As for the relations(tlinks), the current distributed TLINK model is a >>conventional SVM-based model which is tuned to favor close relation >>for high precision, like "12th June 2018 CONTAINS paracetamol². It can >>pick up some longer-distance relationship as well, but obliviously it >>fails for your case for long distance relations. We are currently >>researching a LSTM-based model, which can take more consideration of >>sentence structure and long-distance relationship. Will update more >>details in the future. >> >>Best, >>Chen >> >>On 2/12/18, 7:50 AM, "Gandhi Rajan Natarajan" >><gandhi.natara...@arisglobal.com> wrote: >> >>>Hi, >>> >>>I have a question related to cTAKES Tlinks. >>> >>>Is there any particular reason why >>>FullTemporalExtractionPipeline.CopyPropertiesToTemporalEventAnnotator >>>is commented in TemporalSubPipe.piper? >>> >>>Also if I run the following text - "I had fever due to smoking so >>>took paracetomol on 12th June 2018" in temporal demo link - >>>https://urldefense.proofpoint.com/v2/url?u=http-3A__54.68.117.30-3A80 >>>8 >>>0 >>>_in >>>dex.jsp=DwIFAg=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=PZ24 >>>1 >>>C >>>wYZ >>>3AszaTEBtM2wl3EcIjNNNeKX8q7N_mt-aI=CTJyQ3_c8JXxv2h1gPaq-jZ6hwp-2dKT >>>G q IrJ RkC7Yk=dOVZEd9mrbu0NtEHUZorIzmzEFC_ghg3l8l53ed4O_M= , I >>>get the following TLinks: >>> >>>"June 2018 CONTAINS fever , June 2018 CONTAINS smoking , June 2018 >>>CONTAINS paracetomol" >>> >>>But if I run the same text in the latest cTAKES using >>>TemporalSubPipe.piper, I get onl
