Re: AggregateCdaUmlsprocessor only annotates last section of CDA document [EXTERNAL]

[gandhi rajan]

Sean, Thanks for the response. I m interested in testing this but
unfortunately the infrastructure issues I have currently is stopping me
from doing this. That's why the apology for my helplessness currently. Will
have a check once my setup is ready but at this point the timelines are not
clear to me.

On Friday, January 11, 2019, Finan, Sean 
wrote:

> Hi Gandhi,
>
> It would be great if you could test this!  Why are you sorry?
>
> - I shouldn't have said "let me know", but rather "let the group know" or
> something like that.
>
>
> Cheers,
> Sean
>
> 
> From: gandhi rajan 
> Sent: Friday, January 11, 2019 12:35 PM
> To: dev@ctakes.apache.org
> Subject: Re: AggregateCdaUmlsprocessor only annotates last section of CDA
> document [EXTERNAL]
>
> Sorry Sean. I would love to do this but my old setup is all gone with my
> previous organization. Gotta do a fresh setup in my personal machine all
> over again. Sorry about that.
>
> On Friday, January 11, 2019, Finan, Sean  >
> wrote:
>
> > Hi Sana,
> >
> > This might be related to
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_CTAKES-2D450-3Ffilter-3D=DwIBaQ=
> qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=
> fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao=GSH42JyRAyTSQdrgKBYhzxD-
> 6j2sEhEwivjJTCh6YOQ=9rfxDksaU1nnkKao1IAuhngpM8fW0eVT8YWMcY-gne0=
> > -5=project%20%3D%20CTAKES%20AND%20resolution%20%3D%
> > 20Unresolved%20AND%20%22Attachment%20count%22%20%3C%
> 3D%20%222%22%20AND%20%
> > 22Attachment%20count%22%20%3E%3D%20%221%22%20order%20by%
> > 20priority%20DESC%2Cupdated%20DESC
> >
> > If anybody has time to test and approve the patch attached to that tar
> > please let me know so that it can be checked in.
> >
> > Thanks,
> > Sean
> > 
> > From: Sana Riaz 
> > Sent: Friday, January 11, 2019 5:33 AM
> > To: dev@ctakes.apache.org
> > Subject: AggregateCdaUmlsprocessor only annotates last section of CDA
> > document [EXTERNAL]
> >
> > Hi,
> > I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
> > descriptor (clinical-pipeline). The cda document includes sections like
> > problems, medications, allergies, tests etc. In the plain_view, all these
> > section are visible in CVD but all the annotations extracted by
> > AggregateCdaUMLSProcessor are only on last section. i.e. there's no
> > annotation on the medications or problems.
> >
> > I've looked into CdaCasInitializer output , and it only passes one
> segment
> > (the last one) so all the other annotators only process on that. In
> > addition to that, every section's id (including last) is assigned null as
> > [start section id="null"]
> > 
> > [end section id="null"]
> >
> > Do i have to assign section id's myself? Any suggestion would be very
> > helpful.
> >
> > Warm Regards,
> >
> > Sana Riaz
> >
>
>
> --
> Regards,
> Gandhi
>
> "The best way to find urself is to lose urself in the service of others
> !!!"
>


-- 
Regards,
Gandhi

"The best way to find urself is to lose urself in the service of others !!!"

Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]

[gandhi rajan]

Completely agree. Kalyani, you could give it a try testing with MySQL DB.

On Friday, January 11, 2019, Finan, Sean 
wrote:

> Most likely.   That is the old dictionary lookup, which keeps open access
> to a file database.
> 
> From: gandhi rajan 
> Sent: Friday, January 11, 2019 12:39 PM
> To: dev@ctakes.apache.org
> Subject: Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]
>
> Hi, Have faced issues related to connectivity timeout using mysql db. But
> never faced concurrency issue in our testing. Is it hsql db specific by any
> chance?
>
> On Friday, January 11, 2019, Finan, Sean  >
> wrote:
>
> > Hi Kalyani,
> >
> > At a guess, each parallel instance  of ctakes is attempting to write some
> > log file concurrently and the system does not approve.
> >
> > Have other people out there witnessed and overcome this problem?
> >
> > Sean
> > 
> > From: Kalyani Unnikrishnan 
> > Sent: Friday, January 11, 2019 5:15 AM
> > To: dev@ctakes.apache.org
> > Subject: Not able to run cTAKES UMLS in parallel [EXTERNAL]
> >
> > Hi,
> >
> > I am trying to make cTAKES accessible by multiple programs in parallel. I
> > was able to remove the error on umls.lck by setting PARAMS_ISOLATION to
> > TRANSACTION_SERIALIZABLE. But I am still getting the following error when
> > multiple programs are running the pipeline at the same time (it works
> fine
> > if it is run in sequence).
> >
> > Caused by: java.sql.SQLException: file input/output error:
> > org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> > at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> > at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> > at org.hsqldb.jdbc.JDBCConnection.(Unknown Source)
> > at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> > at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> > at java.sql.DriverManager.getConnection(DriverManager.java:664)
> > at java.sql.DriverManager.getConnection(DriverManager.java:247)
> > at org.apache.ctakes.core.resource.JdbcConnectionResourceImpl.
> > load(JdbcConnectionResourceImpl.java:109)
> > ... 47 more
> > Caused by: org.hsqldb.HsqlException: file input/output error:
> > org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> > at org.hsqldb.error.Error.error(Unknown Source)
> > at org.hsqldb.error.Error.error(Unknown Source)
> > at org.hsqldb.persist.Log.openLog(Unknown Source)
> > at org.hsqldb.persist.Log.open(Unknown Source)
> > at org.hsqldb.persist.Logger.open(Unknown Source)
> > at org.hsqldb.Database.reopen(Unknown Source)
> > at org.hsqldb.Database.open(Unknown Source)
> > at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
> > at org.hsqldb.DatabaseManager.newSession(Unknown Source)
> > ... 53 more
> >
> > Appreciate any inputs.
> >
> > Regards,
> >
> > Kalyani
> >
>
>
> --
> Regards,
> Gandhi
>
> "The best way to find urself is to lose urself in the service of others
> !!!"
>


-- 
Regards,
Gandhi

"The best way to find urself is to lose urself in the service of others !!!"

Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]

[Finan, Sean]

Most likely.   That is the old dictionary lookup, which keeps open access to a 
file database.

From: gandhi rajan 
Sent: Friday, January 11, 2019 12:39 PM
To: dev@ctakes.apache.org
Subject: Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]

Hi, Have faced issues related to connectivity timeout using mysql db. But
never faced concurrency issue in our testing. Is it hsql db specific by any
chance?

On Friday, January 11, 2019, Finan, Sean 
wrote:

> Hi Kalyani,
>
> At a guess, each parallel instance  of ctakes is attempting to write some
> log file concurrently and the system does not approve.
>
> Have other people out there witnessed and overcome this problem?
>
> Sean
> 
> From: Kalyani Unnikrishnan 
> Sent: Friday, January 11, 2019 5:15 AM
> To: dev@ctakes.apache.org
> Subject: Not able to run cTAKES UMLS in parallel [EXTERNAL]
>
> Hi,
>
> I am trying to make cTAKES accessible by multiple programs in parallel. I
> was able to remove the error on umls.lck by setting PARAMS_ISOLATION to
> TRANSACTION_SERIALIZABLE. But I am still getting the following error when
> multiple programs are running the pipeline at the same time (it works fine
> if it is run in sequence).
>
> Caused by: java.sql.SQLException: file input/output error:
> org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> at org.hsqldb.jdbc.JDBCConnection.(Unknown Source)
> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:247)
> at org.apache.ctakes.core.resource.JdbcConnectionResourceImpl.
> load(JdbcConnectionResourceImpl.java:109)
> ... 47 more
> Caused by: org.hsqldb.HsqlException: file input/output error:
> org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> at org.hsqldb.error.Error.error(Unknown Source)
> at org.hsqldb.error.Error.error(Unknown Source)
> at org.hsqldb.persist.Log.openLog(Unknown Source)
> at org.hsqldb.persist.Log.open(Unknown Source)
> at org.hsqldb.persist.Logger.open(Unknown Source)
> at org.hsqldb.Database.reopen(Unknown Source)
> at org.hsqldb.Database.open(Unknown Source)
> at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
> at org.hsqldb.DatabaseManager.newSession(Unknown Source)
> ... 53 more
>
> Appreciate any inputs.
>
> Regards,
>
> Kalyani
>


--
Regards,
Gandhi

"The best way to find urself is to lose urself in the service of others !!!"

Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]

[gandhi rajan]

Hi, Have faced issues related to connectivity timeout using mysql db. But
never faced concurrency issue in our testing. Is it hsql db specific by any
chance?

On Friday, January 11, 2019, Finan, Sean 
wrote:

> Hi Kalyani,
>
> At a guess, each parallel instance  of ctakes is attempting to write some
> log file concurrently and the system does not approve.
>
> Have other people out there witnessed and overcome this problem?
>
> Sean
> 
> From: Kalyani Unnikrishnan 
> Sent: Friday, January 11, 2019 5:15 AM
> To: dev@ctakes.apache.org
> Subject: Not able to run cTAKES UMLS in parallel [EXTERNAL]
>
> Hi,
>
> I am trying to make cTAKES accessible by multiple programs in parallel. I
> was able to remove the error on umls.lck by setting PARAMS_ISOLATION to
> TRANSACTION_SERIALIZABLE. But I am still getting the following error when
> multiple programs are running the pipeline at the same time (it works fine
> if it is run in sequence).
>
> Caused by: java.sql.SQLException: file input/output error:
> org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
> at org.hsqldb.jdbc.JDBCConnection.(Unknown Source)
> at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
> at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:247)
> at org.apache.ctakes.core.resource.JdbcConnectionResourceImpl.
> load(JdbcConnectionResourceImpl.java:109)
> ... 47 more
> Caused by: org.hsqldb.HsqlException: file input/output error:
> org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
> at org.hsqldb.error.Error.error(Unknown Source)
> at org.hsqldb.error.Error.error(Unknown Source)
> at org.hsqldb.persist.Log.openLog(Unknown Source)
> at org.hsqldb.persist.Log.open(Unknown Source)
> at org.hsqldb.persist.Logger.open(Unknown Source)
> at org.hsqldb.Database.reopen(Unknown Source)
> at org.hsqldb.Database.open(Unknown Source)
> at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
> at org.hsqldb.DatabaseManager.newSession(Unknown Source)
> ... 53 more
>
> Appreciate any inputs.
>
> Regards,
>
> Kalyani
>


-- 
Regards,
Gandhi

"The best way to find urself is to lose urself in the service of others !!!"

Re: AggregateCdaUmlsprocessor only annotates last section of CDA document [EXTERNAL]

[Finan, Sean]

Hi Gandhi,

It would be great if you could test this!  Why are you sorry?

- I shouldn't have said "let me know", but rather "let the group know" or 
something like that.   


Cheers,
Sean


From: gandhi rajan 
Sent: Friday, January 11, 2019 12:35 PM
To: dev@ctakes.apache.org
Subject: Re: AggregateCdaUmlsprocessor only annotates last section of CDA 
document [EXTERNAL]

Sorry Sean. I would love to do this but my old setup is all gone with my
previous organization. Gotta do a fresh setup in my personal machine all
over again. Sorry about that.

On Friday, January 11, 2019, Finan, Sean 
wrote:

> Hi Sana,
>
> This might be related to
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CTAKES-2D450-3Ffilter-3D=DwIBaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao=GSH42JyRAyTSQdrgKBYhzxD-6j2sEhEwivjJTCh6YOQ=9rfxDksaU1nnkKao1IAuhngpM8fW0eVT8YWMcY-gne0=
> -5=project%20%3D%20CTAKES%20AND%20resolution%20%3D%
> 20Unresolved%20AND%20%22Attachment%20count%22%20%3C%3D%20%222%22%20AND%20%
> 22Attachment%20count%22%20%3E%3D%20%221%22%20order%20by%
> 20priority%20DESC%2Cupdated%20DESC
>
> If anybody has time to test and approve the patch attached to that tar
> please let me know so that it can be checked in.
>
> Thanks,
> Sean
> 
> From: Sana Riaz 
> Sent: Friday, January 11, 2019 5:33 AM
> To: dev@ctakes.apache.org
> Subject: AggregateCdaUmlsprocessor only annotates last section of CDA
> document [EXTERNAL]
>
> Hi,
> I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
> descriptor (clinical-pipeline). The cda document includes sections like
> problems, medications, allergies, tests etc. In the plain_view, all these
> section are visible in CVD but all the annotations extracted by
> AggregateCdaUMLSProcessor are only on last section. i.e. there's no
> annotation on the medications or problems.
>
> I've looked into CdaCasInitializer output , and it only passes one segment
> (the last one) so all the other annotators only process on that. In
> addition to that, every section's id (including last) is assigned null as
> [start section id="null"]
> 
> [end section id="null"]
>
> Do i have to assign section id's myself? Any suggestion would be very
> helpful.
>
> Warm Regards,
>
> Sana Riaz
>


--
Regards,
Gandhi

"The best way to find urself is to lose urself in the service of others !!!"

Re: AggregateCdaUmlsprocessor only annotates last section of CDA document [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]

[Finan, Sean]

Thanks Tim.
There is a patch in the jira item, but I don't know if it was ever applied 
(checked in).
Does anybody know?


From: Miller, Timothy 
Sent: Friday, January 11, 2019 11:32 AM
To: dev@ctakes.apache.org
Subject: Re: AggregateCdaUmlsprocessor only annotates last section of CDA 
document [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]

Looks like someone fixed that as part of a different issue:
https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CTAKES-2D500=DwIGaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao=1stOSUyvduxWyRf5ExkM407a94mlzdOtd52cdYQb3WQ=eSMr6VoSvH5sGvOa4s3OzFHOWPSUigB1hED4x2wu27A=
Tim


-Original Message-
From: "Finan, Sean" 
mailto:%22Finan,%20sean%22%20%3csean.fi...@childrens.harvard.edu%3e>>
Reply-to: 
To: dev@ctakes.apache.org 
mailto:%22...@ctakes.apache.org%22%20%3c...@ctakes.apache.org%3e>>
Subject: Re: AggregateCdaUmlsprocessor only annotates last section of CDA 
document [EXTERNAL] [SUSPICIOUS]
Date: Fri, 11 Jan 2019 16:05:21 +


Hi Sana,

This might be related to

https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CTAKES-2D450-3Ffilter-3D-2D5-26jql-3Dproject-2520-253D-2520CTAKES-2520AND-2520resolution-2520-253D-2520Unresolved-2520AND-2520-2522Attachment-2520count-2522-2520-253C-253D-2520-25222-2522-2520AND-2520-2522Attachment-2520count-2522-2520-253E-253D-2520-25221-2522-2520order-2520by-2520priority-2520DESC-252Cupdated-2520DESC=DwIFAw=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=Ojz-Ww86QvcLG1VBfECfCcNudtXNQIe7c-jJ_UMXtiE=sd2GH6n5nOzk4vtOA4qKh0kULci4rCiBDMWyM0IKU0Y=

If anybody has time to test and approve the patch attached to that tar please 
let me know so that it can be checked in.

Thanks,
Sean

From: Sana Riaz 
mailto:sana.r...@xflowresearch.com>>
Sent: Friday, January 11, 2019 5:33 AM
To: dev@ctakes.apache.org
Subject: AggregateCdaUmlsprocessor only annotates last section of CDA document 
[EXTERNAL]

Hi,
I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
descriptor (clinical-pipeline). The cda document includes sections like
problems, medications, allergies, tests etc. In the plain_view, all these
section are visible in CVD but all the annotations extracted by
AggregateCdaUMLSProcessor are only on last section. i.e. there's no
annotation on the medications or problems.

I've looked into CdaCasInitializer output , and it only passes one segment
(the last one) so all the other annotators only process on that. In
addition to that, every section's id (including last) is assigned null as
[start section id="null"]

[end section id="null"]

Do i have to assign section id's myself? Any suggestion would be very
helpful.

Warm Regards,

Sana Riaz

Re: AggregateCdaUmlsprocessor only annotates last section of CDA document [EXTERNAL] [SUSPICIOUS]

[Miller, Timothy]

Looks like someone fixed that as part of a different issue:
https://issues.apache.org/jira/browse/CTAKES-500
Tim


-Original Message-
From: "Finan, Sean" 
mailto:%22Finan,%20sean%22%20%3csean.fi...@childrens.harvard.edu%3e>>
Reply-to: 
To: dev@ctakes.apache.org 
mailto:%22...@ctakes.apache.org%22%20%3c...@ctakes.apache.org%3e>>
Subject: Re: AggregateCdaUmlsprocessor only annotates last section of CDA 
document [EXTERNAL] [SUSPICIOUS]
Date: Fri, 11 Jan 2019 16:05:21 +


Hi Sana,

This might be related to

https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CTAKES-2D450-3Ffilter-3D-2D5-26jql-3Dproject-2520-253D-2520CTAKES-2520AND-2520resolution-2520-253D-2520Unresolved-2520AND-2520-2522Attachment-2520count-2522-2520-253C-253D-2520-25222-2522-2520AND-2520-2522Attachment-2520count-2522-2520-253E-253D-2520-25221-2522-2520order-2520by-2520priority-2520DESC-252Cupdated-2520DESC=DwIFAw=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=Ojz-Ww86QvcLG1VBfECfCcNudtXNQIe7c-jJ_UMXtiE=sd2GH6n5nOzk4vtOA4qKh0kULci4rCiBDMWyM0IKU0Y=

If anybody has time to test and approve the patch attached to that tar please 
let me know so that it can be checked in.

Thanks,
Sean

From: Sana Riaz 
mailto:sana.r...@xflowresearch.com>>
Sent: Friday, January 11, 2019 5:33 AM
To: dev@ctakes.apache.org
Subject: AggregateCdaUmlsprocessor only annotates last section of CDA document 
[EXTERNAL]

Hi,
I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
descriptor (clinical-pipeline). The cda document includes sections like
problems, medications, allergies, tests etc. In the plain_view, all these
section are visible in CVD but all the annotations extracted by
AggregateCdaUMLSProcessor are only on last section. i.e. there's no
annotation on the medications or problems.

I've looked into CdaCasInitializer output , and it only passes one segment
(the last one) so all the other annotators only process on that. In
addition to that, every section's id (including last) is assigned null as
[start section id="null"]

[end section id="null"]

Do i have to assign section id's myself? Any suggestion would be very
helpful.

Warm Regards,

Sana Riaz

Re: AggregateCdaUmlsprocessor only annotates last section of CDA document [EXTERNAL]

[Finan, Sean]

Hi Sana,

This might be related to

https://issues.apache.org/jira/browse/CTAKES-450?filter=-5=project%20%3D%20CTAKES%20AND%20resolution%20%3D%20Unresolved%20AND%20%22Attachment%20count%22%20%3C%3D%20%222%22%20AND%20%22Attachment%20count%22%20%3E%3D%20%221%22%20order%20by%20priority%20DESC%2Cupdated%20DESC

If anybody has time to test and approve the patch attached to that tar please 
let me know so that it can be checked in.

Thanks,
Sean

From: Sana Riaz 
Sent: Friday, January 11, 2019 5:33 AM
To: dev@ctakes.apache.org
Subject: AggregateCdaUmlsprocessor only annotates last section of CDA document 
[EXTERNAL]

Hi,
I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
descriptor (clinical-pipeline). The cda document includes sections like
problems, medications, allergies, tests etc. In the plain_view, all these
section are visible in CVD but all the annotations extracted by
AggregateCdaUMLSProcessor are only on last section. i.e. there's no
annotation on the medications or problems.

I've looked into CdaCasInitializer output , and it only passes one segment
(the last one) so all the other annotators only process on that. In
addition to that, every section's id (including last) is assigned null as
[start section id="null"]

[end section id="null"]

Do i have to assign section id's myself? Any suggestion would be very
helpful.

Warm Regards,

Sana Riaz

Re: UmlsLookupAnnotator.xml does not give sign/symptom, disease/disorder in identifiedAnnotation for CDA documents [EXTERNAL]

[Finan, Sean]

Hi Sana,

>This was working fine for plaintext but cuased the mentioned problem when I
processed the CDA document after adding CdaCasInitializer annotator to the
pipeline.

Ok, so there is a very good chance that problem is not the dictionary lookup 
module.

>Do i have to enable the annotations types in
the DefaultJCasTermAnnotator class or something like that? 

You don't need to enable type production.  By default the -fast lookup will 
create the following annotations based upon the Semantic TUI of the discovered 
concepts: 
SignSymptomMention,
ProcedureMention,
DiseaseDisorderMention,
MedicationMention,
LabMention,
AnatomicalSiteMention
EntityMention (for unknown semantic type)

See 
https://svn.apache.org/repos/asf/ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/DefaultTermConsumer.java


In summary, the various types will be produced for discovered concepts.
If types are not produced, it means that concepts were not discovered.
There are various reasons for a lack of discovery: empty dictionary, no 
matching synonyms in the dictionary, missing segment, sentence, parts of speech 
or basetokens ...



From: Sana Riaz 
Sent: Thursday, January 10, 2019 3:29 PM
To: dev@ctakes.apache.org
Subject: Re: UmlsLookupAnnotator.xml does not give sign/symptom, 
disease/disorder in identifiedAnnotation for CDA documents [EXTERNAL]

Hi Sean
Thanks For responding.
Yes, I am using xml descriptor in ctakes-dictionary-lookup-fast. I tried
the fix you mentioned but it changed nothing.
To answer your second question, I have build the pipeline using xml
descriptors in java and am testing in java. Initially i was using
DefaultJCasTermAnnotator with The resource (dictionary) configuration file
resources/.../dictionary/lookup/fast/sno_rx_16ab.xml  as following

AggregateBuilder builder = new AggregateBuilder();
> ...
> //other pipeline components//
> ...
> AnalysisEngineDescription dictionarylookup_desc =
> AnalysisEngineFactory.createEngineDescription(
> DefaultJCasTermAnnotator.class,
>AbstractJCasTermAnnotator.PARAM_WINDOW_ANNOT_KEY,
>"org.apache.ctakes.typesystem.type.textspan.Sentence",
>JCasTermAnnotator.DICTIONARY_DESCRIPTOR_KEY,
>"org/apache/ctakes/dictionary/lookup/fast/sno_rx_16ab.xml");
> builder.add(dictionarylookup_desc);
> ...
> //other pipeline components//
> ...
> builder.createAggregateDescription();


This was working fine for plaintext but cuased the mentioned problem when I
processed the CDA document after adding CdaCasInitializer annotator to the
pipeline. So I changed it to UmlsOverlapLookupAnnotator descriptor as

builder.add( AnalysisEngineFactory.createEngineDescription(
> "desc/ctakes-dictionary-lookup-fast/desc/analysis_engine/UmlsLookupAnnotator")
> );



BTW, I experimented with the AggregateCdaUMLSProcessor.xml
(ctakes-clinical-pipeline) and changed the DictionaryLookupAnnotatorUmls
(original dictionary-lookup ) to UmlsLookupAnnotator . This also doesn't
give the the mentioned annotations. So this gaveme the hint that problem is
with UmlsLookupAnnotator maybe.
What do you suggest?

>The java implementation pointed to in that descriptor,
DefaultJCasTermAnnotator does provide the various semantically-distinct
annotation types that you mention.

I am a little confuse there; Do i have to enable the annotations types in
the DefaultJCasTermAnnotator class or something like that? becuase i didnt
had to do anything like that for the plaintext. Can you please elaborate
this point too, I am really naive to cTAKES yet so i might be not getting
it right.

PS I am not getting any error or warning related to this in the compiling
logs.

Warm Regards

Sana Riaz



On Thu, Jan 10, 2019 at 8:25 PM Finan, Sean <
sean.fi...@childrens.harvard.edu> wrote:

> Hi Sana,
>
> When you say:
> >i want to use dictionary_lookup "UmlsLookupAnnotator"
> are you talking about the xml descriptor in
> ctakes-dictionary-lookup-fast?  If so that is great.
>
> >The problem is that identifiedAnnotation given by UmlsLookupAnnotator does
> not include Sign/Symptoms, Disease/Disorder or Procedure Mentions etc.
>
> How are you testing this?  The java implementation pointed to in that
> descriptor, DefaultJCasTermAnnotator does provide the various
> semantically-distinct annotation
> types that you mention.  I use it every day without problem*.  Are you
> seeing any errors at the top of the log?
>
> I just looked at the descriptor UmlsLookupAnnotator.xml and it may have a
> problem:
>
>org.apache.ctakes.dictionary.lookup2.ae
> .DefaultJCasTermAnnotator
>
>
> Notice that the end tag  is on a second line
> in the file.  I have seen this cause problems in uima/ctakes.  I think that
> the xml parser assumes that whitespace is part of the information - which
> in this case is not true.
>
> Try putting the end tag on the same line and running again.
>
> * I never use xml 

Re: Not able to run cTAKES UMLS in parallel [EXTERNAL]

[Finan, Sean]

Hi Kalyani,

At a guess, each parallel instance  of ctakes is attempting to write some log 
file concurrently and the system does not approve.

Have other people out there witnessed and overcome this problem?

Sean

From: Kalyani Unnikrishnan 
Sent: Friday, January 11, 2019 5:15 AM
To: dev@ctakes.apache.org
Subject: Not able to run cTAKES UMLS in parallel [EXTERNAL]

Hi,

I am trying to make cTAKES accessible by multiple programs in parallel. I was 
able to remove the error on umls.lck by setting PARAMS_ISOLATION to 
TRANSACTION_SERIALIZABLE. But I am still getting the following error when 
multiple programs are running the pipeline at the same time (it works fine if 
it is run in sequence).

Caused by: java.sql.SQLException: file input/output error: 
org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCConnection.(Unknown Source)
at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at 
org.apache.ctakes.core.resource.JdbcConnectionResourceImpl.load(JdbcConnectionResourceImpl.java:109)
... 47 more
Caused by: org.hsqldb.HsqlException: file input/output error: 
org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
at org.hsqldb.error.Error.error(Unknown Source)
at org.hsqldb.error.Error.error(Unknown Source)
at org.hsqldb.persist.Log.openLog(Unknown Source)
at org.hsqldb.persist.Log.open(Unknown Source)
at org.hsqldb.persist.Logger.open(Unknown Source)
at org.hsqldb.Database.reopen(Unknown Source)
at org.hsqldb.Database.open(Unknown Source)
at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
at org.hsqldb.DatabaseManager.newSession(Unknown Source)
... 53 more

Appreciate any inputs.

Regards,

Kalyani

AggregateCdaUmlsprocessor only annotates last section of CDA document

[Sana Riaz]

Hi,
I am trying to process CDA documents with AggregateCdaUMLSProcessor.xml
descriptor (clinical-pipeline). The cda document includes sections like
problems, medications, allergies, tests etc. In the plain_view, all these
section are visible in CVD but all the annotations extracted by
AggregateCdaUMLSProcessor are only on last section. i.e. there's no
annotation on the medications or problems.

I've looked into CdaCasInitializer output , and it only passes one segment
(the last one) so all the other annotators only process on that. In
addition to that, every section's id (including last) is assigned null as
[start section id="null"]

[end section id="null"]

Do i have to assign section id's myself? Any suggestion would be very
helpful.

Warm Regards,

Sana Riaz

Not able to run cTAKES UMLS in parallel

[Kalyani Unnikrishnan]

Hi,

I am trying to make cTAKES accessible by multiple programs in parallel. I was 
able to remove the error on umls.lck by setting PARAMS_ISOLATION to 
TRANSACTION_SERIALIZABLE. But I am still getting the following error when 
multiple programs are running the pipeline at the same time (it works fine if 
it is run in sequence).

Caused by: java.sql.SQLException: file input/output error: 
org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCConnection.(Unknown Source)
at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at 
org.apache.ctakes.core.resource.JdbcConnectionResourceImpl.load(JdbcConnectionResourceImpl.java:109)
... 47 more
Caused by: org.hsqldb.HsqlException: file input/output error: 
org/apache/ctakes/dictionary/lookup/umls2011ab/umls.log
at org.hsqldb.error.Error.error(Unknown Source)
at org.hsqldb.error.Error.error(Unknown Source)
at org.hsqldb.persist.Log.openLog(Unknown Source)
at org.hsqldb.persist.Log.open(Unknown Source)
at org.hsqldb.persist.Logger.open(Unknown Source)
at org.hsqldb.Database.reopen(Unknown Source)
at org.hsqldb.Database.open(Unknown Source)
at org.hsqldb.DatabaseManager.getDatabase(Unknown Source)
at org.hsqldb.DatabaseManager.newSession(Unknown Source)
... 53 more

Appreciate any inputs.

Regards,

Kalyani