RE: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Hi Tim,
There is code scattered all throughout ctakes to handle various easy ways,
starting or stopping at different levels of information - I think that you may
have actually created some of the best of it (ctakes to anafora maybe?).
I started working on a document bundle created from discovered mentions,
placing relations (including coref) in arrays of object references. This is
different than the DeepPhe approach which uses a coalescence of ctakes types to
represent a single concept. It would be nice to (at some point) migrate that
into ctakes, perhaps after the next DeepPhe release.
It should be able to produce both json and xml output. I will place it in
sandbox (later) for you and Gandhi and anybody else that wants to take a look.
Sean
-Original Message-
From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
Sent: Friday, December 15, 2017 9:47 AM
To: dev@ctakes.apache.org
Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
[SUSPICIOUS]
Great, that's very helpful.
I'll be happy to help with extracting the information needed from the CAS the
easy way. Sean, am I remembering right that there was an API started for that
somewhere? Or maybe that was part of DeepPhe?
Tim
On Fri, 2017-12-15 at 03:52 +, Gandhi Rajan Natarajan wrote:
> Hi Tim,
>
> Thanks for taking time out and having a look at this. As you
> mentioned, the dictionary descriptor file contains details specific to
> my setup which needs to be changes to 127.0.0.1 by default. Will make
> the change accordingly.
>
> The only reason we went ahead with the approach of parsing XML to JSON
> is due to our lack of in-depth knowledge in cTAKES implementations. If
> I could get some guidance on how to get the required JSON details
> directly from type systems, will be happy to implement the same as it
> will be a huge performance gain.
>
> Also as you said we have two directories names ctakes-web-rest and
> ctakes-rest-service. Ctakes-rest-service directory is no longer active
> and its obsolete. We are just maintaining it for some reference for
> the time being. We will knock it off soon.
>
> Thanks again for the detailed feedback.
>
> Regards,
> Gandhi
>
>
> -Original Message-
> From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
> Sent: Friday, December 15, 2017 1:25 AM
> To: dev@ctakes.apache.org
> Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS]
> [SUSPICIOUS]
>
> I looked at this today. Looks like a great start!
>
> I was able to get as far as deploying to tomcat, seeing the web form,
> and submitting, but didn't get correct feedback because I don't have a
> mysql dictionary set up, which the default descriptor points at. I
> didn't see any instructions for building that and didn't have time to
> figure that out.
>
> I think I mentioned in a different thread that if this whole thing
> could be wrapped in a docker container that would be really powerful,
> but if not, there are a few things that are obvious to you as
> developers but would make it easier for novices (like me) to deploy.
>
> * download tomcat bin and start with bin/startup.sh (check at
> localhost:8080)
> * run mvn install on my ctakes installation to populate jar files in
> the .m2 directory that were missing
> * run mvn package inside the ctakes-web-rest subdirectory
> * copy the .war file into the webapps directory in my tomcat
> installation.
> * While I couldn't get the dictionary to work pointing to mysql, I
> noticed that the dictionary descriptor file has a hardcoded IP address
> when maybe it should be 127.0.0.1?
>
> One other thing I noticed in the code is that in sending back JSON it
> looks like you're turning the JCas into xml and then parsing it
> yourself. It should be easier just to access typesystem objects
> directly. Sean may have some API code laying around to simplify that
> as well.
>
> To iterate over signs/symptoms, for example, you would do:
>
> for(SignSymptomMention ss : JCasUtil.select(jcas,
> SignSymptomMention.class)){
> int begin = ss.getBegin(); // begin offset
> int end = ss.getEnd(): // end offset ...
> }
>
> Using the typesystem directly may help you to speed up that code or
> make it easier to read. But maybe there is a reason to write it to xml
> that I'm not aware of.
>
> Finally, I see there are two sub-directories with similar names,
> ctakes-rest-service and ctakes-web-rest. If they are duplicates can
> you delete the old one?
>
> I'll keep poking around, but hopefully this is helpful feedback for
> you guys. Thanks again for getting this off the ground!
>
> Tim
>
>
>
>
> On Thu, 2017-12-07 at 14:16 +,
RE: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Hi Tim,
As you mentioned, the pipeline will be loaded only on the very first request to
the REST service currently. But it can be changed to load the pipelines on
server start too. Every call will not re-load the pipeline. Will incorporate
the change to load the pipelines on server start.
Regards,
Gandhi
-Original Message-
From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
Sent: Friday, December 15, 2017 2:28 AM
To: dev@ctakes.apache.org
Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
[SUSPICIOUS] [SUSPICIOUS]
Another thought I just had is that it seems to load the pipeline when the first
call is made -- without knowing the REST APIs that well, is it possible to load
the pipelines when the war is deployed? With some of our larger pipelines the
first call may take quite a while. Would every call re-load the pipeline?
Tim
On Thu, 2017-12-14 at 20:16 +, Finan, Sean wrote:
> Hi Tim,
>
> Many thanks for testing the new rest service! And double that for the
> setup instructions!
>
> >
> > if this whole thing could be wrapped in a docker container that
> > would be really powerful
> - Matthew and I have had a short discussion or two on a docker that he
> is working on. It was working, but performed a lot of the spring
> updates and some workarounds that should no longer be needed. The
> next iteration should be cleaner and simpler. We have also talked
> about making the container more compact. He is busy with real work,
> but I think that this is definitely just over the horizon.
>
> >
> > One other thing I noticed in the code is that in sending back JSON
> > it looks like you're turning the JCas into xml and then parsing it
> > yourself. It should be easier just to access typesystem objects
> > directly. Sean may have some API code laying around to simplify that
> > as well.
> - I am actually looking at the rest/util/XmlParser and had the very
> same thought. It is a great start though, and as far as I know it is
> the first publicly available ctakes json writer. If anybody else out
> there already has or knows of another, please share!
>
>
> Cheers all,
> Sean
>
>
> -Original Message-
> From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
> Sent: Thursday, December 14, 2017 2:55 PM
> To: dev@ctakes.apache.org
> Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS]
> [SUSPICIOUS] [SUSPICIOUS]
>
> I looked at this today. Looks like a great start!
>
> I was able to get as far as deploying to tomcat, seeing the web form,
> and submitting, but didn't get correct feedback because I don't have a
> mysql dictionary set up, which the default descriptor points at. I
> didn't see any instructions for building that and didn't have time to
> figure that out.
>
> I think I mentioned in a different thread that if this whole thing
> could be wrapped in a docker container that would be really powerful,
> but if not, there are a few things that are obvious to you as
> developers but would make it easier for novices (like me) to deploy.
>
> * download tomcat bin and start with bin/startup.sh (check at
> localhost:8080)
> * run mvn install on my ctakes installation to populate jar files in
> the .m2 directory that were missing
> * run mvn package inside the ctakes-web-rest subdirectory
> * copy the .war file into the webapps directory in my tomcat
> installation.
> * While I couldn't get the dictionary to work pointing to mysql, I
> noticed that the dictionary descriptor file has a hardcoded IP address
> when maybe it should be 127.0.0.1?
>
> One other thing I noticed in the code is that in sending back JSON it
> looks like you're turning the JCas into xml and then parsing it
> yourself. It should be easier just to access typesystem objects
> directly. Sean may have some API code laying around to simplify that
> as well.
>
> To iterate over signs/symptoms, for example, you would do:
>
> for(SignSymptomMention ss : JCasUtil.select(jcas,
> SignSymptomMention.class)){
> int begin = ss.getBegin(); // begin offset
> int end = ss.getEnd(): // end offset ...
> }
>
> Using the typesystem directly may help you to speed up that code or
> make it easier to read. But maybe there is a reason to write it to xml
> that I'm not aware of.
>
> Finally, I see there are two sub-directories with similar names,
> ctakes-rest-service and ctakes-web-rest. If they are duplicates can
> you delete the old one?
>
> I'll keep poking around, but hopefully this is helpful feedback for
> you guys. Thanks again for getting this off the ground!
>
> Tim
>
>
>
>
> On Thu, 2017-12-07 at 14:16 +, Miller, Timothy wrote:
> >
> > I
RE: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Hi Tim,
The pipelines will soon be preloaded.
Sean
-Original Message-
From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
Sent: Thursday, December 14, 2017 3:58 PM
To: dev@ctakes.apache.org
Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
[SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Another thought I just had is that it seems to load the pipeline when the first
call is made -- without knowing the REST APIs that well, is it possible to load
the pipelines when the war is deployed? With some of our larger pipelines the
first call may take quite a while. Would every call re-load the pipeline?
Tim
On Thu, 2017-12-14 at 20:16 +, Finan, Sean wrote:
> Hi Tim,
>
> Many thanks for testing the new rest service! And double that for the
> setup instructions!
>
> >
> > if this whole thing could be wrapped in a docker container that
> > would be really powerful
> - Matthew and I have had a short discussion or two on a docker that he
> is working on. It was working, but performed a lot of the spring
> updates and some workarounds that should no longer be needed. The
> next iteration should be cleaner and simpler. We have also talked
> about making the container more compact. He is busy with real work,
> but I think that this is definitely just over the horizon.
>
> >
> > One other thing I noticed in the code is that in sending back JSON
> > it looks like you're turning the JCas into xml and then parsing it
> > yourself. It should be easier just to access typesystem objects
> > directly. Sean may have some API code laying around to simplify that
> > as well.
> - I am actually looking at the rest/util/XmlParser and had the very
> same thought. It is a great start though, and as far as I know it is
> the first publicly available ctakes json writer. If anybody else out
> there already has or knows of another, please share!
>
>
> Cheers all,
> Sean
>
>
> -Original Message-
> From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
> Sent: Thursday, December 14, 2017 2:55 PM
> To: dev@ctakes.apache.org
> Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS]
> [SUSPICIOUS] [SUSPICIOUS]
>
> I looked at this today. Looks like a great start!
>
> I was able to get as far as deploying to tomcat, seeing the web form,
> and submitting, but didn't get correct feedback because I don't have a
> mysql dictionary set up, which the default descriptor points at. I
> didn't see any instructions for building that and didn't have time to
> figure that out.
>
> I think I mentioned in a different thread that if this whole thing
> could be wrapped in a docker container that would be really powerful,
> but if not, there are a few things that are obvious to you as
> developers but would make it easier for novices (like me) to deploy.
>
> * download tomcat bin and start with bin/startup.sh (check at
> localhost:8080)
> * run mvn install on my ctakes installation to populate jar files in
> the .m2 directory that were missing
> * run mvn package inside the ctakes-web-rest subdirectory
> * copy the .war file into the webapps directory in my tomcat
> installation.
> * While I couldn't get the dictionary to work pointing to mysql, I
> noticed that the dictionary descriptor file has a hardcoded IP address
> when maybe it should be 127.0.0.1?
>
> One other thing I noticed in the code is that in sending back JSON it
> looks like you're turning the JCas into xml and then parsing it
> yourself. It should be easier just to access typesystem objects
> directly. Sean may have some API code laying around to simplify that
> as well.
>
> To iterate over signs/symptoms, for example, you would do:
>
> for(SignSymptomMention ss : JCasUtil.select(jcas,
> SignSymptomMention.class)){
> int begin = ss.getBegin(); // begin offset
> int end = ss.getEnd(): // end offset ...
> }
>
> Using the typesystem directly may help you to speed up that code or
> make it easier to read. But maybe there is a reason to write it to xml
> that I'm not aware of.
>
> Finally, I see there are two sub-directories with similar names,
> ctakes-rest-service and ctakes-web-rest. If they are duplicates can
> you delete the old one?
>
> I'll keep poking around, but hopefully this is helpful feedback for
> you guys. Thanks again for getting this off the ground!
>
> Tim
>
>
>
>
> On Thu, 2017-12-07 at 14:16 +, Miller, Timothy wrote:
> >
> > I am really interested in this too, just waiting until I have a few
> > free hours to look around. Don't want you to think it's not of
> > interest.
> >
Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Another thought I just had is that it seems to load the pipeline when
the first call is made -- without knowing the REST APIs that well, is
it possible to load the pipelines when the war is deployed? With some
of our larger pipelines the first call may take quite a while. Would
every call re-load the pipeline?
Tim
On Thu, 2017-12-14 at 20:16 +, Finan, Sean wrote:
> Hi Tim,
>
> Many thanks for testing the new rest service! And double that for
> the setup instructions!
>
> >
> > if this whole thing could be wrapped in a docker container that
> > would be really powerful
> - Matthew and I have had a short discussion or two on a docker that
> he is working on. It was working, but performed a lot of the spring
> updates and some workarounds that should no longer be needed. The
> next iteration should be cleaner and simpler. We have also talked
> about making the container more compact. He is busy with real work,
> but I think that this is definitely just over the horizon.
>
> >
> > One other thing I noticed in the code is that in sending back JSON
> > it looks like you're turning the JCas into xml and then parsing it
> > yourself. It should be easier just to access typesystem objects
> > directly. Sean may have some API code laying around to simplify
> > that as well.
> - I am actually looking at the rest/util/XmlParser and had the very
> same thought. It is a great start though, and as far as I know it is
> the first publicly available ctakes json writer. If anybody else out
> there already has or knows of another, please share!
>
>
> Cheers all,
> Sean
>
>
> -Original Message-
> From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
> Sent: Thursday, December 14, 2017 2:55 PM
> To: dev@ctakes.apache.org
> Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS]
> [SUSPICIOUS] [SUSPICIOUS]
>
> I looked at this today. Looks like a great start!
>
> I was able to get as far as deploying to tomcat, seeing the web form,
> and submitting, but didn't get correct feedback because I don't have
> a mysql dictionary set up, which the default descriptor points at. I
> didn't see any instructions for building that and didn't have time to
> figure that out.
>
> I think I mentioned in a different thread that if this whole thing
> could be wrapped in a docker container that would be really powerful,
> but if not, there are a few things that are obvious to you as
> developers but would make it easier for novices (like me) to deploy.
>
> * download tomcat bin and start with bin/startup.sh (check at
> localhost:8080)
> * run mvn install on my ctakes installation to populate jar files in
> the .m2 directory that were missing
> * run mvn package inside the ctakes-web-rest subdirectory
> * copy the .war file into the webapps directory in my tomcat
> installation.
> * While I couldn't get the dictionary to work pointing to mysql, I
> noticed that the dictionary descriptor file has a hardcoded IP
> address when maybe it should be 127.0.0.1?
>
> One other thing I noticed in the code is that in sending back JSON it
> looks like you're turning the JCas into xml and then parsing it
> yourself. It should be easier just to access typesystem objects
> directly. Sean may have some API code laying around to simplify that
> as well.
>
> To iterate over signs/symptoms, for example, you would do:
>
> for(SignSymptomMention ss : JCasUtil.select(jcas,
> SignSymptomMention.class)){
> int begin = ss.getBegin(); // begin offset
> int end = ss.getEnd(): // end offset ...
> }
>
> Using the typesystem directly may help you to speed up that code or
> make it easier to read. But maybe there is a reason to write it to
> xml that I'm not aware of.
>
> Finally, I see there are two sub-directories with similar names,
> ctakes-rest-service and ctakes-web-rest. If they are duplicates can
> you delete the old one?
>
> I'll keep poking around, but hopefully this is helpful feedback for
> you guys. Thanks again for getting this off the ground!
>
> Tim
>
>
>
>
> On Thu, 2017-12-07 at 14:16 +, Miller, Timothy wrote:
> >
> > I am really interested in this too, just waiting until I have a
> > few
> > free hours to look around. Don't want you to think it's not of
> > interest.
> > Tim
> >
> >
> > On Tue, 2017-12-05 at 19:18 +, Finan, Sean wrote:
> > >
> > >
> > > Hi all,
> > >
> > > I am trying to clear a backlog at work. I will most likely not
> > > be
> > > able to do anything with ctakes for another week. Hopefully
> > > some
> > > rest e
RE: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS] [SUSPICIOUS]
Hi Tim,
Many thanks for testing the new rest service! And double that for the setup
instructions!
> if this whole thing could be wrapped in a docker container that would be
> really powerful
- Matthew and I have had a short discussion or two on a docker that he is
working on. It was working, but performed a lot of the spring updates and some
workarounds that should no longer be needed. The next iteration should be
cleaner and simpler. We have also talked about making the container more
compact. He is busy with real work, but I think that this is definitely just
over the horizon.
> One other thing I noticed in the code is that in sending back JSON it looks
> like you're turning the JCas into xml and then parsing it yourself. It should
> be easier just to access typesystem objects directly. Sean may have some API
> code laying around to simplify that as well.
- I am actually looking at the rest/util/XmlParser and had the very same
thought. It is a great start though, and as far as I know it is the first
publicly available ctakes json writer. If anybody else out there already has
or knows of another, please share!
Cheers all,
Sean
-Original Message-
From: Miller, Timothy [mailto:timothy.mil...@childrens.harvard.edu]
Sent: Thursday, December 14, 2017 2:55 PM
To: dev@ctakes.apache.org
Subject: Re: cTAKES as REST service [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]
[SUSPICIOUS]
I looked at this today. Looks like a great start!
I was able to get as far as deploying to tomcat, seeing the web form, and
submitting, but didn't get correct feedback because I don't have a mysql
dictionary set up, which the default descriptor points at. I didn't see any
instructions for building that and didn't have time to figure that out.
I think I mentioned in a different thread that if this whole thing could be
wrapped in a docker container that would be really powerful, but if not, there
are a few things that are obvious to you as developers but would make it easier
for novices (like me) to deploy.
* download tomcat bin and start with bin/startup.sh (check at
localhost:8080)
* run mvn install on my ctakes installation to populate jar files in the .m2
directory that were missing
* run mvn package inside the ctakes-web-rest subdirectory
* copy the .war file into the webapps directory in my tomcat installation.
* While I couldn't get the dictionary to work pointing to mysql, I noticed that
the dictionary descriptor file has a hardcoded IP address when maybe it should
be 127.0.0.1?
One other thing I noticed in the code is that in sending back JSON it looks
like you're turning the JCas into xml and then parsing it yourself. It should
be easier just to access typesystem objects directly. Sean may have some API
code laying around to simplify that as well.
To iterate over signs/symptoms, for example, you would do:
for(SignSymptomMention ss : JCasUtil.select(jcas, SignSymptomMention.class)){
int begin = ss.getBegin(); // begin offset
int end = ss.getEnd(): // end offset ...
}
Using the typesystem directly may help you to speed up that code or make it
easier to read. But maybe there is a reason to write it to xml that I'm not
aware of.
Finally, I see there are two sub-directories with similar names,
ctakes-rest-service and ctakes-web-rest. If they are duplicates can you delete
the old one?
I'll keep poking around, but hopefully this is helpful feedback for you guys.
Thanks again for getting this off the ground!
Tim
On Thu, 2017-12-07 at 14:16 +, Miller, Timothy wrote:
> I am really interested in this too, just waiting until I have a few
> free hours to look around. Don't want you to think it's not of
> interest.
> Tim
>
>
> On Tue, 2017-12-05 at 19:18 +, Finan, Sean wrote:
> >
> > Hi all,
> >
> > I am trying to clear a backlog at work. I will most likely not be
> > able to do anything with ctakes for another week. Hopefully some
> > rest expert out there can prove their worth by testing ...
> >
> > Sean
> >
> > -Original Message-
> > From: Matthew Vita [mailto:matthewvit...@gmail.com]
> > Sent: Tuesday, December 05, 2017 1:58 PM
> > To: dev@ctakes.apache.org
> > Subject: Re: cTAKES as REST service [EXTERNAL]
> >
> >
> > Hi Gandhi, Sean, Tim, Alex, James,
> >
> > I'm still getting back into the swing of things after my trip (I'm
> > on business traveling at the moment, here in the states). I will be
> > jumping right back into cTAKES REST development next week personally
> > and with a new team mate from the open source team.
> >
> > I'm so sorry for my silence/lack of updates!!! Very excited to see
> > what Gandhi's updates are looking like and enriching the JSON
> > response payload.
> >
> > Thanks,
> >
> > Matthe
