Re: UIMA-AS latest version security issues

[Jaroslaw Cwiklik]

Hi Richard, sorry for the delay in responding. Was away on vacation last
week. Will merge your changes soon.
Thanks for the help. I will work on a new uima-as release when I have free
time.

Jerry

On Mon, Jun 21, 2021 at 2:51 PM Richard Eckart de Castilho 
wrote:

> Hi Jerry,
>
> > On 18. Jun 2021, at 19:35, Jaroslaw Cwiklik  wrote:
> >
> > Thanks Richard. I took a look today at Ruta and compared its eclipse
> > plugins setup to uima-as. Seems like many changes are needed to make them
> > work. Actually made some changes to uima-as poms (including one you've
> > suggested) but am not able to build.
>
> Please have a look at the PR:
> https://github.com/apache/uima-async-scaleout/pull/3
>
> -- Richard

Re: UIMA-AS latest version security issues

[Jaroslaw Cwiklik]

Thanks Richard. I took a look today at Ruta and compared its eclipse
plugins setup to uima-as. Seems like many changes are needed to make them
work. Actually made some changes to uima-as poms (including one you've
suggested) but am not able to build.
The latest problem is this:

*ERROR*] Failed to execute goal on project
uimaj-ep-runtime-deployeditor: *Could
not resolve dependencies for project
org.apache.uima:uimaj-ep-runtime-deployeditor:jar:2.10.4-SNAPSHOT: Artifact
not found*:
/var/folders/qv/mb2304nn2yg65q0c3n0vrcpmgn/T/org.openntf.maven.p2.layout.P2RepositoryLayout-org.eclipse.p2.201812-metadata4232512035347814454/1997609141157123
(No such file or directory) -> *[Help 1]*


*Regards, Jerry*


*I *



On Thu, Jun 17, 2021 at 1:55 PM Richard Eckart de Castilho 
wrote:

> Hi Jerry,
>
> On 17. Jun 2021, at 19:11, Jaroslaw Cwiklik  wrote:
> >
> > There is a problem building eclipse plugins though. Not enough free time
> to
> > address this now.
>
> For the UIMA Java SDK and Ruta, we no longer obtain the Eclipse plugin
> dependencies
> from Maven Central but instead use this plugin to pull the Eclipse bundles
> directly
> from the official P2 repositories:
>
>   
> org.openntf.maven
> p2-layout-resolver
> 1.3.0
> true
>   
>
> That protects us now from the volatility of the version range resolving
> due to Eclipse
> publishing new artifacts to Maven Central as we can lock to a particular
> P2 update site
> and those remain stable.
>
> Cheers,
>
> -- Richard

Re: UIMA-AS latest version security issues

[Jaroslaw Cwiklik]

I was able to run UIMA-AS (from
https://github.com/apache/uima-async-scaleout) extended tests with AMQ
5.16.2.
There is a problem building eclipse plugins though. Not enough free time to
address this now.
Regards, Jerry Cwiklik


5.16.2

4.3.30.RELEASE
2.25.2


On Wed, Jun 2, 2021 at 2:38 AM Richard Eckart de Castilho 
wrote:

> Hi,
>
> > On 1. Jun 2021, at 21:52, E Khorasani  wrote:
> >
> > We are using UIMA-AS 2.10.3  which includes ActiveMQ  5.15.2.  But our
> > AppScan report show High and medium severity security in
> > activemq-broker-5.15.2.jar and activemq-client-5.15.2.jar. Is there a
> way
> > to upgrade ActiveMQ  libraries in UIMA-AS? If so, could you please point
> > me to documents as to how to achieve this?
>
> Have you tried checking out the sources, replacing all the ActiveMQ
> dependency
> versions with the latest 5.x version, building and checking if it still
> works?
>
> Cheers,
>
> -- Richard

Re: UIMA-AS with ActiveMQ VM Transport

[Jaroslaw Cwiklik]

The vm is currently reserved for async aggregate AE (collocated) endpoints.
Async aggregate AE communicates with its colocated delegate AE using java
(non-jms) queues to avoid serialization/deserialization of CASes. The vm
keyword in the endpoint is used to identify such collocated AEs. Don't
remember why vm was chosen to represent these endpoints. This decision was
made early in the development of UIMA-AS. Fixing that will require
refactoring of the UIMA-AS aggregate code and possibly other components as
well.
Even if this is fixed, the UIMA-AS client still has to
seriallize/deserialize CASes when messaging the service. Not sure if using
the vm protocol would reduce the latency assuming this is the main reason
for using the AMQ vm transport between the UIMA-AS client and the service.

-jerry


On Tue, Nov 3, 2020 at 3:11 PM Richard Eckart de Castilho 
wrote:

> Hi Jerry,
>
> > On 3. Nov 2020, at 17:52, Jaroslaw Cwiklik  wrote:
> >
> > My apologies for a late reply to your question. The UIMA-AS client does
> not
> > support vm protocol between clients and services. Only tcp and http are
> > currently supported as you've noted. You can manage broker persistence
> > settings in the broker configuration See
> > http://activemq.apache.org/persistence (
> > )
>
> is there a particular (technical) reason why the vm protocol is not
> supported?
>
> Would a potential interested contributor be faced with significant trouble
> in
> trying to add support for it?
>
> Cheers,
>
> -- Richard

Re: UIMA-AS with ActiveMQ VM Transport

[Jaroslaw Cwiklik]

My apologies for a late reply to your question. The UIMA-AS client does not
support vm protocol between clients and services. Only tcp and http are
currently supported as you've noted. You can manage broker persistence
settings in the broker configuration See
http://activemq.apache.org/persistence (
)

Jerry

On Wed, Oct 14, 2020 at 12:55 PM kra...@t-online.de 
wrote:

> Hi,
>
> I'm using UIMA-AS in a single JVM process, with having the activemq
> broker, the uima-as client and the as deployment in the same java process.
> UimaAsynchronousEngine.sendAndReceiveCAS() is used to send and wait for
> the processed CAS.
> This setups works with fine with brokerURL "tcp://localhost:61616".
>
> If I change the brokerURL to "vm://localhost?broker.persistent=false" I
> get this NullPointerException on reply and after successful processing of
> the CAS.
>
> java.lang.NullPointerException: null
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.sendVMMessage(AggregateAnalysisEngineController_impl.java:2351)
> ~[uimaj-as-core-2.10.3.jar!/:2.10.3]
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.sendReplyToCollocatedClient(AggregateAnalysisEngineController_impl.java:2242)
> ~[uimaj-as-core-2.10.3.jar!/:2.10.3]
>
> If I use "vm://localhost?marshal=false=false" as
> brokerURL, there is a similar NPE before processing at
>
> java.lang.NullPointerException: null
> at
> org.apache.uima.aae.controller.BaseAnalysisEngineController.sendMetadata(BaseAnalysisEngineController.java:2952)
> ~[uimaj-as-core-2.10.3.jar!/:2.10.3]
> at
> org.apache.uima.aae.controller.PrimitiveAnalysisEngineController_impl.sendMetadata(PrimitiveAnalysisEngineController_impl.java:1337)
> ~[uimaj-as-core-2.10.3.jar!/:2.10.3]
> at
> org.apache.uima.aae.handler.input.MetadataRequestHandler_impl.handle(MetadataRequestHandler_impl.java:69)
> ~[uimaj-as-core-2.10.3.jar!/:2.10.3]
> at
> org.apache.uima.adapter.jms.activemq.JmsInputChannel.onMessage(JmsInputChannel.java:745)
> ~[uimaj-as-activemq-2.10.3.jar!/:2.10.3]
> at
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.doInvokeListener(UimaDefaultMessageListenerContainer.java:1180)
> ~[uimaj-as-activemq-2.10.3.jar!/:2.10.3]
>
> Both are due to empty dispatchers in
> org.apache.uima.aae.spi.transport.vm.VmTransport for the temp reply queue.
>
> Are these bugs and/or is there any way to avoid tcp/http brokerURL?
>
> Thank you very much,
> John
> 
>

Re: Java 11 - UIMA-AS 2.10.3

[Jaroslaw Cwiklik]

Thanks for the update. Whose Java are you using (Oracle, IBM, OpenJDK,
other)? I've checked out the latest version of uima-as and tried to build
it with OpenJDK 11.
Found a few build issues related to version of things  in the poms. Fixed
those and was able to build uima-as and run uima-as example described in
README:

./runRemoteAsyncAE.sh tcp://localhost:61616 MeetingDetectorTaeQueue -d
../examples/deploy/as/Deploy_MeetingDetectorTAE.xml -c
$UIMA_HOME/examples/descriptors/collection_reader/FileSystemCollectionReader.xml

Saw a few WARNINGs but the application ran (with OpenJDK 11)  fine
producing output files as expected. Most importantly I did not see
dd2spring exceptions you've reported related to XPath api problem

Below, are the changes I've made today as svn diffs. If you cant wait for
the official release I suggest you check out the latest uima-as: svn co
https://svn.apache.org/repos/asf/uima/uima-as/master/ .
Apply the changes to the 2 poms described below, and build it.

 svn diff uima-as-parent/pom.xml
Index: uima-as-parent/pom.xml
===
--- uima-as-parent/pom.xml  (revision 1874699)
+++ uima-as-parent/pom.xml  (working copy)
@@ -119,12 +119,15 @@
uima-as
${project.artifactId}

-   2.10.3
+   2.10.4


5.15.8


4.3.18.RELEASE

+   1.8
+   1.8
+
 2.19.3


1.1.1
@@ -480,7 +483,7 @@


   maven-compiler-plugin
-  3.1
+  3.8.0
   
  1.8
  1.8
@@ -527,7 +530,7 @@
  
   org.apache.maven.plugins
   maven-javadoc-plugin
-  2.10.4
+  3.1.1
   
   -Xdoclint:none
 ${maven.compiler.source}


 svn diff uimaj-as-core/pom.xml
Index: uimaj-as-core/pom.xml
===
--- uimaj-as-core/pom.xml   (revision 1874699)
+++ uimaj-as-core/pom.xml   (working copy)
@@ -174,6 +174,9 @@
org.codehaus.mojo

xmlbeans-maven-plugin
2.3.3
+
+   1.8
+   


gen-beans-id


On Fri, Feb 28, 2020 at 5:50 PM Hai-Son Nguyen 
wrote:

> Thanks Jerry,
>
> We've had moving to Java 11 from 8 on our agenda for a while; since we
> just ended a release cycle and so the question came up when/if it could
> occur this next cycle...
>
> There are other changes that go with the latest Java that make it crucial
> for us to move forward this year but it is not currently critical, I think
> when the end of support (public updates) Dec 2020 for Java 8 occurs it will
> become critical. Commercial support ended already.
>
>
>
> On 2020/02/27 21:43:48, Jaroslaw Cwiklik  wrote:
> > The UIMA-AS has been tested with Java 8 only. Is it critical for you to
> run
> > uima-as (and core uima) with Java 11?
> > Not sure what else may break with uma-as with a newer java. I will take a
> > look tomorrow at the Dd2Spring problem.
> > Looks like SAXTransformerFactory has changed.
> >
> > - Jerry
> >
> > On Wed, Feb 26, 2020 at 5:15 PM Hai-Son Nguyen <
> haison.nguyen.t...@gmail.com>
> > wrote:
> >
> > > Thanks Marshall,
> > >
> > > I updated the bootstrap.jar file with the one from UIMA 2.10.4 and the
> > > problem went away and a new one arose with using the dd2spring.xml file
> > > (not sure if the issue is coupled or independent:
> > >
> > > INFO: Using Saxon Java API - Java Vendor:Oracle Corporation
> > > Feb 26, 2020 8:24:23 AM org.apache.uima.internal.util.XMLUtils
> > > createSaxTransformerFactory(614)
> > > WARNING: SAXTransformerFactory didn't recognize setting attribute
> > > http://javax.xml.XMLConstants/property/accessExternalDTD
> > > Feb 26, 2020 8:24:23 AM org.apache.uima.internal.util.XMLUtils
> > > createSaxTransformerFactory(621)
> > > WARNING: SAXTransformerFactory didn't recognize setting attribute
> > > http://javax.xml.XMLConstants/property/accessExternalStylesheet
> > > Error at xsl:variable on line 98 column 52 of dd2spring.xsl:
> > >   XPST0017: XPath syntax error at char 12 on line 98 in {x2:getGUID()}:
> > > Cannot find a matching 0-argument function named
> > >   {org.apache.uima.aae.deploymentDescriptor.XsltGUIDgenerator}getGUID()
> > > Error at xsl:variable on line 2405 column 77 of dd2spring.xsl:
> > >   XPST0017: XPath syntax error at char 3

Re: Java 11 - UIMA-AS 2.10.3

[Jaroslaw Cwiklik]

The UIMA-AS has been tested with Java 8 only. Is it critical for you to run
uima-as (and core uima) with Java 11?
Not sure what else may break with uma-as with a newer java. I will take a
look tomorrow at the Dd2Spring problem.
Looks like SAXTransformerFactory has changed.

- Jerry

On Wed, Feb 26, 2020 at 5:15 PM Hai-Son Nguyen 
wrote:

> Thanks Marshall,
>
> I updated the bootstrap.jar file with the one from UIMA 2.10.4 and the
> problem went away and a new one arose with using the dd2spring.xml file
> (not sure if the issue is coupled or independent:
>
> INFO: Using Saxon Java API - Java Vendor:Oracle Corporation
> Feb 26, 2020 8:24:23 AM org.apache.uima.internal.util.XMLUtils
> createSaxTransformerFactory(614)
> WARNING: SAXTransformerFactory didn't recognize setting attribute
> http://javax.xml.XMLConstants/property/accessExternalDTD
> Feb 26, 2020 8:24:23 AM org.apache.uima.internal.util.XMLUtils
> createSaxTransformerFactory(621)
> WARNING: SAXTransformerFactory didn't recognize setting attribute
> http://javax.xml.XMLConstants/property/accessExternalStylesheet
> Error at xsl:variable on line 98 column 52 of dd2spring.xsl:
>   XPST0017: XPath syntax error at char 12 on line 98 in {x2:getGUID()}:
> Cannot find a matching 0-argument function named
>   {org.apache.uima.aae.deploymentDescriptor.XsltGUIDgenerator}getGUID()
> Error at xsl:variable on line 2405 column 77 of dd2spring.xsl:
>   XPST0017: XPath syntax error at char 35 on line 2405 in
> {...solveByName($importNode/@na...}:
> Cannot find a matching 1-argument function named
>
> {org.apache.uima.aae.deploymentDescriptor.XsltImportByName}resolveByName()
> Error at xsl:sequence on line 2459 column 104 of dd2spring.xsl:
>   XPST0017: XPath syntax error at char 69 on line 2459 in
> {...x1:resolveByName($node/@nam...}:
> Cannot find a matching 1-argument function named
>
> {org.apache.uima.aae.deploymentDescriptor.XsltImportByName}resolveByName()
> Error - dd2spring transformation failed:
> javax.xml.transform.TransformerConfigurationException: Failed to compile
> stylesheet. 3 errors detected.
> at
> net.sf.saxon.PreparedStylesheet.prepare(PreparedStylesheet.java:176)
> at
> net.sf.saxon.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:139)
> at
> net.sf.saxon.TransformerFactoryImpl.newTransformer(TransformerFactoryImpl.java:91)
> at
> org.apache.uima.adapter.jms.service.Dd2spring$SaxonJavaInterface.convertDD2Spring(Dd2spring.java:420)
> at
> org.apache.uima.adapter.jms.service.Dd2spring.convertDd2Spring(Dd2spring.java:184)
> at
> org.apache.uima.adapter.jms.service.Dd2spring.convertDd2Spring(Dd2spring.java:105)
> at
> org.apache.uima.adapter.jms.service.UIMA_Service.initialize(UIMA_Service.java:113)
> at
> org.apache.uima.adapter.jms.service.UIMA_Service.main(UIMA_Service.java:420)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.uima.bootstrap.UimaBootstrap.main(UimaBootstrap.java:87)
> Feb 26, 2020 8:24:24 AM org.apache.uima.adapter.jms.service.Dd2spring
> convertDD2Spring
> SEVERE: Unexpected, unhandled exception while calling Saxon
> Feb 26, 2020 8:24:24 AM org.apache.uima.adapter.jms.service.UIMA_Service
> main
> WARNING:
> javax.xml.transform.TransformerConfigurationException: Failed to compile
> stylesheet. 3 errors detected.
> at
> net.sf.saxon.PreparedStylesheet.prepare(PreparedStylesheet.java:176)
> at
> net.sf.saxon.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:139)
> at
> net.sf.saxon.TransformerFactoryImpl.newTransformer(TransformerFactoryImpl.java:91)
> at
> org.apache.uima.adapter.jms.service.Dd2spring$SaxonJavaInterface.convertDD2Spring(Dd2spring.java:420)
> at
> org.apache.uima.adapter.jms.service.Dd2spring.convertDd2Spring(Dd2spring.java:184)
> at
> org.apache.uima.adapter.jms.service.Dd2spring.convertDd2Spring(Dd2spring.java:105)
> at
> org.apache.uima.adapter.jms.service.UIMA_Service.initialize(UIMA_Service.java:113)
> at
> org.apache.uima.adapter.jms.service.UIMA_Service.main(UIMA_Service.java:420)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.uima.bootstrap.UimaBootstrap.main(UimaBootstrap.java:87)
>
> Just a 

Re: Java 11 - UIMA-AS 2.10.3

[Jaroslaw Cwiklik]

Hi, the uima-as release is on my plate. No ETA though. I am still waiting
for Apache Infra to make a switch from SVN to GIT. Not sure how long this
will take. Also the release takes some time to happen.
So you have a few options:

1) If Java 11 is not absolutely necessary you can run with Java 8

2) You can build your own uima-as runtime from SVN trunk. It already has a
dependency on uima 2.10.3 (
2.10.3)

3) You can replace uimaj-core.jar in your uima-as deployment with this jar:
https://repo.maven.apache.org/maven2/org/apache/uima/uimaj-core/2.10.4/uimaj-core-2.10.4.jar

 (in $UIMA_HOME/lib). Not sure if this is the only jar you need. In
case you need other (newer) uima jars just check here:
https://repo.maven.apache.org/maven2/org/apache/uima/uimaj-core/2.10.4


- Jerry

On Wed, Feb 26, 2020 at 11:33 AM Marshall Schor  wrote:

> Hi,
>
> This problem was fixed in core uima (uimaj) in version 2.10.3, see Jira
> issue:
> https://issues.apache.org/jira/browse/UIMA-5754
>
> But uima-as version 2.10.3 was built/delivered with a previous version of
> core
> uima (uimaj), and doesn't have this fix.
>
> We'll look into fixes/workarounds for this earlier version.
>
> -Marshall
>
>
> On 2/26/2020 12:58 AM, Hai-Son Nguyen wrote:
> > Hi,
> > I receiving an exception running:
> >bin/runUimaClass.sh org.apache.uima.adapter.jms.service.UIMA_Service
> ...
> > using Java 11 both the Oracle
> >java 11.0.6 2020-01-14 LTS
> > and the OpenJDK versions:
> >   openjdk 11.0.3 2019-04-16
> >
> > Exception in thread "main" java.lang.ClassCastException: class
> jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
> java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader
> and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
> >   at
> org.apache.uima.bootstrap.UimaBootstrap.addUrlsToSystemLoader(UimaBootstrap.java:146)
> >   at
> org.apache.uima.bootstrap.UimaBootstrap.main(UimaBootstrap.java:74)
> >
> > Thanks!
> > Hai-Son
>

Re: Uima-AS 3 ?

[Jaroslaw Cwiklik]

UIMA Ducc v3 includes UIMA-AS v 2.10.2 currently. Ducc deploys and manages
lifecycle of UIMA-AS services.

Jerry



On Tue, Jan 7, 2020 at 10:57 AM Peter Klügl 
wrote:

> Hi,
>
>
> may I ask how the relationship between UIMA-AS and UIMA-DUCC is?
>
> I always thought that DUCC uses/builds on UIMA-AS, and there is already
> DUCC v3.
>
>
> Best,
>
>
> Peter
>
>
> Am 07.01.2020 um 15:52 schrieb Jaroslaw Cwiklik:
> > Hi Matthias, I will start working on v3 release soon.
> > Jerry
> >
> > On Tue, Jan 7, 2020 at 9:28 AM koch  wrote:
> >
> >> Hi
> >>
> >> Is it planned to release v3 of uima-as ?
> >>
> >> best regards,
> >>
> >> Matthias
> >>
> >>
> >>
> >>
> --
> Dr. Peter Klügl
> R Text Mining/Machine Learning
>
> Averbis GmbH
> Salzstr. 15
> 79098 Freiburg
> Germany
>
> Fon: +49 761 708 394 0
> Fax: +49 761 708 394 10
> Email: peter.klu...@averbis.com
> Web: https://averbis.com
>
> Headquarters: Freiburg im Breisgau
> Register Court: Amtsgericht Freiburg im Breisgau, HRB 701080
> Managing Directors: Dr. med. Philipp Daumke, Dr. Kornél Markó
>
>

Re: Uima-AS 3 ?

[Jaroslaw Cwiklik]

Hi Matthias, I will start working on v3 release soon.
Jerry

On Tue, Jan 7, 2020 at 9:28 AM koch  wrote:

> Hi
>
> Is it planned to release v3 of uima-as ?
>
> best regards,
>
> Matthias
>
>
>
>

Re: UIMA and ActiveMQ?

[Jaroslaw Cwiklik]

Hi, yes UIMA-AS service has multiple async consumers. The GetMeta consumer
responds to clients asking for service metadata.
The Process consumer handles client's process requests (invoking processing
pipelines). And the Target consumer enables clients to
target specific service instance.These consumers are independently scaled
and configured in a deployment descriptor.

Each consumer segments messages it wants to receive from a common queue
using JMS Selector. Both Process and GetMeta requests
support error handling which can also be defined in the deployment
descriptor. The service uses Spring JMS default listener to ingest JMS
messages from a service queue. The general flow of messages is a follows:

Listener > InputChannel > MessageHandler > Controller > AnalysisEngine >
OutputChannel

Currently, there is no documentation about UIMA-AS internals.

The general documentation for UIMA, UIMA-AS and DUCC are here:
https://uima.apache.org/documentation.html

-Jerry

On Mon, Feb 4, 2019 at 5:27 PM armin dadras 
wrote:

> Hey there,
> We are currently looking into ActiveMQ and what the Broker does when we
> start a service instance. UIMA seems to start three consumers for each
> instance, and we don't really get behind what each of them is doing. We
> have a few questions: -in which UIMA class can we look up the function of
> those three consumers / where they are created
> -is it possible to regulate consumer regarding recourses used on them,
> does this even make sense or is it allready optimized
> -do you have generell documentation about the dataflow/architecture of
> UIMA/Ducc when processing data? Like the single steps during communication
> with ActiveMQ?
> Thank in advancceFlorian and Armin
>
>
>
>

Re: Problem in running DUCC Job for Arabic Language

[Jaroslaw Cwiklik]

Forgot to mention that if you have a shared file system the best practice
is not to serialize your content (SOFA)
from JD to service. Instead, in a CR add a path to the file containing
Subject of Analysis to the CAS and have
the CM in the pipeline read the content from the shared file system.
-jerry


On Tue, Nov 6, 2018 at 9:37 AM Jaroslaw Cwiklik  wrote:

> Can you try setting -Dfile.encoding=ISO-8859-1 for the service (job)
> process and -Djavax.servlet.request.encoding=ISO-8859-1
> -Dfile.encoding=ISO-8859-1 for the JD process.
>
> The JD actually uses Jetty webserver to serve service requests over HTTP.
> I went as far as extracting Jetty server code from JD into a simple http
> server process and also extracted HttpClient related code from the service
> into a simple client process to be able to test.
>
> So on the server side I have:
> String text = new String("استعرض المتحدث باسم قوات «التحالف العربي
> لدعم".getBytes("UTF-8"),"ISO-8859-1");
> response.setHeader("content-type", "text/xml");
> String body = marshall(text);   // XStream serialization
> response.getWriter().write(body);
>
> On the client side:
>   System.out.println("Default Locale:   " + Locale.getDefault());
>   System.out.println("Default Charset:  " + Charset.defaultCharset());
>   System. out.println("file.encoding;" +
> System.getProperty("file.encoding"));
>
> HttpResponse response = httpClient.execute(postMethod);
> HttpEntity entity = response.getEntity();
> String content = EntityUtils.toString(entity);
>  String result = (String) unmarshall(content); //XStream unmarshall
>   String o = new String(result.getBytes() );
> System.out.println(o);
>
> When I run with the above -D settings the client console shows:
> Default Locale:   en_US
> Default Charset:  ISO-8859-1
> file.encoding;ISO-8859-1
>
> استعرض المتحدث باسم قوات «التحالف العربي لدعم
>
> Without the -D's I dont see arabic text and instead see garbage on the
> console.
>
> On Fri, Jul 6, 2018 at 3:00 AM rohit14csu...@ncuindia.edu <
> rohit14csu...@ncuindia.edu> wrote:
>
>> Yes if i run the AE as a DUCC UIMA-AS Service and send it CASes from
>> UIMA-AS client it works fine.
>> Infact the enviornment i.e the LANG argument is same for UIMA-AS Service
>> and DUCC JOB.
>>
>> Environ[3] = LANG=en_IN
>>
>> And if i change the LANG=ar then while getting the data coming in JD the
>> arabic text is already replaced with ???(Question Mark) and the encoding of
>> the data coming in JD or CR  shows ASCII encoding.
>> I don't understand why is this happening.
>>
>> Best
>> Rohit
>>
>>
>> On 2018/07/05 13:35:11, Eddie Epstein  wrote:
>> > So if you run the AE as a DUCC UIMA-AS service and send it CASes from
>> some
>> > UIMA-AS client it works OK? The full environment for all processes that
>> > DUCC launches are available via ducc-mon under the Specification or
>> > Registry tab for that job or managed reservation or service. Please see
>> if
>> > the LANG setting for the service is different from the LANG setting for
>> the
>> > job.
>> >
>> > One can also see the LANG setting for a linux process-id by doing:
>> >
>> > cat /proc//environ
>> >
>> > The LANG to be used for a DUCC process can be set by adding to the
>> > --environment argument "LANG=xxx" as needed
>> >
>> > Thanks,
>> > Eddie
>> >
>> >
>> >
>> > On Thu, Jul 5, 2018 at 6:47 AM, rohit14csu...@ncuindia.edu <
>> > rohit14csu...@ncuindia.edu> wrote:
>> >
>> > > Hey,
>> > >  Yeah you got it right the first snippet comes in CR before the data
>> goes
>> > > in CAS.
>> > > And the second snippet is in the first annotator or analysis
>> engine(AE) of
>> > > my Aggregate Desciptor.
>> > > I am pretty sure this is an issue of the CAS used by DUCC because if
>> i use
>> > > service of DUCC in which we are supposed to send the CAS and receive
>> the
>> > > same CAS with added features from DUCC i get accurate results.
>> > >
>> > > But the only problem comes in submitting a job where the cas is
>> generated
>> > > by DUCC.
>> > > This can also be a issue of the enviornment(Language) of DUCC because
>> the
>> > > default language is english.
>> > >
>> > > Bets Regards
>> > > Rohit
>> > >

Re: Problem in running DUCC Job for Arabic Language

[Jaroslaw Cwiklik]

Can you try setting -Dfile.encoding=ISO-8859-1 for the service (job)
process and -Djavax.servlet.request.encoding=ISO-8859-1
-Dfile.encoding=ISO-8859-1 for the JD process.

The JD actually uses Jetty webserver to serve service requests over HTTP. I
went as far as extracting Jetty server code from JD into a simple http
server process and also extracted HttpClient related code from the service
into a simple client process to be able to test.

So on the server side I have:
String text = new String("استعرض المتحدث باسم قوات «التحالف العربي
لدعم".getBytes("UTF-8"),"ISO-8859-1");
response.setHeader("content-type", "text/xml");
String body = marshall(text);   // XStream serialization
response.getWriter().write(body);

On the client side:
  System.out.println("Default Locale:   " + Locale.getDefault());
  System.out.println("Default Charset:  " + Charset.defaultCharset());
  System. out.println("file.encoding;" +
System.getProperty("file.encoding"));

HttpResponse response = httpClient.execute(postMethod);
HttpEntity entity = response.getEntity();
String content = EntityUtils.toString(entity);
 String result = (String) unmarshall(content); //XStream unmarshall
  String o = new String(result.getBytes() );
System.out.println(o);

When I run with the above -D settings the client console shows:
Default Locale:   en_US
Default Charset:  ISO-8859-1
file.encoding;ISO-8859-1

استعرض المتحدث باسم قوات «التحالف العربي لدعم

Without the -D's I dont see arabic text and instead see garbage on the
console.

On Fri, Jul 6, 2018 at 3:00 AM rohit14csu...@ncuindia.edu <
rohit14csu...@ncuindia.edu> wrote:

> Yes if i run the AE as a DUCC UIMA-AS Service and send it CASes from
> UIMA-AS client it works fine.
> Infact the enviornment i.e the LANG argument is same for UIMA-AS Service
> and DUCC JOB.
>
> Environ[3] = LANG=en_IN
>
> And if i change the LANG=ar then while getting the data coming in JD the
> arabic text is already replaced with ???(Question Mark) and the encoding of
> the data coming in JD or CR  shows ASCII encoding.
> I don't understand why is this happening.
>
> Best
> Rohit
>
>
> On 2018/07/05 13:35:11, Eddie Epstein  wrote:
> > So if you run the AE as a DUCC UIMA-AS service and send it CASes from
> some
> > UIMA-AS client it works OK? The full environment for all processes that
> > DUCC launches are available via ducc-mon under the Specification or
> > Registry tab for that job or managed reservation or service. Please see
> if
> > the LANG setting for the service is different from the LANG setting for
> the
> > job.
> >
> > One can also see the LANG setting for a linux process-id by doing:
> >
> > cat /proc//environ
> >
> > The LANG to be used for a DUCC process can be set by adding to the
> > --environment argument "LANG=xxx" as needed
> >
> > Thanks,
> > Eddie
> >
> >
> >
> > On Thu, Jul 5, 2018 at 6:47 AM, rohit14csu...@ncuindia.edu <
> > rohit14csu...@ncuindia.edu> wrote:
> >
> > > Hey,
> > >  Yeah you got it right the first snippet comes in CR before the data
> goes
> > > in CAS.
> > > And the second snippet is in the first annotator or analysis
> engine(AE) of
> > > my Aggregate Desciptor.
> > > I am pretty sure this is an issue of the CAS used by DUCC because if i
> use
> > > service of DUCC in which we are supposed to send the CAS and receive
> the
> > > same CAS with added features from DUCC i get accurate results.
> > >
> > > But the only problem comes in submitting a job where the cas is
> generated
> > > by DUCC.
> > > This can also be a issue of the enviornment(Language) of DUCC because
> the
> > > default language is english.
> > >
> > > Bets Regards
> > > Rohit
> > >
> > > On 2018/07/03 13:11:50, Eddie Epstein  wrote:
> > > > Rohit,
> > > >
> > > > Before sending the data into jcas if i force encode it :-
> > > > >
> > > > > String content2 = null;
> > > > > content2 = new String(content.getBytes("UTF-8"), "ISO-8859-1");
> > > > > jcas.setDocumentText(content2);
> > > > >
> > > >
> > > > Where is this code, in the job CR?
> > > >
> > > >
> > > >
> > > > >
> > > > > And when i go in my first annotator i force decode it:-
> > > > >
> > > > > String content = null;
> > > > > content = new String(jcas.getDocumentText.getBytes("ISO-8859-1"),
> > > > > "UTF-8");
> > > > >
> > > >
> > > > And is this in the first annotator of the job process, i.e. the CM?
> > > >
> > > > Please be as specific as possible.
> > > >
> > > > Thanks,
> > > > Eddie
> > > >
> > >
> >
>

Re: UIMA-AS on Kubernetes

[Jaroslaw Cwiklik]

Patrick, service monitoring is left to applications. I think Kubernetes has
some basic monitoring of pods and
if more is needed I am sure there are third party monitoring tools
available.

The UIMA-AS does provide you with info about which service instance is
processing each CAS.
If you register a callback listener, you get node IP, pid and CAS via

Callback.onBeforeProcessCAS(status, nodeIP, pid);

You can use this to check if the service is running. Recent UIMA-AS
supports targeting specific service instance.
https://uima.apache.org/d/uima-as-2.10.3/uima_async_scaleout.html#ugr.ref.async.api.usage_targetservice
You can send a small test CAS to the specific instance at regular intervals
to see if it is viable.

The UIMA-AS holds on to each CAS which is in-process because it needs it to
deserialize a reply and
to support retry. The only controls you have are the size of the pool and
-Xmx

Jerry

On Thu, Oct 11, 2018 at 5:26 AM Huy, Patrick  wrote:

> Hi,
>
> we are running UIMA-AS on Kubernetes and we are facing some issues: when
> the endpoint a BaseUIMAAsynchronousEngine_impl is connected to goes missing
> (because it's being restarted by Kubernetes) while a CAS is being processed
> it seems like the AsynchronousEngine never recovers and becomes unusable
> from that point (it seems that the CAS is "gone"), currently we are using
> the "failover" activemq protocol but it does not seem to always help with
> the situation.
>
> When the components processing a CAS go down while a CAS is being
> processed we are currently unable to detect whether the process is just
> taking long or whether the component failed - we are currently waiting on
> processing Timeouts to catch this state but something better/faster (like
> healthiness pings or something) would be more desirable.
>
> Also it seems like our component using the  BaseUIMAAsynchronousEngines
> requires a very high amount of RAM to hold the CASes for each CAS Pool
> (around 5MB per CAS) is there anything that can be done to optimize this?
>
> Thanks in advance!
> Patrick
>

Re: DUCC services statistics

[Jaroslaw Cwiklik]

Daniel, you can monitor AMQ to discover client connections and also monitor
queue activity.

Jerry


On Thu, Jul 19, 2018 at 7:31 AM Daniel Baumartz <
bauma...@stud.uni-frankfurt.de> wrote:

> Hi,
>
> we have a DUCC installation with different services that are being used
> by jobs and external clients. We want to collect some statistics for a
> monitoring tool that are not included in the DUCC dashboard, e.g. how
> often and when a service/queue has been used, which services are used by
> which client at the moment...
>
> It looks like I could get these information by monitoring the ActiveMQ
> messages with an external program, or by using a custom Pinger? What
> would be the best way to handle this?
>
> Thanks,
> Daniel
>

Re: UIMA AS deploy: get MessageTimeoutException object

[Jaroslaw Cwiklik]

Daniel, looks like UIMA-AS client receives just the
ResourceInitializationException
and MessageTimeoutException is not available. I will create JIRA for this
soon.
As a workaround you can try changing delegate's error handling thresholds:


  
  
  


  
 

specifically maxRetries and timeout ( may need to adjust client
getMetaTimeout as well).


On Mon, May 28, 2018 at 3:29 PM, Daniel Baumartz <
bauma...@stud.uni-frankfurt.de> wrote:

> Hi,
>
> I can see an MessageTimeoutException in the log when calling
> BaseUIMAAsynchronousEngine_impl.deploy():
>
>
> org.apache.uima.aae.error.MessageTimeoutException: Delegate
> Service:ServiceName Has Timed Out While Processing CAS:null
> at org.apache.uima.aae.delegate.Delegate$2.run(Delegate.java:827)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
>
>
> The timeout is actually expected because the service is not running, but I
> can only catch an exception like this, without the details:
>
>
> org.apache.uima.resource.ResourceInitializationException
> at org.apache.uima.adapter.jms.activemq.SpringContainerDeployer
> .waitForServiceNotification(SpringContainerDeployer.java:530)
> at org.apache.uima.adapter.jms.activemq.SpringContainerDeployer
> .initializeContainer(SpringContainerDeployer.java:314)
> at org.apache.uima.adapter.jms.activemq.SpringContainerDeployer
> .deploy(SpringContainerDeployer.java:434)
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngin
> e_impl.deploy(BaseUIMAAsynchronousEngine_impl.java:916)
> ...
> Caused by: org.apache.uima.resource.ResourceInitializationException
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> stop(BaseAnalysisEngineController.java:2001)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> stop(BaseAnalysisEngineController.java:1898)
> at org.apache.uima.aae.controller.AggregateAnalysisEngineContro
> ller_impl.stop(AggregateAnalysisEngineController_impl.java:3272)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> terminate(BaseAnalysisEngineController.java:2184)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> terminate(BaseAnalysisEngineController.java:2150)
> at org.apache.uima.aae.error.handler.GetMetaErrorHandler.handle
> Error(GetMetaErrorHandler.java:104)
> at org.apache.uima.aae.error.ErrorHandlerChain.handle(ErrorHand
> lerChain.java:57)
> at org.apache.uima.aae.delegate.ControllerDelegate.handleError(
> ControllerDelegate.java:61)
> at org.apache.uima.aae.delegate.Delegate$2.run(Delegate.java:839)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
>
>
> I tried to use the callback listeners, but they do not seem to get called,
> am I doing something wrong here? Or is there another way to get the
> MessageTimeoutException object during deploy?
>
> Thanks,
> Daniel
>

Re: Deploy Async Service without XML

[Jaroslaw Cwiklik]

You can generate UIMA-AS deployment descriptor programmatically. You still
need AE descriptor though.

Please check UIMA-AS documentation:
https://uima.apache.org/d/uima-as-2.10.3/uima_async_scaleout.html#ugr.ref.async.api.usage
section 4.10 Generating Deployment Descriptor Programmatically

Jerry

On Mon, May 14, 2018 at 9:47 PM, Osborne, John D  wrote:

> Is it possible to deploy an UIMA-AS service without an XML descriptor
> similar to how UIMA-FIT works? I currently deploy services using
> deployAsyncService.sh
>
> I have multiple long running services that need to work in different
> (production, testing, dev) environments and would prefer to avoid having an
> XML file for each service. I realize that with some refactoring (like
> removing environment specific parameters) this number of XML files could be
> reduced, but I've become spoiled with UIMA-FIT. :)
>
> I'm looking at the toXML()  function so I can potentially generate the
> aggregate analysis engine with UIMA-FIT.
>
> -John
>
>

Re: Apache UIMA Java sdk 3.0.0 released

[Jaroslaw Cwiklik]

If I recall Marshall has a branch of UIMA-AS that is compatible with UIMA
3.X I think it is based on an older version of UIMA-AS though. Most likely
v.2.9.0.

SVN link is here: https://svn.apache.org/repos/asf/uima/uv3/uima-as-v3/trunk

I recently released v.2.10.2 and currently 2.10.3 RC1 is pending votes.
These two are not compatible with UIMA 3.X. I am working on a new UIMA-AS
which will be. The main feature in this new version will be support for
clients to communicate with collocated services without a JMS broker. There
are other changes like dropping dd2spring and Saxon in favor of parsing DD
in java. Most of the code is in place and I am testing and fixing bugs as
the time allows.
No ETA for the release yet.

Jerry



On Wed, Mar 21, 2018 at 3:58 PM, D. Heinze  wrote:

> Thanks.  I've spent a bit of time trying to ensure that all the XML
> dependencies are up-to-date.  Xalan and Xerces (both of which are pulled in
> as dependencies by other jars) are both up-to-date.  These warnings seem to
> be something that comes around every few years.
>
> I see from the developers list that UIMA-AS may be expected sometime soon?
>
> Thanks / Dan
>
> -Original Message-
> From: Richard Eckart de Castilho [mailto:r...@apache.org]
> Sent: Wednesday, March 21, 2018 10:11 AM
> To: user@uima.apache.org
> Subject: Re: Apache UIMA Java sdk 3.0.0 released
>
> On 20.03.2018, at 15:31, D. Heinze  wrote:
> >
> > Yes, I fixed some old dependencies and got rid of the XML errors.  Now
> > I am getting the following warning when calling the code below...
> > Seems this was an old problem that was fixed several UIMA releases
> > ago.  The only old UIMA thing I have in dependencies is uimafit1.4
> > which is for some deprecated code that will eventually be removed.
> >
> > WARN  [  uima] SAXTransformerFactory didn't
> > recognize setting attribute
> > http://javax.xml.XMLConstants/property/accessExternalDTD
> > WARN  [  uima] SAXTransformerFactory didn't
> > recognize setting attribute
> > http://javax.xml.XMLConstants/property/accessExternalStylesheet
>
> UIMA is trying to configure the XML parser to avoid potential security
> problems. These messages are issued when the underlying XML parser does not
> support the respective configuration properties. The code should work
> nevertheless although it might be a good idea to check if you have any
> additional "old" XML dependencies which might cause the default JDK XML
> implementations not to be used, maybe a Xalan?
>
> -- Richard=
>
>

Re: UIMA AS - ResourceInitializationException

[Jaroslaw Cwiklik]

Try adding a callback listener to the client via

public void addStatusCallbackListener(UimaAsBaseCallbackListener aListener)

There is try ... catch block in the client code

} catch (ResourceInitializationException e) {
  state = ClientState.FAILED;
  notifyOnInitializationFailure(e);
  throw e;
}

which should notify your code of the problem. Are you deploying your
services in
the same process as the application? Or are these services deployed in their
own processes?

Jerry


On Wed, Nov 29, 2017 at 12:00 PM, Wahed Hemati 
wrote:

> Hi,
>
> i am trying to run  a pipeline of 4 annotators in UIMA AS. For this i use
> the BaseUIMAAsynchronousEngine_impl.  The 4 annotators run as services.
>
> I am creating a scenario, where i shut down one of these services. When i
> try to run the same pipeline with the 4 annotators, obviously the pipeline
> can not be initialized, because one of the delegates is not reachable.
>
> I get the following exception:
>
> org.apache.uima.resource.ResourceInitializationException
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> stop(BaseAnalysisEngineController.java:2001)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> stop(BaseAnalysisEngineController.java:1898)
> at org.apache.uima.aae.controller.AggregateAnalysisEngineContro
> ller_impl.stop(AggregateAnalysisEngineController_impl.java:3272)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> terminate(BaseAnalysisEngineController.java:2184)
> at org.apache.uima.aae.controller.BaseAnalysisEngineController.
> terminate(BaseAnalysisEngineController.java:2150)
> at org.apache.uima.aae.error.handler.GetMetaErrorHandler.handle
> Error(GetMetaErrorHandler.java:104)
> at org.apache.uima.aae.error.ErrorHandlerChain.handle(ErrorHand
> lerChain.java:57)
> at org.apache.uima.aae.delegate.ControllerDelegate.handleError(
> ControllerDelegate.java:61)
> at org.apache.uima.aae.delegate.Delegate$2.run(Delegate.java:839)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
>
>
> The strange part is, that i cant get can not catch this exception in my
> client application. My client application just shuts down after this
> exception.
>
> What is the best approach to catch this exception.
>
> Is it possible to first "ping" each delegate, to see if its alive?
>
>
> -Best
>
> Wahed
>
>

Re: How to have client, service and broker co-located using UIMA-AS

[Jaroslaw Cwiklik]

Manuel, the current uima-as code requires a broker between a client and a
service even if all are colocated in the same process. This implies CAS
serialization/deserialization between endpoints. I am refactoring uima-as
code as part of JIRA  https://issues.apache.org/jira/browse/UIMA-5280. Once
the changes are complete the client and a service will use java queue
messaging, passing CASes by reference. Don't have ETA yet. This has been an
ongoing effort. Most of the new code is in place to support the JIRA plus a
few other enhancements. Testing and debugging will take some time.
Jerry

On Thu, Oct 5, 2017 at 11:30 AM, Manuel Fiorelli <manuel.fiore...@gmail.com>
wrote:

> Hi Jaroslaw,
>
> I applied your suggestions to my program, and indeed it seems to work
> without the need of an external broker.
>
> However, it seems to me that timings are more or less the same with the
> co-located and the remote (still on the same machine) broker.
>
> In both cases client and services are launched by the same process.
>
> So my question is: "Is my program passing the CASes by reference?"
> Additionally, "Is this optimization enabled whenever client and service are
> co-located irrespective of the position of the broker?"
>
> Regards,
> Manuel
>
> 2017-10-04 22:23 GMT+02:00 Manuel Fiorelli <manuel.fiore...@gmail.com>:
>
> > Hi Jaroslaw
> >
> > Thank you for your answer. Tomorrow, I will apply your suggestions and
> > tell you what happens.
> >
> > Il 4 ott 2017 19:50, "Jaroslaw Cwiklik" <cwik...@apache.org> ha scritto:
> >
> >> Manuel, small correction. Towards the end replace this
> >>
> >> client.sendCAS(cas);
> >>
> >> with
> >>
> >> client.sendAndReceiveCAS(cas);
> >>
> >> Simpler, synchronous msg exchange between client and a service.
> >>
> >> Jerry Cwiklik
> >>
> >> On Wed, Oct 4, 2017 at 1:00 PM, Jaroslaw Cwiklik <cwik...@apache.org>
> >> wrote:
> >>
> >> > Hi Manuel, you *can* colocate broker, client and a service in the same
> >> > process. Here is the code to get this done. Sorry did not test this.
> You
> >> > need to create a deployment descriptor (xml file) to be able to
> deploy a
> >> > service.
> >> >
> >> > // Deploy in-process broker
> >> > String uri = "tcp://"localhost:61616";
> >> > BrokerService broker =
> >> >  BrokerFactory.createBroker(new URI("broker:()/localhost?
> >> > persistent=false"));
> >> > TransportConnector tcpConnector = broker.addConnector(uri);
> >> > broker.start();
> >> > //--
> >> >
> >> > UimaAsynchronousEngine client =
> >> > new BaseUIMAAsynchronousEngine_impl();
> >> >
> >> > Map<String,String> appCtx = new HashMap();
> >> > appCtx.put(UimaAsynchronousEngine.DD2SpringXsltFilePath,
> >> >$UIMA_HOME+"/bin/dd2spring.xsl");
> >> > appCtx.put(UimaAsynchronousEngine.SaxonClasspath,
> >> >"file:"+$UIMA_HOME+"/saxon/saxon8.jar");
> >> >
> >> > String aDeploymentDescriptorPath =
> >> >   
> >> >
> >> > // deploy in-process service
> >> > String serviceId =
> >> >   client.deploy(aDeploymentDescriptorPath, appCtx);
> >> >
> >> > String endpoint = ; // from the deployment descriptor
> >> >
> >> > appCtx.put(UimaAsynchronousEngine.ServerUri, uri);
> >> > appCtx.put(UimaAsynchronousEngine.ENDPOINT, endpoint);
> >> >
> >> > // connect client with an in-process service
> >> > client.initialize(appCtx);
> >> >
> >> > CAS cas = client.getCAS();
> >> > cas.setDocumentText("Some Text");
> >> > client.sendCAS(cas);
> >> >
> >> > client.stop();
> >> >
> >> >
> >> >
> >> > On Wed, Oct 4, 2017 at 11:37 AM, Manuel Fiorelli <
> >> > manuel.fiore...@gmail.com> wrote:
> >> >
> >> >> Hi everybody
> >> >>
> >> >> Within the documentation of UIMA-AS, specifically within the section
> >> >> Terminology (
> >> >> https://uima.apache.org/d/uima-as-2.9.0/uima_async_scaleout.
> >> >> html#ugr.async.ov.terminology),
> >> >> the description of Queue Broker says:
> >> >>
> >> >> 
>

Re: How to have client, service and broker co-located using UIMA-AS

[Jaroslaw Cwiklik]

Hi Manuel, you *can* colocate broker, client and a service in the same
process. Here is the code to get this done. Sorry did not test this. You
need to create a deployment descriptor (xml file) to be able to deploy a
service.

// Deploy in-process broker
String uri = "tcp://"localhost:61616";
BrokerService broker =
 BrokerFactory.createBroker(new
URI("broker:()/localhost?persistent=false"));
TransportConnector tcpConnector = broker.addConnector(uri);
broker.start();
//--

UimaAsynchronousEngine client =
new BaseUIMAAsynchronousEngine_impl();

Map appCtx = new HashMap();
appCtx.put(UimaAsynchronousEngine.DD2SpringXsltFilePath,
   $UIMA_HOME+"/bin/dd2spring.xsl");
appCtx.put(UimaAsynchronousEngine.SaxonClasspath,
   "file:"+$UIMA_HOME+"/saxon/saxon8.jar");

String aDeploymentDescriptorPath =
  

// deploy in-process service
String serviceId =
  client.deploy(aDeploymentDescriptorPath, appCtx);

String endpoint = ; // from the deployment descriptor

appCtx.put(UimaAsynchronousEngine.ServerUri, uri);
appCtx.put(UimaAsynchronousEngine.ENDPOINT, endpoint);

// connect client with an in-process service
client.initialize(appCtx);

CAS cas = client.getCAS();
cas.setDocumentText("Some Text");
client.sendCAS(cas);

client.stop();



On Wed, Oct 4, 2017 at 11:37 AM, Manuel Fiorelli 
wrote:

> Hi everybody
>
> Within the documentation of UIMA-AS, specifically within the section
> Terminology (
> https://uima.apache.org/d/uima-as-2.9.0/uima_async_
> scaleout.html#ugr.async.ov.terminology),
> the description of Queue Broker says:
>
> 
> Queue brokers manage one or more named queues. The brokers are identified
> using a URL, representing where they are on the network. When the queue
> broker is co-located with the AS client and service, CASes are passed by
> reference, avoiding serialization / deserialization.
> 
>
> Unfortunately, I could not find in the documentation how to place broker,
> client and service in the same JVM (co-location).
>
> After some web searches, experiments and exploration of the source code of
> UIMA-AS, I found that the solution may require the use of the URL
> "vm://localhost?broker.persistent=false" for the broker.
>
> In my sample program, I deploy a service (an asynchronous aggregate AE) and
> then create an asynchronous client using the URL above, but then I receive
> the following NullPointerException:
>
> 
> ott 04, 2017 5:29:28 PM
> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl
> replyToClient
> AVVERTENZA: Service: Top Level Aggregate Service Runtime Exception
> ott 04, 2017 5:29:28 PM
> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl
> replyToClient
> AVVERTENZA:
> java.lang.NullPointerException
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.sendVMMessage(AggregateAnalysisEngineController_impl.java:2393)
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.
> sendReplyToCollocatedClient(AggregateAnalysisEngineControl
> ler_impl.java:2287)
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.replyToClient(AggregateAnalysisEngineController_impl.java:2344)
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.finalStep(AggregateAnalysisEngineController_impl.java:1862)
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.executeFlowStep(AggregateAnalysisEngineController_impl.java:2489)
> at
> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.process(AggregateAnalysisEngineController_impl.java:1271)
> at
> org.apache.uima.aae.handler.HandlerBase.invokeProcess(
> HandlerBase.java:118)
> at
> org.apache.uima.aae.handler.input.ProcessResponseHandler.
> cancelTimerAndProcess(ProcessResponseHandler.java:117)
> at
> org.apache.uima.aae.handler.input.ProcessResponseHandler.
> handleProcessResponseWithCASReference(ProcessResponseHandler.java:485)
> at
> org.apache.uima.aae.handler.input.ProcessResponseHandler.
> handle(ProcessResponseHandler.java:767)
> at
> org.apache.uima.aae.handler.HandlerBase.delegate(HandlerBase.java:149)
> at
> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(
> ProcessRequestHandler_impl.java:1113)
> at
> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.onMessage(
> UimaVmMessageListener.java:107)
> at
> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher$1.run(
> UimaVmMessageDispatcher.java:70)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at
> org.apache.uima.aae.UimaAsThreadFactory$1.run(
> UimaAsThreadFactory.java:132)
> at java.lang.Thread.run(Thread.java:745)
> 
>
> The exception above does not happen when I use a remote broker, so I
> suspect 

Re: XMl generated by AnalysisEngineDescription.toXML not accepted by UIMA-as

[Jaroslaw Cwiklik]

Just a clarification on this xml element (annotatorImplementationName),
from UIMA documentation:

"The next subelement,  is how the UIMA
framework determines which annotator class to use. This should contain a
fully-qualified Java class name for Java implementations, or the name of a
.dll or .so file for C++ implementations."

So add the fully class name which implements your AE in the
annotatorImplementationName.

Jerry

On Tue, Sep 19, 2017 at 3:40 PM, Jaroslaw Cwiklik <cwik...@apache.org>
wrote:

> OK, this is much clearer now. It looks like the NPE comes from the uima-as
> code while trying to determine name for the service. Its trying to use your
> analysis engine  descriptor   WHATEVER annotatorImplementationName>
> Is this missing in your AE descriptor? If so add it and give it a
> meaningful name.
>
> Jerry
>
> On Tue, Sep 19, 2017 at 1:38 PM, r. r. <rob...@abv.bg> wrote:
>
>> Great - I added this and the file is preserved now
>> posted it here: https://pastebin.com/Z2pHa0nN
>>
>> And there is a uima.log indeed! posted it here:
>> https://pastebin.com/dQ4vUW5T
>>
>> It mentions 'primitive_ctrl_...', is that correct? Because the AE is an
>> aggregate one.
>> Before running this test, I ran the uima-as examples and they work fine
>>
>> Thanks!
>> -Robert
>>
>>
>>
>>
>>
>>
>>
>>  > Оригинално писмо 
>>
>>  >От: Jaroslaw Cwiklik cwik...@apache.org
>>
>>  >Относно: Re: XMl generated by AnalysisEngineDescription.toXML not
>> accepted by UIMA-as
>>
>>  >До: user@uima.apache.org
>>
>>  >Изпратено на: 19.09.2017 20:07
>>
>>
>>
>>
>> > I setup my classpath with all the jars needed by the uima-as runtime.
>> What
>>
>> > is strange is that you report a NullPointerException in a constructor of
>>
>> > one of the UIMA-AS classes. I would have expected
>> ClassNotFoundException if
>>
>> > a jar was missing. Unless the actual cause is swallowed somewhere. Do
>> you
>>
>> > have uima.log in a directory where u start the service?
>>
>> >
>>
>> > To preserve a Spring file add: -uimaEeDebug right after -saxonURL and
>> -xslt
>>
>> >  When you get the failure look for a path to it as in your original
>>
>> > error:
>>
>> >
>>
>> > ...Error creating bean with name 'primitive_ctlr_DKProStanfordQueue_1'
>>
>> > defined in URL [*file:/var/folders/dt/**vmntv
>> 00526g5s84b85rqwmm5qbfkqb/T/*
>>
>> > UIMAdd2springOutput2528967577403356774.xml]
>>
>> >
>>
>> > Jerry
>>
>> >
>>
>> > On Tue, Sep 19, 2017 at 12:46 PM, r. r. <rob...@abv.bg> wrote:
>>
>> >
>>
>> > > Thanks Jerry,
>>
>> > > No, that file is gone after the command fails.
>>
>> > > Do I have to setup the jars in use in the class path or is UIMA-as
>> getting
>>
>> > > all the info to locate jars from the descriptor xmls?
>>
>> > > Or maybe a way to see a yet more detailed error description by the
>> deploy
>>
>> > > tool?
>>
>> > >
>>
>> > > I'm running this with Java 8 on OSX Sierra:
>>
>> > > java version "1.8.0_45"
>>
>> > > Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
>>
>> > > Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
>>
>> > >
>>
>> > > Best regards
>>
>> > > -Robert
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >  > Оригинално писмо 
>>
>> > >
>>
>> > >  >От: Jaroslaw Cwiklik cwik...@apache.org
>>
>> > >
>>
>> > >  >Относно: Re: XMl generated by AnalysisEngineDescription.toXML not
>>
>> > > accepted by UIMA-as
>>
>> > >
>>
>> > >  >До: user@uima.apache.org
>>
>> > >
>>
>> > >  >Изпратено на: 19.09.2017 19:28
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > >
>>
>> > > > Robert, the DD looks ok to me. I actually stripped down one of my
>> DD to
>>
>> > >
>>
>> > > > look just as yours and i

Re: XMl generated by AnalysisEngineDescription.toXML not accepted by UIMA-as

[Jaroslaw Cwiklik]

OK, this is much clearer now. It looks like the NPE comes from the uima-as
code while trying to determine name for the service. Its trying to use your
analysis engine  descriptor
WHATEVER
Is this missing in your AE descriptor? If so add it and give it a
meaningful name.

Jerry

On Tue, Sep 19, 2017 at 1:38 PM, r. r. <rob...@abv.bg> wrote:

> Great - I added this and the file is preserved now
> posted it here: https://pastebin.com/Z2pHa0nN
>
> And there is a uima.log indeed! posted it here:
> https://pastebin.com/dQ4vUW5T
>
> It mentions 'primitive_ctrl_...', is that correct? Because the AE is an
> aggregate one.
> Before running this test, I ran the uima-as examples and they work fine
>
> Thanks!
> -Robert
>
>
>
>
>
>
>
>  > Оригинално писмо 
>
>  >От: Jaroslaw Cwiklik cwik...@apache.org
>
>  >Относно: Re: XMl generated by AnalysisEngineDescription.toXML not
> accepted by UIMA-as
>
>  >До: user@uima.apache.org
>
>  >Изпратено на: 19.09.2017 20:07
>
>
>
>
> > I setup my classpath with all the jars needed by the uima-as runtime.
> What
>
> > is strange is that you report a NullPointerException in a constructor of
>
> > one of the UIMA-AS classes. I would have expected ClassNotFoundException
> if
>
> > a jar was missing. Unless the actual cause is swallowed somewhere. Do you
>
> > have uima.log in a directory where u start the service?
>
> >
>
> > To preserve a Spring file add: -uimaEeDebug right after -saxonURL and
> -xslt
>
> >  When you get the failure look for a path to it as in your original
>
> > error:
>
> >
>
> > ...Error creating bean with name 'primitive_ctlr_DKProStanfordQueue_1'
>
> > defined in URL [*file:/var/folders/dt/**vmntv00526g5s84b85rqwmm5qbfkqb
> /T/*
>
> > UIMAdd2springOutput2528967577403356774.xml]
>
> >
>
> > Jerry
>
> >
>
> > On Tue, Sep 19, 2017 at 12:46 PM, r. r. <rob...@abv.bg> wrote:
>
> >
>
> > > Thanks Jerry,
>
> > > No, that file is gone after the command fails.
>
> > > Do I have to setup the jars in use in the class path or is UIMA-as
> getting
>
> > > all the info to locate jars from the descriptor xmls?
>
> > > Or maybe a way to see a yet more detailed error description by the
> deploy
>
> > > tool?
>
> > >
>
> > > I'm running this with Java 8 on OSX Sierra:
>
> > > java version "1.8.0_45"
>
> > > Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
>
> > > Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
>
> > >
>
> > > Best regards
>
> > > -Robert
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >  > Оригинално писмо 
>
> > >
>
> > >  >От: Jaroslaw Cwiklik cwik...@apache.org
>
> > >
>
> > >  >Относно: Re: XMl generated by AnalysisEngineDescription.toXML not
>
> > > accepted by UIMA-as
>
> > >
>
> > >  >До: user@uima.apache.org
>
> > >
>
> > >  >Изпратено на: 19.09.2017 19:28
>
> > >
>
> > >
>
> > >
>
> > >
>
> > > > Robert, the DD looks ok to me. I actually stripped down one of my DD
> to
>
> > >
>
> > > > look just as yours and it deployed fine. Do you still have this file:
>
> > >
>
> > > > file:/var/folders/dt/vmntv00526g5s84b85rqwmm5qbfkqb/T/
>
> > >
>
> > > > UIMAdd2springOutput2528967577403356774.xml
>
> > >
>
> > > >
>
> > >
>
> > > > This is the transformed file which is fed to Spring factory. Can you
>
> > > share
>
> > >
>
> > > > this via pastbin?
>
> > >
>
> > > > Also which version of java are you using?
>
> > >
>
> > > >
>
> > >
>
> > > > Jerry
>
> > >
>
> > > >
>
> > >
>
> > > >
>
> > >
>
> > > > On Tue, Sep 19, 2017 at 11:45 AM, Jaroslaw Cwiklik <
> cwik...@apache.org>
>
> > >
>
> > > > wrote:
>
> > >
>
> > > >
>
> > >
>
> > > > > Hi Robert, any chance u can paste the deployment descriptor for a
>
> > > review?
>
> > >
>
> > > > > Were there any other errors/warnings before

Re: XMl generated by AnalysisEngineDescription.toXML not accepted by UIMA-as

[Jaroslaw Cwiklik]

I setup my classpath with all the jars needed by the uima-as runtime. What
is strange is that you report a NullPointerException in a constructor of
one of the UIMA-AS classes. I would have expected ClassNotFoundException if
a jar was missing. Unless the actual cause is swallowed somewhere. Do you
have uima.log in a directory where u start the service?

To preserve a Spring file add: -uimaEeDebug right after -saxonURL and -xslt
 When you get the failure look for a path to it as in your original
error:

...Error creating bean with name 'primitive_ctlr_DKProStanfordQueue_1'
defined in URL [*file:/var/folders/dt/**vmntv00526g5s84b85rqwmm5qbfkqb/T/*
UIMAdd2springOutput2528967577403356774.xml]

Jerry

On Tue, Sep 19, 2017 at 12:46 PM, r. r. <rob...@abv.bg> wrote:

> Thanks Jerry,
> No, that file is gone after the command fails.
> Do I have to setup the jars in use in the class path or is UIMA-as getting
> all the info to locate jars from the descriptor xmls?
> Or maybe a way to see a yet more detailed error description by the deploy
> tool?
>
> I'm running this with Java 8 on OSX Sierra:
> java version "1.8.0_45"
> Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
>
> Best regards
> -Robert
>
>
>
>
>
>
>
>  > Оригинално писмо 
>
>  >От: Jaroslaw Cwiklik cwik...@apache.org
>
>  >Относно: Re: XMl generated by AnalysisEngineDescription.toXML not
> accepted by UIMA-as
>
>  >До: user@uima.apache.org
>
>  >Изпратено на: 19.09.2017 19:28
>
>
>
>
> > Robert, the DD looks ok to me. I actually stripped down one of my DD to
>
> > look just as yours and it deployed fine. Do you still have this file:
>
> > file:/var/folders/dt/vmntv00526g5s84b85rqwmm5qbfkqb/T/
>
> > UIMAdd2springOutput2528967577403356774.xml
>
> >
>
> > This is the transformed file which is fed to Spring factory. Can you
> share
>
> > this via pastbin?
>
> > Also which version of java are you using?
>
> >
>
> > Jerry
>
> >
>
> >
>
> > On Tue, Sep 19, 2017 at 11:45 AM, Jaroslaw Cwiklik <cwik...@apache.org>
>
> > wrote:
>
> >
>
> > > Hi Robert, any chance u can paste the deployment descriptor for a
> review?
>
> > > Were there any other errors/warnings before? The UIMA-AS deployment
>
> > > descriptor undergoes a transformation under the covers producing Spring
>
> > > Framework compatible file which is fed to Spring Factory for
> instantiating
>
> > > java code. In this
>
> > > last step it appears that there is an exception as you described.
> Wonder
>
> > > if there is other relevant info before this error
>
> > >
>
> > > Jerry
>
> > >
>
> > > On Tue, Sep 19, 2017 at 11:23 AM, r. r. <rob...@abv.bg> wrote:
>
> > >
>
> > >> Hello
>
> > >> I wrote an AE pipeline with uimaFit (and DKPro), which works fine in
> the
>
> > >> command line (mvn exec:java ...).
>
> > >> I successfully exported the xml descriptor via
>
> > >> AnalysisEngineDescription.toXML and want to use it with UIMA-as
>
> > >> (apache-uima-as-2.9.0).
>
> > >> My first attempt with deployAsyncService.sh failed with:
>
> > >>
>
> > >> "SEPM0004: When 'standalone' or 'doctype-system' is specified, the
>
> > >> document must be
>
> > >>   well-formed; but this document contains a top-level text node"
>
> > >>
>
> > >> but then I realised I need to supply an analysisEngineDeploymentDescri
> ption
>
> > >> to UIMA-as, so I amended this with a descriptor pointing to the one
>
> > >> generated by toXML().
>
> > >>
>
> > >> Now the error I'm getting is:
>
> > >>
>
> > >> "Exception encountered during context initialization - cancelling
> refresh
>
> > >> attempt: org.springframework.beans.factory.BeanCreationException:
> Error
>
> > >> creating bean with name 'primitive_ctlr_DKProStanfordQueue_1'
> defined in
>
> > >> URL [file:/var/folders/dt/vmntv00526g5s84b85rqwmm5qbfkqb/T/UIMAd
>
> > >> d2springOutput2528967577403356774.xml]: Bean instantiation via
>
> > >> constructor failed; nested exception is org.springframework.beans.
> BeanInstantiationException:
>
> > >> Failed to instantiate [org.apache.uima.aae.controlle
>
> > >> r.PrimitiveAnalysisEngineController_impl]: Constructor threw
> exception;
>
> > >> nested exception is java.lang.NullPointerException"
>
> > >>
>
> > >> Any idea what might be causing this?
>
> > >>
>
> > >> Thanks!
>
> > >>
>
> > >> Robert
>
> > >>
>
> > >>
>
> > >
>

Re: XMl generated by AnalysisEngineDescription.toXML not accepted by UIMA-as

[Jaroslaw Cwiklik]

Hi Robert, any chance u can paste the deployment descriptor for a review?
Were there any other errors/warnings before? The UIMA-AS deployment
descriptor undergoes a transformation under the covers producing Spring
Framework compatible file which is fed to Spring Factory for instantiating
java code. In this
last step it appears that there is an exception as you described. Wonder if
there is other relevant info before this error

Jerry

On Tue, Sep 19, 2017 at 11:23 AM, r. r.  wrote:

> Hello
> I wrote an AE pipeline with uimaFit (and DKPro), which works fine in the
> command line (mvn exec:java ...).
> I successfully exported the xml descriptor via AnalysisEngineDescription.toXML
> and want to use it with UIMA-as (apache-uima-as-2.9.0).
> My first attempt with deployAsyncService.sh failed with:
>
> "SEPM0004: When 'standalone' or 'doctype-system' is specified, the
> document must be
>   well-formed; but this document contains a top-level text node"
>
> but then I realised I need to supply an analysisEngineDeploymentDescription
> to UIMA-as, so I amended this with a descriptor pointing to the one
> generated by toXML().
>
> Now the error I'm getting is:
>
> "Exception encountered during context initialization - cancelling refresh
> attempt: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'primitive_ctlr_DKProStanfordQueue_1' defined in
> URL [file:/var/folders/dt/vmntv00526g5s84b85rqwmm5qbfkqb/T/
> UIMAdd2springOutput2528967577403356774.xml]: Bean instantiation via
> constructor failed; nested exception is 
> org.springframework.beans.BeanInstantiationException:
> Failed to instantiate [org.apache.uima.aae.controller.
> PrimitiveAnalysisEngineController_impl]: Constructor threw exception;
> nested exception is java.lang.NullPointerException"
>
> Any idea what might be causing this?
>
> Thanks!
>
> Robert
>
>

[ANNOUNCE] Apache UIMA DUCC 2.2.1 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the Apache
UIMA-DUCC version 2.2.1.

The Unstructured Information Management Architecture (UIMA) is a component
framework supporting development, discovery, composition, and deployment of
multi-modal analytics tasked with the analysis of unstructured information.

Apache UIMA is an Apache licensed open source implementation of the UIMA
specification which is being developed by a technical committee within
OASIS, a standards organization. The implementation comprises an SDK and
tooling for composing and running analytic components written in Java and
C++, with some support for Perl, Python and TCL.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling,management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources.
DUCC defines a formal job model that closely maps to a standard UIMA
pipeline. Around this job model DUCC provides cluster management services
to automate the scale-out of UIMA pipelines over computing clusters.


This release contains a number of improvements and bug fixes. Notable
updates in this release include:

-The userid of a privileged DUCC installation does not have to be "ducc"
-ducc-mon login can be used on systems where users do not have password
login
-The DUCC head-node daemons may be moved to another host without breaking
working applications
-The deployment descriptor for a UIMA-AS service can be loaded from the
classpath
-Interactive applications run correctly with viaducc (fixed lost inputs)
-Files created by DUCC jobs inherit the permissions of the launching
shell's umask.
-DUCC performance breakdown for scaled synchronous pipelines is now correct
-Fixed javadoc method headers to enable ducc build with java 8
-Fixed JP communication threads wait logic when JD returns no work
-GC stats not available sometimes from remote JP

For a complete list of bugs and improvements included in this release
please see
https://uima.apache.org/d/uima-ducc-2.2.1/issuesFixed/jira-report.html

-- Jerry Cwiklik, for the Apache UIMA development team

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Thanks Nelson for testing this change. I will commit it soon.
Jerry

On Wed, Aug 30, 2017 at 1:57 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> jerry , i tested the new snapshot of uima-as that you pushed into
> apache snapshot repo to fix problem with the type priorities, and the
> problem is solved. I tested with uima as service with priority
> configuration in the aggregate's descriptor and every things is ok.
> thanks for your quick attention
>
> 2017-08-25 16:47 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Nelson, I just pushed a new snapshot of uima-as to apache snapshot repo.
> > This is a trunk level code not a release. Please deploy it and see if the
> > problem with the type priorites is fixed. I did test it with a simple
> type
> > priority configuration.
> >
> > https://repository.apache.org/content/repositories/
> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-
> as-2.9.1-20170825.204106-5.tar.gz
> >
> > Jerry
> >
> >
> > On Fri, Aug 25, 2017 at 8:25 AM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> Thanks for your attention Jerry, i wait the solution to integrate it
> >> in production
> >>
> >> 2017-08-24 14:27 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> > Nelson, I was able to recreate this problem and am working on a fix
> >> > now.
> >> > Jerry
> >> >
> >> > On Wed, Aug 16, 2017 at 3:39 PM, nelson rivera
> >> > <nelsonriver...@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> Jerry, i understand and wait. Thanks for your attention.
> >> >>
> >> >> 2017-08-16 15:09 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> >> > Nelson, I need more time to look into this. Quite busy at a moment.
> >> >> Unless
> >> >> > someone else wants to jump in here I will address the problem when
> I
> >> am
> >> >> > done with current work load,
> >> >> >
> >> >> > Jerry
> >> >> >
> >> >> > On Mon, Aug 14, 2017 at 4:36 PM, Jaroslaw Cwiklik
> >> >> > <cwik...@apache.org
> >> >
> >> >> > wrote:
> >> >> >
> >> >> >> Thanks, I will try to test this scenario and get back to ya.
> >> >> >>
> >> >> >> Jerry
> >> >> >>
> >> >> >> On Mon, Aug 14, 2017 at 4:27 PM, nelson rivera <
> >> >> nelsonriver...@gmail.com>
> >> >> >> wrote:
> >> >> >>
> >> >> >>> Yes, that is happening
> >> >> >>>
> >> >> >>> 2017-08-14 15:40 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org
> >:
> >> >> >>> > Please confirm that this is happening:
> >> >> >>> >
> >> >> >>> > 1) You deploy an aggregate and specify TypePriorities in
> >> >> >>> > aggregate
> >> >> >>> analysis
> >> >> >>> > engine descriptor.
> >> >> >>> > 2) After uima-as client initilizes, you call client's
> >> >> >>> > public ProcessingResourceMetaData getMetaData();
> >> >> >>> > 3) You check the returned meta and the type priorities are
> >> >> >>> > missing
> >> >> >>> >
> >> >> >>> > You are observing correct behavior when you declare type
> >> priorities
> >> >> in
> >> >> >>> > a delegate analysis engine descriptor of the aggregate
> >> >> >>> >
> >> >> >>> > Jerry
> >> >> >>> >
> >> >> >>> > On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera <
> >> >> >>> nelsonriver...@gmail.com>
> >> >> >>> > wrote:
> >> >> >>> >
> >> >> >>> >> i have configured a priorityList in the descriptor of my
> >> aggregate
> >> >> >>> >> engine, with the objetive of use correctly subiterator()
> >> >> >>> >> function
> >> >> >>> >> in
> >> >> >>> >> my client code over the result CAS.
> >> >> >>> >>
> >> >> >>> >> After deploy the aggregate in uima-as, in my client code that
> >> >> >>> >> consumes
> >> >> >>> >> de uima-as service aggregate to check the priorityList
> >> >> >>> >> configured
> >> >> >>> >> i
> >> >> >>> >> use the instruccion
> >> >> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and
> >> not
> >> >> >>> >> appear the priority configured. This does that the use of
> >> >> subiterator
> >> >> >>> >> not have the expected behavior in the uima-as client.
> >> >> >>> >>
> >> >> >>> >> If the configuration of priorityList is made in the descriptor
> >> >> >>> >> of
> >> >> one
> >> >> >>> >> of the delegate of the aggregate, then if the instruccion
> >> >> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in
> >> >> client
> >> >> >>> >> code shows the priorityList configured and then the  behavior
> >> >> >>> >> is
> >> >> >>> >> the
> >> >> >>> >> expected.
> >> >> >>> >>
> >> >> >>> >
> >> >> >>>
> >> >> >>
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Nelson, I just pushed a new snapshot of uima-as to apache snapshot repo.
This is a trunk level code not a release. Please deploy it and see if the
problem with the type priorites is fixed. I did test it with a simple type
priority configuration.

https://repository.apache.org/content/repositories/snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-as-2.9.1-20170825.204106-5.tar.gz

Jerry


On Fri, Aug 25, 2017 at 8:25 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Thanks for your attention Jerry, i wait the solution to integrate it
> in production
>
> 2017-08-24 14:27 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Nelson, I was able to recreate this problem and am working on a fix now.
> > Jerry
> >
> > On Wed, Aug 16, 2017 at 3:39 PM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> Jerry, i understand and wait. Thanks for your attention.
> >>
> >> 2017-08-16 15:09 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> > Nelson, I need more time to look into this. Quite busy at a moment.
> >> Unless
> >> > someone else wants to jump in here I will address the problem when I
> am
> >> > done with current work load,
> >> >
> >> > Jerry
> >> >
> >> > On Mon, Aug 14, 2017 at 4:36 PM, Jaroslaw Cwiklik <cwik...@apache.org
> >
> >> > wrote:
> >> >
> >> >> Thanks, I will try to test this scenario and get back to ya.
> >> >>
> >> >> Jerry
> >> >>
> >> >> On Mon, Aug 14, 2017 at 4:27 PM, nelson rivera <
> >> nelsonriver...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Yes, that is happening
> >> >>>
> >> >>> 2017-08-14 15:40 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> >>> > Please confirm that this is happening:
> >> >>> >
> >> >>> > 1) You deploy an aggregate and specify TypePriorities in aggregate
> >> >>> analysis
> >> >>> > engine descriptor.
> >> >>> > 2) After uima-as client initilizes, you call client's
> >> >>> > public ProcessingResourceMetaData getMetaData();
> >> >>> > 3) You check the returned meta and the type priorities are missing
> >> >>> >
> >> >>> > You are observing correct behavior when you declare type
> priorities
> >> in
> >> >>> > a delegate analysis engine descriptor of the aggregate
> >> >>> >
> >> >>> > Jerry
> >> >>> >
> >> >>> > On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera <
> >> >>> nelsonriver...@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> >> i have configured a priorityList in the descriptor of my
> aggregate
> >> >>> >> engine, with the objetive of use correctly subiterator() function
> >> >>> >> in
> >> >>> >> my client code over the result CAS.
> >> >>> >>
> >> >>> >> After deploy the aggregate in uima-as, in my client code that
> >> >>> >> consumes
> >> >>> >> de uima-as service aggregate to check the priorityList configured
> >> >>> >> i
> >> >>> >> use the instruccion
> >> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and
> not
> >> >>> >> appear the priority configured. This does that the use of
> >> subiterator
> >> >>> >> not have the expected behavior in the uima-as client.
> >> >>> >>
> >> >>> >> If the configuration of priorityList is made in the descriptor of
> >> one
> >> >>> >> of the delegate of the aggregate, then if the instruccion
> >> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in
> >> client
> >> >>> >> code shows the priorityList configured and then the  behavior is
> >> >>> >> the
> >> >>> >> expected.
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
> >
>

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Nelson, I was able to recreate this problem and am working on a fix now.
Jerry

On Wed, Aug 16, 2017 at 3:39 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Jerry, i understand and wait. Thanks for your attention.
>
> 2017-08-16 15:09 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Nelson, I need more time to look into this. Quite busy at a moment.
> Unless
> > someone else wants to jump in here I will address the problem when I am
> > done with current work load,
> >
> > Jerry
> >
> > On Mon, Aug 14, 2017 at 4:36 PM, Jaroslaw Cwiklik <cwik...@apache.org>
> > wrote:
> >
> >> Thanks, I will try to test this scenario and get back to ya.
> >>
> >> Jerry
> >>
> >> On Mon, Aug 14, 2017 at 4:27 PM, nelson rivera <
> nelsonriver...@gmail.com>
> >> wrote:
> >>
> >>> Yes, that is happening
> >>>
> >>> 2017-08-14 15:40 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >>> > Please confirm that this is happening:
> >>> >
> >>> > 1) You deploy an aggregate and specify TypePriorities in aggregate
> >>> analysis
> >>> > engine descriptor.
> >>> > 2) After uima-as client initilizes, you call client's
> >>> > public ProcessingResourceMetaData getMetaData();
> >>> > 3) You check the returned meta and the type priorities are missing
> >>> >
> >>> > You are observing correct behavior when you declare type priorities
> in
> >>> > a delegate analysis engine descriptor of the aggregate
> >>> >
> >>> > Jerry
> >>> >
> >>> > On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera <
> >>> nelsonriver...@gmail.com>
> >>> > wrote:
> >>> >
> >>> >> i have configured a priorityList in the descriptor of my aggregate
> >>> >> engine, with the objetive of use correctly subiterator() function in
> >>> >> my client code over the result CAS.
> >>> >>
> >>> >> After deploy the aggregate in uima-as, in my client code that
> >>> >> consumes
> >>> >> de uima-as service aggregate to check the priorityList configured i
> >>> >> use the instruccion
> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and not
> >>> >> appear the priority configured. This does that the use of
> subiterator
> >>> >> not have the expected behavior in the uima-as client.
> >>> >>
> >>> >> If the configuration of priorityList is made in the descriptor of
> one
> >>> >> of the delegate of the aggregate, then if the instruccion
> >>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in
> client
> >>> >> code shows the priorityList configured and then the  behavior is the
> >>> >> expected.
> >>> >>
> >>> >
> >>>
> >>
> >>
> >
>

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Nelson, I need more time to look into this. Quite busy at a moment. Unless
someone else wants to jump in here I will address the problem when I am
done with current work load,

Jerry

On Mon, Aug 14, 2017 at 4:36 PM, Jaroslaw Cwiklik <cwik...@apache.org>
wrote:

> Thanks, I will try to test this scenario and get back to ya.
>
> Jerry
>
> On Mon, Aug 14, 2017 at 4:27 PM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> Yes, that is happening
>>
>> 2017-08-14 15:40 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
>> > Please confirm that this is happening:
>> >
>> > 1) You deploy an aggregate and specify TypePriorities in aggregate
>> analysis
>> > engine descriptor.
>> > 2) After uima-as client initilizes, you call client's
>> > public ProcessingResourceMetaData getMetaData();
>> > 3) You check the returned meta and the type priorities are missing
>> >
>> > You are observing correct behavior when you declare type priorities in
>> > a delegate analysis engine descriptor of the aggregate
>> >
>> > Jerry
>> >
>> > On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera <
>> nelsonriver...@gmail.com>
>> > wrote:
>> >
>> >> i have configured a priorityList in the descriptor of my aggregate
>> >> engine, with the objetive of use correctly subiterator() function in
>> >> my client code over the result CAS.
>> >>
>> >> After deploy the aggregate in uima-as, in my client code that consumes
>> >> de uima-as service aggregate to check the priorityList configured i
>> >> use the instruccion
>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and not
>> >> appear the priority configured. This does that the use of subiterator
>> >> not have the expected behavior in the uima-as client.
>> >>
>> >> If the configuration of priorityList is made in the descriptor of one
>> >> of the delegate of the aggregate, then if the instruccion
>> >> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in client
>> >> code shows the priorityList configured and then the  behavior is the
>> >> expected.
>> >>
>> >
>>
>
>

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Thanks, I will try to test this scenario and get back to ya.

Jerry

On Mon, Aug 14, 2017 at 4:27 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Yes, that is happening
>
> 2017-08-14 15:40 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Please confirm that this is happening:
> >
> > 1) You deploy an aggregate and specify TypePriorities in aggregate
> analysis
> > engine descriptor.
> > 2) After uima-as client initilizes, you call client's
> > public ProcessingResourceMetaData getMetaData();
> > 3) You check the returned meta and the type priorities are missing
> >
> > You are observing correct behavior when you declare type priorities in
> > a delegate analysis engine descriptor of the aggregate
> >
> > Jerry
> >
> > On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera <
> nelsonriver...@gmail.com>
> > wrote:
> >
> >> i have configured a priorityList in the descriptor of my aggregate
> >> engine, with the objetive of use correctly subiterator() function in
> >> my client code over the result CAS.
> >>
> >> After deploy the aggregate in uima-as, in my client code that consumes
> >> de uima-as service aggregate to check the priorityList configured i
> >> use the instruccion
> >> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and not
> >> appear the priority configured. This does that the use of subiterator
> >> not have the expected behavior in the uima-as client.
> >>
> >> If the configuration of priorityList is made in the descriptor of one
> >> of the delegate of the aggregate, then if the instruccion
> >> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in client
> >> code shows the priorityList configured and then the  behavior is the
> >> expected.
> >>
> >
>

Re: TypePriorities configured in the descriptor of an aggregate engine are not added in uima-as

[Jaroslaw Cwiklik]

Please confirm that this is happening:

1) You deploy an aggregate and specify TypePriorities in aggregate analysis
engine descriptor.
2) After uima-as client initilizes, you call client's
public ProcessingResourceMetaData getMetaData();
3) You check the returned meta and the type priorities are missing

You are observing correct behavior when you declare type priorities in
a delegate analysis engine descriptor of the aggregate

Jerry

On Fri, Aug 11, 2017 at 12:12 PM, nelson rivera 
wrote:

> i have configured a priorityList in the descriptor of my aggregate
> engine, with the objetive of use correctly subiterator() function in
> my client code over the result CAS.
>
> After deploy the aggregate in uima-as, in my client code that consumes
> de uima-as service aggregate to check the priorityList configured i
> use the instruccion
> uimaAsynchronousEngine.getMetaData().getTypePriorities(); and not
> appear the priority configured. This does that the use of subiterator
> not have the expected behavior in the uima-as client.
>
> If the configuration of priorityList is made in the descriptor of one
> of the delegate of the aggregate, then if the instruccion
> uimaAsynchronousEngine.getMetaData().getTypePriorities()   in client
> code shows the priorityList configured and then the  behavior is the
> expected.
>

Re: Overhead in running UIMA Asynchronous Scaleout

[Jaroslaw Cwiklik]

Do you deploy both the uima-as client and a service in the same process?
How do you get your timing? There is a CAS serialization/deserialization
penalty and latency in a broker but should not be huge. The design goal is
to support scaleout of analytics across multiple processes on multiple
machines. For maximum performance we recommend sending a CAS with a
reference to data to be analyzed instead of sending data in the CAS.

How big is the CAS you are sending from the client? If you are using a
default FC, there should be no significant delay there.

Jerry

On Mon, Jul 31, 2017 at 5:01 AM, ajay kumar 
wrote:

> Hello,
> I am trying to compare the runtimes for pipelines running locally and
> pipelines using UIMA AS.
> I have a setup of each annotators in a pipeline running on different cores.
> It shows from the results pipelines in UIMA AS takes around 4seconds to
> complete processing which is a huge delay compared to local pipelines. I
> wanted to know how much delay is induced from the flow controller
> forwarding each CAS and the reply queues.
>

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

Your application should not be calling connectionOpen() to test the
connection. The uima-as client has been designed to recover from connection
loss automatically. It looks like this method was made public by mistake.
Your application should use UimaAsynchronousEngine interface instead to
access documented API. Looks like we need to better document this and also
provide a factory to enforce UIMA-AS client contract.

Can you retest your app without calling connectionOpen().

Regarding the timeouts, the client code is designed to create new temp
queue for every uima-as client instance when a connection is recovered.
Need to do more testing to see if there is a bug in the code that deals
with recovery.

Jerry

On Wed, Jul 19, 2017 at 1:52 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Exactly, only one instance of all clients works after the broker is
> restarted. I don't know if this first client recovered and included
> correct temp queue for replie, what I do see is that this first client
> in my source code return "false" to the method connectionOpen()" and
> because of that, i initializes the client again (uimaEEEngine = new
> BaseUIMAAsynchronousEngine_impl() and
> uimaEEEngine.initialize(appCtx);).
>
> The rest of the clients return "true" to the method connectionOpen()
> because of that, i don't initializes again.
>
> The following logs are of one of the client failed, the first time
> that is initialized, processing the first request and  before the
> broker is stopped and restarted
>
> INFORMACIÓN: UIMA Version 2.9.0 UIMA-AS Version 2.9.1
> jul 19, 2017 1:31:32 PM
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl
> initializeProducer
> INFORMACIÓN: Initializing JMS Message Producer. Broker:
> tcp://172.24.10.193:61616 Queue Name: XCharsetDetector
> jul 19, 2017 1:31:32 PM
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl
> initializeConsumer
> INFORMACIÓN: Initializing JMS Message Consumer. Broker:
> tcp://172.24.10.193:61616 Queue Name:
> ID:nelson-XPS-8700-39640-1500485371575-1:1:2
> jul 19, 2017 1:31:32 PM
> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_impl
> initialize
> INFORMACIÓN: Asynchronous Client Has Been Initialized. Serialization
> Strategy: [SerializationStrategy] Ready To Process.
> jul 19, 2017 1:31:32 PM
> org.rp.platform.charset.business.CharsetBusiness
> getProcessingCharsetDetector
> INFORMACIÓN: Clase encargada de la detección de charset
> CharsetDetectorRemoteUIMAProcessing
> jul 19, 2017 1:32:54 PM
> org.rp.platform.charset.uima.CharsetDetectorRemoteUIMAProcessing
> process
> INFORMACIÓN: Inicio de procesamiento en detección de charset.
> jul 19, 2017 1:32:54 PM
> cu.datys.xinetica.uima.listener.StatusCalbackListenerPerformance
> onBeforeProcessCAS
> INFORMACIÓN: null
> jul 19, 2017 1:32:54 PM
> org.rp.platform.charset.uima.CharsetDetectorRemoteUIMAProcessing
> process
> INFORMACIÓN: Fin de procesamiento en detección de charset
>
>
>
>
> And this following logs are of the uima-as service that corresponds to
> the client, after the broker is stopped and processing the second
> request
>
> 01:34:43.215 - 27:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XCharsetDetector Listener Recovered
> Connection to Broker: tcp://127.0.0.1:61616 - Ready to Process Again
> 01:34:43.255 - 25:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XCharsetDetector Listener Recovered
> Connection to Broker: tcp://127.0.0.1:61616 - Ready to Process Again
> 01:38:50.709 - 1191:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send:
> WARNING: Controller: XCharsetDetector Failed To Send Message to
> Delegates  Queue: null
> 01:38:50.710 - 17:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> Entrando al process
> 01:38:50.710 - 17:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> XCharsetDetector::Encontrado el objecto para detectar charset...
> 01:38:50.715 - 1191:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Service:XCharsetDetector JMS unable to Deliver
> CAS:6e78005b:15d5be5c629:-7ff6 -
> Error:temp-queue://ID:nelson-XPS-8700-39640-1500485371575-1:1:2
> 01:38:50.715 - 1191:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Controller: XCharsetDetector Handling Release CAS Request.
> Releasing CAS Reference Id: 6e78005b:15d5be5c629:-7ff6
>
>
> 2017-07-19 9:12 GMT

Re: DUCC Security Model for Service Deployment

[Jaroslaw Cwiklik]

John, you can use the broker shipped with UIMA-AS installation for your
services.

Jerry

On Wed, Jul 19, 2017 at 10:55 AM, Osborne, John David (Campus) <
ozb...@uab.edu> wrote:

> Jerry, my services would span across multiple jobs so a service model is
> preferable for me.
>
> With regards to a secondary broker - I'm thinking to use a UIMA-AS
> installation on the same machine rather than download the full Apache AMQ
> runtime standalone. I'm worried about potential compatibility issues
> between DUCC broker and downloaded broker (unfounded concern?) and I am
> unfamiliar with AMQ in isolation.  It should also allow for comparison
> between AS and DUCC. Is this a reasonable approach?
>
> Thanks for all your help so far,
>
>  -John
>
>
> -Original Message-
> From: uim...@gmail.com [mailto:uim...@gmail.com] On Behalf Of Jaroslaw
> Cwiklik
> Sent: Tuesday, July 18, 2017 3:40 PM
> To: user@uima.apache.org
> Subject: Re: DUCC Security Model for Service Deployment
>
> John, services are typically long running and independent of jobs. They
> provide a specific function and may take a long time to initialize. They
> are meant to be reusable across multiple jobs. If your analytics are fast
> initializing and not meant to be shared you can just use jobs.
>
> We should probably modify the documentation to talk about recommending a
> secondary broker for services. My recommendation is to download and install
> the full AMQ runtime for your services. DUCC only bundles essential parts
> of AMQ distro.
>
> You decide about how secure your DUCC should be. We lock DUCC's broker by
> default to protect it from a malicious user code. If this is not your
> concern you can use DUCC's broker if you want. Keep in mind that if you use
> DUCC's broker for services its load will increase and if this is high
> enough it may effect how DUCC responds. Another benefit of having a
> secondary broker is that you can keep your services up while DUCC is being
> shutdown for update or maintenance. This is especially important if your
> services take hours to initialize.
>
> Jerry
>
> On Tue, Jul 18, 2017 at 4:07 PM, Osborne, John David (Campus) <
> ozb...@uab.edu> wrote:
>
> > Thanks Jerry!
> >
> > I can run it using the insecure option (just tried it - works great!)
> > - the broker port is closed to all but localhost as another
> > application feeds requests to DUCC. From a non-security operational
> > point of few though I am violating your recommendation to use a
> > secondary broker... I hope this won't get me into too much trouble...
> >
> > From the documentation, I had no idea there was a need for a secondary
> > broker to run services. If I wanted to do this, is this something that
> > can be set up in the current ducc distribution or would it be better
> > to go ahead and run my legacy services independently? To do this,  I'm
> > presuming I would instantiate a secondary broker (presumably still
> > 61616) but submit the deployment descriptor to the DUCC broker (61617)
> > but with the inputQueue brokerURL pointing to the secondary (61616)
> broker?
> >
> > I was hoping to make a clean migration to DUCC - maybe it would be
> > easier to get rid of services altogether and just submit jobs? Is this
> > the current trend or are a lot of folks running secondary brokers and
> services?
> >
> >  -John
> >
> >
> >
> > -Original Message-
> > From: uim...@gmail.com [mailto:uim...@gmail.com] On Behalf Of Jaroslaw
> > Cwiklik
> > Sent: Tuesday, July 18, 2017 2:19 PM
> > To: user@uima.apache.org
> > Subject: Re: DUCC Security Model for Service Deployment
> >
> > Hi, as Lou stated its recommended to use a secondary broker for
> > services to keep this separate from a DUCC broker. The DUCC broker is
> > protected by default as it is used for internal communication.
> >
> > If you want to run without broker credentials change this setting in
> > default.ducc.properties
> >
> > ducc.broker.configuration = conf/activemq-ducc-unsecure.xml
> >
> > Jerry
> >
> >
> >
> > On Tue, Jul 18, 2017 at 3:05 PM, Osborne, John David (Campus) <
> > ozb...@uab.edu> wrote:
> >
> > > Thanks Lou,
> > >
> > > I'm still not totally following.
> > >
> > > In UIMA-AS, I could deploy services (as the administrator) using:
> > > deployAsyncService.sh $SOME_DEPLOYMENT_DESCRIPTOR $BROKER_URL
> > >
> > > I thought the equivalent in DUCC was (for example):  ducc_services
> > > --register  --process_descriptor $SOME_DEPLOYMENT_

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

Hi, can you include part of a client logging just before the broker is
stopped and restarted until timeouts start. When a connection is recovered,
the client creates a new temp queue and includes it in an outgoing messages
to services so that replies can come back. Seems like the client outgoing
messages contain a reference to an old temp queue which explain the
timeouts. You say you have multiple clients in your app and only one
instance works after the broker is restarted. The rest of the clients
continuously fail after the broker restart (true?). All UIMA-AS clients
share the same broker connection so the question is why only one of the
clients recovered and included correct temp queue for replies and the
others did not. Can yo confirm that you keep sending messages from your
clients and only one of them succeeds with send and receive.

Jerry

On Tue, Jul 18, 2017 at 4:09 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> i just tested the snapshot build from
> https://svn.apache.org/repos/asf/uima/uima-as/trunk, i continue using
> 8 service uima-as remote, 2 are used only for
> aggregates services, and created 6 uima-as client instances in my
> client application general.
>
> Now the behavior is the follow:
>
> After restart the broker, the first uima-as client that is consumed
> (no matter which),
> has a correct behavior, the instruccion to uima-as client
> "BaseUIMAAsynchronousEngine_impl.connectionOpen()" return "false" and
> correctly initializes the client again and process the request.
>
> All next uima-as client to consume returns "true" to the instruccion
> "BaseUIMAAsynchronousEngine_impl.connectionOpen()", send the request,
> and then the uima-as service process the request but after shows lines
> similar to this in the log
>
> 03:24:52.338 - 1283:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Service:XCharsetDetector JMS unable to Deliver
> CAS:46623412:15d57234659:-7fea -
> Error:temp-queue://ID:nelson-XPS-8700-40232-1500405516005-1:1:5
> 03:24:52.338 - 1283:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Controller: XCharsetDetector Handling Release CAS Request.
> Releasing CAS Reference Id: 46623412:15d57234659:-7fea
>
>
> and then in my mechanism de timeout in client side, after 10 seconds
> without get any notification  on onBeforeProcessCAS() metod of uima-as
> client's UimaAsBaseCallbackListener y throws timeout exception.
>
>
>
>
>
> 2017-07-17 13:37 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Yes. the changes are in the svn so you prefer building from source just
> do
> > svn update to pickup the latest code.
> >
> > Jerry
> >
> > On Mon, Jul 17, 2017 at 1:18 PM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> Sorry for the delay, I took holidays days, i will test the snapshot very
> >> now.
> >>
> >> I can build the snapshot from
> >> https://svn.apache.org/repos/asf/uima/uima-as/trunk?, the change is
> >> there?
> >>
> >> 2017-07-14 9:28 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> > Hi, please download a new snapshot
> >> >
> >> > https://repository.apache.org/content/repositories/
> >> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-
> >> as-2.9.1-20170714.131835-3.tgz
> >> > or
> >> > https://repository.apache.org/content/repositories/
> >> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-
> >> as-2.9.1-20170714.131915-4.zip
> >> >
> >> > You can check signatures of the above here:
> >> > https://repository.apache.org/content/repositories/
> >> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/
> >> >
> >> > uima-as-2.9.1-20170714.131835-3.tgz.md5
> >> > uima-as-2.9.1-20170714.131835-3.tgz.sha1
> >> >
> >> > uima-as-2.9.1-20170714.131915-4.zip.md5
> >> > uima-as-2.9.1-20170714.131915-4.zip.sha1
> >> >
> >> >
> >> >
> >> > On Tue, Jul 11, 2017 at 10:17 AM, nelson rivera <
> >> nelsonriver...@gmail.com>
> >> > wrote:
> >> >
> >> >> Thanks very much, actually i have my application in production but
> >> >> with this bug as reported. i wait
> >> >>
> >> >> 2017-07-11 9:59 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> >> > Hi, I am investigating this issue now. Are you having

Re: DUCC Security Model for Service Deployment

[Jaroslaw Cwiklik]

John, services are typically long running and independent of jobs. They
provide a specific function and may take a long time to initialize. They
are meant to be reusable across multiple jobs. If your analytics are fast
initializing and not meant to be shared you can just use jobs.

We should probably modify the documentation to talk about recommending a
secondary broker for services. My recommendation is to download and install
the full AMQ runtime for your services. DUCC only bundles essential parts
of AMQ distro.

You decide about how secure your DUCC should be. We lock DUCC's broker by
default to protect it from a malicious user code. If this is not your
concern you can use DUCC's broker if you want. Keep in mind that if you use
DUCC's broker for services its load will increase and if this is high
enough it may effect how DUCC responds. Another benefit of having a
secondary broker is that you can keep your services up while DUCC is being
shutdown for update or maintenance. This is especially important if your
services take hours to initialize.

Jerry

On Tue, Jul 18, 2017 at 4:07 PM, Osborne, John David (Campus) <
ozb...@uab.edu> wrote:

> Thanks Jerry!
>
> I can run it using the insecure option (just tried it - works great!) -
> the broker port is closed to all but localhost as another application feeds
> requests to DUCC. From a non-security operational point of few though I am
> violating your recommendation to use a secondary broker... I hope this
> won't get me into too much trouble...
>
> From the documentation, I had no idea there was a need for a secondary
> broker to run services. If I wanted to do this, is this something that can
> be set up in the current ducc distribution or would it be better to go
> ahead and run my legacy services independently? To do this,  I'm presuming
> I would instantiate a secondary broker (presumably still 61616) but submit
> the deployment descriptor to the DUCC broker (61617) but with the
> inputQueue brokerURL pointing to the secondary (61616) broker?
>
> I was hoping to make a clean migration to DUCC - maybe it would be easier
> to get rid of services altogether and just submit jobs? Is this the current
> trend or are a lot of folks running secondary brokers and services?
>
>  -John
>
>
>
> -Original Message-
> From: uim...@gmail.com [mailto:uim...@gmail.com] On Behalf Of Jaroslaw
> Cwiklik
> Sent: Tuesday, July 18, 2017 2:19 PM
> To: user@uima.apache.org
> Subject: Re: DUCC Security Model for Service Deployment
>
> Hi, as Lou stated its recommended to use a secondary broker for services
> to keep this separate from a DUCC broker. The DUCC broker is protected by
> default as it is used for internal communication.
>
> If you want to run without broker credentials change this setting in
> default.ducc.properties
>
> ducc.broker.configuration = conf/activemq-ducc-unsecure.xml
>
> Jerry
>
>
>
> On Tue, Jul 18, 2017 at 3:05 PM, Osborne, John David (Campus) <
> ozb...@uab.edu> wrote:
>
> > Thanks Lou,
> >
> > I'm still not totally following.
> >
> > In UIMA-AS, I could deploy services (as the administrator) using:
> > deployAsyncService.sh $SOME_DEPLOYMENT_DESCRIPTOR $BROKER_URL
> >
> > I thought the equivalent in DUCC was (for example):  ducc_services
> > --register  --process_descriptor $SOME_DEPLOYMENT_DESCRIPTOR
> > --classpath $CLASSPATH --autostart true
> >
> > In the first case, I don't recall worrying about passwords - I presume
> > deployAsyncService was privileged and knew any broker credentials. Are
> > you saying the command line utility ducc_services is unaware of
> > ducc-broker-credentionals.properties and there is no other way for the
> > ducc user to pass these credentionals to the broker when
> > registering/auto-start services?
> >
> > Sorry for the confusion, new to DUCC. I can see the
> > $DUCC_HOME/resources.private/ducc-broker-credentials.properties - I
> > just don't know how to use them when deploying (same thing as
> > registering and
> > starting?) a service.
> >
> >  -John
> >
> >
> >
> > -Original Message-
> > From: Lou DeGenaro [mailto:lou.degen...@gmail.com]
> > Sent: Tuesday, July 18, 2017 1:34 PM
> > To: user@uima.apache.org
> > Subject: Re: DUCC Security Model for Service Deployment
> >
> > The DUCC broker is for DUCC.  User services should employ a separate
> > user broker, not the DUCC broker.
> >
> > That said, with proper permissions you can discover the DUCC broker
> > user and pw in
> > $DUCC_HOME/resources.private/ducc-broker-credentials.properties
> >
> > Lou.
> >
> > On 

Re: DUCC Security Model for Service Deployment

[Jaroslaw Cwiklik]

Hi, as Lou stated its recommended to use a secondary broker for services to
keep this separate from a DUCC broker. The DUCC broker is protected by
default as it is used for internal communication.

If you want to run without broker credentials change this setting in
default.ducc.properties

ducc.broker.configuration = conf/activemq-ducc-unsecure.xml

Jerry



On Tue, Jul 18, 2017 at 3:05 PM, Osborne, John David (Campus) <
ozb...@uab.edu> wrote:

> Thanks Lou,
>
> I'm still not totally following.
>
> In UIMA-AS, I could deploy services (as the administrator) using:
> deployAsyncService.sh $SOME_DEPLOYMENT_DESCRIPTOR $BROKER_URL
>
> I thought the equivalent in DUCC was (for example):  ducc_services
> --register
>  --process_descriptor $SOME_DEPLOYMENT_DESCRIPTOR  --classpath $CLASSPATH
> --autostart true
>
> In the first case, I don't recall worrying about passwords - I presume
> deployAsyncService was privileged and knew any broker credentials. Are you
> saying the command line utility ducc_services is unaware of
> ducc-broker-credentionals.properties and there is no other way for the
> ducc user to pass these credentionals to the broker when
> registering/auto-start services?
>
> Sorry for the confusion, new to DUCC. I can see the
> $DUCC_HOME/resources.private/ducc-broker-credentials.properties - I just
> don't know how to use them when deploying (same thing as registering and
> starting?) a service.
>
>  -John
>
>
>
> -Original Message-
> From: Lou DeGenaro [mailto:lou.degen...@gmail.com]
> Sent: Tuesday, July 18, 2017 1:34 PM
> To: user@uima.apache.org
> Subject: Re: DUCC Security Model for Service Deployment
>
> The DUCC broker is for DUCC.  User services should employ a separate user
> broker, not the DUCC broker.
>
> That said, with proper permissions you can discover the DUCC broker user
> and pw in $DUCC_HOME/resources.private/ducc-broker-credentials.properties
>
> Lou.
>
> On Tue, Jul 18, 2017 at 2:14 PM, Osborne, John David (Campus) <
> ozb...@uab.edu> wrote:
>
> > Yes - the idea (based on a previous UIMA-AS workflow) was to attached
> > services to the broker and have other application/s call them as needed.
> >
> >
> > I thought this was still done with DUCC?
> >
> >
> > I see what I *think* is the active security configuration in
> > activemq.xml
> > below:
> >
> >
> > 
> > 
> >  > password="${ducc.broker.admin.password}"
> > groups="ducc-admin"/>
> > 
> > 
> >
> >
> > I'm registering services and running ducc as user ducc.
> >
> >
> >  -John
> >
> >
> > 
> > From: Lou DeGenaro 
> > Sent: Tuesday, July 18, 2017 12:19:21 PM
> > To: user@uima.apache.org
> > Subject: Re: DUCC Security Model for Service Deployment
> >
> > DUCC has its own password protected AMQ broker used for daemon
> > communications.  Are your services trying to use DUCC's broker?
> >
> > Lou.
> >
> > On Tue, Jul 18, 2017 at 1:02 PM, Osborne, John David (Campus) <
> > ozb...@uab.edu> wrote:
> >
> > > I am having deploying services with DUCC using a single system,
> > > single user setup. I can run test jobs, so I *think* my system is
> > > set up
> > correctly
> > > but I can't register and then start services due to
> > > "SecurityException on Connect".
> > >
> > >
> > > I don't understand how when registering services the credentionals
> > > are passed to DUCC, I know it uses activemq credentionals, but there
> > > is no parameter to pass them on the command line.
> > >
> > >
> > > I am registering the service like this:
> > >
> > > /web/servers/apache-uima-ducc-2.2.0/bin/ducc_services --register
> > > --process_descriptor_DD  ../resources/desc/deployment/
> > > ImportDocumentsDeploymentDescriptorSimple.xml --description "Pull
> > > Documents" --classpath $CLASSPATH --autostart true
> > >
> > >
> > > I have not messed with any of the configuration files
> > > (credentional.properties, etc...) under apache-activemq.
> > >
> > >
> > > My error (actually a warning) is below:
> > >
> > >
> > > Any help appreciated,
> > >
> > >
> > >  -John
> > >
> > >
> > > INFO: Controller: MedicsClobsConsumer Trying to Start Listener on
> > > Endpoint: queue://MRN_Document_Pull_Queue Selector: Command=2001
> Broker:
> > > tcp://localhost:61617
> > > >>> Service Container Deployed Successfully
> > > DuccAbstractProcessContainer.deploy()  User Container
> > > deployed  Deployed Processing Container - Initialization
> > > Successful - Thread 1
> > > 18 Jul 2017 11:40:22,858  INFO AgentSession - T[1]
> > > notifyAgentWithStatus ... Job Process State Changed - PID:5803.
> > > Process State: Running. JMX
> > > Url:service:jmx:rmi:///jndi/rmi://uimaprapp1.hs.uab.edu:2105/jmxrmi
> > > Dispatched State Update Event to Agent with IP:10.23.142.165 Jul 18,
> > > 2017 11:40:23 AM org.apache.uima.adapter.jms.activemq.
> > > UimaDefaultMessageListenerContainer onException
> > > WARNING: Service: MedicsClobsConsumer 

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

Yes. the changes are in the svn so you prefer building from source just do
svn update to pickup the latest code.

Jerry

On Mon, Jul 17, 2017 at 1:18 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Sorry for the delay, I took holidays days, i will test the snapshot very
> now.
>
> I can build the snapshot from
> https://svn.apache.org/repos/asf/uima/uima-as/trunk?, the change is
> there?
>
> 2017-07-14 9:28 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> > Hi, please download a new snapshot
> >
> > https://repository.apache.org/content/repositories/
> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-
> as-2.9.1-20170714.131835-3.tgz
> > or
> > https://repository.apache.org/content/repositories/
> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/uima-
> as-2.9.1-20170714.131915-4.zip
> >
> > You can check signatures of the above here:
> > https://repository.apache.org/content/repositories/
> snapshots/org/apache/uima/uima-as/2.9.1-SNAPSHOT/
> >
> > uima-as-2.9.1-20170714.131835-3.tgz.md5
> > uima-as-2.9.1-20170714.131835-3.tgz.sha1
> >
> > uima-as-2.9.1-20170714.131915-4.zip.md5
> > uima-as-2.9.1-20170714.131915-4.zip.sha1
> >
> >
> >
> > On Tue, Jul 11, 2017 at 10:17 AM, nelson rivera <
> nelsonriver...@gmail.com>
> > wrote:
> >
> >> Thanks very much, actually i have my application in production but
> >> with this bug as reported. i wait
> >>
> >> 2017-07-11 9:59 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
> >> > Hi, I am investigating this issue now. Are you having this problem in
> >> > production or testing? The multi-client recovery is quite complicated
> >> > and
> >> > subject to race conditions. I need to first re-create the problem and
> >> work
> >> > on a solution if this in fact is a bug. From what you sent it looks
> >> > like
> >> > there is a bug in client recovery. This may take some time to sort
> out.
> >> > Will let you know what I find.
> >> >
> >> > Jerry
> >> >
> >> > On Tue, Jul 11, 2017 at 9:02 AM, nelson rivera
> >> > <nelsonriver...@gmail.com
> >> >
> >> > wrote:
> >> >
> >> >> analyzing more deep the issue, I have observed that after restart the
> >> >> broker, the first uima-as client that is consumed (no matter which),
> >> >> has a correct behavior, the instruccion to uima-as client
> >> >> "BaseUIMAAsynchronousEngine_impl.connectionOpen()" return "false"
> and
> >> >> correctly initializes the client again and process the request.
> >> >>
> >> >> All next uima-as client to consume returns "true" to the instruccion
> >> >> "BaseUIMAAsynchronousEngine_impl.connectionOpen()" and them get the
> >> >> error:
> >> >>
> >> >> jul 11, 2017 8:11:52 AM
> >> >> org.apache.uima.adapter.jms.client.ActiveMQMessageSender
> createSession
> >> >> INFORMACIÓN: UIMA AS Client Failed Creating Session To Destination:
> >> >> XCharsetDetector Managed By Broker: tcp://nelson-XPS-8700:61616
> >> >> jul 11, 2017 8:11:52 AM
> >> >> org.apache.uima.adapter.jms.client.ActiveMQMessageSender
> createSession
> >> >> INFORMACIÓN: UIMA AS Client Connection To Destination:
> >> >> XCharsetDetector Managed By Broker: tcp://nelson-XPS-8700:61616 Is
> >> >> Closed
> >> >> jul 11, 2017 8:11:52 AM
> >> >> org.apache.uima.adapter.jms.client.ActiveMQMessageSender run
> >> >> ADVERTENCIA:
> >> >> org.apache.activemq.ConnectionClosedException: The connection is
> >> already
> >> >> closed
> >> >> at org.apache.activemq.ActiveMQConnection.checkClosed(
> >> >> ActiveMQConnection.java:1459)
> >> >> at org.apache.activemq.ActiveMQConnection.
> checkClosedOrFailed(
> >> >> ActiveMQConnection.java:1446)
> >> >> at org.apache.activemq.ActiveMQConnection.createSession(
> >> >> ActiveMQConnection.java:328)
> >> >> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.
> >> >> createSession(ActiveMQMessageSender.java:108)
> >> >> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.
> >> >> initializeProducer(ActiveMQMessageSender.java:143)
> >> >>  

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

Hi, I am investigating this issue now. Are you having this problem in
production or testing? The multi-client recovery is quite complicated and
subject to race conditions. I need to first re-create the problem and work
on a solution if this in fact is a bug. From what you sent it looks like
there is a bug in client recovery. This may take some time to sort out.
Will let you know what I find.

Jerry

On Tue, Jul 11, 2017 at 9:02 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> analyzing more deep the issue, I have observed that after restart the
> broker, the first uima-as client that is consumed (no matter which),
> has a correct behavior, the instruccion to uima-as client
> "BaseUIMAAsynchronousEngine_impl.connectionOpen()" return "false" and
> correctly initializes the client again and process the request.
>
> All next uima-as client to consume returns "true" to the instruccion
> "BaseUIMAAsynchronousEngine_impl.connectionOpen()" and them get the error:
>
> jul 11, 2017 8:11:52 AM
> org.apache.uima.adapter.jms.client.ActiveMQMessageSender createSession
> INFORMACIÓN: UIMA AS Client Failed Creating Session To Destination:
> XCharsetDetector Managed By Broker: tcp://nelson-XPS-8700:61616
> jul 11, 2017 8:11:52 AM
> org.apache.uima.adapter.jms.client.ActiveMQMessageSender createSession
> INFORMACIÓN: UIMA AS Client Connection To Destination:
> XCharsetDetector Managed By Broker: tcp://nelson-XPS-8700:61616 Is
> Closed
> jul 11, 2017 8:11:52 AM
> org.apache.uima.adapter.jms.client.ActiveMQMessageSender run
> ADVERTENCIA:
> org.apache.activemq.ConnectionClosedException: The connection is already
> closed
> at org.apache.activemq.ActiveMQConnection.checkClosed(
> ActiveMQConnection.java:1459)
> at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(
> ActiveMQConnection.java:1446)
> at org.apache.activemq.ActiveMQConnection.createSession(
> ActiveMQConnection.java:328)
> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.
> createSession(ActiveMQMessageSender.java:108)
> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.
> initializeProducer(ActiveMQMessageSender.java:143)
> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.
> createTextMessage(ActiveMQMessageSender.java:191)
> at org.apache.uima.adapter.jms.client.BaseMessageSender.run(
> BaseMessageSender.java:330)
> at java.lang.Thread.run(Thread.java:748)
>
> 2017-07-11 8:50 GMT-04:00, nelson rivera <nelsonriver...@gmail.com>:
> > My client application not receive any exception in any time. I use
> > both, in some uima-as client  use sendAndReceive() and in others use
> > send(), each uima-as client has try-catch and none of them receive a
> > exception, the following lines are printed in the console output,but
> > the uima-as client is not notified and keeps waiting.
> >
> > 2017-07-10 16:48 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >> it looks like the UIMA-AS client is setup to throw an exception back to
> >> the
> >> application in this case. Did your application receive an exception? Are
> >> you using sendAndReceive() or send()? The UIMA-AS client rejects
> requests
> >> while a connection is down.
> >>
> >> Jerry
> >>
> >> On Mon, Jul 10, 2017 at 4:19 PM, Jaroslaw Cwiklik <cwik...@apache.org>
> >> wrote:
> >>
> >>> Hmm, this is a different exception from Session closed. This time it is
> >>> a
> >>> Connection is closed. Seems like a race condition. How many UIMA-AS
> >>> clients
> >>> do you have running in this process?
> >>>
> >>> Jerry
> >>>
> >>> On Mon, Jul 10, 2017 at 3:51 PM, nelson rivera
> >>> <nelsonriver...@gmail.com>
> >>> wrote:
> >>>
> >>>> i just tested the snapshot build from
> >>>> https://svn.apache.org/repos/asf/uima/uima-as/trunk, and the behavior
> >>>> still are similar for many uima-as client, end in a exception, but now
> >>>> show this stacktrace:
> >>>>
> >>>> org.apache.activemq.ConnectionClosedException: The connection is
> >>>> already
> >>>> closed
> >>>> at org.apache.activemq.ActiveMQConnection.
> checkClosed(ActiveMQC
> >>>> onnection.java:1459)
> >>>> at org.apache.activemq.ActiveMQConnection.
> checkClosedOrFailed(A
> >>>> ctiveMQConnection.java:1446)
> >>>> at org.apache.activemq.ActiveMQConnection.
>

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

it looks like the UIMA-AS client is setup to throw an exception back to the
application in this case. Did your application receive an exception? Are
you using sendAndReceive() or send()? The UIMA-AS client rejects requests
while a connection is down.

Jerry

On Mon, Jul 10, 2017 at 4:19 PM, Jaroslaw Cwiklik <cwik...@apache.org>
wrote:

> Hmm, this is a different exception from Session closed. This time it is a
> Connection is closed. Seems like a race condition. How many UIMA-AS clients
> do you have running in this process?
>
> Jerry
>
> On Mon, Jul 10, 2017 at 3:51 PM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> i just tested the snapshot build from
>> https://svn.apache.org/repos/asf/uima/uima-as/trunk, and the behavior
>> still are similar for many uima-as client, end in a exception, but now
>> show this stacktrace:
>>
>> org.apache.activemq.ConnectionClosedException: The connection is already
>> closed
>> at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQC
>> onnection.java:1459)
>> at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(A
>> ctiveMQConnection.java:1446)
>> at org.apache.activemq.ActiveMQConnection.createSession(ActiveM
>> QConnection.java:328)
>> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.cre
>> ateSession(ActiveMQMessageSender.java:108)
>> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.ini
>> tializeProducer(ActiveMQMessageSender.java:143)
>> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.cre
>> ateTextMessage(ActiveMQMessageSender.java:191)
>> at org.apache.uima.adapter.jms.client.BaseMessageSender.run(Bas
>> eMessageSender.java:330)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> 2017-07-07 15:33 GMT-04:00, nelson rivera <nelsonriver...@gmail.com>:
>> > Sorry for the delay, i am building the framework from the source at
>> > https://svn.apache.org/repos/asf/uima/uima-as/trunk, when test the
>> > issue, i will comment the result.
>> >
>> > 2017-07-06 10:04 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
>> >> Hello, the snapshot binaries (zip + tar) are here:
>> >>
>> >> https://repository.apache.org/content/repositories/snapshots
>> /org/apache/uima/uima-as/2.9.1-SNAPSHOT/
>> >>
>> >> Jerry
>> >>
>> >> On Thu, Jul 6, 2017 at 8:51 AM, Jaroslaw Cwiklik <cwik...@apache.org>
>> >> wrote:
>> >>
>> >>> Hi, instructions how to build from source are here:
>> >>> https://uima.apache.org/building-uima.html
>> >>>
>> >>> It's a bit involved process so I will try to post a snapshot of
>> uima-as
>> >>> to
>> >>> Apache snapshot repository and will provide you with a link later
>> today.
>> >>>
>> >>> Jerry
>> >>>
>> >>> On Wed, Jul 5, 2017 at 4:32 PM, nelson rivera <
>> nelsonriver...@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Please can you give me the url to source code with the changes, to
>> >>>> test.
>> >>>>
>> >>>> 2017-07-05 16:19 GMT-04:00, Jaroslaw Cwiklik <cwik...@apache.org>:
>> >>>> > Just delivered a few changes to deal with a stale Session. Can you
>> >>>> > build
>> >>>> > from source and test.
>> >>>> > Jerry
>> >>>> >
>> >>>> > On Wed, Jul 5, 2017 at 2:21 PM, Jaroslaw Cwiklik <
>> cwik...@apache.org>
>> >>>> > wrote:
>> >>>> >
>> >>>> >> Thanks. Will check the code and work on a fix
>> >>>> >>
>> >>>> >> Jerry
>> >>>> >>
>> >>>> >> On Wed, Jul 5, 2017 at 2:10 PM, nelson rivera <
>> >>>> nelsonriver...@gmail.com>
>> >>>> >> wrote:
>> >>>> >>
>> >>>> >>> Here the stack trace for the error that asked me:
>> >>>> >>> jul 05, 2017 2:08:50 PM
>> >>>> >>> org.apache.uima.adapter.jms.client.ActiveMQMessageSender run
>> >>>> >>> ADVERTENCIA:
>> >>>> >>> javax.jms.IllegalStateException: The Session is closed
>> >>>> >>> at org.apache.activemq.ActiveMQSe
>> >&g

Re: How know when "Session is closed" after establish connections between uima-as client and ActiveMQ?

[Jaroslaw Cwiklik]

Just delivered a few changes to deal with a stale Session. Can you build
from source and test.
Jerry

On Wed, Jul 5, 2017 at 2:21 PM, Jaroslaw Cwiklik <cwik...@apache.org> wrote:

> Thanks. Will check the code and work on a fix
>
> Jerry
>
> On Wed, Jul 5, 2017 at 2:10 PM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> Here the stack trace for the error that asked me:
>> jul 05, 2017 2:08:50 PM
>> org.apache.uima.adapter.jms.client.ActiveMQMessageSender run
>> ADVERTENCIA:
>> javax.jms.IllegalStateException: The Session is closed
>> at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSess
>> ion.java:771)
>> at org.apache.activemq.ActiveMQSession.configureMessage(ActiveM
>> QSession.java:759)
>> at org.apache.activemq.ActiveMQSession.createTextMessage(Active
>> MQSession.java:440)
>> at org.apache.uima.adapter.jms.client.ActiveMQMessageSender.cre
>> ateTextMessage(ActiveMQMessageSender.java:184)
>> at org.apache.uima.adapter.jms.client.BaseMessageSender.run(Bas
>> eMessageSender.java:307)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> 2017-07-05 12:16 GMT-04:00, nelson rivera <nelsonriver...@gmail.com>:
>> > This behavior is random, not occurs always. In the most of the simple
>> > programs and simple test case tested, after restart the UIMA-AS Broker
>> > without restart service uima-as, the instruccion to uima-as client
>> > "BaseUIMAAsynchronousEngine_impl.connectionOpen()" return "false" and
>> > correctly i initialize the client again.
>> >
>> > The incorrect behavior is most frequently with many uima-as client
>> > instances. Is the case of my general application in which i have many
>> > service uima-as, and for each one creates one instace of uima-as
>> > client (BaseUIMAAsynchronousEngine_impl) and keep it alive to process
>> > all requests of the service uima-as.
>> >
>> > Is in this scenario with many instancies of uima-as client
>> > (BaseUIMAAsynchronousEngine_impl) in the same program (a monolithic
>> > application web in tomcat) where after restart the UIMA-AS Broker
>> > without restart any service uima-as, many uima-as client returns
>> > "true" to the instruccion
>> > "BaseUIMAAsynchronousEngine_impl.connectionOpen()" but them throw
>> > "javax.jms.IllegalStateException: The Session is closed"
>> > when proces the request.
>> >
>> > 2017-07-04 16:13 GMT-04:00, nelson rivera <nelsonriver...@gmail.com>:
>> >> One peculiarity is that, is a primitive annotator implented in c++,
>> >> but is deployed with de deployAsyncService.sh de uima-as directly.
>> >>
>> >> 2017-07-04 14:53 GMT-04:00, nelson rivera <nelsonriver...@gmail.com>:
>> >>> In my program main, i initialize a uimas-as client only once and I use
>> >>> this client to consume all requests to a displayed service uima-as.
>> >>>
>> >>> Before process a request i ask to uima-as client with:
>> >>> BaseUIMAAsynchronousEngine_impl.connectionOpen()
>> >>> and if the result es "false", i stop the uima-as client and initialize
>> >>> it
>> >>> again.
>> >>>
>> >>> With some steps, still undefined, but after restart the UIMA-AS Broker
>> >>> without restart service uima-as.
>> >>> The instruccion to uima-as client
>> >>> BaseUIMAAsynchronousEngine_impl.connectionOpen() return "true", but
>> >>> them i get a "javax.jms.IllegalStateException: The Session is closed"
>> >>> when proces the request.
>> >>>
>> >>> Wich is the best way to know when connections betewen cliente uima-as
>> >>> and ActiveMQ is broken? and i should restart uima-as client.
>> >>>
>> >>
>> >
>>
>
>

Re: Broken conections after ACTIVEMQ restart with service uima-as CasMultiplier and delegate remote

[Jaroslaw Cwiklik]

Hi, the above problem is a different issue from the Session Closed reported
earlier which occurs in the uima-as client. Please provide a stack trace
for that.

The issue you are reporting above is an expected behavior. The remote is
trying to deliver msgs to a temp reply queue which no longer exists due to
a broker restart. JMS temp queues are transient and not recoverable. If a
broker is restarted while uima-as service is processing a CAS, that service
will not be able to reply. Any new CASes (sent after broker restart) will
have a valid temp queue and replies will succeed.

Jerry

On Wed, Jul 5, 2017 at 9:55 AM, nelson rivera 
wrote:

> i have a aggregate service uima-as that it contains a CasMultiplier,
> after this casmultiplier in the flow one of the delegates is remote.
> The delegate remote is a primitive annotator implented in c++,but is
> deployed with de deployAsyncService.sh de uima-as directly.
>
> With these step i get the error that shows
>
> 1- Start all components (broker , agregate service, remote service).
> 2- First request, process ok
> 3- Restart broker (without restart services uima-as). //Logs shows
> Ready to Process Again
> 4- Second request, process ok
> 5- Restart broker, second (without restart services uima-as). //Logs
> show Ready to Process Again
> 6- I get a error, shows following lines in the log of the delegate
> sevice uima-as remote.
> Attachs images of the queues created at all time in case it is of
> help. The remote service delegate is XLangdetector and service
> agregate casmultiplier is XClusterAnalyzerAggregate.
>
> Errors Lines:
> Recovered Connection to Broker: tcp://127.0.0.1:61616 - Ready to Process
> Again
> 09:27:19.705 - 18:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> Begin -> XLangDetector::process --- OK
> 09:27:19.707 - 18:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: : End
> -> XLangDetector::process --- OK
> 09:27:19.710 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send:
> WARNING: Controller: XLangDetector Failed To Send Message to Delegates
>  Queue: null
> 09:27:19.721 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Service:XLangDetector JMS unable to Deliver
> CAS:-4f7adbed:15d12e82d61:-7fc7 -
> Error:temp-queue://ID:nelson-XPS-8700-46518-1499260793324-13:1:1
> 09:27:19.721 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Controller: XLangDetector Handling Release CAS Request.
> Releasing CAS Reference Id: -4f7adbed:15d12e82d61:-7fc7
> 09:27:19.722 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send:
> WARNING: Controller: XLangDetector Failed To Send Message to Delegates
>  Queue: null
> 09:27:19.722 - 19:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> Begin -> XLangDetector::process --- OK
> 09:27:19.723 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send:
> WARNING: Controller: XLangDetector Failed To Send Message to Delegates
>  Queue: null
> 09:27:19.724 - 17:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> Begin -> XLangDetector::process --- OK
> 09:27:19.725 - 17:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: : End
> -> XLangDetector::process --- OK
> 09:27:19.727 - 19:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: : End
> -> XLangDetector::process --- OK
> 09:27:19.732 - 20:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> Begin -> XLangDetector::process --- OK
> 09:27:19.734 - 20:
> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: : End
> -> XLangDetector::process --- OK
> 09:27:19.737 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.send:
> WARNING: Controller: XLangDetector Failed To Send Message to Delegates
>  Queue: null
> 09:27:19.738 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Service:XLangDetector JMS unable to Deliver
> CAS:-4f7adbed:15d12e82d61:-7fc8 -
> Error:temp-queue://ID:nelson-XPS-8700-46518-1499260793324-13:1:1
> 09:27:19.738 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Controller: XLangDetector Handling Release CAS Request.
> Releasing CAS Reference Id: -4f7adbed:15d12e82d61:-7fc8
> 09:27:19.738 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():
> INFO: Service:XLangDetector JMS unable to Deliver
> CAS:-4f7adbed:15d12e82d61:-7fc5 -
> Error:temp-queue://ID:nelson-XPS-8700-46518-1499260793324-13:1:1
> 09:27:19.739 - 3645:
> org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> UimaAsAsyncCallbackListener.onException():

Re: Synchonizing Batches AE and StatusCallbackListener

[Jaroslaw Cwiklik]

Because I am not the only one using
> the cluster.
> And then I have to stop all AE “services”. Until then, they will use
> memory because they just idle when there is nothing more to do.
>
> In contrast, CPEs are self-contained projects in my case which I can
> distribute easily through our job system (SLURM).
>
> I thought all the setup for UIMA-AS would pay out in better performance.
> But in my - admittedly limited - tests there was not much of a performance
> difference. CPEs seemed to be a bit faster due to the lack of CAS
> serialization between reader and AEs.
>
> Of course, this was years in the past. Is the process a bit simpler today?
> Or perhaps I got it wrong to begin with, that’s possible. But I read the
> documentation back then and couldn’t see how to do things much simpler.
>
> BUT if CPEs can’t solve my issue and UIMA-AS can, then perhaps I will try
> it again.
>
> Another question: You said “CPE was replaced by UIMA-AS”. Does that mean
> that CPEs will eventually be removed from UIMA? Are they still a part of
> UIMA 3?
>
> Sorry for all the text!
>
> Best regards and thanks!
>
> Erik
>
> > On 20 Apr 2017, at 20:31, Jaroslaw Cwiklik <uim...@gmail.com> wrote:
> >
> > Hi Erik, sorry for a delay responding to your question. This seems like a
> > CPE question is this right? I am not quite following what is the issue
> you
> > are running into. Could you explain this better? With a clearer problem
> > description perhaps others will jump in with an answer  :)
> >
> > Just FYI, the CPE was replaced by the UIMA-AS quite a long time ago.
> > Perhaps UIMA-AS can work better for you. You can read about it here:
> > https://uima.apache.org/d/uima-as-2.9.0/uima_async_scaleout.html
> >
> > Jerry
> > UIMA Team
> >
> > On Tue, Apr 18, 2017 at 5:56 AM, Erik Fäßler <erik.faess...@uni-jena.de>
> > wrote:
> >
> >> Hi all,
> >>
> >> I have a use case where a consumer of mine sends CAS XMI data into a
> >> database in batchProcessComplete(). I also use a StatusCallbackListener
> >> that logs into the database whether a document has been completed
> >> processing, this is also done batch wise.
> >> Now the issue is, if the pipeline crashes for any reason, I must start
> >> over because the “completion” flag from the CallbackListener and the
> data
> >> actually sent by the XMI consumer is not synchronised, i.e. I don’t
> know if
> >> the data has actually been sent for a document that has completed
> >> processing because everything is done batch-wise and not immediately for
> >> performance reasons. I also cannot just look into the database which XMI
> >> data is there because it only gets sent on a met condition.
> >>
> >> I would like to somehow communicate between the consumer and the
> >> CallbackListener to send their data for the same documents in
> agreement. Is
> >> there anything I can do to achieve this?
> >>
> >> Best,
> >>
> >> Erik
>
>

Re: Synchonizing Batches AE and StatusCallbackListener

[Jaroslaw Cwiklik]

Hi Erik, sorry for a delay responding to your question. This seems like a
CPE question is this right? I am not quite following what is the issue you
are running into. Could you explain this better? With a clearer problem
description perhaps others will jump in with an answer  :)

Just FYI, the CPE was replaced by the UIMA-AS quite a long time ago.
Perhaps UIMA-AS can work better for you. You can read about it here:
https://uima.apache.org/d/uima-as-2.9.0/uima_async_scaleout.html

Jerry
UIMA Team

On Tue, Apr 18, 2017 at 5:56 AM, Erik Fäßler 
wrote:

> Hi all,
>
> I have a use case where a consumer of mine sends CAS XMI data into a
> database in batchProcessComplete(). I also use a StatusCallbackListener
> that logs into the database whether a document has been completed
> processing, this is also done batch wise.
> Now the issue is, if the pipeline crashes for any reason, I must start
> over because the “completion” flag from the CallbackListener and the data
> actually sent by the XMI consumer is not synchronised, i.e. I don’t know if
> the data has actually been sent for a document that has completed
> processing because everything is done batch-wise and not immediately for
> performance reasons. I also cannot just look into the database which XMI
> data is there because it only gets sent on a met condition.
>
> I would like to somehow communicate between the consumer and the
> CallbackListener to send their data for the same documents in agreement. Is
> there anything I can do to achieve this?
>
> Best,
>
> Erik

Re: Reordering Analysis Engines

[Jaroslaw Cwiklik]

Take a look at UIMA-AS:

"UIMA Asynchronous Scaleout (AS) is a set of capabilities supported in the
UIMA Framework for achieving scaleout that is more general than the
approaches provided for in the Collection Processing Manager (CPM). AS is a
second generation design, replacing the CPM and Vinci Services. The CPM and
Vinci are still available and are not being deprecated, but new designs are
encouraged to use AS for scalability, and current designs reaching
limitations may want to move to AS.

AS is integrated with the flow controller architecture, and can be applied
to both primitive and aggregate analysis engines."

https://uima.apache.org/d/uima-as-2.9.0/uima_async_scaleout.html

-jerry



On Mon, Feb 27, 2017 at 6:22 AM, James Baker 
wrote:

> Is it possible to reorder the analysis engines in a CPE once it is created?
>
> I have a CPE consisting of a large number of analysis engines, and I'd like
> to automatically optimise the order. However, some of the parameters needed
> to perform the optimisation are only calculated once the analysis engines
> are initialised. So I will need to initialise the CPE and then reorder the
> analysis engines. Is this possible?
>
> Thanks,
> James
>

[ANNOUNCE] Apache UIMA DUCC 2.2.0 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the Apache
UIMA-DUCC version 2.2.0.

The Unstructured Information Management Architecture (UIMA) is a component
framework supporting development, discovery, composition, and deployment of
multi-modal analytics tasked with the analysis of unstructured information.

Apache UIMA is an Apache licensed open source implementation of the UIMA
specification which is being developed by a technical committee within
OASIS, a standards organization. The implementation comprises an SDK and
tooling for composing and running analytic components written in Java and
C++, with some support for Perl, Python and TCL.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling,management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources.
DUCC defines a formal job model that closely maps to a standard UIMA
pipeline. Around this job model DUCC provides cluster management services
to automate the scale-out of UIMA pipelines over computing clusters.


This release contains a number of improvements and bug fixes. Notable
updates in this release include:

 - Ships with the latest UIMA-AS v2.9.0 and UIMA SDK 2.9.0
 - Ships with ActiveMQ v5.14.0
 - Added support for static failover and capability to move the head node
 - Fixed DUCC OR "warm" restart issues
 - Fixed DUCC startup script to fail if the DB doesn't start
 - Fixed DUCC shutdown sequence bug which prevented agents from stopping if
Broker was shutdown first
 - Fixed Rogue process detector to detect and cleanup orphan services
 - Deprecated ducc.agent.node.metrics.sys.gid.max and replaced with
ducc.agent.rogue.process.sys.uid.max
 - Enhanced DUCC Job Driver (JD) to provide individual work item
performance breakdowns
 - Modified DUCC to restrict broker use to ducc user only
 - On process launch failure, agent supplies a reason for failure for
display in ducc mon
 - Added duplicate daemon detector to prevent starting duplicate DUCC daemon
 - Many DUCC Database improvements
 - Many DUCC webpage improvements

For a complete list of bugs and improvements included in this release
please see
https://uima.apache.org/d/uima-ducc-2.2.0/issuesFixed/jira-report.html

-- Jerry Cwiklik, for the Apache UIMA development team

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Nelson, I agree with Marshall. If you can produce a small test case that
shows the problem I will find time to debug it.

-jerry

On Fri, Feb 17, 2017 at 2:14 PM, Marshall Schor <m...@schor.com> wrote:

> Hi Nelson,
>
> Due to local holidays, you may not see activity here until mid-next week.
>
> Meanwhile, it would help to have a small test case which reproduces this
> issue.
>
> If you could make one and zip it up and attach it to a Jira, that would
> enable
> great progress :-)
>
> -Marshall
>
>
> On 2/16/2017 8:52 AM, nelson rivera wrote:
> > I tested the solution that proposes, add processParentLast="true" to
> > the  Cas Multiplier delegate's configuration. The behavior is the
> > same. The other alternative that you say, is that i have a bug
> > somewhere in my code which allows a CAS to be accessed in two separate
> > threads, but i not have idea,because the cas generated in the second
> > cas multipler is composed of all views and then this cas go to the end
> > annotator that have only 1 instance, and here finish the flow.
> >
> > 2017-02-15 16:31 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >> Nelson, change Cas Multiplier in your deployment descriptor as follows:
> >>
> >> 
> >>  >> processParentLast="true"/>
> >> 
> >>
> >> Note: processParentLast="true".
> >>
> >> In UIMA-AS async aggregate its possible for a child CAS and its parent
> CAS
> >> to flow through the pipeline at the same time and the parent CAS may
> reach
> >> the end before its child(ren). The above setting will ensure the parent
> CAS
> >> does not flow ahead of its children. From UIMA-AS documentation:
> >>
> >> "The processParentLast attribute on the  element is
> >> optional, and specifies processing order of an input CAS relative to its
> >> children. If true, a flow of an input CAS will be suspended after it is
> >> returned from a Cas Multiplier delegate until all its child CASes have
> >> finished processing. If false, an input CAS can be processed in parallel
> >> with its children."
> >>
> >>
> >> If the above change does not fix the NPE, I suspect you may have a bug
> >> somewhere in your code which allows a CAS to be accessed in two separate
> >> threads.
> >>
> >> -jerry
> >>
> >> On Wed, Feb 15, 2017 at 12:43 PM, Jaroslaw Cwiklik <uim...@gmail.com>
> >> wrote:
> >>
> >>> Nelson, I can try to setup a simple pipeline with one AE which will add
> >>> 20
> >>> views and than test serialization. Not sure if I get to it today. If
> not
> >>> this will have to wait till Monday next week. I've already mentioned
> this
> >>> before, don't operate on a CAS once it leaves an AE. The contract is
> >>> CAS-In
> >>> CAS-out. A CAS instance can only be operated on by one AE at a time.
> >>>
> >>> -jerry
> >>>
> >>> On Wed, Feb 15, 2017 at 11:06 AM, Marshall Schor <m...@schor.com>
> wrote:
> >>>
> >>>> On 2/15/2017 9:51 AM, Jaroslaw Cwiklik wrote:
> >>>>> Not exactly sure how to debug this.
> >>>> a small-ish test case we could run would enable debugging...
> >>>>
> >>>>> The UIMA-AS does not touch contents of
> >>>>> a CAS directly. Are there any other errors in the log besides NPE?
> The
> >>>>> UIMA-AS uses uima-sdk to serialize CASes. Since you are getting null
> >>>> from
> >>>>> getView(N), this view must have been deleted somehow.
> >>>>>
> >>>>> -jerry
> >>>>>
> >>>>> On Mon, Feb 13, 2017 at 11:43 AM, nelson rivera <
> >>>> nelsonriver...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> I was able to check your email just today. The agregegate is async,
> >>>>>> but only process one input CAS at the same time,default
> >>>>>> numberOfCASes.
> >>>>>> I read your possible explanation but i have no idea that another
> >>>>>> thread can modificate the cas, because the last annotator's
> execution
> >>>>>> is correct and only missing that the framework uima-as serializes
> the
> >>>>>> cas.
> >>>>

Re: Passing object from cpe to collection reader

[Jaroslaw Cwiklik]

Oops, by mistake I included a link to very old uima documentation. Sorry
about that. The Java Docs can be found here:

https://uima.apache.org/d/uimaj-current/apidocs/index.html

Looks for UimaFramework and produceCollectionReader(...)

-jerry

On Wed, Feb 22, 2017 at 1:40 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Not sure if this is the best way to do it but can you first instantiate
> and initialize your CR and plug it in to the CPM? Something like this:
>
>Map additionalParams = new HashMap();
>
>   additionalParams.put("object1", object1);
>
>  BaseCollectionReader colreader = (BaseCollectionReader)
>   UIMAFramework.produceResource(resourceSpecifier, additionalParams);
>
> And than use an instance of BaseCPMImpl to call
>
>   public void setCollectionReader(BaseCollectionReader aCollectionReader)
> {
>
>
> Check documentation for details:
> https://uima.apache.org/downloads/releaseDocs/2.3.0-
> incubating/docs/api/org/apache/uima/UIMAFramework.html
>
> -jerry
>
>
> On Wed, Feb 22, 2017 at 2:44 AM, Prerna Bhatia <prerna.i...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> At the time of initialization of the CPE, I am passing list of objects to
>> CPE. I want these objects to be used by collection reader to prepare the
>> documents and run AAE on it. How can this be done?
>>
>> Thanks in advance,
>> Prerna
>>
>
>

Re: Passing object from cpe to collection reader

[Jaroslaw Cwiklik]

Not sure if this is the best way to do it but can you first instantiate and
initialize your CR and plug it in to the CPM? Something like this:

   Map additionalParams = new HashMap();

  additionalParams.put("object1", object1);

 BaseCollectionReader colreader = (BaseCollectionReader)
  UIMAFramework.produceResource(resourceSpecifier, additionalParams);

And than use an instance of BaseCPMImpl to call

  public void setCollectionReader(BaseCollectionReader aCollectionReader) {


Check documentation for details:
https://uima.apache.org/downloads/releaseDocs/2.3.0-incubating/docs/api/org/apache/uima/UIMAFramework.html

-jerry


On Wed, Feb 22, 2017 at 2:44 AM, Prerna Bhatia 
wrote:

> Hi All,
>
> At the time of initialization of the CPE, I am passing list of objects to
> CPE. I want these objects to be used by collection reader to prepare the
> documents and run AAE on it. How can this be done?
>
> Thanks in advance,
> Prerna
>

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Nelson, change Cas Multiplier in your deployment descriptor as follows:





Note: processParentLast="true".

In UIMA-AS async aggregate its possible for a child CAS and its parent CAS
to flow through the pipeline at the same time and the parent CAS may reach
the end before its child(ren). The above setting will ensure the parent CAS
does not flow ahead of its children. From UIMA-AS documentation:

"The processParentLast attribute on the  element is
optional, and specifies processing order of an input CAS relative to its
children. If true, a flow of an input CAS will be suspended after it is
returned from a Cas Multiplier delegate until all its child CASes have
finished processing. If false, an input CAS can be processed in parallel
with its children."


If the above change does not fix the NPE, I suspect you may have a bug
somewhere in your code which allows a CAS to be accessed in two separate
threads.

-jerry

On Wed, Feb 15, 2017 at 12:43 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Nelson, I can try to setup a simple pipeline with one AE which will add 20
> views and than test serialization. Not sure if I get to it today. If not
> this will have to wait till Monday next week. I've already mentioned this
> before, don't operate on a CAS once it leaves an AE. The contract is CAS-In
> CAS-out. A CAS instance can only be operated on by one AE at a time.
>
> -jerry
>
> On Wed, Feb 15, 2017 at 11:06 AM, Marshall Schor <m...@schor.com> wrote:
>
>> On 2/15/2017 9:51 AM, Jaroslaw Cwiklik wrote:
>> > Not exactly sure how to debug this.
>>
>> a small-ish test case we could run would enable debugging...
>>
>> > The UIMA-AS does not touch contents of
>> > a CAS directly. Are there any other errors in the log besides NPE? The
>> > UIMA-AS uses uima-sdk to serialize CASes. Since you are getting null
>> from
>> > getView(N), this view must have been deleted somehow.
>> >
>> > -jerry
>> >
>> > On Mon, Feb 13, 2017 at 11:43 AM, nelson rivera <
>> nelsonriver...@gmail.com>
>> > wrote:
>> >
>> >> I was able to check your email just today. The agregegate is async,
>> >> but only process one input CAS at the same time,default numberOfCASes.
>> >> I read your possible explanation but i have no idea that another
>> >> thread can modificate the cas, because the last annotator's execution
>> >> is correct and only missing that the framework uima-as serializes the
>> >> cas.
>> >>
>> >> This is the configuration of deploy of the aggregate:
>> >>
>> >> 
>> >> > >> xmlns="http://uima.apache.org/resourceSpecifier;>
>> >>
>> >> XClusterAnalyzerAE Deploy Descriptor
>> >> Deploys XClusterAnalyzerAE
>> >>
>> >> 
>> >>
>> >> 
>> >> > >> brokerURL="${defaultBrokerURL}"/>
>> >> 
>> >> 
>> >> 
>> >> 
>> >> > >> internalReplyQueueScaleout="3">
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> > >> continueOnRetryFailure="true"/>
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> > >> continueOnRetryFailure="true"/>
>> >> 
>> >> 
>> >> 
>> >> > >> brokerURL="${defaultBrokerURL}"/>
>> >> 
>> >> 
>> >> > >> continueOnRetryFailure="true"/>
>> >>     
>> >> 
>> >> 
>> >> 
>> >> 
>> >> > >> continueOnRetryFailure="true"/>
>> >> 
>> >> 
>> >>

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Nelson, I can try to setup a simple pipeline with one AE which will add 20
views and than test serialization. Not sure if I get to it today. If not
this will have to wait till Monday next week. I've already mentioned this
before, don't operate on a CAS once it leaves an AE. The contract is CAS-In
CAS-out. A CAS instance can only be operated on by one AE at a time.

-jerry

On Wed, Feb 15, 2017 at 11:06 AM, Marshall Schor <m...@schor.com> wrote:

> On 2/15/2017 9:51 AM, Jaroslaw Cwiklik wrote:
> > Not exactly sure how to debug this.
>
> a small-ish test case we could run would enable debugging...
>
> > The UIMA-AS does not touch contents of
> > a CAS directly. Are there any other errors in the log besides NPE? The
> > UIMA-AS uses uima-sdk to serialize CASes. Since you are getting null from
> > getView(N), this view must have been deleted somehow.
> >
> > -jerry
> >
> > On Mon, Feb 13, 2017 at 11:43 AM, nelson rivera <
> nelsonriver...@gmail.com>
> > wrote:
> >
> >> I was able to check your email just today. The agregegate is async,
> >> but only process one input CAS at the same time,default numberOfCASes.
> >> I read your possible explanation but i have no idea that another
> >> thread can modificate the cas, because the last annotator's execution
> >> is correct and only missing that the framework uima-as serializes the
> >> cas.
> >>
> >> This is the configuration of deploy of the aggregate:
> >>
> >> 
> >>  >> xmlns="http://uima.apache.org/resourceSpecifier;>
> >>
> >> XClusterAnalyzerAE Deploy Descriptor
> >> Deploys XClusterAnalyzerAE
> >>
> >> 
> >>
> >> 
> >>  >> brokerURL="${defaultBrokerURL}"/>
> >> 
> >> 
> >> 
> >> 
> >>  >> internalReplyQueueScaleout="3">
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >> 
> >> 
> >> 
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >> 
> >> 
> >> 
> >>  >> brokerURL="${defaultBrokerURL}"/>
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >> 
> >> 
> >> 
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >> 
> >> 
> >> 
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >>  >> continueOnRetryFailure="true"/>
> >>     
> >> 
> >> 
> >> 
> >> 
> >> 
> >>
> >> 
> >>
> >> 2017-02-10 16:43 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >>> Just a bit more evidence. The caller of the gerSofaAddr()
> >>>
> >>> public void writeViewsCommons() throws Exception {
> >>>   // Get indexes for each SofaFS in the CAS
> >>>   int numViews = cas.getBaseSofaCount();
> >>>
> >>>   for (int sofaNum = 1; sofaNum <= numViews; sofaNum++) {
> >>> FSIndexRepositoryImpl loopIR = (FSIndexRepositoryImpl)
> >>>  cas.getBaseCAS().getSofaIndexRepository(sofaNum);
> >>

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Not exactly sure how to debug this. The UIMA-AS does not touch contents of
a CAS directly. Are there any other errors in the log besides NPE? The
UIMA-AS uses uima-sdk to serialize CASes. Since you are getting null from
getView(N), this view must have been deleted somehow.

-jerry

On Mon, Feb 13, 2017 at 11:43 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> I was able to check your email just today. The agregegate is async,
> but only process one input CAS at the same time,default numberOfCASes.
> I read your possible explanation but i have no idea that another
> thread can modificate the cas, because the last annotator's execution
> is correct and only missing that the framework uima-as serializes the
> cas.
>
> This is the configuration of deploy of the aggregate:
>
> 
>  xmlns="http://uima.apache.org/resourceSpecifier;>
>
> XClusterAnalyzerAE Deploy Descriptor
> Deploys XClusterAnalyzerAE
>
> 
>
> 
>  brokerURL="${defaultBrokerURL}"/>
> 
> 
> 
> 
>  internalReplyQueueScaleout="3">
> 
> 
> 
> 
> 
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
>  brokerURL="${defaultBrokerURL}"/>
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
> 
>     
> 
> 
> 
>  continueOnRetryFailure="true"/>
> 
> 
> 
> 
> 
> 
>
> 
>
> 2017-02-10 16:43 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Just a bit more evidence. The caller of the gerSofaAddr()
> >
> > public void writeViewsCommons() throws Exception {
> >   // Get indexes for each SofaFS in the CAS
> >   int numViews = cas.getBaseSofaCount();
> >
> >   for (int sofaNum = 1; sofaNum <= numViews; sofaNum++) {
> > FSIndexRepositoryImpl loopIR = (FSIndexRepositoryImpl)
> >  cas.getBaseCAS().getSofaIndexRepository(sofaNum);
> > final int sofaAddr = getSofaAddr(sofaNum);
> >
> > Not an expert of this code, but it smells like another thread is
> changing a
> > CAS which is being serialized.
> >
> > -jerry
> >
> > On Fri, Feb 10, 2017 at 4:31 PM, Jaroslaw Cwiklik <uim...@gmail.com>
> wrote:
> >
> >> Is this a primitive (single-threaded) aggregate or async
> >> (multi-threaded)?
> >> If async, try to simplify and run primitive aggregate with scaleout=1.
> >>
> >> The CAS does not seem to be null in this case. The caller of the
> >> getSerializedCas()
> >> checks for null.
> >>
> >> The code dies here:
> >> Caused by: java.lang.NullPointerException
> >> at org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
> >> er.getSofaAddr(CasSerializerSupport.java:454)
> >>
> >>public int getSofaAddr(int sofaNum) {
> >>   if (sofaNum != 1 || cas.isInitialSofaCreated()) { //skip if
> initial
> >> view && no Sofa yet
> >> // all
> >> non-initial-views must have a sofa
> >>* return ((CASImpl)cas.getView(sofaNum)).getSofaRef();*
> >>   }
> >>   return 0;
> >> }
> >>
> >> Looks to me that getView(sofaNum) is returning null. Is it possible that
> >> two threads are operati

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Just a bit more evidence. The caller of the gerSofaAddr()

public void writeViewsCommons() throws Exception {
  // Get indexes for each SofaFS in the CAS
  int numViews = cas.getBaseSofaCount();

  for (int sofaNum = 1; sofaNum <= numViews; sofaNum++) {
FSIndexRepositoryImpl loopIR = (FSIndexRepositoryImpl)
 cas.getBaseCAS().getSofaIndexRepository(sofaNum);
final int sofaAddr = getSofaAddr(sofaNum);

Not an expert of this code, but it smells like another thread is changing a
CAS which is being serialized.

-jerry

On Fri, Feb 10, 2017 at 4:31 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Is this a primitive (single-threaded) aggregate or async (multi-threaded)?
> If async, try to simplify and run primitive aggregate with scaleout=1.
>
> The CAS does not seem to be null in this case. The caller of the 
> getSerializedCas()
> checks for null.
>
> The code dies here:
> Caused by: java.lang.NullPointerException
> at org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
> er.getSofaAddr(CasSerializerSupport.java:454)
>
>public int getSofaAddr(int sofaNum) {
>   if (sofaNum != 1 || cas.isInitialSofaCreated()) { //skip if initial
> view && no Sofa yet
> // all
> non-initial-views must have a sofa
>* return ((CASImpl)cas.getView(sofaNum)).getSofaRef();*
>   }
>   return 0;
> }
>
> Looks to me that getView(sofaNum) is returning null. Is it possible that
> two threads are operating on the same CAS maybe? One removing a view while
> another trying to serialize. Have no idea what else could it be.
>
> -jerry
>
>
>
> On Fri, Feb 10, 2017 at 8:45 AM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> Hi,  The first thing I did was these tests,i made a simple test case
>> that create a Cas with 17 views and then serialize using
>> XmiCasSerializer.serialize(newJCas.getCas(), fis) and serializes
>> correctly.
>> Also i made other test, initialize the same AE but of local way with
>> UIMA API and process the same input documents and the processing is
>> correct and then serialize the CAS, without problem.
>>
>> The error is with AE deployed in uima-as and consuming it.
>>
>> 2017-02-09 17:30 GMT-05:00, Marshall Schor <m...@schor.com>:
>> > one thing that would help track this down is a small isolated test case.
>> >
>> > Do you think uima-as is needed? I'm wondering if a simple test case
>> which
>> > generated 17 views and then tried to serialize would show the failure...
>> >
>> > If you could supply a small test case that showed the failure so we
>> could
>> > reproduce it, that would enable a rapid resolution.
>> >
>> > -Marshall
>> >
>> >
>> > On 2/9/2017 3:58 PM, Marshall Schor wrote:
>> >>  The line throwing the null pointer exception is :
>> >>
>> >> cas.getView(sofaNum).getSofaRef()
>> >>
>> >> So the NPE is either the cas is null, or the getView(sofaNum) is
>> returning
>> >> null.
>> >>
>> >> I'm not sure what the best way is to debug this...
>> >>
>> >> -Marshall
>> >>
>> >>
>> >>
>> >>
>> >> On 2/9/2017 12:42 PM, nelson rivera wrote:
>> >>> I have a aggregate service uima-as, at the end of aggregate the cas to
>> >>> return is composed of as many views as the number of input files, each
>> >>> view with annotations of processing.
>> >>> With a number of input documents less than 15 the processing is
>> >>> successful always,
>> >>> but if the number of documents is greater than 15, i get a
>> >>> NullPointerException at the aggregate service trying to serialize the
>> >>> cas, not in the processing of AE aggregate.
>> >>> the logs of aggregate service:
>> >>>
>> >>> 11:51:38.815 - 42:
>> >>> cu.datys.xinetica.uima.core.MergerInViewCasMultipler.hasNext(285):
>> >>> INFO: HasNext false
>> >>> 11:51:38.875 - 44:
>> >>> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
>> >>> XClusterAnalyzer::process --- OK
>> >>> 11:51:39.145 - 45:
>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>> ller_impl.replyToClient:
>> >>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
>> >>> 11:51:3

Re: Many views in the cas to serialize cause java.lang.NullPointerException in service uima-as

[Jaroslaw Cwiklik]

Is this a primitive (single-threaded) aggregate or async (multi-threaded)?
If async, try to simplify and run primitive aggregate with scaleout=1.

The CAS does not seem to be null in this case. The caller of the
getSerializedCas()
checks for null.

The code dies here:
Caused by: java.lang.NullPointerException
at org.apache.uima.cas.impl.CasSerializerSupport$
CasDocSerializer.getSofaAddr(CasSerializerSupport.java:454)

   public int getSofaAddr(int sofaNum) {
  if (sofaNum != 1 || cas.isInitialSofaCreated()) { //skip if initial
view && no Sofa yet
// all
non-initial-views must have a sofa
   * return ((CASImpl)cas.getView(sofaNum)).getSofaRef();*
  }
  return 0;
}

Looks to me that getView(sofaNum) is returning null. Is it possible that
two threads are operating on the same CAS maybe? One removing a view while
another trying to serialize. Have no idea what else could it be.

-jerry



On Fri, Feb 10, 2017 at 8:45 AM, nelson rivera 
wrote:

> Hi,  The first thing I did was these tests,i made a simple test case
> that create a Cas with 17 views and then serialize using
> XmiCasSerializer.serialize(newJCas.getCas(), fis) and serializes
> correctly.
> Also i made other test, initialize the same AE but of local way with
> UIMA API and process the same input documents and the processing is
> correct and then serialize the CAS, without problem.
>
> The error is with AE deployed in uima-as and consuming it.
>
> 2017-02-09 17:30 GMT-05:00, Marshall Schor :
> > one thing that would help track this down is a small isolated test case.
> >
> > Do you think uima-as is needed? I'm wondering if a simple test case which
> > generated 17 views and then tried to serialize would show the failure...
> >
> > If you could supply a small test case that showed the failure so we could
> > reproduce it, that would enable a rapid resolution.
> >
> > -Marshall
> >
> >
> > On 2/9/2017 3:58 PM, Marshall Schor wrote:
> >>  The line throwing the null pointer exception is :
> >>
> >> cas.getView(sofaNum).getSofaRef()
> >>
> >> So the NPE is either the cas is null, or the getView(sofaNum) is
> returning
> >> null.
> >>
> >> I'm not sure what the best way is to debug this...
> >>
> >> -Marshall
> >>
> >>
> >>
> >>
> >> On 2/9/2017 12:42 PM, nelson rivera wrote:
> >>> I have a aggregate service uima-as, at the end of aggregate the cas to
> >>> return is composed of as many views as the number of input files, each
> >>> view with annotations of processing.
> >>> With a number of input documents less than 15 the processing is
> >>> successful always,
> >>> but if the number of documents is greater than 15, i get a
> >>> NullPointerException at the aggregate service trying to serialize the
> >>> cas, not in the processing of AE aggregate.
> >>> the logs of aggregate service:
> >>>
> >>> 11:51:38.815 - 42:
> >>> cu.datys.xinetica.uima.core.MergerInViewCasMultipler.hasNext(285):
> >>> INFO: HasNext false
> >>> 11:51:38.875 - 44:
> >>> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
> >>> XClusterAnalyzer::process --- OK
> >>> 11:51:39.145 - 45:
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.replyToClient:
> >>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> >>> 11:51:39.145 - 45:
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.replyToClient:
> >>> WARNING:
> >>> org.apache.uima.aae.error.AsynchAEException:
> >>> org.apache.uima.UIMARuntimeException
> >>> at
> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.
> getSerializedCas(JmsOutputChannel.java:1265)
> >>> at
> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.
> sendReply(JmsOutputChannel.java:800)
> >>> at
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineController_impl.
> sendReplyToRemoteClient(AggregateAnalysisEngineController_impl.java:2173)
> >>> at
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.replyToClient(AggregateAnalysisEngineController_impl.java:2342)
> >>> at
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.finalStep(AggregateAnalysisEngineController_impl.java:1862)
> >>> at
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.executeFlowStep(AggregateAnalysisEngineController_impl.java:2489)
> >>> at
> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> ler_impl.process(AggregateAnalysisEngineController_impl.java:1271)
> >>> at
> >>> org.apache.uima.aae.handler.HandlerBase.invokeProcess(
> HandlerBase.java:118)
> >>> at
> >>> org.apache.uima.aae.handler.input.ProcessResponseHandler.
> cancelTimerAndProcess(ProcessResponseHandler.java:117)
> >>> at
> >>> org.apache.uima.aae.handler.input.ProcessResponseHandler.
> 

Re: Using optional argument -Duima.as.enable.jmx=false

[Jaroslaw Cwiklik]

Nelson, thanks for pointing this out. This parameter
(-Duima.as.enable.jmx=false)
disables ActiveMQ JMX port when running UIMA-AS extended tests. Per JIRA
https://issues.apache.org/jira/browse/UIMA-5166 the fix addresses this
scenario only. The fix was motivated by a complaint from Apache infra team
which noticed an open RMI port while running UIMA-AS Jenkins regression
tests.

The UIMA-AS 2.9.0 README comments are misleading and suggest that this flag
can disable JMX port of the service.I need to rewrite the comment to
reflect the truth. Will open JIRA for this.

As far as I know there is no way in java code to override these

 -Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=$PORT
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

These settings take effect before the jvm hands off control to a user code.
The java JMX Agent's registry which opens the port is not
accessible for the purpose of stopping the port once it is started. So
bottom line the comments are wrong and need to be fixed. I will address
this soon. To manage service jmx port use the above 4 -Dcom.sun.management..
 settings

- Jerry Cwiklik


With the above there is no way to disable the RMI port as far as I know.
The -D's are

On Thu, Dec 22, 2016 at 2:49 PM, nelson rivera 
wrote:

> I am testing recent uima-as 2.9.0 release. I try deploy a service
> uima-as, with the new argument uima.as.enable.jmx with value "false",
> i expected that the UIMA-AS will start with no JMX support and no port
> will not be opened, but a port continue opening and Providing JMX
> statistics of processing.
>
> What should be the behavior of this argument?
>
> This is the complete set of arguments JMX configured in the uima-as
> service start:
>
> -Duima.as.enable.jmx=false
> -Duima.as.enable.jmx=false -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.port=$PORT
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
>

[ANNOUNCE] Apache UIMA AS 2.9.0 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the Apache
UIMA-AS version 2.9.0, which includes asynchronous scaleout capabilities
for the UIMA annotators.

UIMA-AS includes the base UIMA SDK and augments it with scaleout
capability; it is a next-generation replacement for the original CPM
(Collection Processing Management) scaleout that is part of the core UIMA
Framework. For more information, please visit:

http://uima.apache.org/doc-uimaas-what.html

This release contains a number of improvements and bug fixes. Notable
updates in this release include:

  - Updated to use Activemq 5.14.0
  - Added dependency on UIMA SDK 2.9.0
  - Fixed http based service connectivity
  - Added support for automatic recovery of temp queues after broker
restart.
  - Fixed per CAS Performance Metrics breakdown
  - Fixed per process CPU & RSS reporting
  - Fixed example runtime configurations
  - Fixed error recovery on exception while deserializing a CAS
  - "Pinned" JMX MBeans to a specific deployment to enable orderly cleanup
  - Fixed support of AMQ white listing of packages
  - Added support to disable JMX via a new argument
-Duima.as.enable.jmx=false
  - Fixed dd2spring issues
  - Updated version checker to test compatibility with UIMA SDK.

For a complete list of bugs and improvements included in this release
please see
https://uima.apache.org/d/uima-as-2.9.0/issuesFixed/jira-report.html.

-- Jerry Cwiklik, for the Apache UIMA development team

Re: Possible cause of nullPointerException in casmultiplier?

[Jaroslaw Cwiklik]

An input CAS should not be released in the AE (user code). The Flow
Controller should be used
to prevent a CAS from flowing downstream. UIMA-AS will try to release a CAS
in a Final Step
of processing and this may cause an error like

"The system tried to return a common analysis structure to the pool from
which it was not checked out."

Jerry


On Mon, Nov 28, 2016 at 12:40 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> I have deployed the aggregate of asynchronous way.
>  And i configure this aggregate to processing one request at the same
> time(numberOfCASes=1,default), because as i commented on previous
> email, in my flow: The first casmultiplier receive a annotation with
> the path of a folder and creates a new child cas for each file in the
> folder, the child cas
> continue the flow in the other primitive annotators and at the end it
> go to the second casmultipler, in the second casmultipler can or not
> create new child cas, in the case of not i release the cas. I am
> assuming (numberOfCASes=1,default) that until all child has finished
> its processing, the aggregate not going processing other input cas of
> other request, it is correct?.
>
>
>
> 2016-11-28 12:00 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Nelson, this error:
> >
> > "The system tried to return a common analysis structure to the pool from
> > which it was not checked out."
> >
> > I think this is caused if you try to release a CAS that has been
> previously
> > released. Sure you
> > are not trying to release it twice? I think releasing an input CAS in AE
> > code is not a good idea. I am assuming
> > you have an asynchronous aggregate (queue in front of each AE). In such
> > deployments, the UIMA-AS is
> > responsible for releasing a CAS once it reaches a Final state. Within a
> > synchronous aggregate I am not sure
> > if you are allowed to release a CAS in your code. Perhaps others can
> chime
> > in. Can you confirm which
> > deployment you chose for this aggregate (synch or asynch)
> > Jerry
> >
> > On Fri, Nov 25, 2016 at 11:14 AM, nelson rivera <
> nelsonriver...@gmail.com>
> > wrote:
> >
> >> The errores related are very random. I found a patron of error ( don't
> >> know if it's the only one): The caspool of the first casmultipler
> >> have size of 8, if i send a request to the aggregate in which the
> >> first casmultipler have to generate more than 8 child cas, i get
> >> random errors, the childs cas generated after the number 8, sometimes
> >> is empty, sometimes have less annotations than it should have (I see
> >> this with a own XCasSerializer annotator) or get nullPointerException.
> >> I don't know why can happen, because in the method "process()" of the
> >> second casmultipler i release the incoming cas (aJCas.release();)
> >> directly in my source code, for that the caspool of the first
> >> casmultipler will not empty. There is something taht i am not doing
> >> well?
> >>
> >> I don't have idea what may be going on?, i will appreciate the help
> >>
> >> 2016-11-24 19:24 GMT-05:00, nelson rivera <nelsonriver...@gmail.com>:
> >> > I continued investigating  and according to the stacktrace and source
> >> > code of of FSIndexRepositoryImpl.ll_addFS_common() the exception is
> >> > from line 2805 , in this part
> >> >
> >> > final ArrayList<IndexIteratorCachePair>
> >> > indexes = this.indexArray[typeCode];
> >> > // Add fsRef to all indexes.
> >> > boolean noIndexOrOnlySetindexes = true;
> >> > for (IndexIteratorCachePair iicp :
> >> > indexes) {} //here
> >> >
> >> > apparently loop on a null value, any idea of why can happen this
> >> exception?
> >> >
> >> > 2016-11-24 16:27 GMT-05:00, nelson rivera <nelsonriver...@gmail.com>:
> >> >> the error continue ocurring , this is the log, my casmultiplier is
> >> >> FileSystemMultiplerCas
> >> >>
> >> >> 04:04:17.520 - 23:
> >> >> cu.datys.xinetica.uima.core.FileSystemMultiplerCas.next(334): INFO:
> >> >> Adicionando annotation tipo cu.datys.xinetica.uima.ContinueOnFailure
> >> >> 04:04:17.522 - 23: org.apache.uima.util.CasPool.releaseCas: WARNING:
> >> >> The system tried to return a common analysis structure to the pool
> >> >> from which it was not checked out.
> >> >> 04:04:17.523 - 23:
> >> >> org.apache.u

Re: Possible cause of nullPointerException in casmultiplier?

[Jaroslaw Cwiklik]

Nelson, this error:

"The system tried to return a common analysis structure to the pool from
which it was not checked out."

I think this is caused if you try to release a CAS that has been previously
released. Sure you
are not trying to release it twice? I think releasing an input CAS in AE
code is not a good idea. I am assuming
you have an asynchronous aggregate (queue in front of each AE). In such
deployments, the UIMA-AS is
responsible for releasing a CAS once it reaches a Final state. Within a
synchronous aggregate I am not sure
if you are allowed to release a CAS in your code. Perhaps others can chime
in. Can you confirm which
deployment you chose for this aggregate (synch or asynch)
Jerry

On Fri, Nov 25, 2016 at 11:14 AM, nelson rivera 
wrote:

> The errores related are very random. I found a patron of error ( don't
> know if it's the only one): The caspool of the first casmultipler
> have size of 8, if i send a request to the aggregate in which the
> first casmultipler have to generate more than 8 child cas, i get
> random errors, the childs cas generated after the number 8, sometimes
> is empty, sometimes have less annotations than it should have (I see
> this with a own XCasSerializer annotator) or get nullPointerException.
> I don't know why can happen, because in the method "process()" of the
> second casmultipler i release the incoming cas (aJCas.release();)
> directly in my source code, for that the caspool of the first
> casmultipler will not empty. There is something taht i am not doing
> well?
>
> I don't have idea what may be going on?, i will appreciate the help
>
> 2016-11-24 19:24 GMT-05:00, nelson rivera :
> > I continued investigating  and according to the stacktrace and source
> > code of of FSIndexRepositoryImpl.ll_addFS_common() the exception is
> > from line 2805 , in this part
> >
> > final ArrayList
> > indexes = this.indexArray[typeCode];
> > // Add fsRef to all indexes.
> > boolean noIndexOrOnlySetindexes = true;
> > for (IndexIteratorCachePair iicp :
> > indexes) {} //here
> >
> > apparently loop on a null value, any idea of why can happen this
> exception?
> >
> > 2016-11-24 16:27 GMT-05:00, nelson rivera :
> >> the error continue ocurring , this is the log, my casmultiplier is
> >> FileSystemMultiplerCas
> >>
> >> 04:04:17.520 - 23:
> >> cu.datys.xinetica.uima.core.FileSystemMultiplerCas.next(334): INFO:
> >> Adicionando annotation tipo cu.datys.xinetica.uima.ContinueOnFailure
> >> 04:04:17.522 - 23: org.apache.uima.util.CasPool.releaseCas: WARNING:
> >> The system tried to return a common analysis structure to the pool
> >> from which it was not checked out.
> >> 04:04:17.523 - 23:
> >> org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.
> callAnalysisComponentNext(549):
> >> SEVERE: Exception occurred
> >> org.apache.uima.analysis_engine.AnalysisEngineProcessException
> >>  at
> >> cu.datys.xinetica.uima.core.FileSystemMultiplerCas.next(
> FileSystemMultiplerCas.java:351)
> >>  at
> >> org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.
> callAnalysisComponentNext(PrimitiveAnalysisEngine_impl.java:526)
> >>  at
> >> org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl$
> AnalysisComponentCasIterator.next(PrimitiveAnalysisEngine_impl.java:647)
> >>  at
> >> org.apache.uima.aae.controller.PrimitiveAnalysisEngineControl
> ler_impl.process(PrimitiveAnalysisEngineController_impl.java:860)
> >>  at
> >> org.apache.uima.aae.handler.HandlerBase.invokeProcess(
> HandlerBase.java:121)
> >>  at
> >> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.
> handleProcessRequestWithCASReference(ProcessRequestHandler_impl.java:795)
> >>  at
> >> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.handle(
> ProcessRequestHandler_impl.java:1055)
> >>  at
> >> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.onMessage(
> UimaVmMessageListener.java:107)
> >>  at
> >> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher$1.run(
> UimaVmMessageDispatcher.java:70)
> >>  at
> >> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
> >>  at
> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
> >>  at
> >> org.apache.uima.aae.UimaAsThreadFactory$1.run(
> UimaAsThreadFactory.java:132)
> >>  at java.lang.Thread.run(Thread.java:745)
> >> Caused by: java.lang.NullPointerException
> >>  at
> >> org.apache.uima.cas.impl.FSIndexRepositoryImpl.ll_addFS_common(
> FSIndexRepositoryImpl.java:2805)
> >>  at
> >> org.apache.uima.cas.impl.FSIndexRepositoryImpl.ll_
> addFS(FSIndexRepositoryImpl.java:2763)
> >>  at
> >> org.apache.uima.cas.impl.FSIndexRepositoryImpl.addFS(
> FSIndexRepositoryImpl.java:2068)
> >>  at
> >> org.apache.uima.cas.impl.FSIndexRepositoryImpl.addFS(
> 

Re: No service reply, after org.xml.sax.SAXParseException; Trying to serialize non-XML 1.0 character:

[Jaroslaw Cwiklik]

Nelson, the fix for this is in the UIMA-AS RC1 announced last week on
uima-dev.
Jerry

On Fri, Nov 25, 2016 at 10:33 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Hi, i want to know if you have worked in issue of JIRA UIMA-5189 which
> addresses error handling when a serializer throws an exception.
>
> 2016-11-21 16:32 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Nelson, a fix for this is part of JIRA UIMA-5189 which addresses error
> > handing when a serializer throws an exception.
> > I will post UIMA-AS 2.9.0 release candidate tomorrow so you can test your
> > use case. Watch for an email on uima dev list.
> > Jerry
> >
> > On Mon, Nov 21, 2016 at 4:17 PM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> I tried to process a input cas in service aggregate deployed in
> >> uima-as. The annotations produced for annotators contains apparently
> >> invalid character, after finalize the processing , when the framework
> >> tries to send the reply, shows a org.xml.sax.SAXParseException error
> >> serializing the cas and in the client side i get not any reply, the
> >> listener associate it is not notified of the error, and the client
> >> program stays waiting
> >>
> >> the log of service aggregate error:
> >>
> >> 03:50:03.578 - 22:
> >> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> >> ler_impl.replyToClient:
> >> WARNING: Service: XDataFileExtractorAggregate Runtime Exception
> >> 03:50:03.579 - 22:
> >> org.apache.uima.aae.controller.AggregateAnalysisEngineControl
> >> ler_impl.replyToClient:
> >> WARNING:
> >> org.apache.uima.aae.error.AsynchAEException:
> >> org.xml.sax.SAXParseException; Trying to serialize non-XML 1.0
> >> character: , 0x1 at offset 0 in string starting with
> >> at org.apache.uima.adapter.jms.activemq.JmsOutputChannel.
> >> getSerializedCas(JmsOutputChannel.java:1258)
> >> at org.apache.uima.adapter.jms.activemq.JmsOutputChannel.
> >> sendReply(JmsOutputChannel.java:793)
> >> at org.apache.uima.aae.controller.
> AggregateAnalysisEngineControl
> >> ler_impl.sendReplyToRemoteClient(AggregateAnalysisEngineControl
> >> ler_impl.java:2166)
> >> at org.apache.uima.aae.controller.
> AggregateAnalysisEngineControl
> >> ler_impl.replyToClient(AggregateAnalysisEngineControl
> ler_impl.java:2335)
> >> at org.apache.uima.aae.controller.
> AggregateAnalysisEngineControl
> >> ler_impl.finalStep(AggregateAnalysisEngineController_impl.java:1855)
> >> at org.apache.uima.aae.controller.
> AggregateAnalysisEngineControl
> >> ler_impl.executeFlowStep(AggregateAnalysisEngineControl
> ler_impl.java:2482)
> >> at org.apache.uima.aae.controller.
> AggregateAnalysisEngineControl
> >> ler_impl.process(AggregateAnalysisEngineController_impl.java:1264)
> >> at org.apache.uima.aae.handler.HandlerBase.invokeProcess(
> >> HandlerBase.java:118)
> >> at org.apache.uima.aae.handler.input.ProcessResponseHandler.
> >> cancelTimerAndProcess(ProcessResponseHandler.java:117)
> >> at org.apache.uima.aae.handler.input.ProcessResponseHandler.
> >> handleProcessResponseWithCASReference(ProcessResponseHandler.java:485)
> >> at org.apache.uima.aae.handler.input.ProcessResponseHandler.
> >> handle(ProcessResponseHandler.java:767)
> >> at org.apache.uima.aae.handler.HandlerBase.delegate(
> >> HandlerBase.java:149)
> >> at org.apache.uima.aae.handler.input.ProcessRequestHandler_
> >> impl.handle(ProcessRequestHandler_impl.java:1085)
> >> at org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.
> >> onMessage(UimaVmMessageListener.java:107)
> >> at org.apache.uima.aae.spi.transport.vm.
> >> UimaVmMessageDispatcher$1.run(UimaVmMessageDispatcher.java:70)
> >> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> >> ThreadPoolExecutor.java:1145)
> >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> >> ThreadPoolExecutor.java:615)
> >> at org.apache.uima.aae.UimaAsThreadFactory$1.run(
> >> UimaAsThreadFactory.java:132)
> >> at java.lang.Thread.run(Thread.java:745)
> >> Caused by: org.xml.sax.SAXParseException; Trying to serialize non-XML
> >> 1.0 character: , 0x1 at offset 0 in string starting with
> >> at org.apache.uima.util.XMLSerializer$
> >

Re: What should happen with the original input cas and child cas when a excption occurs

[Jaroslaw Cwiklik]

Nelson, in a scenario where there is a CM failure all child CASes are auto
released back to
a pool and an input CAS is returned with an exception. Are you observing a
hang due to
depleted cas pool on subsequent input CAS?

Jerry

On Tue, Nov 22, 2016 at 4:15 PM, nelson rivera 
wrote:

> I have a service aggregate deployed in uima-as, the first annotator in
> the aggregate is a casmultipler y then more primitive annotator in the
> flow. When in the middle of processing (Have been generated various
> child cas from the casmultipler and are being processed in other
> anotators in the flow) occurs an exception in casmultipler's method
> next (Ex NullPointerException), what should happen with the original
> input cas and child cas that were being processed?
>
> In practice when this happens, and i get in the listener associated
> method entityComplete, a status of exception and a reference to the
> original input cas, but, what happen with child cas that were being
> processed?. Are release to the casmultipler's caspool automatically?
>
> I would appreciate the explanation
>

Re: TERMINATE Action with org.xml.sax.SAXParseException in deserializeCasFromXmi function

[Jaroslaw Cwiklik]

Nelson, I've created a JIRA for this bug:
https://issues.apache.org/jira/browse/UIMA-5189

This will be fixed soon and will be part of the next UIMA-AS release
(2.9.0).

Thanks for finding the bug.
Jerry

On Fri, Nov 18, 2016 at 3:39 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Hi, looks like a bug. Will take a look on Monday.
> Thanks
> Jerry
>
> On Fri, Nov 18, 2016 at 11:12 AM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> I have a service aggregate deploys in uima-as. When i send a input cas
>> with a text that contains apparently invalid character, occurs an
>> error deserializing the cas and the framework stops the aggregate
>> service
>>
>> this is the complete stacktrace:
>>
>> 09:54:38.24 - 1:
>> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer
>> .doStartListeners:
>> INFO: Controller: XTokenizerAggregate Trying to Start Listener on
>> Endpoint: queue://XTokenizerAggregate Selector: Command=2000 OR
>> Command=2002 Broker: tcp://localhost:61616
>> 09:54:38.193 - 1:
>> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer
>> .doStartListeners:
>> INFO: Controller: XTokenizerAggregate Trying to Start Listener on
>> Endpoint: queue://XTokenizerAggregate Selector: Command=2001 Broker:
>> tcp://localhost:61616
>> 09:55:11.411 - 16:
>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl
>> .handleProcessRequestFromRemoteClient:
>> WARNING: Service: XTokenizerAggregate Runtime Exception
>> 09:55:11.411 - 16:
>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl
>> .handleProcessRequestFromRemoteClient:
>> WARNING:
>> org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 585;
>> Character reference "
<

Re: TERMINATE Action with org.xml.sax.SAXParseException in deserializeCasFromXmi function

[Jaroslaw Cwiklik]

Hi, looks like a bug. Will take a look on Monday.
Thanks
Jerry

On Fri, Nov 18, 2016 at 11:12 AM, nelson rivera 
wrote:

> I have a service aggregate deploys in uima-as. When i send a input cas
> with a text that contains apparently invalid character, occurs an
> error deserializing the cas and the framework stops the aggregate
> service
>
> this is the complete stacktrace:
>
> 09:54:38.24 - 1:
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.
> doStartListeners:
> INFO: Controller: XTokenizerAggregate Trying to Start Listener on
> Endpoint: queue://XTokenizerAggregate Selector: Command=2000 OR
> Command=2002 Broker: tcp://localhost:61616
> 09:54:38.193 - 1:
> org.apache.uima.adapter.jms.activemq.SpringContainerDeployer.
> doStartListeners:
> INFO: Controller: XTokenizerAggregate Trying to Start Listener on
> Endpoint: queue://XTokenizerAggregate Selector: Command=2001 Broker:
> tcp://localhost:61616
> 09:55:11.411 - 16:
> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.
> handleProcessRequestFromRemoteClient:
> WARNING: Service: XTokenizerAggregate Runtime Exception
> 09:55:11.411 - 16:
> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl.
> handleProcessRequestFromRemoteClient:
> WARNING:
> org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 585;
> Character reference "

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.handleTempQueueFailure:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.handleTempQueueFailure:
> WARNING: Jms Listener Failed. Endpoint:
> temp-queue://ID:nelson-H81-M1-35307-1478005075345-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.825 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:58:13.839 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:58:13.844 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 08:58:13.844 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:58:13.876 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:58:16.719 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:59:16.887 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 08:59:16.887 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:59:16.900 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 08:59:16.917 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 08:59:16.948 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
> 09:01:17.93 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed
> 09:01:17.93 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying ...
> 09:01:17.109 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:XClusterAnalyzerAggregate Listener
> Established Connection to Broker: tcp://localhost:61616
>
>
>
> And in the console output of service:
> Could not refresh JMS Connection for destination
> 'queue://XClusterAnalyzerAggregate' - silently 

Re: Removed Mbeans

[Jaroslaw Cwiklik]

Matthias, I've delivered changes to the UIMA-AS to support orderly cleanup
of MBeans. The code will only remove MBeans related to a pipeline being
undeployed. Do you have a way to build UIMA-As from the SVN trunk or do you
want me to post a new UIMA-AS snapshot for you?
Jerry

On Fri, Oct 7, 2016 at 9:29 AM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Matthias, I will take a look at this soon.  The undeploy should only
> remove MBeans related to a pipeline being undeployed. Not sure how
> involved a fix for this will be yet. Just created JIRA for this:
> https://issues.apache.org/jira/browse/UIMA-5136
>
> Jerry
>
> On Fri, Oct 7, 2016 at 3:28 AM, Matthias Koch <matthias.k...@averbis.com>
> wrote:
>
>> Hi,
>>
>> We are currently using Uima-AS 2.8.1.
>> We are deploying several Pipelines on the same tomcat with the
>> baseUimaAsynchronousEngine. Then we can see their statistics via JConsole
>> and everything seems fine.
>> But if we are undeploying on of the pipelines, every jmxMbean is removed.
>> We investigated the 'bug' and it seems that the undeploy call calls a
>> terminate() method and 5 methods later it leads to a
>> JMXManager.destroy call
>>
>> Line 138 -140 in org.apache.uima.aae.jmx.JmxManager
>>
>> public void destroy() throws Exception {
>> unregisterDomainObjects("org.apache.uima:type=ee.jms.services,*");
>>
>> }
>>
>>
>> Can we prevent this from happening?
>>
>> Every time we undeploy a pipeline we lose all statistic of all pipelines.
>>
>>
>> Sincerely,
>>
>> Matthias
>>
>>
>

Re: UIMA DUCC java.lang.SecurityException

[Jaroslaw Cwiklik]

Wahed, I just reconfigured my ducc with a one line change in
DUCC_HOME/resources/default.ducc.properties

from:
ducc.broker.configuration = conf/activemq-ducc.xml

to:
ducc.broker.configuration = conf/activemq-ducc-unsecure.xml

I can deploy services with no errors.

Jerry


On Mon, Oct 10, 2016 at 9:38 AM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Wahed, actually this is not a bug. Ducc by default runs with a secure
> broker. Meaning that all destinations (queues, topics) are password
> protected. The exception you are seeing is coming from a service unable to
> authenticate. I am not sure if this is documented, but to use services you
> need to use a separate broker or run Ducc with an unsecured broker. The
> easiest is the latter. To run in unsecure mode:
>
> In DUCC_HOME/resources/default.ducc.properties find the following
>
> ducc.broker.configuration = conf/activemq-ducc.xml
>
> and change it to:
>
> ducc.broker.configuration = conf/activemq-ducc-unsecure.xml
>
> I am not 100% sure but you may also need to comment out a line in the same
> file
>
> ducc.broker.credentials.file=${ducc.private.resources}/ducc-
> broker-credentials.properties
>
> I will try the above changes and confirm this shortly.
>
> Jerry
>
>
> On Mon, Oct 10, 2016 at 7:55 AM, Wahed Hemati <hem...@em.uni-frankfurt.de>
> wrote:
>
>> Thanks Jerry,
>>
>> i rebuild it from trunk but still get the same error.
>>
>> when i run ./admin/ducc_statedump -p 61617 i get the following:
>>
>> javax.jms.JMSSecurityException: User name [null] or password is invalid.
>> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExcep
>> tionSupport.java:52)
>> at org.apache.activemq.ActiveMQConnection.syncSendPacket(Active
>> MQConnection.java:1393)
>> at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoS
>> ent(ActiveMQConnection.java:1496)
>> at org.apache.activemq.ActiveMQConnection.createSession(ActiveM
>> QConnection.java:324)
>> at org.apache.uima.ducc.tools.DuccPubListener.connect(DuccPubLi
>> stener.java:88)
>> at org.apache.uima.ducc.tools.DuccPubListener.run(DuccPubListen
>> er.java:183)
>> at org.apache.uima.ducc.tools.DuccPubListener.main(DuccPubListe
>> ner.java:283)
>> Caused by: java.lang.SecurityException: User name [null] or password is
>> invalid.
>> at org.apache.activemq.security.SimpleAuthenticationBroker.auth
>> enticate(SimpleAuthenticationBroker.java:103)
>> at org.apache.activemq.security.SimpleAuthenticationBroker.addC
>> onnection(SimpleAuthenticationBroker.java:71)
>> at org.apache.activemq.broker.BrokerFilter.addConnection(Broker
>> Filter.java:98)
>> at org.apache.activemq.broker.MutableBrokerFilter.addConnection
>> (MutableBrokerFilter.java:103)
>> at org.apache.activemq.broker.TransportConnection.processAddCon
>> nection(TransportConnection.java:817)
>> at org.apache.activemq.broker.jmx.ManagedTransportConnection.pr
>> ocessAddConnection(ManagedTransportConnection.java:77)
>> at org.apache.activemq.command.ConnectionInfo.visit(ConnectionI
>> nfo.java:139)
>> at org.apache.activemq.broker.TransportConnection.service(Trans
>> portConnection.java:338)
>> at org.apache.activemq.broker.TransportConnection$1.onCommand(T
>> ransportConnection.java:188)
>> at org.apache.activemq.transport.MutexTransport.onCommand(Mutex
>> Transport.java:50)
>> at org.apache.activemq.transport.TransportFilter.onCommand(Tran
>> sportFilter.java:69)
>> at org.apache.activemq.transport.WireFormatNegotiator.onCommand
>> (WireFormatNegotiator.java:125)
>> at org.apache.activemq.transport.AbstractInactivityMonitor.onCo
>> mmand(AbstractInactivityMonitor.java:300)
>> at org.apache.activemq.transport.TransportSupport.doConsume(Tra
>> nsportSupport.java:83)
>> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTran
>> sport.java:233)
>> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransp
>> ort.java:215)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> is it somehow related?
>>
>>
>> -Wahed
>>
>>
>

Re: UIMA DUCC java.lang.SecurityException

[Jaroslaw Cwiklik]

Wahed, actually this is not a bug. Ducc by default runs with a secure
broker. Meaning that all destinations (queues, topics) are password
protected. The exception you are seeing is coming from a service unable to
authenticate. I am not sure if this is documented, but to use services you
need to use a separate broker or run Ducc with an unsecured broker. The
easiest is the latter. To run in unsecure mode:

In DUCC_HOME/resources/default.ducc.properties find the following

ducc.broker.configuration = conf/activemq-ducc.xml

and change it to:

ducc.broker.configuration = conf/activemq-ducc-unsecure.xml

I am not 100% sure but you may also need to comment out a line in the same
file

ducc.broker.credentials.file=${ducc.private.resources}/ducc-broker-credentials.properties

I will try the above changes and confirm this shortly.

Jerry


On Mon, Oct 10, 2016 at 7:55 AM, Wahed Hemati 
wrote:

> Thanks Jerry,
>
> i rebuild it from trunk but still get the same error.
>
> when i run ./admin/ducc_statedump -p 61617 i get the following:
>
> javax.jms.JMSSecurityException: User name [null] or password is invalid.
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExcep
> tionSupport.java:52)
> at org.apache.activemq.ActiveMQConnection.syncSendPacket(Active
> MQConnection.java:1393)
> at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoS
> ent(ActiveMQConnection.java:1496)
> at org.apache.activemq.ActiveMQConnection.createSession(ActiveM
> QConnection.java:324)
> at org.apache.uima.ducc.tools.DuccPubListener.connect(DuccPubLi
> stener.java:88)
> at org.apache.uima.ducc.tools.DuccPubListener.run(DuccPubListen
> er.java:183)
> at org.apache.uima.ducc.tools.DuccPubListener.main(DuccPubListe
> ner.java:283)
> Caused by: java.lang.SecurityException: User name [null] or password is
> invalid.
> at org.apache.activemq.security.SimpleAuthenticationBroker.auth
> enticate(SimpleAuthenticationBroker.java:103)
> at org.apache.activemq.security.SimpleAuthenticationBroker.addC
> onnection(SimpleAuthenticationBroker.java:71)
> at org.apache.activemq.broker.BrokerFilter.addConnection(Broker
> Filter.java:98)
> at org.apache.activemq.broker.MutableBrokerFilter.addConnection
> (MutableBrokerFilter.java:103)
> at org.apache.activemq.broker.TransportConnection.processAddCon
> nection(TransportConnection.java:817)
> at org.apache.activemq.broker.jmx.ManagedTransportConnection.pr
> ocessAddConnection(ManagedTransportConnection.java:77)
> at org.apache.activemq.command.ConnectionInfo.visit(ConnectionI
> nfo.java:139)
> at org.apache.activemq.broker.TransportConnection.service(Trans
> portConnection.java:338)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(
> TransportConnection.java:188)
> at org.apache.activemq.transport.MutexTransport.onCommand(Mutex
> Transport.java:50)
> at org.apache.activemq.transport.TransportFilter.onCommand(Tran
> sportFilter.java:69)
> at org.apache.activemq.transport.WireFormatNegotiator.onCommand
> (WireFormatNegotiator.java:125)
> at org.apache.activemq.transport.AbstractInactivityMonitor.onCo
> mmand(AbstractInactivityMonitor.java:300)
> at org.apache.activemq.transport.TransportSupport.doConsume(Tra
> nsportSupport.java:83)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTran
> sport.java:233)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransp
> ort.java:215)
> at java.lang.Thread.run(Thread.java:745)
>
> is it somehow related?
>
>
> -Wahed
>
>

Re: UIMA DUCC java.lang.SecurityException

[Jaroslaw Cwiklik]

Looks like a bug in  Ducc. Just rebuilt Ducc from trunk and am seeing this
in the sm.log:

07 Oct 2016 11:47:59,477  INFO SM.PingDriver - 0 run  Starting INTERNAL
ping.
07 Oct 2016 11:47:59,655 ERROR SM.PingDriver - 0 handleStatistics
 Missed_pings  1 endpoint UIMA-AS:FixedSleepAE_1:tcp://bluejws65:61617
Alive[false] Healthy[false] + Inf\
o: QDEPTH[0] AveNQ[0] Consum[0] Prod[0] minNQ[0] maxNQ[0] expCnt[0]
inFlt[0] DQ[0] NQ[0] NDisp[0] MetaNode[N/A] MetaPid[N/A]: Cannot issue
getMeta to: FixedSleepAE_1:tcp:\
//bluejws65:61617: java.lang.SecurityException: User name [null] or
password is invalid.

Will check the code this afternoon. Looks like a regression.

Jerry

On Fri, Oct 7, 2016 at 9:30 AM, Wahed Hemati 
wrote:

> Hi,
>
> I am trying to register a service to ducc. It's failing with the exception
> java.lang.SecurityException: User name [null] or password is invalid. The
> exception is in sm.log file
>
> ERROR SM.PingDriver - 0 handleStatistics  Missed_pings  13 endpoint
> UIMA-AS:FixedSleepAE_0:tcp://wahed-pc:61617 Alive[false] Healthy[false] +
> Info: JMX Failure[Failed to retrieve RMIServer stub:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: wahed-pc; nested
> exception is:
> java.net.ConnectException: Connection refused]]] MetaNode[N/A]
> MetaPid[N/A: Cannot issue getMeta to: FixedSleepAE_0:tcp://wahed-pc:61617:
> java.lang.SecurityException: User name [null] or password is inval
> id.
>
> Thanks in advance
>
> -Wahed
>
>

Re: Removed Mbeans

[Jaroslaw Cwiklik]

Matthias, I will take a look at this soon.  The undeploy should only remove
MBeans related to a pipeline being undeployed. Not sure how involved a fix
for this will be yet. Just created JIRA for this:
https://issues.apache.org/jira/browse/UIMA-5136

Jerry

On Fri, Oct 7, 2016 at 3:28 AM, Matthias Koch 
wrote:

> Hi,
>
> We are currently using Uima-AS 2.8.1.
> We are deploying several Pipelines on the same tomcat with the
> baseUimaAsynchronousEngine. Then we can see their statistics via JConsole
> and everything seems fine.
> But if we are undeploying on of the pipelines, every jmxMbean is removed.
> We investigated the 'bug' and it seems that the undeploy call calls a
> terminate() method and 5 methods later it leads to a
> JMXManager.destroy call
>
> Line 138 -140 in org.apache.uima.aae.jmx.JmxManager
>
> public void destroy() throws Exception {
> unregisterDomainObjects("org.apache.uima:type=ee.jms.services,*");
>
> }
>
>
> Can we prevent this from happening?
>
> Every time we undeploy a pipeline we lose all statistic of all pipelines.
>
>
> Sincerely,
>
> Matthias
>
>

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Nelson, yes I think there is a problem in the UIMA-AS. First, there is
evidence that a service
is trying to use a stale temp queue:

tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
Cannot use a Temporary destination from another Connection.


Also, I think there is another problem related to recovery of temp queues
in the service.
The service cas multiplier uses multiple queues:
1) input queue - receives GetMeta and Process requests from a client
2) temp queue - receives FreeCas requests from a client. This is only
created when a service
is a Cas Multiplier. This is used to free CASes in a cas multiplier.

The input queue listener recovers the connection since it received an input
CAS. I think the
temp queue is not recovered and this leads to repeated retries.

I will work on fixing this today and let you know when this is ready. I
will also create a new
snapshot for you to try.

Thanks for finding these problems.

- jerry


On Wed, Sep 28, 2016 at 10:12 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> ok, only show you the result of last test.
> Now in my test  with uima-as-2.9.0, after complete restart of the
> Broker UIMA-AS log show
>
> 09:51:01.731 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.740 - 46:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.747 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Recovered
> Connection to Broker: tcp://localhost:61616 - Ready to Process Again
> 09:51:01.773 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.773 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.774 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 09:51:01.774 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> WARNING: Uima AS Service:Aggregate Cluster Analyzer Listener Unable To
> Connect To Broker: tcp://localhost:61616 Retrying Until Successful ...
> 09:51:01.777 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Service: Aggregate Cluster Analyzer Runtime Exception
> 09:51:01.777 - 19:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> WARNING: Jms Listener Failed. Endpoint:
> ID:nelson-H81-M1-43454-1475070490428-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.InvalidDestinationException:
> Cannot use a Temporary destination from another Connection.
>
> this lines are continue to repeat in the log indefinitely. And after
> this have the same problem when send input Cas at the service
>
> 2016-09-28 9:27 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Nelson, I will investigate your use case and let you know.
> >
> > -jerry
> >
> > On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> Thank you for describe  how UIMA-AS works, it is very helpful. But my
> >> scenario is not restart the broker while the uima-as service works on
> >> input CAS
> >>
> >> my scenario is
> >> 1- Start Broker and UIMA-AS aggregate service
> >>
> >> 2- Complete Restart of the Broker  (without restart the service).
> >> UIMA-AS log show the lines that previously i said you:
> >> "Jms Listener Failed." 
> >> "The Session is closed." ...
> >> "The Consumer is closed." ...
> >>
> >> I do not see in my uima-as log the lines that you say in your test
> >> after broker is bounced:
> >> Listeners reconnect:
> >>  INFO | Successfully refreshed JMS Connection
> >>  INFO | Successfully refreshed JMS Connection
> >> In my case (uima-as 2.8.1) seems to be a pro

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Nelson, I will investigate your use case and let you know.

-jerry

On Wed, Sep 28, 2016 at 9:01 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> Thank you for describe  how UIMA-AS works, it is very helpful. But my
> scenario is not restart the broker while the uima-as service works on
> input CAS
>
> my scenario is
> 1- Start Broker and UIMA-AS aggregate service
>
> 2- Complete Restart of the Broker  (without restart the service).
> UIMA-AS log show the lines that previously i said you:
> "Jms Listener Failed." 
> "The Session is closed." ...
> "The Consumer is closed." ...
>
> I do not see in my uima-as log the lines that you say in your test
> after broker is bounced:
> Listeners reconnect:
>  INFO | Successfully refreshed JMS Connection
>  INFO | Successfully refreshed JMS Connection
> In my case (uima-as 2.8.1) seems to be a problem with reestablishing
> the connection to the broker after restart.
>
> 4- Then in this point, after the broker was restarted completely,
> uima-as client sends a CAS to UIMA-AS aggregate service. And for this
> petition UIMA-AS client receive the parent CAS, but not child CAS.
>
> In this moment i will trys with uima-as-2.9.0 snapshot
>
>
> 2016-09-27 15:33 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Nelson, in my scenario I do have a Cas Multiplier which produces a child
> > CAS for each input CAS. For the purpose of this
> > test its not important how many internal delegates there are in a
> pipeline.
> > The service is configured as a Cas Multiplier
> > and it does send child CASes when a broker is running.
> >
> > The problem is that once the input CAS is received, its reply queue is
> > bound to a specific temp reply queue. When a broker is
> > bounced, that queue is not valid anymore and cant be used to return child
> > CASes nor the input CAS. To guard against such
> > cases you need to use timeouts on the client side.
> >
> > Like I said in my previous email I dont get how an input CAS can be
> > returned back to the client after a broker bounce. This should
> > not happen since its temp queue does not exist in a new broker.
> >
> > I am still looking at this scenario as I don't like what I see in the
> > service log. I think there might be a bug in the UIMA-AS but I am
> > still investigating. Bug or not, what I described above is how UIMA-AS
> > works. You should not expect any replies from a service
> > working on a CAS that was sent through a broker that is bounced before a
> > reply.
> >
> > -jerry
> >
> >
> > On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera <nelsonriver...@gmail.com
> >
> > wrote:
> >
> >> My service  is a aggregate analysis engine that receives input CAS
> >> with the direcction of a directory of documents, the first component
> >> is a MultiplerCas that for each document generate a child CAS, each
> >> child CAS continues the flow that are other primitive annotators
> >> (primitive annotators in c++ that can throw exceptions), then exists
> >> other MultiplerCas that acts as Merger, waiting  the complete
> >> processing of each child CAS and finally generate a child CAS where
> >> each child CAS previously processed is a view of CAS, this a child CAS
> >> that is composed of view continúa el flujo a otro AE, que espera un
> >> CAS de esta manera.
> >> I will send you my uima-as descriptor,  to understand better my
> >> service. All delegates are co-located.
> >>
> >> i will trys with uima-as-2.9.0 snapshot how you say, to see how it
> >> behaves
> >>
> >> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >> > I was able to stage this scenario in which an aggregate service cas
> >> > multiplier receives a single input CAS and
> >> > produces a child CAS.
> >> >
> >> > Scenario:
> >> > 1) Service receives input CAS
> >> > 2) CasMultiplier generates a child CAS
> >> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> >> > 4) Broker is shutdown
> >> > 5) AE.process() awakes from sleep
> >> > 6) Uima-AS tries to deliver child CAS to a remote client
> >> > 7) Connection is invalid so the child CAS is dropped
> >> > 8) Broker is restarted
> >> > 9) Listeners reconnect:
> >> >  INFO | Successfully refreshed JMS Connection
> >> >  INFO | Successfully refreshed JMS Connection
> >> > 10) Broker log shows:
> >> > WARN  Service 

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Nelson, in my scenario I do have a Cas Multiplier which produces a child
CAS for each input CAS. For the purpose of this
test its not important how many internal delegates there are in a pipeline.
The service is configured as a Cas Multiplier
and it does send child CASes when a broker is running.

The problem is that once the input CAS is received, its reply queue is
bound to a specific temp reply queue. When a broker is
bounced, that queue is not valid anymore and cant be used to return child
CASes nor the input CAS. To guard against such
cases you need to use timeouts on the client side.

Like I said in my previous email I dont get how an input CAS can be
returned back to the client after a broker bounce. This should
not happen since its temp queue does not exist in a new broker.

I am still looking at this scenario as I don't like what I see in the
service log. I think there might be a bug in the UIMA-AS but I am
still investigating. Bug or not, what I described above is how UIMA-AS
works. You should not expect any replies from a service
working on a CAS that was sent through a broker that is bounced before a
reply.

-jerry


On Tue, Sep 27, 2016 at 3:18 PM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> My service  is a aggregate analysis engine that receives input CAS
> with the direcction of a directory of documents, the first component
> is a MultiplerCas that for each document generate a child CAS, each
> child CAS continues the flow that are other primitive annotators
> (primitive annotators in c++ that can throw exceptions), then exists
> other MultiplerCas that acts as Merger, waiting  the complete
> processing of each child CAS and finally generate a child CAS where
> each child CAS previously processed is a view of CAS, this a child CAS
> that is composed of view continúa el flujo a otro AE, que espera un
> CAS de esta manera.
> I will send you my uima-as descriptor,  to understand better my
> service. All delegates are co-located.
>
> i will trys with uima-as-2.9.0 snapshot how you say, to see how it behaves
>
> 2016-09-27 14:27 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > I was able to stage this scenario in which an aggregate service cas
> > multiplier receives a single input CAS and
> > produces a child CAS.
> >
> > Scenario:
> > 1) Service receives input CAS
> > 2) CasMultiplier generates a child CAS
> > 3) AE.process() is called (child CAS) and it sleeps for awhile
> > 4) Broker is shutdown
> > 5) AE.process() awakes from sleep
> > 6) Uima-AS tries to deliver child CAS to a remote client
> > 7) Connection is invalid so the child CAS is dropped
> > 8) Broker is restarted
> > 9) Listeners reconnect:
> >  INFO | Successfully refreshed JMS Connection
> >  INFO | Successfully refreshed JMS Connection
> > 10) Broker log shows:
> > WARN  Service- Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException: The destination
> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> > WARN  Service- Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException: The destination
> > temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
> >
> > The service failed to send a reply.Nothing is delivered to a client as
> > expected. This is the confusing part since you see a parent CAS being
> sent
> > to a client. I just cant imagine how this is possible.
> >
> > When a service receives an input CAS in a message, it extracts a JMS
> > replyTo object which is a JMS temp queue and stores in an internal cache
> > until it is time to send a reply. Temp queues are transient and unique
> each
> > time one is created.
> >
> > If the broker dies and subsequently restarts, the cached replyTo queue
> > object is invalid (temp queue no longer exist) which leads to
> > "Async error occurred:
> > org.apache.activemq.DestinationDoesNotExistException" as shows above.
> >
> > In the service log I see a bunch of these exceptions:
> >
> > 02:05:46.574 - 56:
> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> openChannel:
> > WARNING: Service: Test Aggregate TAE Unable to Open Connection To Broker:
> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
> > Retrying ...
> > 02:05:46.574 - 56:
> > org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.
> openChannel:
> > WARNING:
> > javax.jms.JMSException: Could not connect to broker URL:
> > tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
> > java.net.ConnectException: Connection refused
> >
> &g

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

I was able to stage this scenario in which an aggregate service cas
multiplier receives a single input CAS and
produces a child CAS.

Scenario:
1) Service receives input CAS
2) CasMultiplier generates a child CAS
3) AE.process() is called (child CAS) and it sleeps for awhile
4) Broker is shutdown
5) AE.process() awakes from sleep
6) Uima-AS tries to deliver child CAS to a remote client
7) Connection is invalid so the child CAS is dropped
8) Broker is restarted
9) Listeners reconnect:
 INFO | Successfully refreshed JMS Connection
 INFO | Successfully refreshed JMS Connection
10) Broker log shows:
WARN  Service- Async error occurred:
org.apache.activemq.DestinationDoesNotExistException: The destination
temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.
WARN  Service- Async error occurred:
org.apache.activemq.DestinationDoesNotExistException: The destination
temp-queue://ID:bluejws65-53920-1474997439574-8:1:1 does not exist.

The service failed to send a reply.Nothing is delivered to a client as
expected. This is the confusing part since you see a parent CAS being sent
to a client. I just cant imagine how this is possible.

When a service receives an input CAS in a message, it extracts a JMS
replyTo object which is a JMS temp queue and stores in an internal cache
until it is time to send a reply. Temp queues are transient and unique each
time one is created.

If the broker dies and subsequently restarts, the cached replyTo queue
object is invalid (temp queue no longer exist) which leads to
"Async error occurred:
org.apache.activemq.DestinationDoesNotExistException" as shows above.

In the service log I see a bunch of these exceptions:

02:05:46.574 - 56:
org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
WARNING: Service: Test Aggregate TAE Unable to Open Connection To Broker:
tcp://localhost:61616?wireFormat.maxInactivityDuration=0 - Silently
Retrying ...
02:05:46.574 - 56:
org.apache.uima.adapter.jms.activemq.JmsEndpointConnection_impl.openChannel:
WARNING:
javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616?wireFormat.maxInactivityDuration=0. Reason:
java.net.ConnectException: Connection refused

This is *after* the service is restarted so there seems to be a problem
with reestablishing the connection to a new broker.
I am still investigating that part. I just am confused how you are able to
see a reply in the client.

-Jerry


On Tue, Sep 27, 2016 at 1:13 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> From what I understand there are two issues:
>
> 1) Why service JMS listeners are not reconnecting when a broker is
> bounced. This
> should be done by default. The 2.9.0 snapshot may address this since a
> listener
> uses newer AMQ jars (5.14.0)
>
> 2) What happens to a child CAS when a Broker disappears and the service
> wants
> to deliver it to a client that is not reachable. I need to investigate
> this part. I think
> such CAS is dropped but that should be reflected in a log and also the
> input CAS
> should be marked as failed. Again something for me to investigate.
>
> I will try to set up a test to see if I can create the scenario.It will
> take me a bit of time
> to set this up but I will let you know what happens.
>
> -jerry
>
> On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <uim...@gmail.com>
> wrote:
>
>> Here is the uima-as 2.9.0-SNAPSHOT to try:
>> https://repository.apache.org/content/repositories/snapshots
>> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-201609
>> 23.195233-6.tar.gz
>>
>> Can you briefly describe your service? Does your service have any remote
>> delegates or all parts of the pipeline are in the same process?
>> Do you see and exceptions in the service log?
>>
>> -jerry
>>
>> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <nelsonriver...@gmail.com>
>> wrote:
>>
>>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>>> appears to be broker
>>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
>>> work
>>>
>>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
>>> > Can you install AMQ broker 5.14.0 to see if this is related to broker
>>> > issues.
>>> >
>>> > -jerry
>>> >
>>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>>> nelsonriver...@gmail.com>
>>> > wrote:
>>> >
>>> >> just like that, the parent CAS comes back but the child CAS doesnt.
>>> >>
>>> >> And the child CAS if processed, because  the  uima-as log shows the
>>> >> lines of th

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

>From what I understand there are two issues:

1) Why service JMS listeners are not reconnecting when a broker is bounced.
This
should be done by default. The 2.9.0 snapshot may address this since a
listener
uses newer AMQ jars (5.14.0)

2) What happens to a child CAS when a Broker disappears and the service
wants
to deliver it to a client that is not reachable. I need to investigate
this part. I think
such CAS is dropped but that should be reflected in a log and also the
input CAS
should be marked as failed. Again something for me to investigate.

I will try to set up a test to see if I can create the scenario.It will
take me a bit of time
to set this up but I will let you know what happens.

-jerry

On Tue, Sep 27, 2016 at 12:42 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Here is the uima-as 2.9.0-SNAPSHOT to try:
> https://repository.apache.org/content/repositories/snapshots
> /org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-
> 20160923.195233-6.tar.gz
>
> Can you briefly describe your service? Does your service have any remote
> delegates or all parts of the pipeline are in the same process?
> Do you see and exceptions in the service log?
>
> -jerry
>
> On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> I was testing with AMQ broker 5.14.0 and i get the same error. not
>> appears to be broker
>> issues, appears to be uima-as issues. Any other idea, i am stagnant at
>> work
>>
>> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
>> > Can you install AMQ broker 5.14.0 to see if this is related to broker
>> > issues.
>> >
>> > -jerry
>> >
>> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
>> nelsonriver...@gmail.com>
>> > wrote:
>> >
>> >> just like that, the parent CAS comes back but the child CAS doesnt.
>> >>
>> >> And the child CAS if processed, because  the  uima-as log shows the
>> >> lines of the other annotators primitive of the flow processing the
>> >> child CAS, but not comes back to UIMA-AS client.
>> >>
>> >> uima-as log only shows
>> >> "Jms Listener Failed. Endpoint:
>> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
>> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
>> >> java.io.EOFException" 
>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
>> >> Managed By: tcp://localhost:61616 Reason:
>> >> javax.jms.IllegalStateException: The Consumer is closed..
>> >>
>> >> The CAS is send to service after restart the Broker, and how say
>> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
>> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
>> >> Remaining Number of CASes to Process:1".  I suppose that is the child
>> >> CAS.
>> >>
>> >> the client if can send more CASes to the service, but not receive
>> >> response, for each CAS send after the one CAS that was send after
>> >> restart broker (for which  parent CAS comes back but the child CAS
>> >> doesnt)  not comes back parent CAS or child CAS.
>> >>
>> >> More specific:
>> >> scenario 1:
>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>> >>
>> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
>> >> "Jms Listener Failed." 
>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> >> comes back but the child CAS doesnt, to the uima-as client.)
>> >> If i try quiesce the Aggregate Service console shows : "Quiescing
>> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
>> >>
>> >> scenario 2:
>> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>> >>
>> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
>> >> "Jms Listener Failed." 
>> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
>> >> comes back but the child CAS doesnt, to the uima-as client)
>> >> 4- Client  sends another CAS to UIMA-AS aggregate service.(not comes
>> >> back parent CAS or child CAS , to the uima-as client)
>> >> 5- Client  sends another CAS to UIMA-AS aggregate service.(not comes
>> >> back parent CAS or child CAS , to the 

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Here is the uima-as 2.9.0-SNAPSHOT to try:
https://repository.apache.org/content/repositories/
snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-
as-2.9.0-20160923.195233-6.tar.gz

Can you briefly describe your service? Does your service have any remote
delegates or all parts of the pipeline are in the same process?
Do you see and exceptions in the service log?

-jerry

On Tue, Sep 27, 2016 at 9:07 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> I was testing with AMQ broker 5.14.0 and i get the same error. not
> appears to be broker
> issues, appears to be uima-as issues. Any other idea, i am stagnant at work
>
> 2016-09-22 13:04 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > Can you install AMQ broker 5.14.0 to see if this is related to broker
> > issues.
> >
> > -jerry
> >
> > On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <
> nelsonriver...@gmail.com>
> > wrote:
> >
> >> just like that, the parent CAS comes back but the child CAS doesnt.
> >>
> >> And the child CAS if processed, because  the  uima-as log shows the
> >> lines of the other annotators primitive of the flow processing the
> >> child CAS, but not comes back to UIMA-AS client.
> >>
> >> uima-as log only shows
> >> "Jms Listener Failed. Endpoint:
> >> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
> >> tcp://localhost:61616 Reason: javax.jms.JMSException:
> >> java.io.EOFException" 
> >> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> >> Managed By: tcp://localhost:61616 Reason:
> >> javax.jms.IllegalStateException: The Consumer is closed..
> >>
> >> The CAS is send to service after restart the Broker, and how say
> >> parent CAS comes back but the child CAS doesnt. And if i try quiesce
> >> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> >> Remaining Number of CASes to Process:1".  I suppose that is the child
> >> CAS.
> >>
> >> the client if can send more CASes to the service, but not receive
> >> response, for each CAS send after the one CAS that was send after
> >> restart broker (for which  parent CAS comes back but the child CAS
> >> doesnt)  not comes back parent CAS or child CAS.
> >>
> >> More specific:
> >> scenario 1:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> >> "Jms Listener Failed." 
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client.)
> >> If i try quiesce the Aggregate Service console shows : "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:1"
> >>
> >> scenario 2:
> >> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
> >>
> >> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> >> "Jms Listener Failed." 
> >> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> >> comes back but the child CAS doesnt, to the uima-as client)
> >> 4- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >> 5- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> >> back parent CAS or child CAS , to the uima-as client)
> >>  If i try quiesce the Aggregate Service  console shows: "Quiescing
> >> UIMA-AS Service. Remaining Number of CASes to Process:5"
> >>
> >>
> >>
> >>
> >>
> >> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >> > When a parent CAS comes back (after broker is restored), can the
> client
> >> > send more CASes to the service and receive responses? I am trying to
> >> > determine what the real problem is here.
> >> >
> >> > 1) JMS Listener not recovering connection
> >> > 2) UIMA-AS is not handling CAS reply failures when broker connection
> is
> >> > stale
> >> >
> >> > -jerry
> >> >
> >> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <uim...@gmail.com>
> >> > wrote:
> >> >
> >> >> Strange. You say that the parent CAS comes back but the child CAS
> >> doesnt.
> >> >> Are there any exceptions i

Re: Debugging a NullPointerException in UIMA AS / processing timeouts

[Jaroslaw Cwiklik]

Egbert, the UIMA-AS only supports timeouts on Remotes. You could deploy the
annotator in question as a remote
and define a process timeout on it. You would get a timeout on that
component and a log will show it. The
UIMA-AS does not support timeouts on co-located (in-process) parts of the
pipeline.

To eliminate UIMA-AS as a cause of the NPE, can you define a CasConsumer at
the end of your pipeline and write
out a CAS to disk, and in the reply just send a filename to that serialized
CAS?


>From what I understand so far your service can serialize a CAS in a reply,
but the UIMA-AS client fails to
deserialize it. Not sure if this is caused by a race condition where one
thread begins to deserialize a reply
while another is handling a timeout which resets a CAS.

-jerry





On Mon, Sep 26, 2016 at 6:01 AM, Egbert van der Wal <e...@pointpro.nl>
wrote:

> Hi again,
>
> After some days of testing and debugging I've been able to narrow down the
> problem somewhat.
>
> It turned out that, after all, it was reproducable by feeding a certain
> document. After attemping to disable the enabled annotators one by one, I
> was able to localize the problem to a specific annotator, which, when,
> disabled, 'fixed' the issue. I still don't know exactly what is causing it
> within this annotator, but now I've got a lead so I should be able to fix
> it.
>
> However, the FINEST loglevel doesn't really give any useful information.
> It shows the current waiting queue of CAS ID's that are in the pipeline.
> However, it still doesn't give me a useful error message, it's still an
> exception somewhere in the internals of UIMA, either a
> 'NullPointerException' on
>
> org.apache.uima.cas.impl.CASImpl.ll_getFSForRef(CASImpl.java:3478)
>
> or a a 'IndexOutOfBoundsException' on:
>
> org.apache.uima.cas.impl.XmiCasDeserializer$XmiCasDeserializ
> erHandler.getIndexRepo(XmiCasDeserializer.java:628)
>
>
> It seems both are related to a faulty annotator choking on a too large
> payload and not returning results in time.
>
> However, in this situation I would expect a exception that the annotator
> doesn't respond in time, rather than these messages, so that I would know
> where to look right away. Have I misconfigured some debugging / pipeline
> settings or is this something for the UIMA wishlist?
>
> Thanks again!
>
> Regards,
>
> Egbert
>
>
>
>
>
>
> On 29-08-16 15:28, Jaroslaw Cwiklik wrote:
>
>> Egbert, thanks. I forgot to ask, what version of UIMA-AS are you using?
>> Also, are you using sendCAS() or sendAndReceive() API?
>>
>> Have a great vacation!
>>
>> -jerry
>>
>> On Sun, Aug 28, 2016 at 9:39 AM, Egbert van der Wal <e...@pointpro.nl>
>> wrote:
>>
>> Hi Jerry,
>>>
>>> Thanks for the suggestion. I have the feeling that it's a race condition,
>>> too, but since I'm doing no multi-threading myself, basically all the
>>> threading and synchronization should be UIMA-internal.
>>>
>>> Anyway, I'll have to postpone researching the issue due to going on
>>> vacation. When I get back I'll try to get more information with a
>>> increased
>>> log level, and get back to you.
>>>
>>> Thanks again!
>>>
>>> Regards,
>>>
>>> Egbert
>>>
>>>
>>> Op 25-8-2016 om 17:17 schreef Jaroslaw Cwiklik:
>>>
>>> Hi, I have a feeling that there might be a race condition here. In the
>>>
>>>> client, the timer pops and at the same time a reply is received.
>>>> The timout logic is resetting the CAS while its being deserialized which
>>>> may lead to NPE. Not 100% certain but this might be the problem.
>>>>
>>>> Any chance you can increase UIMA log level to FINEST on the client side?
>>>> It
>>>> would log important information like the internal CAS ID  on each reply
>>>> which can be used to correlate events in the log.
>>>>
>>>> -jerry
>>>>
>>>> On Thu, Aug 25, 2016 at 10:18 AM, Egbert van der Wal <e...@pointpro.nl>
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>>
>>>>> I'm having a problem using UIMA-AS. I have a pipeline set up that
>>>>> processes HTML documents in ~= 10 ms. The total time out value was
>>>>> initially 20 seconds, but I increased it to 120 ms at some point to
>>>>> avoid
>>>>> this problem, it seemed to help.
>>>>>
>>>>> However, sometimes the 2 minutes is still hit and a warning is shown.
>>>>> When
>>>>> t

Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)

[Jaroslaw Cwiklik]

David, here is the next snapshot:

https://repository.apache.org/content/repositories/snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/uima-as-2.9.0-20160923.195233-6.tar.gz


-jerry


On Fri, Sep 23, 2016 at 3:35 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> David, like I said not my code so not quite in-tune what this does.
>
> I've added this to the snapshot you are using:
>
>
>   synchronized public String quiesceAndStop() throws IOException {
> StringBuffer sb = new StringBuffer();
>
>  if (socket != null && !socket.isClosed() ) {
>
>
> In the above I am checking to see if the socket is open before taking an
> action. Since the socket
> is closed, the else block is executing. My fault for not noticing this. It
> looks like quiesceAndStop()
> is called twice. The last time from a ShutdownHook which triggers an
> exception and dumps the stack.
>
> I will remove the
>
>  throw new IOException("Error: no socket connection.");
>
> from the else block in quiesceAndStop().
>
> Hopefully this will be the last bug.
>
> Will ping you when I have the next snapshot ready.
>
> -jerry
>
> On Fri, Sep 23, 2016 at 3:15 PM, David Fox <david@humedica.com> wrote:
>
>> That’s odd… now I get
>>
>> java.io.IOException: Error: no socket connection.
>> at
>> org.apache.uima.aae.controller.UimacppServiceManagement.quie
>> sceAndStop(Uima
>> cppServiceManagement.java:109)
>> at
>> org.apache.uima.aae.controller.UimacppServiceManagement.quie
>> sceAndShutdown(
>> UimacppServiceManagement.java:409)
>> at
>> org.apache.uima.aae.controller.UimacppShutdownHook.run(Uimac
>> ppServiceContro
>> ller.java:1105)
>> UimacppShutdownHook destroy C++ process
>>
>> I can’t tell for sure, because the source code in trunk doesn’t match this
>> latest snapshot, but it looks as if I’m now hitting the else clause from
>> the if statement containing the code which was generating the previous
>> exception.  Full output below:
>>
>>
>> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ...
>> Waiting for Uima C++ service to connect...
>> Waiting for Uima C++ service to connect...
>> Waiting for Uima C++ service to report init status...
>> deployCppService.cpp91 Start receiving messages
>>
>> Startinging GetMetaData instance
>>
>> Starting Annotator instance 0
>>
>> deployCppService.cpp UIMA C++ Service MeetingAnnotator at
>> tcp://localhost:61616 Ready to process...
>>
>> Uima C++ service at MeetingAnnotator Ready to process...
>> UIMA AS Service Initialization Complete
>> Completed 8 documents; 17165 characters
>> Time Elapsed : 6262 ms
>> 0 8 apr_socket_recv command=SHUTDOWN
>>
>> deployCppService.cpp Shutdown started.
>>
>> Stopping GetMetaData instance
>>
>> Stopping Annotator instance 0
>>
>> deployCppService.cpp Shutdown done.
>>
>> UimacppServiceManagement service reports shutdown DONE
>> Error no connection
>> UimacppShutdownHook sending quiesce message
>> java.io.IOException: Error: no socket connection.
>> at
>> org.apache.uima.aae.controller.UimacppServiceManagement.quie
>> sceAndStop(Uima
>> cppServiceManagement.java:109)
>> at
>> org.apache.uima.aae.controller.UimacppServiceManagement.quie
>> sceAndShutdown(
>> UimacppServiceManagement.java:409)
>> at
>> org.apache.uima.aae.controller.UimacppShutdownHook.run(Uimac
>> ppServiceContro
>> ller.java:1105)
>> UimacppShutdownHook destroy C++ process
>>
>>
>>
>>
>> On 9/23/16, 11:31 AM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote:
>>
>> >David, here is another uima-as snapshot for you to test
>> >
>> >https://urldefense.proofpoint.com/v2/url?u=https-3A__
>> repository.apache.org
>> >_content_repositories_snapshots_org_apache_uima_uima-2Das_
>> 2.9.0-2DSNAPSHOT
>> >_uima-2Das-2D2.9.0-2D20160923.152659-2D5.tar.gz=DQIFaQ=
>> 3XrKki35ZWuh8X2
>> >qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=tXj
>> 0BK0pkaseF8BQPk
>> >gw2nEjl9UxDSq736aXsJcDJ0g=iB7ah2MjAK4kl3Ops3aKk9yYcmxk19I
>> cBDuWHgkGN5U=
>> >
>> >
>> >I have not tested the fix. All I did was to modify quiesceAndStop() and
>> >terminate() to catch SocketException and log a message. This code no
>> >longer dumps a stack trace nor re-throws an exception.
>> >
>> >Let me know if this addresses the problem and I will close the JIRA
>> >

Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)

[Jaroslaw Cwiklik]

David, like I said not my code so not quite in-tune what this does.

I've added this to the snapshot you are using:


  synchronized public String quiesceAndStop() throws IOException {
StringBuffer sb = new StringBuffer();

 if (socket != null && !socket.isClosed() ) {


In the above I am checking to see if the socket is open before taking an
action. Since the socket
is closed, the else block is executing. My fault for not noticing this. It
looks like quiesceAndStop()
is called twice. The last time from a ShutdownHook which triggers an
exception and dumps the stack.

I will remove the

 throw new IOException("Error: no socket connection.");

from the else block in quiesceAndStop().

Hopefully this will be the last bug.

Will ping you when I have the next snapshot ready.

-jerry

On Fri, Sep 23, 2016 at 3:15 PM, David Fox <david@humedica.com> wrote:

> That’s odd… now I get
>
> java.io.IOException: Error: no socket connection.
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndStop(Uima
> cppServiceManagement.java:109)
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndShutdown(
> UimacppServiceManagement.java:409)
> at
> org.apache.uima.aae.controller.UimacppShutdownHook.run(
> UimacppServiceContro
> ller.java:1105)
> UimacppShutdownHook destroy C++ process
>
> I can’t tell for sure, because the source code in trunk doesn’t match this
> latest snapshot, but it looks as if I’m now hitting the else clause from
> the if statement containing the code which was generating the previous
> exception.  Full output below:
>
>
> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ...
> Waiting for Uima C++ service to connect...
> Waiting for Uima C++ service to connect...
> Waiting for Uima C++ service to report init status...
> deployCppService.cpp91 Start receiving messages
>
> Startinging GetMetaData instance
>
> Starting Annotator instance 0
>
> deployCppService.cpp UIMA C++ Service MeetingAnnotator at
> tcp://localhost:61616 Ready to process...
>
> Uima C++ service at MeetingAnnotator Ready to process...
> UIMA AS Service Initialization Complete
> Completed 8 documents; 17165 characters
> Time Elapsed : 6262 ms
> 0 8 apr_socket_recv command=SHUTDOWN
>
> deployCppService.cpp Shutdown started.
>
> Stopping GetMetaData instance
>
> Stopping Annotator instance 0
>
> deployCppService.cpp Shutdown done.
>
> UimacppServiceManagement service reports shutdown DONE
> Error no connection
> UimacppShutdownHook sending quiesce message
> java.io.IOException: Error: no socket connection.
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndStop(Uima
> cppServiceManagement.java:109)
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndShutdown(
> UimacppServiceManagement.java:409)
> at
> org.apache.uima.aae.controller.UimacppShutdownHook.run(
> UimacppServiceContro
> ller.java:1105)
> UimacppShutdownHook destroy C++ process
>
>
>
>
> On 9/23/16, 11:31 AM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote:
>
> >David, here is another uima-as snapshot for you to test
> >
> >https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__repository.apache.org
> >_content_repositories_snapshots_org_apache_uima_
> uima-2Das_2.9.0-2DSNAPSHOT
> >_uima-2Das-2D2.9.0-2D20160923.152659-2D5.tar.gz&
> d=DQIFaQ=3XrKki35ZWuh8X2
> >qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-WQSvBt9z6TEbaT3nnNM=
> tXj0BK0pkaseF8BQPk
> >gw2nEjl9UxDSq736aXsJcDJ0g=iB7ah2MjAK4kl3Ops3aKk9yYcmxk19
> IcBDuWHgkGN5U=
> >
> >
> >I have not tested the fix. All I did was to modify quiesceAndStop() and
> >terminate() to catch SocketException and log a message. This code no
> >longer dumps a stack trace nor re-throws an exception.
> >
> >Let me know if this addresses the problem and I will close the JIRA
> >
> >-jerry
> >
> >On Fri, Sep 23, 2016 at 10:24 AM, Jaroslaw Cwiklik <uim...@gmail.com>
> >wrote:
> >
> >> David, so there is progress. :)
> >>
> >> I am not familiar with UimacppServiceController code but a quick glance
> >>at
> >> the source
> >> suggests that this code communicates with a C++ process via a socket:
> >>
> >>
> >>  if (socket != null) {
> >> // System.out.println("UimacppServiceManagement::quiesceAndStop()
> >> // Sending QUIESCEANDSTOP");
> >> writer.write("QUIESCEANDSTOP");
> >> writer.flush();
> >> BufferedReader in = new BufferedReader(new InputStreamReader(socket

Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)

[Jaroslaw Cwiklik]

David, so there is progress. :)

I am not familiar with UimacppServiceController code but a quick glance at
the source
suggests that this code communicates with a C++ process via a socket:


 if (socket != null) {
// System.out.println("UimacppServiceManagement::quiesceAndStop()
// Sending QUIESCEANDSTOP");
writer.write("QUIESCEANDSTOP");
writer.flush();
BufferedReader in = new BufferedReader(new
InputStreamReader(socket.getInputStream()));

It tells the C++ to quiesce and exit which apparently it does very quickly.
It looks like

writer.write("QUIESCEANDSTOP");

succeeded, but the code fails here

BufferedReader in = new BufferedReader(new
InputStreamReader(socket.getInputStream()));
which means that the C++ socket is dead (process exited)

It seems that the code should tolerate such condition and not "scream"
Exception :)

I think a try catch should be added around this code and on SocketException
we should just log some informative
message like "Remote process socket closed - It may have terminated" or
something similar to that.

So this does not seem like a critical error although this should be cleaned
up.

-jerry

On Fri, Sep 23, 2016 at 9:41 AM, David Fox <david@humedica.com> wrote:

> I was able to install that, and I’m not getting the NullPointerException.
> The example annotator still runs successfully, but now I get
> 
> ---
>
> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ...
> Waiting for Uima C++ service to connect...
> Waiting for Uima C++ service to connect...
> Waiting for Uima C++ service to report init status...
> deployCppService.cpp91 Start receiving messages
>
> Startinging GetMetaData instance
>
> Starting Annotator instance 0
>
> deployCppService.cpp UIMA C++ Service MeetingAnnotator at
> tcp://localhost:61616 Ready to process...
>
> Uima C++ service at MeetingAnnotator Ready to process...
> UIMA AS Service Initialization Complete
> Completed 8 documents; 17165 characters
> Time Elapsed : 3705 ms
> 0 8 apr_socket_recv command=SHUTDOWN
>
> deployCppService.cpp Shutdown started.
>
> Stopping GetMetaData instance
>
> Stopping Annotator instance 0
>
> deployCppService.cpp Shutdown done.
>
> UimacppServiceManagement service reports shutdown DONE
> UimacppShutdownHook sending quiesce message
> java.net.SocketException: Socket is closed
> at java.net.Socket.getInputStream(Socket.java:903)
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndStop(Uima
> cppServiceManagement.java:88)
> at
> org.apache.uima.aae.controller.UimacppServiceManagement.
> quiesceAndShutdown(
> UimacppServiceManagement.java:397)
> at
> org.apache.uima.aae.controller.UimacppShutdownHook.run(
> UimacppServiceContro
> ller.java:1105)
> UimacppShutdownHook destroy C++ process
>
> ―
>
> I’ve attached the uima.log
>
> On 9/22/16, 3:54 PM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote:
>
> >David, I've deployed uima-as 2.9.0-SNAPSHOT here:
> >
> >https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__repository.apache.org
> >_content_repositories_snapshots_org_apache_uima_
> uima-2Das_2.9.0-2DSNAPSHOT
> >_=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-
> WQSvBt9z6TEb
> >aT3nnNM=NnQy2jfAk4Dyg-uijgdjHHbg6pxfdvwDkCn3166fVc8&
> s=y7GSTfzyEzhkYgFGy1
> >vLLmDJ82il_hzMP7Y4LQjuBPg=
> >
> >there is a binary tarball (tar.gz) in there.
> >
> >This is a trunk level uima-as code for the upcoming 2.9.0 uima-as release.
> >It has not been thoroughly
> >tested or vetted yet.  It has some fixes and improvements since 2.8.1. It
> >includes a new AMQ broker
> >version 5.14.0. It also has a fix for the problem you've reported.
> >
> >-jerry
> >
> >
> >On Thu, Sep 22, 2016 at 3:15 PM, Jaroslaw Cwiklik <uim...@gmail.com>
> >wrote:
> >
> >> Another possibility is to deploy a snapshot in a public maven repo.
> >> Hopefully you can get the binary tarball
> >> onto your machines somehow. I will to do that know and let you know
> >>where
> >> to get this.
> >>
> >> -jerry
> >>
> >> On Thu, Sep 22, 2016 at 3:01 PM, David Fox <david@humedica.com>
> >>wrote:
> >>
> >>> Building from svn on our linux machines is a bit tricky, because they
> >>>are
> >>> cut off from the internet by a firewall, and maven install wants
> >>>internet
> >>> access to download dependencies.  It looks like there are workarounds
> &g

Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)

[Jaroslaw Cwiklik]

David, I've deployed uima-as 2.9.0-SNAPSHOT here:

https://repository.apache.org/content/repositories/snapshots/org/apache/uima/uima-as/2.9.0-SNAPSHOT/

there is a binary tarball (tar.gz) in there.

This is a trunk level uima-as code for the upcoming 2.9.0 uima-as release.
It has not been thoroughly
tested or vetted yet.  It has some fixes and improvements since 2.8.1. It
includes a new AMQ broker
version 5.14.0. It also has a fix for the problem you've reported.

-jerry


On Thu, Sep 22, 2016 at 3:15 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Another possibility is to deploy a snapshot in a public maven repo.
> Hopefully you can get the binary tarball
> onto your machines somehow. I will to do that know and let you know where
> to get this.
>
> -jerry
>
> On Thu, Sep 22, 2016 at 3:01 PM, David Fox <david@humedica.com> wrote:
>
>> Building from svn on our linux machines is a bit tricky, because they are
>> cut off from the internet by a firewall, and maven install wants internet
>> access to download dependencies.  It looks like there are workarounds for
>> that
>> (http://stackoverflow.com/questions/7233328/how-do-i-configu
>> re-maven-for-of
>> fline-development
>> <http://stackoverflow.com/questions/7233328/how-do-i-configure-maven-for-offline-development>
>> suggests Artifactory), but it may take a day or two
>> before I can get everything set up.
>>
>> I could build it on my dev machine (OS X), but with the binary install, I
>> wasn’t able to get the sample UIMA C++ code to build on Mac OS X.  (My
>> eventual plan is to get linux on my dev machine via virtual box, but that
>> means more software to install…)
>>
>> So anyway, I’ll try to get this working so I can build from svn before the
>> fix, reproduce the problem, and then update and verify the fix.
>>
>> Thanks,
>> David
>>
>> On 9/22/16, 1:23 PM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote:
>>
>> >I think I see the bug. Do you know how to build uima-as from svn?
>> >
>> >https://urldefense.proofpoint.com/v2/url?u=https-3A__uima.
>> apache.org_build
>> >ing-2Duima.html=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K
>> 6Famz8NiMJzvOgY
>> >A-WQSvBt9z6TEbaT3nnNM=9Hf7csP-1jIlxx_vgB3QA47yh0w3i1-
>> XbVER8KQmhT8=Gin5
>> >am-lATmbl6WQFmZrKwjxKa3tiL1qz1ykFh0cdWI=
>> >
>> >I will fix this shortly in the svn trunk. The next release of uima-as
>> >(2.9.0) is in the
>> >works. I will get a release candidate out hopefully next week.
>> >
>> >-jerry
>> >
>> >On Thu, Sep 22, 2016 at 12:08 PM, David Fox <david@humedica.com>
>> >wrote:
>> >
>> >> Thanks for the quick response
>> >>
>> >> Yes, it occurs every time, at the same line in RunRemoteAsyncAE and
>> >> org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_
>> >> impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:964)
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On 9/22/16, 11:15 AM, "Jaroslaw Cwiklik" <uim...@gmail.com> wrote:
>> >>
>> >> >This warning
>> >> >
>> >> >WARN  Transport  - Transport Connection to: tcp://
>> >> >127.0.0.1:56261 failed: java.io.EOFException
>> >> >
>> >> >is from brokers log or console I suspect. It just means that the
>> broker
>> >> >lost a connection to a client. Looks like the client
>> >> >went away without closing a connection. This is probably a side-effect
>> >>of
>> >> >NPE.
>> >> >
>> >> >Is the NPE repeatable? Does it happen each time you run this test?
>> >>Looking
>> >> >for a hint if there is a race
>> >> >condition in the UIMA-AS code.
>> >> >
>> >> >
>> >> >
>> >> >On Thu, Sep 22, 2016 at 10:54 AM, David Fox <david@humedica.com>
>> >> >wrote:
>> >> >
>> >> >> I¹m using the UIMA C++ framework 2.4.0 (32-bit) on linux (CentOS
>> >>6.5),
>> >> >>and
>> >> >> following the instructions
>> >> >>https://urldefense.proofpoint.com/v2/url?u=https-
>> >> 3A__uima.apache.org_d_
>> >> >>=DQIFaQ=3XrKki35ZWuh8X2qbeRISQ=BYS7q6K6Famz8NiMJzvOgYA-
>> >> WQSvBt9z6TEbaT
>> >> >>3nnNM=dFZSJENJkyCDLeCQuBn89fBrTrsdKh4RIWwwPV7r0o8=CaIoLZ_7Je-
>> >> TYsr0ZIP
>>

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Can you install AMQ broker 5.14.0 to see if this is related to broker
issues.

-jerry

On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <nelsonriver...@gmail.com>
wrote:

> just like that, the parent CAS comes back but the child CAS doesnt.
>
> And the child CAS if processed, because  the  uima-as log shows the
> lines of the other annotators primitive of the flow processing the
> child CAS, but not comes back to UIMA-AS client.
>
> uima-as log only shows
> "Jms Listener Failed. Endpoint:
> temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException" 
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed..
>
> The CAS is send to service after restart the Broker, and how say
> parent CAS comes back but the child CAS doesnt. And if i try quiesce
> the Aggregate Service console shows : "Quiescing UIMA-AS Service.
> Remaining Number of CASes to Process:1".  I suppose that is the child
> CAS.
>
> the client if can send more CASes to the service, but not receive
> response, for each CAS send after the one CAS that was send after
> restart broker (for which  parent CAS comes back but the child CAS
> doesnt)  not comes back parent CAS or child CAS.
>
> More specific:
> scenario 1:
> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>
> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> "Jms Listener Failed." 
> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> comes back but the child CAS doesnt, to the uima-as client.)
> If i try quiesce the Aggregate Service console shows : "Quiescing
> UIMA-AS Service. Remaining Number of CASes to Process:1"
>
> scenario 2:
> 1- Start Broker and UIMA-AS aggregate service. Everything is fine.
>
> 2- Restart Broker (without restart the service).  UIMA-AS log show  :
> "Jms Listener Failed." 
> 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS
> comes back but the child CAS doesnt, to the uima-as client)
> 4- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> back parent CAS or child CAS , to the uima-as client)
> 5- Client  sends another CAS to UIMA-AS aggregate service.(not comes
> back parent CAS or child CAS , to the uima-as client)
>  If i try quiesce the Aggregate Service  console shows: "Quiescing
> UIMA-AS Service. Remaining Number of CASes to Process:5"
>
>
>
>
>
> 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> > When a parent CAS comes back (after broker is restored), can the client
> > send more CASes to the service and receive responses? I am trying to
> > determine what the real problem is here.
> >
> > 1) JMS Listener not recovering connection
> > 2) UIMA-AS is not handling CAS reply failures when broker connection is
> > stale
> >
> > -jerry
> >
> > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <uim...@gmail.com>
> > wrote:
> >
> >> Strange. You say that the parent CAS comes back but the child CAS
> doesnt.
> >> Are there any exceptions in the service log? Perhaps the service is
> >> unable
> >> to send the child CAS due to missing broker. There should be something
> in
> >> the uima-as log about this.
> >>
> >> -jerry
> >>
> >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera
> >> <nelsonriver...@gmail.com>
> >> wrote:
> >>
> >>> Yes, it is supposed to do the service, given a
> >>> CAS it produces a child CAS and and this CAS continue in the flow,
> >>> that are other primitive annotators. And I expected that the child CAS
> >>> will sent to  UIMA-AS client, but not do.
> >>>
> >>> the scenario:
> >>> 1- Start Broker and UIMA-AS aggregate service
> >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
> >>> everything is fine. The process is correct.
> >>>
> >>> 3- Restart Broker (without restart the service).  UIMA-AS log show  :
> >>> "Jms Listener Failed." 
> >>> 4- Client sends a CAS to UIMA-AS aggregate service
> >>>
> >>> UIMA-AS client receive the parent CAS, but not child CAS.
> >>>
> >>> Thank  for you quick reply
> >>>
> >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> >>> > Can you 

Re: NullPointerException when RunRemoteAsyncAE tries to unload (UIMA AS and UIMA C++)

[Jaroslaw Cwiklik]

This warning

WARN  Transport  - Transport Connection to: tcp://
127.0.0.1:56261 failed: java.io.EOFException

is from brokers log or console I suspect. It just means that the broker
lost a connection to a client. Looks like the client
went away without closing a connection. This is probably a side-effect of
NPE.

Is the NPE repeatable? Does it happen each time you run this test? Looking
for a hint if there is a race
condition in the UIMA-AS code.



On Thu, Sep 22, 2016 at 10:54 AM, David Fox  wrote:

> I’m using the UIMA C++ framework 2.4.0 (32-bit) on linux (CentOS 6.5), and
> following the instructions  https://uima.apache.org/d/
> uimacpp-2.4.0/docs/overview_and_setup.html.  I’ve successfully built
> DaveDetector and run it with the standalone C++ pipeline (runAECpp) and
> within a Java pipeline via JNI.  I’ve now installed UIMA AS 2.8.1 so I can
> follow the instructions under 2.3 Testing Interoperability with UIMA AS
>
>
> cd $UIMACPP_HOME/examples/tutorial
> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator \
> -d descriptors/Deploy_MeetingAnnotator.xml
>
>
> I get a NullPointerException and a transport warning about a
> java.io.EOFException
>
>
> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator -d
> descriptors/Deploy_MeetingAnnotator.xml -o `pwd`/out
>
> Attempting to deploy descriptors/Deploy_MeetingAnnotator.xml ...
>
> Waiting for Uima C++ service to connect...
>
> Waiting for Uima C++ service to connect...
>
> Waiting for Uima C++ service to report init status...
>
> deployCppService.cpp91 Start receiving messages
>
>
> Startinging GetMetaData instance
>
>
> Starting Annotator instance 0
>
>
> deployCppService.cpp UIMA C++ Service MeetingAnnotator at
> tcp://localhost:61616 Ready to process...
>
>
> Uima C++ service at MeetingAnnotator Ready to process...
>
> UIMA AS Service Initialization Complete
>
> .Completed 1 documents
>
> Time Elapsed : 2184 ms
>
> java.lang.NullPointerException
>
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_
> impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:964)
>
> at org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngine_
> impl.undeploy(BaseUIMAAsynchronousEngine_impl.java:906)
>
> at org.apache.uima.examples.as.RunRemoteAsyncAE.run(
> RunRemoteAsyncAE.java:260)
>
> at org.apache.uima.examples.as.RunRemoteAsyncAE.main(
> RunRemoteAsyncAE.java:314)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:497)
>
> at org.apache.uima.bootstrap.UimaBootstrap.main(
> UimaBootstrap.java:87)
>
> WARN  Transport  - Transport Connection to: tcp://
> 127.0.0.1:56221 failed: java.io.EOFException
>
> I'm
> I can also run with a collection reader:
>
>
> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingAnnotator -d
> descriptors/Deploy_MeetingAnnotator.xml -o `pwd`/out -c
> /misc/NLP/dfox/scratch/uima/apache-uima-as-2.8.1/examples/
> descriptors/collection_reader/FileSystemCollectionReader.mine.xml
>
> (where the modified collection reader points to $UIMA_HOME/examples/data)
> in which case I get “Completed 8 documents”, and xmi output for each in the
> out/ directory, but still get the same exception and warning.  Based on
> that (plus looking at the RunRemoteAsyncAE java source), it looks like my
> analysis engine is running fine, but then hitting some problem when
> RunRemoteAsyncAE tries to undeploy the C++ annotator.
>
> I’m trying to figure out if this is something I should worry about
> (especially if it reflects a problem with running UIMA C++ 2.4 and UIMA AS
> 2.8.1), or if it is normal or harmless.
>
> When I run the similar test with one of the Java-only UIMA AS samples:
>
>
> runRemoteAsyncAE.sh tcp://localhost:61616 MeetingDetectorTaeQueue -d
> deploy/as/Deploy_MeetingDetectorTAE.xml -c descriptors/collection_reader/
> FileSystemCollectionReader.mine.xml
>
> I still get the transport warning
>
>
> WARN  Transport  - Transport Connection to: tcp://
> 127.0.0.1:56261 failed: java.io.EOFException
>
> but not the NullPointerException
>
> Thanks,
> David
>
>
>
>
>
>

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

When a parent CAS comes back (after broker is restored), can the client
send more CASes to the service and receive responses? I am trying to
determine what the real problem is here.

1) JMS Listener not recovering connection
2) UIMA-AS is not handling CAS reply failures when broker connection is
stale

-jerry

On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:

> Strange. You say that the parent CAS comes back but the child CAS doesnt.
> Are there any exceptions in the service log? Perhaps the service is unable
> to send the child CAS due to missing broker. There should be something in
> the uima-as log about this.
>
> -jerry
>
> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera <nelsonriver...@gmail.com>
> wrote:
>
>> Yes, it is supposed to do the service, given a
>> CAS it produces a child CAS and and this CAS continue in the flow,
>> that are other primitive annotators. And I expected that the child CAS
>> will sent to  UIMA-AS client, but not do.
>>
>> the scenario:
>> 1- Start Broker and UIMA-AS aggregate service
>> 2- Client sends a CAS to UIMA-AS aggregate service to verify
>> everything is fine. The process is correct.
>>
>> 3- Restart Broker (without restart the service).  UIMA-AS log show  :
>> "Jms Listener Failed." 
>> 4- Client sends a CAS to UIMA-AS aggregate service
>>
>> UIMA-AS client receive the parent CAS, but not child CAS.
>>
>> Thank  for you quick reply
>>
>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning
>> given a
>> > CAS it produces child CASes
>> > which are then sent to UIMA-AS client. Is this what this service is
>> > supposed to do if the broker is running?
>> >
>> > Can you confirm the scenario:
>> > 1) Client sends a CAS to UIMA-AS aggregate service
>> > 2) Service receives the CAS
>> > 3) Broker dies
>> > 4) Broker is restarted
>> >
>> > Does the UIMA-AS client receive the CAS back in a reply when a broker is
>> > restarted?
>> >
>> > -Jerry
>> >
>> >
>> >
>> >
>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera <
>> nelsonriver...@gmail.com>
>> > wrote:
>> >
>> >> I will try the variants that you say. I was deepening the problem:
>> >>
>> >> I have a Aggregate AE that contains CasMultiplier, the original CAS
>> >> is finish when it arrives to this CasMultiplier,  this generates new
>> >> CAS and is this CAS that continue in the flow of Aggregate.
>> >>
>> >> After restart broker (without restart the service of annotator) and
>> >> send a CAS to process,  the listener associated to the client api
>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified  when the
>> >> original CAS finalizes, but the new Cas generated in the
>> >> CasMultiplier, It is not notifies when finalizes.
>> >>
>> >> with Aggregate AE simple the log is the same, but how the original CAS
>> >>  is notifies into the listener when finalizes, not have problem
>> >> because is not expected for another new CAS.
>> >>
>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <uim...@gmail.com>:
>> >> > If possible can you download AMQ 5.14.0 (latest) and use that broker
>> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty
>> bug
>> >> > in
>> >> > AMQ related to keep-alive protocol not working. I am not sure if this
>> >> > is
>> >> > the cause but worth trying.
>> >> >
>> >> > You can download AMQ from here:
>> >> > http://activemq.apache.org/download.html
>> >> >
>> >> > -Jerry
>> >> >
>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik <uim...@gmail.com>
>> >> wrote:
>> >> >
>> >> >> Not sure what is causing this. I need to stage this with UIMA-AS
>> >> >> 2.8.1.
>> >> I
>> >> >> am currently testing with trunk (latest) code. The most current code
>> >> does
>> >> >> have newer broker.
>> >> >>
>> >> >> I've started a broker and a sample service:
>> >> >>
>> >> >> $UIMA_HOME/bin/deployAsyncService.sh $UIMA_HOME/examples/deploy/as/
>> >> >> Deploy_MeetingDe

Re: Broken conections after ACTIVEMQ restart

[Jaroslaw Cwiklik]

Which version of UIMA-AS are you using?

-Jerry

On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera 
wrote:

> When start ActiveMQ and deploys an annotator as service, processing is
> executed correctly, but  when the broker is stopped and subsequently
> restarted. UIMA-AS log show the lines below  and after this the api
> client uima-as is not notified in the listener when a cas is
> processed, any more.
> can help me?:
>
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 02:08:25.570 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> Unable To Connect To Broker: tcp://localhost:61616 Retrying ...
> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint:
> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 Managed By:
> tcp://localhost:61616 Reason: javax.jms.JMSException:
> java.io.EOFException
> 02:09:02.222 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerContainer.
> handleListenerSetupFailure:
> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer Listener
> Established Connection to Broker: tcp://localhost:61616
>
> 02:09:02.317 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:09:02.317 - 14:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> org.apache.activemq.ConnectionClosedException: The connection is
> already closed02:09:02.356 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime Exception
> 02:09:02.357 - 15:
> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont
> ainer.onException:
> ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate
> Managed By: tcp://localhost:61616 Reason:
> javax.jms.IllegalStateException: The Consumer is closed.
>

Re: initiating CpeComponentDescriptor from String or InputStream

[Jaroslaw Cwiklik]

Can you just write out the component descriptor to a file and pass that to
the factory? I think you need a path since
the underlying code needs uima-style include which supports import by name
or location. Perhaps CPE can do this for you
with a new API you are suggesting but I the quickest path for you is to
create a file from string.

  public static CpeComponentDescriptor produceComponentDescriptor(String
aPath) {

CpeComponentDescriptor componentDescriptor = new
CpeComponentDescriptorImpl();
CpeInclude include = new CpeIncludeImpl();
include.set(aPath);
componentDescriptor.setInclude(include);
return componentDescriptor;
  }

-jerry


On Fri, Sep 9, 2016 at 11:15 AM, Chen Xiaobin  wrote:

> Hi,
> I am wondering if there is a way to initiate a CpeComponentDescriptor from
> an InputStream or a String, instead of from a physical descriptor file.
> I am using the following code originally:
>
> CpeCasProcessor casProcessor = CpeDescriptorFactory.
> produceCasProcessor(ae.getName());
> CpeComponentDescriptor componentDescriptor = CpeDescriptorFactory.
> produceComponentDescriptor("path/to/aeDescriptor.xml");
> casProcessor.setCpeComponentDescriptor(componentDescriptor);
> cpeDescription.addCasProcessor(casProcessor);
>
> But now in my application, all AE descriptors are stored in a database as
> Strings. I need to construct a CPE and add some AEs to the CPE.
>
> Is there a way to substitute the second line of the above code to something
> like:
>  CpeComponentDescriptor componentDescriptor =
> CpeDescriptorFactory.produceComponentDescriptor(**
> descriptorFromAnInputStream
> or String**);
> The UIMA API does not provide such a method in the CpeDescriptorFactor.
>
> Thank you!
>
> Xiaobin
> --
> --
> Eberhard Karls Universität Tübingen
> LEAD Graduate School
> Doctoral Candidate
> Gartenstraße 29A · 72074 Tübingen · Germany
> Phone +49 1765 7634 683
>

Re: Debugging a NullPointerException in UIMA AS / processing timeouts

[Jaroslaw Cwiklik]

Egbert, thanks. I forgot to ask, what version of UIMA-AS are you using?
Also, are you using sendCAS() or sendAndReceive() API?

Have a great vacation!

-jerry

On Sun, Aug 28, 2016 at 9:39 AM, Egbert van der Wal <e...@pointpro.nl>
wrote:

> Hi Jerry,
>
> Thanks for the suggestion. I have the feeling that it's a race condition,
> too, but since I'm doing no multi-threading myself, basically all the
> threading and synchronization should be UIMA-internal.
>
> Anyway, I'll have to postpone researching the issue due to going on
> vacation. When I get back I'll try to get more information with a increased
> log level, and get back to you.
>
> Thanks again!
>
> Regards,
>
> Egbert
>
>
> Op 25-8-2016 om 17:17 schreef Jaroslaw Cwiklik:
>
> Hi, I have a feeling that there might be a race condition here. In the
>> client, the timer pops and at the same time a reply is received.
>> The timout logic is resetting the CAS while its being deserialized which
>> may lead to NPE. Not 100% certain but this might be the problem.
>>
>> Any chance you can increase UIMA log level to FINEST on the client side?
>> It
>> would log important information like the internal CAS ID  on each reply
>> which can be used to correlate events in the log.
>>
>> -jerry
>>
>> On Thu, Aug 25, 2016 at 10:18 AM, Egbert van der Wal <e...@pointpro.nl>
>> wrote:
>>
>> Hi,
>>>
>>> I'm having a problem using UIMA-AS. I have a pipeline set up that
>>> processes HTML documents in ~= 10 ms. The total time out value was
>>> initially 20 seconds, but I increased it to 120 ms at some point to avoid
>>> this problem, it seemed to help.
>>>
>>> However, sometimes the 2 minutes is still hit and a warning is shown.
>>> When
>>> this occurs, it will usually be accompanied with NullPointerExceptions in
>>> combination with Xerces, somewhere in the internals of UIMA. See the
>>> attached log-file excerpt for the errors I'm seeing. The first 5 lines
>>> are
>>> the 'normal' output, which was repeated for several thousand lines before
>>> during the succesful operation of the pipeline.
>>>
>>> The SOFA that is being sent out during this particular exception is a
>>> quite small HTML-document, just a couple of kilobytes, and it's not
>>> actually reproducible with the same document; if I run the program again
>>> it
>>> will eventually fail, but at some other point.
>>>
>>> How can I go about solving this issue? Since the part of my own code in
>>> the stacktrace is limited to the point where 'sendCAS' is called, I can't
>>> really think of any additional debugging I can do.
>>>
>>> Any suggestions are highly appreciated!
>>>
>>> Thanks,
>>>
>>> Egbert van der Wal
>>>
>>>
>>

Re: Debugging a NullPointerException in UIMA AS / processing timeouts

[Jaroslaw Cwiklik]

Hi, I have a feeling that there might be a race condition here. In the
client, the timer pops and at the same time a reply is received.
The timout logic is resetting the CAS while its being deserialized which
may lead to NPE. Not 100% certain but this might be the problem.

Any chance you can increase UIMA log level to FINEST on the client side? It
would log important information like the internal CAS ID  on each reply
which can be used to correlate events in the log.

-jerry

On Thu, Aug 25, 2016 at 10:18 AM, Egbert van der Wal 
wrote:

> Hi,
>
> I'm having a problem using UIMA-AS. I have a pipeline set up that
> processes HTML documents in ~= 10 ms. The total time out value was
> initially 20 seconds, but I increased it to 120 ms at some point to avoid
> this problem, it seemed to help.
>
> However, sometimes the 2 minutes is still hit and a warning is shown. When
> this occurs, it will usually be accompanied with NullPointerExceptions in
> combination with Xerces, somewhere in the internals of UIMA. See the
> attached log-file excerpt for the errors I'm seeing. The first 5 lines are
> the 'normal' output, which was repeated for several thousand lines before
> during the succesful operation of the pipeline.
>
> The SOFA that is being sent out during this particular exception is a
> quite small HTML-document, just a couple of kilobytes, and it's not
> actually reproducible with the same document; if I run the program again it
> will eventually fail, but at some other point.
>
> How can I go about solving this issue? Since the part of my own code in
> the stacktrace is limited to the point where 'sendCAS' is called, I can't
> really think of any additional debugging I can do.
>
> Any suggestions are highly appreciated!
>
> Thanks,
>
> Egbert van der Wal
>

[ANNOUNCE] Apache UIMA DUCC 2.1.0 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the UIMA DUCC,
version 2.1.0.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling, management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources. DUCC defines a formal job model
that closely maps to a standard UIMA pipeline. Around this job model DUCC
provides cluster management services to automate the scale-out of UIMA
pipelines over computing clusters.

The main changes in this release are:

- DUCC framework now runs on Java 7 or Java 8
- Ubuntu and RHEL 7 support
- cgroup enhancements
 - uses standard cgroups organization
 - supports cgroup swappiness setting, restricting any swapping if
desired
- DUCC state and history storage moved from flat files to Cassandra DB,
reducing storage size 5x
- Ships with the latest UIMA-AS v2.8.1
- Ships with recent ActiveMQ v5.13.2
- DUCC's UIMA-AS services support failover and ssl connectors
- Many DUCC webpage improvements
- Clear user display of DUCC classes and relation to machines
- Robust handling of dynamic changes to DUCC class and nodepool definitions
- Full support of nodepools with different quantum
- DUCC broker access restricted to user ducc
- Eliminate need for user home directories located on a shared filesystem
- Built-in Job error handler programmable per job
- Migration utility for DUCC updates
- Change to vary-off behavior to facilitate cluster management
- Horizontal stacking of services instance allocations
- java-viaducc improvements including separation of stdout from stderr
respoonses
- An alert banner is displayed on ducc-mon pages if daemons are down
- Promoted DUCC from sandbox to the regular Apache project in the SVN

For a full list of the changes, please refer to Jira report:
http://uima.apache.org/d/uima-ducc-2.1.0/issuesFixed/jira-report.html

More information about DUCC can be found here:
http://uima.apache.org/doc-uimaducc-whatitam.html

-Jerry Cwiklik, for the Apache UIMA community

Re: SaxParse Exception

[Jaroslaw Cwiklik]

Matthias, I will create a JIRA for this so that it can be addressed for the
next UIMA-AS release For some reason I have not seen this type of problem
when running regression tests. Thank you for discovering this and finding
the fix.

Jerry

On Fri, Aug 5, 2016 at 3:16 AM, Matthias Koch 
wrote:

> Hi
>
> http://forum.spring.io/forum/spring-projects/aop/22379-why-d
> oes-scope-attribute-cause-xml-parsing-exception
>
> This post mentions that the dtd has to be version 2.0.
>
> I change the dd2spring.xsl line:87 - 90 to this
>
>  doctype-public="-//Spring//DTD BEAN 2.0//EN"
> doctype-system="http://www.springframework.org/dtd/spring-beans-2.0.dtd;
> />
>
> now the exception disappeared and everything looks fine.
>
> Sincere regards,
> Matthias
>
>

Re: Basic question about UIMA as service deployment

[Jaroslaw Cwiklik]

Hello Yamen, you can deploy uima-as service on a node which does not have a
broker. One of the benefits of JMS is location transparency. The broker can
run on any node in a cluster and mediates messages between client(s) and
service(s). A service connects to a broker and registers its queue. Client
which needs the service creates a connection to the same broker and sends
messages to the service queue.

The uima-as client deploy function does not exec a service process. It
actually deploys a service in the same jvm. To externalize the service use
$UIMA_HOME/bin/deployAsyncService.sh script and provide your deployment
descriptor with appropriate values for endpoint name and brokerURL. The
following is a relevant xml snippet from a deployment descriptor:

"/>

Jerry



On Fri, Jul 8, 2016 at 9:53 AM, Yamen Ajjour  wrote:

> Hello ,
>
> I have a node on which I want to deploy an asynchronous analysis engine.
> After running ActiveMq broker there I deployed the analysis engine on that
> machine from the client as specified in the specification. Even though
> everything seems to be working correclty I got the feeling that the
> analysis engines are still running in my the client's JVM and not on the
> node only the queue seems to be running there . Is deploying a UIMA as
> service implies its execution on the computer where the broker is ?
>
> Cheers,
> Yamen
>

[ANNOUNCE] Apache UIMA-AS 2.8.1 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the Apache
UIMA-AS version 2.8.1, which includes asynchronous scaleout capabilities
for the UIMA annotators.

UIMA-AS includes the base UIMA SDK and augments it with scaleout
capability; it is a next-generation replacement for the original CPM
(Collection Processing Management) scaleout that is part of the core UIMA
Framework. For more information, please visit:

http://uima.apache.org/doc-uimaas-what.html

This release contains a number of improvements and bug fixes. Notable
updates
in this release include:

- Replaced Activemq 5.7.0 with 5.13.2
- Added dependency on UIMA SDK 2.8.1
- Fixed per CAS Performance Metrics breakdown for async deployments
- Added new feature to allow warm up of a JVM service instance before real
processing begins,  by feeding it a specified set of CASes before the
instance connects to the service input queue.
- Allow dd2spring to use a custom XML parser

For a complete list of bugs and improvements included in this release
please see
https://uima.apache.org/d/uima-as-2.8.1/issuesFixed/jira-report.html.

-- Jerry Cwiklik, for the Apache UIMA development team

Re: UIMAfit - cannot find type system

[Jaroslaw Cwiklik]

I think this is import by name so no need for .xml at the end.

Try: classpath*:Users/anni/ /typesystem

Jerry

On Thu, Apr 28, 2016 at 5:06 PM, Anni R Coden  wrote:

> Hi -
>
> I am using UIMAfit
>
> I created a a file: META-INF/org.apache.uima.fit/types.txt
>
> in the file I put
>
> classpath*:Users/anni/ /typesystem.xml
>
>
> However I get a enrror that TypeSystemMgr requires a particular type
> (specified in typesystem.xml) which was not found in the CAS
>
> Here is the stack
>
> Exception in thread "main"
> com.ibm.langware.annotator.jFrostAEProcessException: Annotator class
> com.ibm.dltj.uima_annotator.lex_analysis.TypeSystemMgr requires Type
> uima.tt.ParagraphAnnotation, which was not found in the CAS.
> at
> com.ibm.dltj.uima_annotator.implBase.TypeSystemMgr_ImplBase.initTypeCode(
> TypeSystemMgr_ImplBase.java:66)
> at
> com.ibm.dltj.uima_annotator.lex_analysis.TypeSystemMgr.initTypeSystem(
> TypeSystemMgr.java:141)
> at
>
> com.ibm.dltj.uima_annotator.implBase.AnalyzerContext_ImplBase.setTypeSystem(
> AnalyzerContext_ImplBase.java:89)
> at
>
> com.ibm.dltj.uima_annotator.implBase.jFrostAnnotator_ImplBase.typeSystemInit(
> jFrostAnnotator_ImplBase.java:70)
> at
>
> org.apache.uima.analysis_component.CasAnnotator_ImplBase.checkTypeSystemChange(
> CasAnnotator_ImplBase.java:100)
> at
> org.apache.uima.analysis_component.CasAnnotator_ImplBase.process(
> CasAnnotator_ImplBase.java:55)
> at
>
> org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(
> PrimitiveAnalysisEngine_impl.java:378)
> at
>
> org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.processAndOutputNewCASes(
> PrimitiveAnalysisEngine_impl.java:298)
> at
>
> org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOutputCas(
> ASB_impl.java:568)
> at
>
> org.apache.uima.analysis_engine.asb.impl.ASB_impl$AggregateCasIterator.(
> ASB_impl.java:410)
> at org.apache.uima.analysis_engine.asb.impl.ASB_impl.process(
> ASB_impl.java:343)
> at
>
> org.apache.uima.analysis_engine.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes(
> AggregateAnalysisEngine_impl.java:265)
> at
> org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase.process(
> AnalysisEngineImplBase.java:267)
> at
> org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase.process(
> AnalysisEngineImplBase.java:280)
> at org.apache.uima.fit.pipeline.SimplePipeline.runPipeline(
> SimplePipeline.java:285)
> at com.ibm.medtas.runNegator.RunMedNeg.main(RunMedNeg.java:53)
>
> any hints are appreciated.
>
> Thanks, Anni
>
> Dr. Anni R. Coden
> IBM T.J. Watson Research Center
> 1101 Kitchawan Road, Route 134
> Yorktown Heights, NY 10598
> USA
> Tel: (914) 945 2359 t/l 862 2359
> a...@us.ibm.com
>
>

Re: UIMA-AS and RabbitMQ

[Jaroslaw Cwiklik]

Hi, the UIMA-AS currently works with ActiveMQ only. Perhaps we should take
a look at refactoring UIMA-AS to make it possible to use a different JMS
provider. So far there was no demand for this.
Jerry


On Mon, Dec 21, 2015 at 12:26 PM, Sylvain Surcin 
wrote:

> Hi,
>
> Is it possible to configure UIMA-AS to use another AMQP rather than
> ActiveMQ, let's say RabbitMQ for instance?
>
> Thanks and season's greetings.
> Sylvain
>

Re: UIMA-AS and RabbitMQ

[Jaroslaw Cwiklik]

No promises but will think about it. What special features in Rabbit MQ do
you rely on? Any reason why AMQ is not adequate for your needs? Just
curious.
Thanks
Jerry

On Mon, Dec 21, 2015 at 2:07 PM, Sylvain Surcin <sylvain.sur...@gmail.com>
wrote:

> Well, if by chance you happen to make that possible, I would be very
> interested.
> Sylvain
>
> 2015-12-21 19:55 GMT+01:00 Jaroslaw Cwiklik <uim...@gmail.com>:
>
> > Hi, the UIMA-AS currently works with ActiveMQ only. Perhaps we should
> take
> > a look at refactoring UIMA-AS to make it possible to use a different JMS
> > provider. So far there was no demand for this.
> > Jerry
> >
> >
> > On Mon, Dec 21, 2015 at 12:26 PM, Sylvain Surcin <
> sylvain.sur...@gmail.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > Is it possible to configure UIMA-AS to use another AMQP rather than
> > > ActiveMQ, let's say RabbitMQ for instance?
> > >
> > > Thanks and season's greetings.
> > > Sylvain
> > >
> >
>

Re: UIMA AS roadmap

[Jaroslaw Cwiklik]

Yes, the new release will have the latest ActiveMQ v.5.12.1 which comes
with Spring Framwork v.3.2.11.
Jerry

On Tue, Dec 8, 2015 at 10:52 AM, Peter Klügl <peter.klu...@averbis.com>
wrote:

> Hi,
>
> are there plans concerning the spring versions?
>
> Best,
>
> Peter
>
> Am 08.12.2015 um 16:31 schrieb Jaroslaw Cwiklik:
> > Mihaela, I am working on the new UIMA-AS release right now. It will
> include
> > the UIMA SDK 2.8.1, ActiveMQ 5.12.1, and a few fixes and features like:
> >
> > - Fixed Performance Metrics for Async Aggregates
> > - Prime AEs before processing
> > - Fixed Eclipse Runtime Configurations
> >
> > The new release should be available within a few weeks as.
> > Jerry
> >
> >
> > On Tue, Dec 8, 2015 at 8:05 AM, mihaela olteanu <
> > mihaela...@yahoo.com.invalid> wrote:
> >
> >> Hello,
> >> Does anyone know whether a new version of UIMA AS will be release that
> >> will use the latest version of UIMA framework which is 2.8.1, and if so
> >> when? Is there any reason for UIMA AS not being kept in sync with UIMA
> >> framework?
> >> Thanks.Mihaela
>
>

Re: UIMA AS roadmap

[Jaroslaw Cwiklik]

Mihaela, I am working on the new UIMA-AS release right now. It will include
the UIMA SDK 2.8.1, ActiveMQ 5.12.1, and a few fixes and features like:

- Fixed Performance Metrics for Async Aggregates
- Prime AEs before processing
- Fixed Eclipse Runtime Configurations

The new release should be available within a few weeks as.
Jerry


On Tue, Dec 8, 2015 at 8:05 AM, mihaela olteanu <
mihaela...@yahoo.com.invalid> wrote:

> Hello,
> Does anyone know whether a new version of UIMA AS will be release that
> will use the latest version of UIMA framework which is 2.8.1, and if so
> when? Is there any reason for UIMA AS not being kept in sync with UIMA
> framework?
> Thanks.Mihaela

[ANNOUNCE] Apache UIMA DUCC 2.0.1 released

[Jaroslaw Cwiklik]

The Apache UIMA team is pleased to announce the release of the UIMA DUCC,
version 2.0.1.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling, management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources. DUCC defines a formal job model
that closely maps to a standard UIMA pipeline. Around this job model DUCC
provides cluster management services to automate the scale-out of UIMA
pipelines over computing clusters.

This is a bug release, addressing bugs found since DUCC 2.0.0 release. For
a full list of the changes, please refer to Jira report:
http://uima.apache.org/d/uima-ducc-2.0.1/issuesFixed/jira-report.html

More information about DUCC can be found here:
http://uima.apache.org/doc-uimaducc-whatitam.html

-Jerry Cwiklik, for the Apache UIMA community

Re: DUCC - Work Item Queue Time Management

[Jaroslaw Cwiklik]

Hi, can you try Ducc 2.0.0? It was recently released into Apache. One of
the key changes was to remove queues as means of transport between JD (Job
Driver) and JP (Job Process). Instead, each JP uses HTTP to request a Work
Item (CAS) from a JD.

DUCC 1.1.0 has a concept of a WI timeout which I think is 24 hours by
default. A timer is started in a JD when each WI is dispatched to a JP. If
the WI does not come back for whatever reason, the timer pops and a JD will
attempt to retry that WI.

To debug your problem with DUCC 1.1.0 I suggest attaching JMX console to a
running JP to see where its threads are. Before doing this, check JP logs
to see if there is an exception.

Jerry



On Thu, Sep 17, 2015 at 4:32 AM, reshu.agarwal 
wrote:

> My DUCC version is 1.1.0.
>
>
> On 09/17/2015 11:35 AM, reshu.agarwal wrote:
>
>>
>> Hi,
>>
>> I am facing a problem in DUCC that some documents were shown in queue but
>> did not get processed. In Job, work item list shows a particular work
>> item's status "queued" and queueing time is "4115 seconds".
>>
>> I want to set queueing time of work item not more then 1 minute. What is
>> the reason for the same? Is there any method to solve this? How can I set
>> maximum queueing time for work item?
>>
>> Thanks in advance.
>>
>> Reshu.
>>
>
>

[ANNOUNCE] Apache UIMA DUCC 2.0.0 released

[Jaroslaw Cwiklik]

The Apache Maven team is pleased to announce the release of the UIMA DUCC,
version 2.0.0.

DUCC stands for Distributed UIMA Cluster Computing. DUCC is a cluster
management system providing tooling, management, and scheduling facilities
to automate the scale-out of applications written to the UIMA framework.
Core UIMA provides a generalized framework for applications that process
unstructured information such as human language, but does not provide a
scale-out mechanism. UIMA-AS provides a scale-out mechanism to distribute
UIMA pipelines over a cluster of computing resources, but does not provide
job or cluster management of the resources. DUCC defines a formal job model
that closely maps to a standard UIMA pipeline. Around this job model DUCC
provides cluster management services to automate the scale-out of UIMA
pipelines over computing clusters.

This is a major release containing new features and bug fixes. Please visit
http://uima.apache.org/news.html for more details.

-Jerry Cwiklik, for the Apache UIMA community

Re: UIMAj3 ideas

[Jaroslaw Cwiklik]

The UIMA-AS *does* have an API to generate deployment descriptors although
its not documented. Its an internal API for now and most likely will be
documented in the next release of UIMA-AS. The API is implemented by
 DeploymentDescriptorFactory.java. in the uimaj-as-core project.

Jerry

On Thu, Jul 16, 2015 at 4:56 PM, Thomas Ginter thomas.gin...@utah.edu
wrote:

 Richard,

 There is an API in UIMA for generating Analysis Engine Descriptors as well
 as Aggregates and Type System descriptions.  I use that API to generate the
 xml descriptor at runtime after the configuration has been completed.  I
 wrote my own logic to track the delegates of an Aggregate descriptor in
 order to propagate updates to/from delegates to allow the user to
 dynamically specify Analysis Engine parameters.  I also merged the scale
 out parameters for UIMA-AS into the Analysis Engine object for ease of
 configuration.

 In addition I wrote my own code to generate the deployment descriptor from
 the programmatic parameters provided.  The resulting XML is what the
 framework uses to generate the Spring Bean file you mentioned.

 That being said the existing API definitely has a learning curve which was
 part of the motivation for creating Leo.

 Thanks,

 Thomas Ginter
 801-448-7676
 thomas.gin...@utah.edu




  On Jul 16, 2015, at 1:51 PM, Richard Eckart de Castilho r...@apache.org
 wrote:
 
  Hi Thomas,
 
  On 16.07.2015, at 21:42, Thomas Ginter thomas.gin...@utah.edu wrote:
 
  Have you looked into using Leo?  It allows you to programmatically
 create Analysis Engines, Aggregates, the type system, and launch everything
 in UIMA-AS without having to manage any XML descriptors at all.
 Furthermore it is available via Maven so your code can compile an run.
 
  Did you find an API in UIMA AS to handle the programmatic generation of
 descriptors, or did you implement that yourself in Leo (as I had tried to
 in DKPro Lab)?
 
  If I remember correctly, then UIMA AS loaded plain XML descriptor files,
 transforms them to a Spring Bean file using XSLT and then used Spring to
 instantiate it. But I may have missed something.
 
  Cheers,
 
  -- Richard