Can you install AMQ broker 5.14.0 to see if this is related to broker issues.
-jerry On Thu, Sep 22, 2016 at 11:29 AM, nelson rivera <[email protected]> wrote: > just like that, the parent CAS comes back but the child CAS doesnt. > > And the child CAS if processed, because the uima-as log shows the > lines of the other annotators primitive of the flow processing the > child CAS, but not comes back to UIMA-AS client. > > uima-as log only shows > "Jms Listener Failed. Endpoint: > temp-queue://ID:localhsot-H81-M1-33610-1474481197190-1:1:1 Managed By: > tcp://localhost:61616 Reason: javax.jms.JMSException: > java.io.EOFException" ........ > ADVERTENCIA: Jms Listener Failed. Endpoint: XClusterAnalyzerAggregate > Managed By: tcp://localhost:61616 Reason: > javax.jms.IllegalStateException: The Consumer is closed.............. > > The CAS is send to service after restart the Broker, and how say > parent CAS comes back but the child CAS doesnt. And if i try quiesce > the Aggregate Service console shows : "Quiescing UIMA-AS Service. > Remaining Number of CASes to Process:1". I suppose that is the child > CAS. > > the client if can send more CASes to the service, but not receive > response, for each CAS send after the one CAS that was send after > restart broker (for which parent CAS comes back but the child CAS > doesnt) not comes back parent CAS or child CAS. > > More specific: > scenario 1: > 1- Start Broker and UIMA-AS aggregate service. Everything is fine. > > 2- Restart Broker (without restart the service). UIMA-AS log show : > "Jms Listener Failed." .... > 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS > comes back but the child CAS doesnt, to the uima-as client.) > If i try quiesce the Aggregate Service console shows : "Quiescing > UIMA-AS Service. Remaining Number of CASes to Process:1" > > scenario 2: > 1- Start Broker and UIMA-AS aggregate service. Everything is fine. > > 2- Restart Broker (without restart the service). UIMA-AS log show : > "Jms Listener Failed." .... > 3- Client sends a CAS to UIMA-AS aggregate service.(The parent CAS > comes back but the child CAS doesnt, to the uima-as client) > 4- Client sends another CAS to UIMA-AS aggregate service.(not comes > back parent CAS or child CAS , to the uima-as client) > 5- Client sends another CAS to UIMA-AS aggregate service.(not comes > back parent CAS or child CAS , to the uima-as client) > If i try quiesce the Aggregate Service console shows: "Quiescing > UIMA-AS Service. Remaining Number of CASes to Process:5" > > > > > > 2016-09-22 10:50 GMT-04:00, Jaroslaw Cwiklik <[email protected]>: > > When a parent CAS comes back (after broker is restored), can the client > > send more CASes to the service and receive responses? I am trying to > > determine what the real problem is here. > > > > 1) JMS Listener not recovering connection > > 2) UIMA-AS is not handling CAS reply failures when broker connection is > > stale > > > > -jerry > > > > On Thu, Sep 22, 2016 at 10:38 AM, Jaroslaw Cwiklik <[email protected]> > > wrote: > > > >> Strange. You say that the parent CAS comes back but the child CAS > doesnt. > >> Are there any exceptions in the service log? Perhaps the service is > >> unable > >> to send the child CAS due to missing broker. There should be something > in > >> the uima-as log about this. > >> > >> -jerry > >> > >> On Thu, Sep 22, 2016 at 10:19 AM, nelson rivera > >> <[email protected]> > >> wrote: > >> > >>> Yes, it is supposed to do the service, given a > >>> CAS it produces a child CAS and and this CAS continue in the flow, > >>> that are other primitive annotators. And I expected that the child CAS > >>> will sent to UIMA-AS client, but not do. > >>> > >>> the scenario: > >>> 1- Start Broker and UIMA-AS aggregate service > >>> 2- Client sends a CAS to UIMA-AS aggregate service to verify > >>> everything is fine. The process is correct. > >>> > >>> 3- Restart Broker (without restart the service). UIMA-AS log show : > >>> "Jms Listener Failed." .... > >>> 4- Client sends a CAS to UIMA-AS aggregate service > >>> > >>> UIMA-AS client receive the parent CAS, but not child CAS. > >>> > >>> Thank for you quick reply > >>> > >>> 2016-09-22 9:48 GMT-04:00, Jaroslaw Cwiklik <[email protected]>: > >>> > Can you clarify if the UIMA-AS service is a Cas Multiplier. Meaning > >>> given a > >>> > CAS it produces child CASes > >>> > which are then sent to UIMA-AS client. Is this what this service is > >>> > supposed to do if the broker is running? > >>> > > >>> > Can you confirm the scenario: > >>> > 1) Client sends a CAS to UIMA-AS aggregate service > >>> > 2) Service receives the CAS > >>> > 3) Broker dies > >>> > 4) Broker is restarted > >>> > > >>> > Does the UIMA-AS client receive the CAS back in a reply when a broker > >>> > is > >>> > restarted? > >>> > > >>> > -Jerry > >>> > > >>> > > >>> > > >>> > > >>> > On Thu, Sep 22, 2016 at 9:33 AM, nelson rivera < > >>> [email protected]> > >>> > wrote: > >>> > > >>> >> I will try the variants that you say. I was deepening the problem: > >>> >> > >>> >> I have a Aggregate AE that contains CasMultiplier, the original CAS > >>> >> is finish when it arrives to this CasMultiplier, this generates new > >>> >> CAS and is this CAS that continue in the flow of Aggregate. > >>> >> > >>> >> After restart broker (without restart the service of annotator) and > >>> >> send a CAS to process, the listener associated to the client api > >>> >> uima-as (BaseUIMAAsynchronousEngine_impl) is notified when the > >>> >> original CAS finalizes, but the new Cas generated in the > >>> >> CasMultiplier, It is not notifies when finalizes. > >>> >> > >>> >> with Aggregate AE simple the log is the same, but how the original > >>> >> CAS > >>> >> is notifies into the listener when finalizes, not have problem > >>> >> because is not expected for another new CAS. > >>> >> > >>> >> 2016-09-21 17:23 GMT-04:00, Jaroslaw Cwiklik <[email protected]>: > >>> >> > If possible can you download AMQ 5.14.0 (latest) and use that > >>> >> > broker > >>> >> > instead of the one the uima-as 2.8.1 ships with? There was a nasty > >>> bug > >>> >> > in > >>> >> > AMQ related to keep-alive protocol not working. I am not sure if > >>> >> > this > >>> >> > is > >>> >> > the cause but worth trying. > >>> >> > > >>> >> > You can download AMQ from here: > >>> >> > http://activemq.apache.org/download.html > >>> >> > > >>> >> > -Jerry > >>> >> > > >>> >> > On Wed, Sep 21, 2016 at 4:52 PM, Jaroslaw Cwiklik > >>> >> > <[email protected]> > >>> >> wrote: > >>> >> > > >>> >> >> Not sure what is causing this. I need to stage this with UIMA-AS > >>> >> >> 2.8.1. > >>> >> I > >>> >> >> am currently testing with trunk (latest) code. The most current > >>> >> >> code > >>> >> does > >>> >> >> have newer broker. > >>> >> >> > >>> >> >> I've started a broker and a sample service: > >>> >> >> > >>> >> >> $UIMA_HOME/bin/deployAsyncService.sh > >>> >> >> $UIMA_HOME/examples/deploy/as/ > >>> >> >> Deploy_MeetingDetectorTAE.xml > >>> >> >> > >>> >> >> After the service started, I stopped the broker, waited a bit, > and > >>> >> >> restarted the broker. > >>> >> >> > >>> >> >> With UIMA-AS from trunk I see the following in the log: > >>> >> >> > >>> >> >> 04:21:35.496 - 1: org.apache.uima.adapter.jms.activemq. > >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: > >>> >> >> Meeting > >>> >> >> Detector TAE Trying to Start Listener on Endpoint: > >>> >> >> queue://MeetingDetectorTaeQueue > >>> >> >> Selector: Command=2000 OR Command=2002 Broker: > >>> >> >> tcp://localhost:61616 > >>> >> >> 04:21:35.873 - 1: org.apache.uima.adapter.jms.activemq. > >>> >> >> SpringContainerDeployer.doStartListeners: INFO: Controller: > >>> >> >> Meeting > >>> >> >> Detector TAE Trying to Start Listener on Endpoint: > >>> >> >> queue://MeetingDetectorTaeQueue > >>> >> >> Selector: Command=2001 Broker: tcp://localhost:61616 > >>> >> >> 04:21:45.621 - 22: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: > Service: > >>> >> >> Meeting Detector TAE Runtime Exception > >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms > >>> Listener > >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By: > >>> >> >> tcp://localhost:61616 > >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException > >>> >> >> 04:21:45.622 - 22: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: > >>> >> WARNING: > >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect > To > >>> >> >> Broker: > >>> >> >> tcp://localhost:61616 Retrying Until Successful ... > >>> >> >> 04:21:45.638 - 27: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: > Service: > >>> >> >> Meeting Detector TAE Runtime Exception > >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.onException: WARNING: Jms > >>> Listener > >>> >> >> Failed. Endpoint: MeetingDetectorTaeQueue Managed By: > >>> >> >> tcp://localhost:61616 > >>> >> >> Reason: javax.jms.JMSException: java.io.EOFException > >>> >> >> 04:21:45.639 - 27: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: > >>> >> WARNING: > >>> >> >> Uima AS Service:Meeting Detector TAE Listener Unable To Connect > To > >>> >> >> Broker: > >>> >> >> tcp://localhost:61616 Retrying Until Successful ... > >>> >> >> 04:22:07.439 - 22: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: > >>> >> WARNING: > >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered > Connection > >>> to > >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again > >>> >> >> 04:22:07.454 - 27: org.apache.uima.adapter.jms.activemq. > >>> >> >> UimaDefaultMessageListenerContainer.handleListenerSetupFailure: > >>> >> WARNING: > >>> >> >> Uima AS Service:Meeting Detector TAE Listener Recovered > Connection > >>> to > >>> >> >> Broker: tcp://localhost:61616 - Ready to Process Again > >>> >> >> > >>> >> >> > >>> >> >> I will repeat the above test with v 2.8.1 > >>> >> >> > >>> >> >> -Jerry > >>> >> >> > >>> >> >> > >>> >> >> On Wed, Sep 21, 2016 at 4:34 PM, nelson rivera < > >>> >> [email protected]> > >>> >> >> wrote: > >>> >> >> > >>> >> >>> I'm using UIMA-AS 2.8.1, any idea that what happen? > >>> >> >>> > >>> >> >>> 2016-09-21 16:09 GMT-04:00, Jaroslaw Cwiklik <[email protected] > >: > >>> >> >>> > Which version of UIMA-AS are you using? > >>> >> >>> > > >>> >> >>> > -Jerry > >>> >> >>> > > >>> >> >>> > On Wed, Sep 21, 2016 at 3:42 PM, nelson rivera < > >>> >> >>> [email protected]> > >>> >> >>> > wrote: > >>> >> >>> > > >>> >> >>> >> When start ActiveMQ and deploys an annotator as service, > >>> >> >>> >> processing > >>> >> >>> >> is > >>> >> >>> >> executed correctly, but when the broker is stopped and > >>> >> >>> >> subsequently > >>> >> >>> >> restarted. UIMA-AS log show the lines below and after this > >>> >> >>> >> the > >>> >> >>> >> api > >>> >> >>> >> client uima-as is not notified in the listener when a cas is > >>> >> >>> >> processed, any more. > >>> >> >>> >> can help me?: > >>> >> >>> >> > >>> >> >>> >> 02:08:25.570 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime > >>> Exception > >>> >> >>> >> 02:08:25.570 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: > >>> >> XClusterAnalyzerAggregate > >>> >> >>> >> Managed By: tcp://localhost:61616 Reason: > >>> javax.jms.JMSException: > >>> >> >>> >> java.io.EOFException > >>> >> >>> >> 02:08:25.570 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> >> >>> nerContainer. > >>> >> >>> >> handleListenerSetupFailure: > >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer > >>> >> >>> >> Listener > >>> >> >>> >> Unable To Connect To Broker: tcp://localhost:61616 Retrying > >>> >> >>> >> ... > >>> >> >>> >> QueueFailure: ADVERTENCIA: Jms Listener Failed. Endpoint: > >>> >> >>> >> temp-queue://ID:localhost-H81-M1-33610-1474481197190-1:1:1 > >>> Managed > >>> >> >>> >> By: > >>> >> >>> >> tcp://localhost:61616 Reason: javax.jms.JMSException: > >>> >> >>> >> java.io.EOFException > >>> >> >>> >> 02:09:02.222 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> >> >>> nerContainer. > >>> >> >>> >> handleListenerSetupFailure: > >>> >> >>> >> ADVERTENCIA: Uima AS Service:Aggregate Cluster Analyzer > >>> >> >>> >> Listener > >>> >> >>> >> Established Connection to Broker: tcp://localhost:61616 > >>> >> >>> >> > >>> >> >>> >> 02:09:02.317 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime > >>> Exception > >>> >> >>> >> 02:09:02.317 - 14: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: > >>> >> XClusterAnalyzerAggregate > >>> >> >>> >> Managed By: tcp://localhost:61616 Reason: > >>> >> >>> >> org.apache.activemq.ConnectionClosedException: The > connection > >>> is > >>> >> >>> >> already closed02:09:02.356 - 15: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Service: Aggregate Cluster Analyzer Runtime > >>> Exception > >>> >> >>> >> 02:09:02.357 - 15: > >>> >> >>> >> org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListe > >>> nerCont > >>> >> >>> >> ainer.onException: > >>> >> >>> >> ADVERTENCIA: Jms Listener Failed. Endpoint: > >>> >> XClusterAnalyzerAggregate > >>> >> >>> >> Managed By: tcp://localhost:61616 Reason: > >>> >> >>> >> javax.jms.IllegalStateException: The Consumer is closed. > >>> >> >>> >> > >>> >> >>> > > >>> >> >>> > >>> >> >> > >>> >> >> > >>> >> > > >>> >> > >>> > > >>> > >> > >> > > >
