There were no long GCs for this service. Note (from above) that many other services running older versions of uima-as were also showing the same behavior.
Eddie On Tue, Jan 2, 2018 at 9:21 AM, Jaroslaw Cwiklik <cwik...@apache.org> wrote: > Looks like the client lost connection to the broker for some reason. Long > GC perhaps? But eventually the service should have recovered. Any > indication of connection > recovery later? > > Jerry > > On Sun, Dec 24, 2017 at 11:34 AM, Eddie Epstein <eaepst...@gmail.com> > wrote: > > > Jerry, > > > > Testing a real analytic running 28 sync pipelines as a DUCC managed > UIMA-AS > > service, the DUCC pinger is reporting failures for an idle service 2 or 3 > > times a day. > > > > sm.log, an error surrounded by good pings, the ping normally done every 2 > > minutes: > > > > Dec 2017 07:49:42,740 INFO SM.PingDriver - 10668 T[3588945] > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616 > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1] > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1] > > MetaNode[192.168.7.101] MetaPid[71061:91] > > Dec 2017 07:51:42,844 INFO SM.PingDriver - 10668 T[3588945] > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616 > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[0] Consum[3] Prod[1] > > minNQ[0] maxNQ[0] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1] > > MetaNode[192.168.7.101] MetaPid[71061:91] > > Dec 2017 07:54:02,912 ERROR SM.PingDriver - 10668 T[3588945] > > handleStatistics Missed_pings 1 endpoint > > UIMA-AS:eae-r1.16:tcp://broker1:61616 Alive[false] Healthy[false] + > Info: > > QDEPTH[0] AveNQ[0] Consum[3] Prod[1] minNQ[0] maxNQ[0] expCnt[0] inFlt[0] > > DQ[1] NQ[1] NDisp[1] MetaNode[N/A] MetaPid[N/A]: Cannot issue getMeta to: > > eae-r1.16:tcp://broker1:61616: java.util.concurrent.TimeoutException > > Dec 2017 07:56:03,016 INFO SM.PingDriver - 10668 T[3588945] > > handleStatistics Ping ok: UIMA-AS:eae-r1.16:tcp://broker1:61616 > > Alive[true] Healthy[true] + Info: QDEPTH[0] AveNQ[89570] Consum[3] > Prod[1] > > minNQ[89570] maxNQ[89570] expCnt[0] inFlt[0] DQ[1] NQ[1] NDisp[1] > > MetaNode[192.168.7.101] MetaPid[71061:91] > > > > The broker logfile, complaining about all three service listeners around > > the same time (the service runs on 192.168.7.101): > > > > 2017-12-24 07:54:48,257 [yMonitor Worker] WARN Transport > > - Transport Connection to: tcp://192.168.7.101:43895 failed: > > org.apache.activemq.transport.InactivityIOException: Channel was > inactive > > for too (>30000) long: tcp://192.168.7.101:43895 > > 2017-12-24 07:54:48,287 [yMonitor Worker] WARN Transport > > - Transport Connection to: tcp://192.168.7.101:43898 failed: > > org.apache.activemq.transport.InactivityIOException: Channel was > inactive > > for too (>30000) long: tcp://192.168.7.101:43898 > > 2017-12-24 07:54:48,288 [yMonitor Worker] WARN Transport > > - Transport Connection to: tcp://192.168.7.101:43899 failed: > > org.apache.activemq.transport.InactivityIOException: Channel was > inactive > > for too (>30000) long: tcp://192.168.7.101:43899 > > > > The service logfile also reporting problems for all three listeners: > > > > 07:55:12.457 - 91: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Service: Processing2 Runtime Exception > > 07:55:12.457 - 91: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By: > > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException > > 07:55:12.457 - 91: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > ainer. > > handleListenerSetupFailure: > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To > Broker: > > tcp://broker1:61616 Retrying Until Successful ... > > 07:55:12.457 - 2336: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Service: Processing2 Runtime Exception > > 07:55:12.457 - 2336: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By: > > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException > > 07:55:12.457 - 2335: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Service: Processing2 Runtime Exception > > 07:55:12.457 - 2335: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > > ainer.onException: > > WARNING: Jms Listener Failed. Endpoint: eae-r1.16 Managed By: > > tcp://broker1:61616 Reason: javax.jms.JMSException: java.io.EOFException > > 07:55:12.457 - 2336: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > ainer. > > handleListenerSetupFailure: > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To > Broker: > > tcp://broker1:61616 Retrying Until Successful ... > > 07:55:12.457 - 2335: > > org.apache.uima.adapter.jms.activemq.UimaDefaultMessageListenerCont > ainer. > > handleListenerSetupFailure: > > WARNING: Uima AS Service:Processing2 Listener Unable To Connect To > Broker: > > tcp://broker1:61616 Retrying Until Successful ... > > > > > > sm.log is showing the same kind of ping failures for previous versions of > > UIMA-AS as well, not all at the same time as this event, but spread out > > widely. The clocks on all three machines are synchronized so it could be > > meaningful that the pinger client registered the error before the broker > or > > the service. > > > > Don't think this is a blocker, just wanted to check if there is a problem > > in UIMA-AS. > > > > Eddie > > > > > > On Thu, Dec 14, 2017 at 4:52 PM, Jaroslaw Cwiklik <cwik...@apache.org> > > wrote: > > > > > Hi, > > > > > > the UIMA-AS 2.10.2 release candidate 5 is ready for voting. > > > > > > This version contains the following fixes: > > > - fixed documentation > > > > > > The list of changes in Jira: > > > http://issues.apache.org/jira/issues/?jql=fixVersion%20%3D% > > > 202.10.2AS%20AND%20project%20%3D%20UIMA > > > > > > The source and binary zip/tars are staged to > > > http://dist.apache.org/repos/dist/dev/uima/uima-as/2.10.2/RC5/ > > > > > > The eclipse update subsite is here: > > > http://dist.apache.org/repos/dist/dev/uima/eclipse-update- > site/uima-as/ > > > > > > The Maven artifacts are here: > > > http://repository.apache.org/content/repositories/orgapacheuima-1177 > > > > > > The SVN tags are here: > > > http://svn.apache.org/repos/asf/uima/uima-as/tags/uima-as-2.10.2/ > > > > > > See http://uima.apache.org/testing-builds.html for suggestions on how > to > > > test release candidates. > > > > > > Please vote on release: > > > > > > [ ] +1 OK to release > > > [ ] 0 Don't care > > > [ ] -1 Not OK to release, because ... > > > > > >
