[
https://issues.apache.org/jira/browse/UIMA-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484937#comment-15484937
]
Jerry Cwiklik commented on UIMA-4210:
-------------------------------------
I have instrumented junit test (testPrimitiveServiceProcessPingFailure) to
force the scenario described. Not seeing a hang when the code in question is
run. Specifically, added tracing in a block of code that was referenced in the
problem report (BaseUIMAAsynchronousEngineCommon_impl.notifyOnTimeout())
System.out.println("isSynchronousCall="+isSynchronousCall+"
serviceDelegate.getCasPendingReplyListSize()="+serviceDelegate.getCasPendingReplyListSize());
if ( !isSynchronousCall &&
serviceDelegate.getCasPendingReplyListSize() > 0) {
String nextOutstandingCasReferenceId =
serviceDelegate.getOldestCasIdFromOutstandingList();
if ( nextOutstandingCasReferenceId != null ) {
cachedRequest = (ClientRequest)
clientCache.get(nextOutstandingCasReferenceId);
if ( cachedRequest != null &&
cachedRequest.getCAS() != null ) {
try {
System.out.println("Sending CAS
Again");
sendCAS(cachedRequest.getCAS());
} catch( Exception e) {
UIMAFramework.getLogger(CLASS_NAME).logrb(Level.WARNING, getClass().getName(),
"notifyOnTimout",
UIMAEE_Constants.JMS_LOG_RESOURCE_BUNDLE,
"UIMAEE_exception__WARNING", e);
}
}
}
}
When I ran a junit test I could see the trace msgs showing each time a timeout
occurred. I can see the sendCAS() is being called and no hang has been
observed.
org.apache.uima.aae.error.UimaASProcessCasTimeout: UIMA AS Client Timed Out
Waiting For CAS:-e5dc9fa:1571f849883:-55c1 Reply From a Service On
Queue:PersonTitleAnnotatorQueue
runTest: Received Reply from CAS -e5dc9fa:1571f849883:-55c1 Containing 1
Exception(s)
at
org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl.notifyOnTimout(BaseUIMAAsynchronousEngineCommon_impl.java:2372)
at
org.apache.uima.adapter.jms.client.ClientServiceDelegate.handleError(ClientServiceDelegate.java:144)
at org.apache.uima.aae.delegate.Delegate$1.run(Delegate.java:790)
at java.util.TimerThread.mainLoop(Timer.java:566)
at java.util.TimerThread.run(Timer.java:516)
isSynchronousCall=false serviceDelegate.getCasPendingReplyListSize()=3
Sending CAS Again
runTest: Received Reply from CAS -e5dc9fa:1571f849883:-55c0 Containing 1
Exception(s)
org.apache.uima.aae.error.UimaASProcessCasTimeout: UIMA AS Client Timed Out
Waiting For CAS:-e5dc9fa:1571f849883:-55c0 Reply From a Service On
Queue:PersonTitleAnnotatorQueue
at
org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl.notifyOnTimout(BaseUIMAAsynchronousEngineCommon_impl.java:2372)
at
org.apache.uima.adapter.jms.client.ClientServiceDelegate.handleError(ClientServiceDelegate.java:144)
at org.apache.uima.aae.delegate.Delegate$1.run(Delegate.java:790)
at java.util.TimerThread.mainLoop(Timer.java:566)
at java.util.TimerThread.run(Timer.java:516)
isSynchronousCall=false serviceDelegate.getCasPendingReplyListSize()=2
Sending CAS Again
runTest: Received Reply from CAS -e5dc9fa:1571f849883:-55bf Containing 1
Exception(s)
org.apache.uima.aae.error.UimaASProcessCasTimeout: UIMA AS Client Timed Out
Waiting For CAS:-e5dc9fa:1571f849883:-55bf Reply From a Service On
Queue:PersonTitleAnnotatorQueue
at
org.apache.uima.adapter.jms.client.BaseUIMAAsynchronousEngineCommon_impl.notifyOnTimout(BaseUIMAAsynchronousEngineCommon_impl.java:2372)
at
org.apache.uima.adapter.jms.client.ClientServiceDelegate.handleError(ClientServiceDelegate.java:144)
at org.apache.uima.aae.delegate.Delegate$1.run(Delegate.java:790)
at java.util.TimerThread.mainLoop(Timer.java:566)
at java.util.TimerThread.run(Timer.java:516)
isSynchronousCall=false serviceDelegate.getCasPendingReplyListSize()=1
Sending CAS Again
runTest: Received Reply from CAS -e5dc9fa:1571f849883:-55bd Containing 1
Exception(s)
runTest: Ping Timeout - service Not Responding To Ping
org.apache.uima.aae.error.UimaASProcessCasTimeout: Service Not Responding to
Ping - CAS:-e5dc9fa:1571f849883:-55bd
at
org.apache.uima.adapter.jms.client.ClientServiceDelegate.handleError(ClientServiceDelegate.java:175)
at org.apache.uima.aae.delegate.Delegate$2.run(Delegate.java:835)
at java.util.TimerThread.mainLoop(Timer.java:566)
at java.util.TimerThread.run(Timer.java:516)
Caused by: org.apache.uima.aae.error.UimaASPingTimeout: Forced Timeout on CAS
in PendingDispatch list. The CAS Has Not Been Dispatched since the Service
Appears to be Unavailable
... 4 more
The hang was observed with UIMA-AS 2.4.2 so it could be that the bug was there
which was fixed in a later version. Went back through JIRAs and found a few
that may have caused this hang in 2.4.2:
UIMA-4830
UIMA-3572
UIMA-3393
All of the above have been fixed in more recent versions of the UIMA-AS. I will
keep this JIRA open for a few days and will close with Wont Fix if there are no
complains or comments to justify keeping the JIRA open.
-jerry
> Client hangs with more than 1 time-out
> --------------------------------------
>
> Key: UIMA-4210
> URL: https://issues.apache.org/jira/browse/UIMA-4210
> Project: UIMA
> Issue Type: Bug
> Components: Async Scaleout
> Affects Versions: 2.4.2AS
> Environment: Java 7, Mac OS
> Reporter: Frank Xu
> Labels: client, hangs
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> The client hangs if the execution has two time-outs. After debugging into the
> issue, we figure out that the resending mechanism has some bugs in it. Here
> are the detailed description.
> Please review the necessity for the invocation of sendCAS(). In our system,
> we don't have to resend the CAS to process again. Please provide a
> configuration so that we don't have to resend the CAS every time there is a
> time out.
> Whenever there is the first time-out,
> BaseUIMAAsynchronousEngine_impl#notifyOnTimout() is invoked and it hangs when
> it tries to invoke sendCas() at line 2385. I believe the reason is that the
> sendCAS() is a synchronized method and a potential threading issue causes
> this thread hang over there. Please be noted that this block is also
> synchronized.
> Then when there is a second time-out, it will be hanging in the very
> beginning of the method notifyOnTimeout() and cannot clear the time-out CAS
> from the CAS list, which hangs the entire client.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
