[
https://issues.apache.org/jira/browse/AXIS2-6030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875191#comment-17875191
]
David Steindorfer edited comment on AXIS2-6030 at 8/20/24 3:05 PM:
-------------------------------------------------------------------
Thank you for the workaround [~bbehling]!
It was working locally so we decided to take it live, but for some reason the
jobs on our live system got stuck.
When we forced a shutdown of the server we noticed following stack traces
(shortened to leave out sensitive information)
-
org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:393)
-
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:306)
-
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
-
org.apache.axis2.transport.http.impl.httpclient4.RequestImpl.execute(RequestImpl.java:210)
- org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:431)
-
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:399)
-
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
-
org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:579)
-
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:373)
-
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invoke(JAXWSProxyHandler.java:171)
It seemed there was a different setup with the other system we communicate than
what I had locally.
After more investigations I noticed that Axis2 does not close the initial input
stream it gets from the http client execute.
Here is a screenshot of an assignment of the input stream and storing it inside
Axis2 internal code:
{code:java}
org.apache.axis2.kernel.TransportUtils#createSOAPMessage{code}
!inputStreamAssignment.png!
(!) here I think it would make sense to also close the input stream when the
"detach" flag is true. (but it might already close the input stream in this
case, which needs to be investigated)
—
*Working solution when "detach" flag is false*
After fidling around for a while I think I found a good place to close the
input stream and make sure that connection can be released.
A new function was added to close the input stream for the Http Connection to
be released
!closeInputStreamFn.png!
{code:java}
private void closeInputStream(MessageContext responseContext) {
// accessing the input stream is not possible via get
// workaround using entry set
Iterator var2 = responseContext.getMEPContext().entrySet().iterator();
while(var2.hasNext()) {
Object entry = var2.next();
if (entry instanceof Map.Entry &&
"TRANSPORT_IN".equals(((Map.Entry)entry).getKey())) {
Object prop = ((Map.Entry)entry).getValue();
if (prop instanceof InputStream) {
try {
InputStream inputStream = (InputStream)prop;
inputStream.close();
} catch (Exception var6) {
log.error(var6.getMessage(), var6);
}
}
break;
}
}
}
{code}
This function was used in the "finally" block of the "createResponse" function
to clean up the resources.
Affected class is
{code:java}
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler{code}
!usageInCreateResponseFn.png!
Also if local debugging is possible you can always peek into the following
function to see how many connections are leased and how many are available.
{code:java}
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection
{code}
this -> pool (variable of class)
- leased
- available
Hope this helps other people as well if they ever stumble upon this issue!
was (Author: JIRAUSER287101):
Thank you for the workaround [~bbehling]!
It was working locally so we decided to take it live, but for some reason the
jobs on our live system got stuck.
When we forced a shutdown of the server we noticed following stack traces
(shortened to leave out sensitive information)
-
org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:393)
-
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:306)
-
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
-
org.apache.axis2.transport.http.impl.httpclient4.RequestImpl.execute(RequestImpl.java:210)
- org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:431)
-
org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:399)
-
org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
-
org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:579)
-
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:373)
-
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invoke(JAXWSProxyHandler.java:171)
It seemed there was a different setup with the other system we communicate than
what I had locally.
After more investigations I noticed that Axis2 does not close the initial input
stream it gets from the http client execute.
Here is a screenshot of an assignment of the input stream and storing it inside
Axis2 internal code:
{code:java}
org.apache.axis2.kernel.TransportUtils#createSOAPMessage{code}
!inputStreamAssignment.png!
(!) here I think it would make sense to also close the input stream when the
"detach" flag is true.
---
*Working solution when detach flag is false*
After fidling around for a while I think I found a good place to close the
input stream and make sure that connection can be released.
A new function was added to close the input stream for the Http Connection to
be released
!closeInputStreamFn.png!
{code:java}
private void closeInputStream(MessageContext responseContext) {
// accessing the input stream is not possible via get
// workaround using entry set
Iterator var2 = responseContext.getMEPContext().entrySet().iterator();
while(var2.hasNext()) {
Object entry = var2.next();
if (entry instanceof Map.Entry &&
"TRANSPORT_IN".equals(((Map.Entry)entry).getKey())) {
Object prop = ((Map.Entry)entry).getValue();
if (prop instanceof InputStream) {
try {
InputStream inputStream = (InputStream)prop;
inputStream.close();
} catch (Exception var6) {
log.error(var6.getMessage(), var6);
}
}
break;
}
}
}
{code}
This function was used in the "finally" block of the "createResponse" function
to clean up the resources.
Affected class is
{code:java}
org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler{code}
!usageInCreateResponseFn.png!
Also if local debugging is possible you can always peek into the following
function to see how many connections are leased and how many are available.
{code:java}
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection
{code}
this -> pool (variable of class)
- leased
- available
Hope this helps other people as well if they ever stumble upon this issue!
> Axis2 connections are not returned to connection pool on 1.8.0 with JAXWS
> -------------------------------------------------------------------------
>
> Key: AXIS2-6030
> URL: https://issues.apache.org/jira/browse/AXIS2-6030
> Project: Axis2
> Issue Type: Bug
> Components: jaxws
> Affects Versions: 1.8.0
> Reporter: David Steindorfer
> Assignee: Robert Lazarski
> Priority: Critical
> Fix For: 1.8.3
>
> Attachments: EchoTest.java, XMLSpineImpl_Fix.zip,
> closeInputStreamFn.png, echoTestInterruptionException.txt,
> inputStreamAssignment.png, jaxws-samples.png, screenshot-1.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png,
> usageInCreateResponseFn.png
>
>
> Good Afternoon,
>
> I have upgraded Axis2 from 1.7.9 to 1.8.0 on one of our projects.
> However with the upgrade the requests seem to get stuck.
> On further analysis I figured out that the connection pool ran dry as the
> connections were not returned.
>
> It took a while to reproduce the issue reliably and isolated from the other
> code of our project but under very similar circumstances.
> Thus I created a public git repo with the code I wrote to reproduce the issue:
> [https://github.com/dsteindo/Axis2Examples]
>
> I think the affected component is the "axis2-transport-http" library.
> For now I reverted the project's libraries back to 1.7.9.
> But it would be awesome to use the latest libraries, considering that you did
> quite a lot of changes and improvements :)
>
> Please feel free to contact me if more information is needed, but I guess
> that everything should be documented with the git repository.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org
For additional commands, e-mail: java-dev-h...@axis.apache.org
