[
https://issues.apache.org/jira/browse/HTTPASYNC-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831992#comment-15831992
]
Oleg Kalnichevski commented on HTTPASYNC-117:
---------------------------------------------
Michael
HttpAsyncClient 4.x has not been designed to handle such use case. Generally
one _ought_ to keep code executed in callbacks as light as possible given the
same I/O dispatch thread has to handle I/O events of multiple connections. This
especially implies avoidance of potentially blocking or time consuming
operations in callbacks. So, one _ought_ not attempt to submit a new request
directly from the result callback of another request. There is nothing I can
about the problem other than suggesting to use an intermediate request queue
managed by a dedicated thread for request submission and revisiting the problem
in the course of 5.0 development.
Oleg
> Blocking thread when executing a request in a FutureCallback
> ------------------------------------------------------------
>
> Key: HTTPASYNC-117
> URL: https://issues.apache.org/jira/browse/HTTPASYNC-117
> Project: HttpComponents HttpAsyncClient
> Issue Type: Bug
> Affects Versions: 4.1.2
> Environment: Environment Oracle JDK 8u121
> Reporter: Michael Ernst
> Priority: Critical
> Attachments: blocking-thread-example-application.zip
>
>
> When executing a request within a {{FutureCallback}} (inner request) the
> invoking thread is blocked. The inner request will not be executed and
> processing the request terminates after the write event.
> If the inner request is executed in another thread, the application works
> fine (see the thenApplyAsync note in the code).
> This issue seems already be an issue for another user who reported this in a
> StackOverflow question:
> http://stackoverflow.com/questions/32031846/apaches-httpasyncclient-never-returns-after-execute
> See the following log when it blocks:
> {noformat}
> IM-WEBAPP 2017-01-20 09:27:44,506 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.client.MainClientExec - [exchange: 4] start
> execution
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected:
> default
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.client.protocol.RequestAuthCache - Auth cache not set in the
> context
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 4]
> Request connection for
> {tls}->http://localhost:3128->https://permissions-api.apps.bosch-iot-cloud.com:443
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager -
> Connection request: [route:
> {tls}->http://localhost:3128->https://permissions-api.apps.bosch-iot-cloud.com:443][total
> kept alive: 1; route allocated: 1 of 20; total allocated: 1 of 200]
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl -
> http-outgoing-0
> 127.0.0.1:57860<->127.0.0.1:3128[ACTIVE][r:r][ACTIVE][r][NOT_HANDSHAKING][0][0][0][0]:
> Set timeout 0
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager -
> Connection leased: [id: http-outgoing-0][route:
> {tls}->http://localhost:3128->https://permissions-api.apps.bosch-iot-cloud.com:443][total
> kept alive: 0; route allocated: 1 of 20; total allocated: 1 of 200]
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 4]
> Connection allocated: CPoolProxy{http-outgoing-0 [ACTIVE]}
> IM-WEBAPP 2017-01-20 09:27:44,507 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl -
> http-outgoing-0
> 127.0.0.1:57860<->127.0.0.1:3128[ACTIVE][r:r][ACTIVE][r][NOT_HANDSHAKING][0][0][0][0]:
> Set attribute http.nio.exchange-handler
> IM-WEBAPP 2017-01-20 09:27:44,508 [I/O dispatcher 1] DEBUG
> org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl -
> http-outgoing-0
> 127.0.0.1:57860<->127.0.0.1:3128[ACTIVE][rw:r][ACTIVE][rw][NOT_HANDSHAKING][0][0][0][0]:
> Event set [w]
> {noformat}
> The stack of the blocking thread:
> {noformat}
> Thread [I/O dispatcher 1] (Suspended)
> sun.misc.Unsafe.park(boolean, long) line: not available [native method]
> java.util.concurrent.locks.LockSupport.park(java.lang.Object) line: 175
> java.util.concurrent.CompletableFuture$Signaller.block() line: 1693
>
> java.util.concurrent.ForkJoinPool.managedBlock(java.util.concurrent.ForkJoinPool$ManagedBlocker)
> line: 3323
> java.util.concurrent.CompletableFuture<T>.waitingGet(boolean) line:
> 1729
> java.util.concurrent.CompletableFuture<T>.get() line: 1895
>
> BlockingThreadMainApplication.lambda$0(org.apache.http.impl.nio.client.CloseableHttpAsyncClient,
> java.io.InputStream) line: 60
>
> BlockingThreadMainApplication$$Lambda$4.1112414583.apply(java.lang.Object)
> line: not available
>
> java.util.concurrent.CompletableFuture<T>.uniApply(java.util.concurrent.CompletableFuture<S>,
> java.util.function.Function<? super S,? extends T>,
> java.util.concurrent.CompletableFuture.UniApply<S,T>) line: 602
> java.util.concurrent.CompletableFuture$UniApply<T,V>.tryFire(int) line:
> 577
> java.util.concurrent.CompletableFuture<T>.postComplete() line: 474
> java.util.concurrent.CompletableFuture<T>.complete(T) line: 1962
> BlockingThreadMainApplication$1.completed(org.apache.http.HttpResponse)
> line: 83
> BlockingThreadMainApplication$1.completed(java.lang.Object) line: 1
> org.apache.http.concurrent.BasicFuture<T>.completed(T) line: 119
>
> org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl<T>.responseCompleted()
> line: 177
>
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(org.apache.http.nio.NHttpClientConnection,
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor$State,
> org.apache.http.nio.protocol.HttpAsyncClientExchangeHandler) line: 436
>
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(org.apache.http.nio.NHttpClientConnection,
> org.apache.http.nio.ContentDecoder) line: 326
>
> org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(org.apache.http.nio.NHttpClientConnection,
> org.apache.http.nio.ContentDecoder) line: 83
>
> org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl(org.apache.http.impl.nio.DefaultNHttpClientConnection).consumeInput(org.apache.http.nio.NHttpClientEventHandler)
> line: 265
>
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(org.apache.http.impl.nio.DefaultNHttpClientConnection)
> line: 81
>
> org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(java.lang.Object)
> line: 39
>
> org.apache.http.impl.nio.client.InternalIODispatch(org.apache.http.impl.nio.reactor.AbstractIODispatch<T>).inputReady(org.apache.http.nio.reactor.IOSession)
> line: 121
>
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(java.nio.channels.SelectionKey)
> line: 162
>
> org.apache.http.impl.nio.reactor.BaseIOReactor(org.apache.http.impl.nio.reactor.AbstractIOReactor).processEvent(java.nio.channels.SelectionKey)
> line: 337
>
> org.apache.http.impl.nio.reactor.BaseIOReactor(org.apache.http.impl.nio.reactor.AbstractIOReactor).processEvents(java.util.Set<java.nio.channels.SelectionKey>)
> line: 315
>
> org.apache.http.impl.nio.reactor.BaseIOReactor(org.apache.http.impl.nio.reactor.AbstractIOReactor).execute()
> line: 276
>
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(org.apache.http.nio.reactor.IOEventDispatch)
> line: 104
>
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run()
> line: 588
> java.lang.Thread.run() line: 745
> {noformat}
> Be aware that this issue could sometimes be a Heisenbug and the application
> terminates successfully in debugging. When running normally it fails on my
> machine with 100%.
> You can simply reproduce this issue with the following code or use the
> attached maven project:
> {code:java}
> import java.io.IOException;
> import java.io.InputStream;
> import java.util.concurrent.CompletableFuture;
> import java.util.concurrent.ExecutionException;
> import org.apache.http.HttpHost;
> import org.apache.http.HttpResponse;
> import org.apache.http.client.config.RequestConfig;
> import org.apache.http.client.methods.HttpGet;
> import org.apache.http.concurrent.FutureCallback;
> import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
> import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
> import org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager;
> import org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor;
> import org.apache.http.nio.reactor.IOReactorException;
> public class BlockingThreadMainApplication
> {
> private static final int POOL_PER_ROUTE = 20;
> private static final int POOL_MAX = 200;
> private static final int CONNECT_TIMEOUT = 10000;
> private static final int SOCKET_TIMEOUT = 60000;
> private static final String PUBLIC_KEYS_URL =
> "https://permissions-api.apps.bosch-iot-cloud.com/2/rest/publickeys";;
> private static HttpHost proxy = null;
> private int requestCounter = 0;
> public static void main(final String[] args) throws InterruptedException,
> ExecutionException, IOException
> {
> // include this lines if you are behind a proxy server
> // proxy = new HttpHost("localhost", 3128);
> final BlockingThreadMainApplication application = new
> BlockingThreadMainApplication();
> try (CloseableHttpAsyncClient asyncClient =
> application.createAsynClient())
> {
> // these request work like they should
> application.createRequest(asyncClient).get();
> application.createRequest(asyncClient).get();
> System.out.println("Execute request with inner request");
> final Object object =
> application.createRequestWithInnerRequest(asyncClient).get();
> // will never be invoked
> System.out.println("Yeaha, application completed. Resolved last
> response " + object.getClass().getName());
> }
> }
> private CompletableFuture<?> createRequestWithInnerRequest(final
> CloseableHttpAsyncClient asyncClient)
> throws InterruptedException, ExecutionException
> {
> // it works like a charm when using thenApplyAsync so it is executed in
> another thread
> return createRequest(asyncClient).thenApply(i ->
> {
> // execute another request and get its result
> try
> {
> System.out.println("executing request in thenApply");
> createRequest(asyncClient).get(); // never comes back
> System.out.println("executed request in thenApply");
> }
> catch (InterruptedException | ExecutionException e)
> {
> throw new IllegalStateException(e);
> }
> return i;
> });
> }
> public CompletableFuture<InputStream> createRequest(final
> CloseableHttpAsyncClient asyncClient)
> {
> final int requestId = ++requestCounter;
> final CompletableFuture<InputStream> toBeCompleted = new
> CompletableFuture<>();
> final FutureCallback<HttpResponse> callback = new
> FutureCallback<HttpResponse>()
> {
> @Override
> public void completed(final HttpResponse response)
> {
> System.out.println("completed " + requestId);
> try
> {
> toBeCompleted.complete(response.getEntity().getContent());
> }
> catch (UnsupportedOperationException | IOException e)
> {
> System.out.println("failed to get InputStream from response "
> + requestId);
> toBeCompleted.completeExceptionally(e);
> }
> }
> @Override
> public void cancelled()
> {
> System.out.println("cancelled " + requestId);
> toBeCompleted.cancel(true);
> }
> @Override
> public void failed(final Exception e)
> {
> System.out.println("failed " + requestId);
> toBeCompleted.completeExceptionally(e);
> }
> };
> asyncClient.execute(getGetRequest(), callback);
> return toBeCompleted;
> }
> private CloseableHttpAsyncClient createAsynClient()
> {
> // Create common default configuration
> final RequestConfig clientConfig =
> RequestConfig.custom().setConnectTimeout(CONNECT_TIMEOUT)
>
> .setSocketTimeout(SOCKET_TIMEOUT).setConnectionRequestTimeout(SOCKET_TIMEOUT).setProxy(proxy).build();
> DefaultConnectingIOReactor ioreactor;
> PoolingNHttpClientConnectionManager asyncConnectionManager;
> try
> {
> ioreactor = new DefaultConnectingIOReactor();
> asyncConnectionManager = new
> PoolingNHttpClientConnectionManager(ioreactor);
> asyncConnectionManager.setMaxTotal(POOL_MAX);
> asyncConnectionManager.setDefaultMaxPerRoute(POOL_PER_ROUTE);
> }
> catch (final IOReactorException e)
> {
> throw new RuntimeException(e);
> }
> final CloseableHttpAsyncClient client =
> HttpAsyncClientBuilder.create().setDefaultRequestConfig(clientConfig)
> .setConnectionManager(asyncConnectionManager).build();
> client.start();
> return client;
> }
> private HttpGet getGetRequest()
> {
> return new HttpGet(PUBLIC_KEYS_URL);
> }
> }
> {code}
> I don't think it is related to HTTPASYNC-35 or HTTPASYNC-34 who sounds
> similiar.
> Just one last note: Why is 4.1.2 released but not in this Jira project?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
