Thanks Oleg. I looked at your example, but it was not clear to me as to how I can avoid passing an input stream from the http request. So in our case the incoming data from the client is obtained in form of an input stream and we create a HttpAsyncRequestProducer and pass the entity to it. Do you have any examples for such a use case to see how we can improve our code?
regards, Nitu On Tue, Nov 28, 2017 at 1:18 PM, Oleg Kalnichevski <[email protected]> wrote: > On Tue, Nov 28, 2017 at 7:43 PM, Nitu Choudhary <[email protected]> > wrote: > >> sorry, I am still a little confused. Can you explain the message exchange >> in more detail? >> >> Say I get a future when I make a request. Now I will typically do a >> future.get and get the future http response and consume it. However, in >> some cases I might not be able to call a get on the future, but the future >> is still holding the response. When does this memory get released? >> >> regards, >> Nitu >> > > Hi Nitu > > The response future does not necessarily hold a reference to the entire > response message (though it might in the most over-simplistic scenario, > which I would recommend to avoid). It rather holds reference to a result of > response processing generated by the associated HttpAsyncResponseConsumer. > See this sample for a reference > > http://hc.apache.org/httpcomponents-asyncclient-4.1.x/ > httpasyncclient/examples/org/apache/http/examples/nio/clien > t/AsyncClientHttpExchangeStreaming.java > > HttpAsyncResponseConsumer is responsible for resource allocation and > deallocation. HttpAsyncClient makes sure HttpAsyncResponseConsumer-s get to > complete their work (normally or abnormally) regardless of what happens to > the response future. As long as the result object does not hold any > resources beyond objects in the heap, it does not need to be explicitly > closed or released. > > Hope this helps > > Oleg > > > > >> On Tue, Nov 28, 2017 at 12:15 AM, Oleg Kalnichevski <[email protected]> >> wrote: >> >> >>> >>> On Mon, Nov 27, 2017 at 6:57 PM, Nitu Choudhary <[email protected]> >>> wrote: >>> >>> thanks Oleg. What happens if in a successful case the future is not >>>> consumed by the application? When will that memory be released? >>>> >>>> >>> Upon completion of the message exchange (assuming both request producer >>> and response consumer correctly deallocate their resources). >>> >>> Oleg >>> >>> >>> regards, >>>> Nitu >>>> >>>> On Thu, Nov 23, 2017 at 3:08 AM, Oleg Kalnichevski <[email protected]> >>>> wrote: >>>> >>>> >>>> >>>>> On Wed, Nov 22, 2017 at 9:19 PM, Nitu Choudhary >>>>> <[email protected]> >>>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>>> >>>>>> Under high concurrency we see cases where I/O reactor is getting >>>>>> terminated >>>>>> when closing the input stream. In our application we also close the >>>>>> input >>>>>> stream as we want to cleanup the memory under various exception >>>>>> conditions. >>>>>> What is the recommended way to consume the http entity? It shows >>>>>> that >>>>>> on >>>>>> timeout the nio client is trying to consume the entity, but it has >>>>>> already >>>>>> been consumed by our application as it got a done signal, and we >>>>>> retrieved >>>>>> the entity and closed it. >>>>>> >>>>>> >>>>>> >>>>> Hi Nitu >>>>> >>>>> Unlike its blocking counterpart HttpAsyncClient automatically >>>>> deallocates >>>>> resources once a message exchange is complete (successfully or >>>>> unsuccessfully). One does not need to ensure the response entity is >>>>> fully >>>>> consumed in order to ensure proper deallocation of the connection >>>>> associated with it. This is taken care of by the framework. >>>>> >>>>> However I would strongly recommend to avoid mixing non-blocking I/O >>>>> with >>>>> inherently blocking InputStream / OutputStream based APIs. I would >>>>> recommend using proper non-blocking HttpAsyncRequestProducer and >>>>> HttpAsyncResponseConsumer implementations for any non-trivial >>>>> request / >>>>> response exchanges with HttpAsyncClient. In other words try getting >>>>> rid >>>>> of >>>>> SequenceInputStream entirely. >>>>> >>>>> Hope this helps >>>>> >>>>> Oleg >>>>> >>>>> >>>>> >>>>> Here is the stack for your reference: >>>>> >>>>>> >>>>>> org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O >>>>>> reactor >>>>>> terminated abnormally java.util.NoSuchElementException: null at >>>>>> java.util. >>>>>> ArrayList$Itr.next(ArrayList.java:854) at >>>>>> java.util.Collections$3.nextEl >>>>>> ement(Collections.java:5216) at java.io.SequenceInputStream.ne >>>>>> xtStream( >>>>>> SequenceInputStream.java:110) at java.io.SequenceInputStream.close( >>>>>> SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent >>>>>> ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) >>>>>> at >>>>>> org. >>>>>> apache.http.nio.protocol.BasicAsyncRequestProducer.close( >>>>>> BasicAsyncRequestProducer.java:155) at >>>>>> org.apache.http.impl.nio.clien >>>>>> t. >>>>>> DefaultClientExchangeHandlerImpl.releaseResources(DefaultC >>>>>> lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.clien >>>>>> t. >>>>>> AbstractClientExchangeHandler.failed(AbstractClientExchangeH >>>>>> andler.java:429) >>>>>> at org.apache.http.impl.nio.client.AbstractClientExchangeHandler. >>>>>> connectionRequestFailed(AbstractClientExchangeHandler.java:348) at >>>>>> org. >>>>>> apache.http.impl.nio.client.AbstractClientExchangeHandler.ac >>>>>> cess$100( >>>>>> AbstractClientExchangeHandler.java:62) at >>>>>> org.apache.http.impl.nio.clien >>>>>> t. >>>>>> AbstractClientExchangeHandler$1.failed(AbstractClientExchang >>>>>> eHandler.java: >>>>>> 392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture. >>>>>> >>>>>> java:134) >>>>>> at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionMa >>>>>> nager$1. >>>>>> failed(PoolingNHttpClientConnectionManager.java:289) at >>>>>> org.apache.http. >>>>>> concurrent.BasicFuture.failed(BasicFuture.java:134) at >>>>>> org.apache.http.nio. >>>>>> pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at >>>>>> org.apache. >>>>>> http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO >>>>>> ConnPool.java: >>>>>> 584) at org.apache.http.nio.pool.Abstr >>>>>> actNIOConnPool$InternalSession >>>>>> RequestCallback.timeout(AbstractNIOConnPool.java:851) at >>>>>> org.apache.http. >>>>>> impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestI >>>>>> mpl.java:183) >>>>>> at >>>>>> org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor. >>>>>> processTimeouts( >>>>>> DefaultConnectingIOReactor.java:210) at >>>>>> org.apache.http.impl.nio.react >>>>>> or. >>>>>> DefaultConnectingIOReactor.processEvents(DefaultConnectingIO >>>>>> Reactor.java:155) >>>>>> at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReacto >>>>>> r.execute( >>>>>> AbstractMultiworkerIOReactor.java:348) at >>>>>> org.apache.http.impl.nio.conn. >>>>>> PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie >>>>>> ntConnectionManager.java:194) at org.apache.http.impl.nio.client. >>>>>> CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClient >>>>>> Base.java:64) >>>>>> at >>>>>> java.lang.Thread.run(Thread.java:748) >>>>>> >>>>>> regards, >>>>>> nitu >>>>>> >>>>>> >>>>>> >>>>> ------------------------------------------------------------ >>>>> --------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>>> >>>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
