Re: Sockets not closing
On 7/7/22 11:58, Gordon Ross wrote: I’ve tried doing the bare minimum of: InputStream is = response.getEntity().getContent(); is.close(); response.close(); I remember having some problems with connections staying open, leading the client to run out. I was advised to make sure the entity is fully consumed. Try this code instead of those three lines: HttpEntity entity = response.getEntity(); InputStream is = entity.getContent(); is.close(); EntityUtils.consumeQuietly(entity); response.close(); I am not in any way an expert or even very knowledgeable about httpclient, but this looks like what I ended up doing on advice from this list, which did fix my problem. I can't find the code where I did this, so I can't verify. https://hc.apache.org/httpcomponents-core-5.1.x/current/httpcore5/apidocs/org/apache/hc/core5/http/io/entity/EntityUtils.html#consumeQuietly(org.apache.hc.core5.http.HttpEntity) Based on the javadoc, that might close the InputStream too, but someone who knows more than I do will have to confirm. Thanks, Shawn - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Sockets not closing
How do I do that? Gordon. From: Oleg Kalnichevski Date: Thursday, 7 July 2022 at 19:30 To: HttpClient User Discussion Subject: Re: Sockets not closing On Thu, 2022-07-07 at 18:17 +, Gordon Ross wrote: > If I don’t do the getContent() then the socket is closed as soon as I > do response.close(). This is expected and per design. HttpClient has no choice but to close the connection assuming it may be left in a inconsistent state (there are still bits of the last response message in transit). > > If I do getContent(), the socket stays open for several minutes (in > ESTABLISHED state) and then the socket sits in CLOSE_WAIT for > hours/days. Eventually I can’t open any more connections and I have > to restart the app > You might want to configure HttpClient to evict connections that have expired and been idle too long or evict them manually based on a custom strategy (for instance, at the end of a logical session). Oleg > Gordon. > > From: Oleg Kalnichevski > Date: Thursday, 7 July 2022 at 19:13 > To: HttpClient User Discussion > Subject: Re: Sockets not closing > On Thu, 2022-07-07 at 14:10 -0400, Carter Kozak wrote: > > Hi, > > > > I think you may be seeing the same issue as > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHTTPCLIENT-2221data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C77cf04e8c9194868e32708da6046c99b%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928154380248437%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=MEjL5RtkWCAtg9QiyqNJF0Zdzd%2FnT3KnDzYpTl%2FNIzY%3Dreserved=0 > > , although in a > > slightly different way. > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhttpcomponents-client%2Fpull%2F371data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C77cf04e8c9194868e32708da6046c99b%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928154380248437%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=fTyKc81qVova%2BLaglaMOCdW0s8rlgjT88xs5Z9PoPRQ%3Dreserved=0 > > > > If the socket isn't closed, it's possible that the response has > > been > > drained, and the connection is still pooled awaiting reuse. In the > > response.getEntity().close() case, it's bypassing the response > > draining code-path, and closing the socket. Once the above commit > > has > > been released, both codepaths will produce the same behavior, and I > > suspect in your case neither will close sockets as they're meant to > > be reused later on. Connection reuse can be configured/disabled if > > you prefer, such that closing the response/entity/stream will close > > the socket in all cases. > > > > -ck > > > > I strongly suspect that the connection was simply kept alive and > pooled. That can be established by tuning on connection management > context logging. > > Oleg > > > > On Thu, Jul 7, 2022, at 13:58, Gordon Ross wrote: > > > I’m using HttpClient 5.1.3 and I’m having problems with the > > > underlying socket not closing. This problem only occurs if I call > > > response.getEntity().getContent() > > > > > > If I only call response.getEntity() the underlying socket gets > > > closed fine. > > > > > > I’ve tried doing the bare minimum of: > > > > > > InputStream is = response.getEntity().getContent(); > > > is.close(); > > > response.close(); > > > > > > But even with this, the underlying socket remains open. > > > > > > I’ve tried looking at the samples and I can’t see I’m doing > > > anything different. What am I missing? > > > > > > > > > Gordon Ross (he/him) > > > Collaboration Tools Team > > > University Information Services > > > University of Cambridge > > > > > > While it suits me to email outside normal working hours, I do not > > > expect a response outside your own. > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Sockets not closing
On Thu, 2022-07-07 at 18:17 +, Gordon Ross wrote: > If I don’t do the getContent() then the socket is closed as soon as I > do response.close(). This is expected and per design. HttpClient has no choice but to close the connection assuming it may be left in a inconsistent state (there are still bits of the last response message in transit). > > If I do getContent(), the socket stays open for several minutes (in > ESTABLISHED state) and then the socket sits in CLOSE_WAIT for > hours/days. Eventually I can’t open any more connections and I have > to restart the app > You might want to configure HttpClient to evict connections that have expired and been idle too long or evict them manually based on a custom strategy (for instance, at the end of a logical session). Oleg > Gordon. > > From: Oleg Kalnichevski > Date: Thursday, 7 July 2022 at 19:13 > To: HttpClient User Discussion > Subject: Re: Sockets not closing > On Thu, 2022-07-07 at 14:10 -0400, Carter Kozak wrote: > > Hi, > > > > I think you may be seeing the same issue as > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHTTPCLIENT-2221data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C5a20fddb8b2f4e4664a808da60445714%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928143896007094%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=9VDc8xztk2AIP6jYanWbEXjJGkxca4ey3sZdPGNzgwk%3Dreserved=0 > > , although in a > > slightly different way. > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhttpcomponents-client%2Fpull%2F371data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C5a20fddb8b2f4e4664a808da60445714%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928143896007094%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=mLTk8YnYrZbgYJUMCYe8nL5EzOgHM%2Bpiyd8NOOVO6Kc%3Dreserved=0 > > > > If the socket isn't closed, it's possible that the response has > > been > > drained, and the connection is still pooled awaiting reuse. In the > > response.getEntity().close() case, it's bypassing the response > > draining code-path, and closing the socket. Once the above commit > > has > > been released, both codepaths will produce the same behavior, and I > > suspect in your case neither will close sockets as they're meant to > > be reused later on. Connection reuse can be configured/disabled if > > you prefer, such that closing the response/entity/stream will close > > the socket in all cases. > > > > -ck > > > > I strongly suspect that the connection was simply kept alive and > pooled. That can be established by tuning on connection management > context logging. > > Oleg > > > > On Thu, Jul 7, 2022, at 13:58, Gordon Ross wrote: > > > I’m using HttpClient 5.1.3 and I’m having problems with the > > > underlying socket not closing. This problem only occurs if I call > > > response.getEntity().getContent() > > > > > > If I only call response.getEntity() the underlying socket gets > > > closed fine. > > > > > > I’ve tried doing the bare minimum of: > > > > > > InputStream is = response.getEntity().getContent(); > > > is.close(); > > > response.close(); > > > > > > But even with this, the underlying socket remains open. > > > > > > I’ve tried looking at the samples and I can’t see I’m doing > > > anything different. What am I missing? > > > > > > > > > Gordon Ross (he/him) > > > Collaboration Tools Team > > > University Information Services > > > University of Cambridge > > > > > > While it suits me to email outside normal working hours, I do not > > > expect a response outside your own. > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Sockets not closing
If I don’t do the getContent() then the socket is closed as soon as I do response.close(). If I do getContent(), the socket stays open for several minutes (in ESTABLISHED state) and then the socket sits in CLOSE_WAIT for hours/days. Eventually I can’t open any more connections and I have to restart the app Gordon. From: Oleg Kalnichevski Date: Thursday, 7 July 2022 at 19:13 To: HttpClient User Discussion Subject: Re: Sockets not closing On Thu, 2022-07-07 at 14:10 -0400, Carter Kozak wrote: > Hi, > > I think you may be seeing the same issue as > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHTTPCLIENT-2221data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C5a20fddb8b2f4e4664a808da60445714%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928143896007094%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=9VDc8xztk2AIP6jYanWbEXjJGkxca4ey3sZdPGNzgwk%3Dreserved=0, > although in a > slightly different way. > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhttpcomponents-client%2Fpull%2F371data=05%7C01%7Cgr306%40universityofcambridgecloud.onmicrosoft.com%7C5a20fddb8b2f4e4664a808da60445714%7C49a50445bdfa4b79ade3547b4f3986e9%7C0%7C0%7C637928143896007094%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=mLTk8YnYrZbgYJUMCYe8nL5EzOgHM%2Bpiyd8NOOVO6Kc%3Dreserved=0 > > If the socket isn't closed, it's possible that the response has been > drained, and the connection is still pooled awaiting reuse. In the > response.getEntity().close() case, it's bypassing the response > draining code-path, and closing the socket. Once the above commit has > been released, both codepaths will produce the same behavior, and I > suspect in your case neither will close sockets as they're meant to > be reused later on. Connection reuse can be configured/disabled if > you prefer, such that closing the response/entity/stream will close > the socket in all cases. > > -ck > I strongly suspect that the connection was simply kept alive and pooled. That can be established by tuning on connection management context logging. Oleg > On Thu, Jul 7, 2022, at 13:58, Gordon Ross wrote: > > I’m using HttpClient 5.1.3 and I’m having problems with the > > underlying socket not closing. This problem only occurs if I call > > response.getEntity().getContent() > > > > If I only call response.getEntity() the underlying socket gets > > closed fine. > > > > I’ve tried doing the bare minimum of: > > > > InputStream is = response.getEntity().getContent(); > > is.close(); > > response.close(); > > > > But even with this, the underlying socket remains open. > > > > I’ve tried looking at the samples and I can’t see I’m doing > > anything different. What am I missing? > > > > > > Gordon Ross (he/him) > > Collaboration Tools Team > > University Information Services > > University of Cambridge > > > > While it suits me to email outside normal working hours, I do not > > expect a response outside your own. > > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Sockets not closing
On Thu, 2022-07-07 at 14:10 -0400, Carter Kozak wrote: > Hi, > > I think you may be seeing the same issue as > https://issues.apache.org/jira/browse/HTTPCLIENT-2221, although in a > slightly different way. > https://github.com/apache/httpcomponents-client/pull/371 > > If the socket isn't closed, it's possible that the response has been > drained, and the connection is still pooled awaiting reuse. In the > response.getEntity().close() case, it's bypassing the response > draining code-path, and closing the socket. Once the above commit has > been released, both codepaths will produce the same behavior, and I > suspect in your case neither will close sockets as they're meant to > be reused later on. Connection reuse can be configured/disabled if > you prefer, such that closing the response/entity/stream will close > the socket in all cases. > > -ck > I strongly suspect that the connection was simply kept alive and pooled. That can be established by tuning on connection management context logging. Oleg > On Thu, Jul 7, 2022, at 13:58, Gordon Ross wrote: > > I’m using HttpClient 5.1.3 and I’m having problems with the > > underlying socket not closing. This problem only occurs if I call > > response.getEntity().getContent() > > > > If I only call response.getEntity() the underlying socket gets > > closed fine. > > > > I’ve tried doing the bare minimum of: > > > > InputStream is = response.getEntity().getContent(); > > is.close(); > > response.close(); > > > > But even with this, the underlying socket remains open. > > > > I’ve tried looking at the samples and I can’t see I’m doing > > anything different. What am I missing? > > > > > > Gordon Ross (he/him) > > Collaboration Tools Team > > University Information Services > > University of Cambridge > > > > While it suits me to email outside normal working hours, I do not > > expect a response outside your own. > > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: Sockets not closing
Hi, I think you may be seeing the same issue as https://issues.apache.org/jira/browse/HTTPCLIENT-2221, although in a slightly different way. https://github.com/apache/httpcomponents-client/pull/371 If the socket isn't closed, it's possible that the response has been drained, and the connection is still pooled awaiting reuse. In the response.getEntity().close() case, it's bypassing the response draining code-path, and closing the socket. Once the above commit has been released, both codepaths will produce the same behavior, and I suspect in your case neither will close sockets as they're meant to be reused later on. Connection reuse can be configured/disabled if you prefer, such that closing the response/entity/stream will close the socket in all cases. -ck On Thu, Jul 7, 2022, at 13:58, Gordon Ross wrote: > I’m using HttpClient 5.1.3 and I’m having problems with the underlying socket > not closing. This problem only occurs if I call > response.getEntity().getContent() > > If I only call response.getEntity() the underlying socket gets closed fine. > > I’ve tried doing the bare minimum of: > > InputStream is = response.getEntity().getContent(); > is.close(); > response.close(); > > But even with this, the underlying socket remains open. > > I’ve tried looking at the samples and I can’t see I’m doing anything > different. What am I missing? > > > Gordon Ross (he/him) > Collaboration Tools Team > University Information Services > University of Cambridge > > While it suits me to email outside normal working hours, I do not expect a > response outside your own. >
Sockets not closing
I’m using HttpClient 5.1.3 and I’m having problems with the underlying socket not closing. This problem only occurs if I call response.getEntity().getContent() If I only call response.getEntity() the underlying socket gets closed fine. I’ve tried doing the bare minimum of: InputStream is = response.getEntity().getContent(); is.close(); response.close(); But even with this, the underlying socket remains open. I’ve tried looking at the samples and I can’t see I’m doing anything different. What am I missing? Gordon Ross (he/him) Collaboration Tools Team University Information Services University of Cambridge While it suits me to email outside normal working hours, I do not expect a response outside your own.
