Re: HTTP/2 client and closed connections
On 18/03/2024 03:01, Jonathan Amir wrote: Thank you again for your detailed answer. I am wondering about the pros and cons of validateAfterInactivity vs. evictIdleConnections. validateAfterInactivity - I am guessing, for lack of any other information, that inactivity here is meant to be the same as idleness. Thus, after the connection is idle for a while it will be validated. If it is valid then it is kept. If it isn't, then it is dropped and at the next request httpclient will create a new connection. #validateAfterInactivity is performed by the connection manager while processing connection lease requests. Basically if the client is idle and there are no lease requests stale connections can get stuck in the pool indefinitely. evictIdleConnections - after a while the connection is dropped (evicted) unconditionally and then at the next request we pay the price to create a new connection. #evictIdleConnections proactively evicts idle connections irrespective of client activity (or inactivity). It seems to me that the performance characteristics are as follows. If the connection is no longer valid: - validate - we pay the price to validate the connection and the price to create a new one. - evict - we only pay the price to create a new connection. If the connection is still good: - validate - we pay the price of validating it and that's it. - evict - we pay the price of creating a new connection, but it was not really necessary. Is the cost of validating a connection comparable to the cost of creating a new one? Certainly new connection creation is more expensive than a single validation check, especially if TLS is involved. However if one executes thousands of requests over the same connection the total cost of validation can become significant. The whole point of having the #evictIdleConnections setting is to reduce instances of validation checks. Let's say if the connection was operational 1 second ago, it is quite likely to be still valid. After 30 seconds, who knows? So, if I think that idle connections will mostly stay valid, the validation option is better. If I think they'll mostly go bad, the eviction is better. Pretty much. One should probably employ both settings, but keep #evictIdleConnections invocations infrequent, say, every 3 minutes of so, just to make sure the connection pool gets flushed if there is no activity for a long period of time and there are no lease requests that could cause stale connections to get evicted. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HTTP/2 client and closed connections
Thank you again for your detailed answer. I am wondering about the pros and cons of validateAfterInactivity vs. evictIdleConnections. validateAfterInactivity - I am guessing, for lack of any other information, that inactivity here is meant to be the same as idleness. Thus, after the connection is idle for a while it will be validated. If it is valid then it is kept. If it isn't, then it is dropped and at the next request httpclient will create a new connection. evictIdleConnections - after a while the connection is dropped (evicted) unconditionally and then at the next request we pay the price to create a new connection. It seems to me that the performance characteristics are as follows. If the connection is no longer valid: - validate - we pay the price to validate the connection and the price to create a new one. - evict - we only pay the price to create a new connection. If the connection is still good: - validate - we pay the price of validating it and that's it. - evict - we pay the price of creating a new connection, but it was not really necessary. Is the cost of validating a connection comparable to the cost of creating a new one? So, if I think that idle connections will mostly stay valid, the validation option is better. If I think they'll mostly go bad, the eviction is better. Thanks, Jonathan On Fri, Mar 15, 2024 at 4:44 AM Oleg Kalnichevski wrote: > > > On 14/03/2024 21:53, Jonathan Amir wrote: > > Thanks for your answers, they help a lot. > > So far what I am understanding about the connection closed situation is > > that it is recoverable, but it is the responsibility of the caller to > > implement its own retry mechanism. > > It is potentially recoverable but the exact recovery strategy tends to > be application specific. > > > The retry isn't built-in to the http client code itself but the recovery > of > > the internally managed connection is handled by the http client. > > Is that right? > > > > No, it is not. There a default retry-mechanism in place but it applies > to idempotent methods only. Please take a look at > DefaultHttpRequestRetryStrategy. HttpClient should automatically re-try > idempotent methods, but non-idempotent method recovery is application > specific and is considered a responsibility of the caller. > > ConnectionClosedException is presently handled as non-recoverable by > default. I am actually not sure this should be the case, but this is how > it is now. > > @Michael, do you happen to remember why we ended up treating > ConnectionClosedException as non-recoverable? > > However, It is always advisable to have one's own application specific > recovery strategy. > > > > Also, a small follow-up question about the TTL, how is it related (or > not) > > to ConnectionConfig.setValidateAfterInactivity, > > Validate-after-inactivity and total-time-to-live are unrelated. Both, > however, can cause the connection to be considered expired. > > and are those two related > > to client's builder evictIdleConnections method? > > Those two settings are unrelated to idle connection eviction. A > connection can be perfectly valid but it stays idle too long, it can get > dropped. > > > > Between idleness, TTL, and validation, what is supposed to be the correct > > way to use these three configurations together? > > > > This is entirely up to the caller. One may want to have a fairly long > TTL, say 15 minutes, to ensure connections get refreshed every once in a > while. The validate-after-inactivity check is not cheap. One should use > it sparingly. It is usually up to the caller to decide, what is more > preferable, a certain performance hit due to the > validate-after-inactivity check or an occasional i/o exception due to > the connection being stale. What connections are considered idle is > entirely up to the caller. > > Oleg > > > > On Wed, Mar 13, 2024 at 5:14 AM Oleg Kalnichevski > wrote: > > > >> On Tue, 2024-03-12 at 21:58 -0400, Jonathan Amir wrote: > >>> Hello, > >>> I am building an HTTP/2 only client for running multiple requests in > >>> parallel. > >>> I understand that there is no connection pool internally, rather > >>> there is > >>> one connection per host. For simplicity, let's say all my requests go > >>> to > >>> the same host. > >>> > >> > >> This is correct. It is still technically a pool though (per host > >> basis). > >> > >> > >>> I have a situation where under stress there are some errors. It > >>> starts with > >>> socket timeout (several threads in parallel), and after a while there > >>> is a > >>> ConnectionClosedException. > >> > >> Even though HTTP/2 has a proper connection termination handshake, the > >> handshake is potentially racy. Under high load > >> ConnectionClosedException can and will happen. Your application code > >> must be prepared to handle those. > >> > >> > >>> I am not sure what is the flow of events that leads to this, and what > >>> is > >>> the relationship between those errors. I also don't know if it is my > >>>
Re: HTTP/2 client and closed connections
On 14/03/2024 21:53, Jonathan Amir wrote: Thanks for your answers, they help a lot. So far what I am understanding about the connection closed situation is that it is recoverable, but it is the responsibility of the caller to implement its own retry mechanism. It is potentially recoverable but the exact recovery strategy tends to be application specific. The retry isn't built-in to the http client code itself but the recovery of the internally managed connection is handled by the http client. Is that right? No, it is not. There a default retry-mechanism in place but it applies to idempotent methods only. Please take a look at DefaultHttpRequestRetryStrategy. HttpClient should automatically re-try idempotent methods, but non-idempotent method recovery is application specific and is considered a responsibility of the caller. ConnectionClosedException is presently handled as non-recoverable by default. I am actually not sure this should be the case, but this is how it is now. @Michael, do you happen to remember why we ended up treating ConnectionClosedException as non-recoverable? However, It is always advisable to have one's own application specific recovery strategy. Also, a small follow-up question about the TTL, how is it related (or not) to ConnectionConfig.setValidateAfterInactivity, Validate-after-inactivity and total-time-to-live are unrelated. Both, however, can cause the connection to be considered expired. and are those two related to client's builder evictIdleConnections method? Those two settings are unrelated to idle connection eviction. A connection can be perfectly valid but it stays idle too long, it can get dropped. Between idleness, TTL, and validation, what is supposed to be the correct way to use these three configurations together? This is entirely up to the caller. One may want to have a fairly long TTL, say 15 minutes, to ensure connections get refreshed every once in a while. The validate-after-inactivity check is not cheap. One should use it sparingly. It is usually up to the caller to decide, what is more preferable, a certain performance hit due to the validate-after-inactivity check or an occasional i/o exception due to the connection being stale. What connections are considered idle is entirely up to the caller. Oleg On Wed, Mar 13, 2024 at 5:14 AM Oleg Kalnichevski wrote: On Tue, 2024-03-12 at 21:58 -0400, Jonathan Amir wrote: Hello, I am building an HTTP/2 only client for running multiple requests in parallel. I understand that there is no connection pool internally, rather there is one connection per host. For simplicity, let's say all my requests go to the same host. This is correct. It is still technically a pool though (per host basis). I have a situation where under stress there are some errors. It starts with socket timeout (several threads in parallel), and after a while there is a ConnectionClosedException. Even though HTTP/2 has a proper connection termination handshake, the handshake is potentially racy. Under high load ConnectionClosedException can and will happen. Your application code must be prepared to handle those. I am not sure what is the flow of events that leads to this, and what is the relationship between those errors. I also don't know if it is my client or the server that closed the connection. It, of source, would help greatly to know what exactly happens and leads to ConnectionClosedException. My initial question is, since there is only one connection maintained internally, how does one recover from ConnectionClosedException? The connection life-cycle is opaque to me - there is no pool, and no eviction strategy, so no concept of creating a new connection. So what am I missing? Is the httpClient object still usable after a ConnectionClosedException? The internal connection pool can automatically re-establish closed connection once the connection termination handshake completes or the connection gets dropped abnormally. Somewhat related, I am looking at the sample here: https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/migration-to-async-http2.html What is the difference between the two socket timeout configurations, on IOReactorConfig and ConnectionConfig? Both represent the same timeout but apply at different levels. IOReactorConfig apply at the i/o reactor level and are specific to the async i/o model. ConnectionConfig apply at the connection management level and is not specific to any i/o model. What is the time to live? You mean TTL, total time to live? The maximum period of time connections can be kept alive and re-used. Once past TTL connections get automatically closed out. Hope this helps Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HTTP/2 client and closed connections
Thanks for your answers, they help a lot. So far what I am understanding about the connection closed situation is that it is recoverable, but it is the responsibility of the caller to implement its own retry mechanism. The retry isn't built-in to the http client code itself but the recovery of the internally managed connection is handled by the http client. Is that right? Also, a small follow-up question about the TTL, how is it related (or not) to ConnectionConfig.setValidateAfterInactivity, and are those two related to client's builder evictIdleConnections method? Between idleness, TTL, and validation, what is supposed to be the correct way to use these three configurations together? On Wed, Mar 13, 2024 at 5:14 AM Oleg Kalnichevski wrote: > On Tue, 2024-03-12 at 21:58 -0400, Jonathan Amir wrote: > > Hello, > > I am building an HTTP/2 only client for running multiple requests in > > parallel. > > I understand that there is no connection pool internally, rather > > there is > > one connection per host. For simplicity, let's say all my requests go > > to > > the same host. > > > > This is correct. It is still technically a pool though (per host > basis). > > > > I have a situation where under stress there are some errors. It > > starts with > > socket timeout (several threads in parallel), and after a while there > > is a > > ConnectionClosedException. > > Even though HTTP/2 has a proper connection termination handshake, the > handshake is potentially racy. Under high load > ConnectionClosedException can and will happen. Your application code > must be prepared to handle those. > > > > I am not sure what is the flow of events that leads to this, and what > > is > > the relationship between those errors. I also don't know if it is my > > client > > or the server that closed the connection. > > > > It, of source, would help greatly to know what exactly happens and > leads to ConnectionClosedException. > > > > My initial question is, since there is only one connection maintained > > internally, how does one recover from ConnectionClosedException? The > > connection life-cycle is opaque to me - there is no pool, and no > > eviction > > strategy, so no concept of creating a new connection. So what am I > > missing? Is > > the httpClient object still usable after a ConnectionClosedException? > > > > The internal connection pool can automatically re-establish closed > connection once the connection termination handshake completes or the > connection gets dropped abnormally. > > > > Somewhat related, I am looking at the sample here: > > > https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/migration-to-async-http2.html > > What is the difference between the two socket timeout configurations, > > on > > IOReactorConfig and ConnectionConfig? > > Both represent the same timeout but apply at different levels. > IOReactorConfig apply at the i/o reactor level and are specific to the > async i/o model. ConnectionConfig apply at the connection management > level and is not specific to any i/o model. > > > > What is the time to live? > > You mean TTL, total time to live? The maximum period of time > connections can be kept alive and re-used. Once past TTL connections > get automatically closed out. > > Hope this helps > > Oleg > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Re: HTTP/2 client and closed connections
On Tue, 2024-03-12 at 21:58 -0400, Jonathan Amir wrote: > Hello, > I am building an HTTP/2 only client for running multiple requests in > parallel. > I understand that there is no connection pool internally, rather > there is > one connection per host. For simplicity, let's say all my requests go > to > the same host. > This is correct. It is still technically a pool though (per host basis). > I have a situation where under stress there are some errors. It > starts with > socket timeout (several threads in parallel), and after a while there > is a > ConnectionClosedException. Even though HTTP/2 has a proper connection termination handshake, the handshake is potentially racy. Under high load ConnectionClosedException can and will happen. Your application code must be prepared to handle those. > I am not sure what is the flow of events that leads to this, and what > is > the relationship between those errors. I also don't know if it is my > client > or the server that closed the connection. > It, of source, would help greatly to know what exactly happens and leads to ConnectionClosedException. > My initial question is, since there is only one connection maintained > internally, how does one recover from ConnectionClosedException? The > connection life-cycle is opaque to me - there is no pool, and no > eviction > strategy, so no concept of creating a new connection. So what am I > missing? Is > the httpClient object still usable after a ConnectionClosedException? > The internal connection pool can automatically re-establish closed connection once the connection termination handshake completes or the connection gets dropped abnormally. > Somewhat related, I am looking at the sample here: > https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/migration-to-async-http2.html > What is the difference between the two socket timeout configurations, > on > IOReactorConfig and ConnectionConfig? Both represent the same timeout but apply at different levels. IOReactorConfig apply at the i/o reactor level and are specific to the async i/o model. ConnectionConfig apply at the connection management level and is not specific to any i/o model. > What is the time to live? You mean TTL, total time to live? The maximum period of time connections can be kept alive and re-used. Once past TTL connections get automatically closed out. Hope this helps Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
HTTP/2 client and closed connections
Hello, I am building an HTTP/2 only client for running multiple requests in parallel. I understand that there is no connection pool internally, rather there is one connection per host. For simplicity, let's say all my requests go to the same host. I have a situation where under stress there are some errors. It starts with socket timeout (several threads in parallel), and after a while there is a ConnectionClosedException. I am not sure what is the flow of events that leads to this, and what is the relationship between those errors. I also don't know if it is my client or the server that closed the connection. My initial question is, since there is only one connection maintained internally, how does one recover from ConnectionClosedException? The connection life-cycle is opaque to me - there is no pool, and no eviction strategy, so no concept of creating a new connection. So what am I missing? Is the httpClient object still usable after a ConnectionClosedException? Somewhat related, I am looking at the sample here: https://hc.apache.org/httpcomponents-client-5.3.x/migration-guide/migration-to-async-http2.html What is the difference between the two socket timeout configurations, on IOReactorConfig and ConnectionConfig? What is the time to live? Thanks, Jonathan
