michaeljmarshall commented on code in PR #19377:
URL: https://github.com/apache/pulsar/pull/19377#discussion_r1092344168
##########
pulsar-io/elastic-search/src/main/java/org/apache/pulsar/io/elasticsearch/client/RestClient.java:
##########
@@ -83,7 +83,6 @@ public RestClient(ElasticSearchConfig elasticSearchConfig,
BulkProcessor.Listene
// idle+expired connection evictor thread
this.executorService = Executors.newSingleThreadScheduledExecutor();
this.executorService.scheduleAtFixedRate(() -> {
- configCallback.connectionManager.closeExpiredConnections();
configCallback.connectionManager.closeIdleConnections(
Review Comment:
Good question. The motivation of this PR assumes that closing these
connections is necessary, however, I am not sure that it is. The fundamental
risk is that something between the client and the server closes the connection.
In my mind, the canonical example is a network load balancer with a 4 or 5
minute timeout.
Closing expired and idle connections is one solution to prevent such errors
due to inactivity.
While troubleshooting the underlying behavior this PR aims to fix, I came
across https://github.com/elastic/elasticsearch/issues/65213, which indicates
that an alternative solution is to enable socket keepalives and to decrease the
`net.ipv4.tcp_keepalive_time` in order to make sure those keepalives are sent
before any intermediate server closes the connection due to inactivity. Since
that solution requires modifying OS settings, I think this solution might be
easier to maintain, even though it'll be less efficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
