[
https://issues.apache.org/jira/browse/CALCITE-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375825#comment-17375825
]
Josh Elser commented on CALCITE-4676:
-------------------------------------
Validated this with a naive test:
{code:java}
public static void main(String[] args) throws Exception {
for (int i = 0; i < 1000; i++) {
try (Connection conn =
DriverManager.getConnection("jdbc:avatica:remote:url=http://localhost:55000;serialization=protobuf";))
{
DatabaseMetaData metadata = conn.getMetaData();
ResultSet tables = metadata.getTables(null, null, null, null);
ArrayList<String> tableList = new ArrayList<>();
while (tables.next()) {
String table = String.format("%s:%s", tables.getString(2),
tables.getString(3));
tableList.add(table);
}
tables.close();
System.out.println(tableList.size() + " tables seen");
}
}
System.out.println("Waiting...");
Thread.sleep(5 * 60 * 1000);
}
{code}
Used the hsqldb standalone server here, ala {{docker run --rm -P -it
avatica-hsqldb-server}} (sadly, this was broken during the Gradle migration and
never re-automated)
Without the fix, we can see lots of TCP connections for this one program:
{noformat}
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
79
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
83
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
87
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
242 {noformat}
With Istvan's fix, we can see:
{noformat}
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
1
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
1
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
1
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
1
% lsof -Pp $(jps -m | fgrep ConnectionLeak | awk '{print $1}') | fgrep -c TCP
1 {noformat}
> Avatica client leaks TCP connections
> ------------------------------------
>
> Key: CALCITE-4676
> URL: https://issues.apache.org/jira/browse/CALCITE-4676
> Project: Calcite
> Issue Type: Bug
> Components: avatica
> Affects Versions: avatica-1.18.0
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
>
> The default Http client implmentation uses
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager to pool
> connections, and does not close the TCP connections explicitly when the JDBC
> connection is closed.
> However, the http client pools are created *per JDBC Connection*, so the
> pooling is effectively only used when concurrent statements are executed from
> the same Connection object.
> This also means that when the application opens and closes a lot of
> Connections, then every Connection leaves behind active (typically in
> CLOSE_WAIT) TCP connections, which are only cleaned up on GC.
> This wastes resources, and eventually breaks the application, as connection
> limits are reached.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
