This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new c46f1d38c1b [improvement](jdbc catalog)Optimize JDBC Catalog refresh
to reduce frequent client creation. (#40261)
c46f1d38c1b is described below
commit c46f1d38c1b4be585546dbc7a5d97339102c4ad9
Author: zy-kkk <[email protected]>
AuthorDate: Mon Sep 2 23:42:16 2024 +0800
[improvement](jdbc catalog)Optimize JDBC Catalog refresh to reduce frequent
client creation. (#40261)
In the previous JDBC Catalog refresh behavior, each refresh would close
and recreate the JdbcClient. During this process, we observed that some
JDBC drivers create classloader-level shared threads when the JdbcClient
is repeatedly instantiated. These threads cannot be garbage collected.
Since we use the JdbcClient as the context class loader for the JDBC
driver, frequent creation leads to a buildup of non-recyclable shared
threads in the JVM. This PR changes the refresh behavior to update the
cache only, rather than recreating the client. Recreating the client is
unnecessary when the Catalog configuration has not changed.
---
.../apache/doris/datasource/jdbc/JdbcExternalCatalog.java | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
index 8f9bca79fcc..aabe9d140ee 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
@@ -101,11 +101,9 @@ public class JdbcExternalCatalog extends ExternalCatalog {
}
@Override
- public void onRefresh(boolean invalidCache) {
- super.onRefresh(invalidCache);
- if (jdbcClient != null) {
- jdbcClient.closeClient();
- }
+ public void notifyPropertiesUpdated(Map<String, String> updatedProps) {
+ super.notifyPropertiesUpdated(updatedProps);
+ this.onClose();
}
@Override
@@ -113,6 +111,7 @@ public class JdbcExternalCatalog extends ExternalCatalog {
super.onClose();
if (jdbcClient != null) {
jdbcClient.closeClient();
+ jdbcClient = null;
}
}
@@ -205,6 +204,9 @@ public class JdbcExternalCatalog extends ExternalCatalog {
@Override
protected void initLocalObjectsImpl() {
+ if (jdbcClient != null) {
+ return;
+ }
JdbcClientConfig jdbcClientConfig = new JdbcClientConfig()
.setCatalog(this.name)
.setUser(getJdbcUser())
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
