This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git
commit f01813750069556434fbd6eb36a12b214cb62bc6 Author: zy-kkk <[email protected]> AuthorDate: Sat Feb 3 18:43:38 2024 +0800 [doc](jdbc catalog) add connection pool doc (#30641) --- docs/en/docs/lakehouse/multi-catalog/jdbc.md | 16 ++++++++++++++++ docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 16 ++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/docs/en/docs/lakehouse/multi-catalog/jdbc.md b/docs/en/docs/lakehouse/multi-catalog/jdbc.md index dc45c247d54..f9e21b2080b 100644 --- a/docs/en/docs/lakehouse/multi-catalog/jdbc.md +++ b/docs/en/docs/lakehouse/multi-catalog/jdbc.md @@ -159,6 +159,22 @@ set enable_odbc_transcation = true; The transaction mechanism ensures the atomicity of data writing to JDBC External Tables, but it reduces performance to a certain extent. You may decide whether to enable transactions based on your own tradeoff. +## JDBC Connection Pool Configuration + +In Doris, each Frontend (FE) and Backend (BE) node maintains a connection pool, thus avoiding the need to frequently open and close individual connections to data sources. Each connection within this pool can be used to establish a connection to a data source and perform queries. After operations are completed, connections are returned to the pool for reuse. This not only enhances performance but also reduces system load during connection establishment, and helps to prevent hitting the m [...] + +The following Catalog configuration properties are available for tuning the behavior of the connection pool: + +| Parameter Name | Default Value | Description and Behavior | +|---------------------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `connection_pool_min_size` | 1 | Defines the minimum number of connections that the pool will maintain, ensuring that this number of connections remains active when the keep-alive mechanism is enabled. | +| `connection_pool_max_size` | 10 | Specifies the maximum number of connections allowed in the pool. Each Catalog corresponding to every FE or BE node can hold up to this number of connections. | +| `connection_pool_max_wait_time` | 5000 | Determines the maximum amount of time, in milliseconds, that the client will wait for a connection from the pool if none is immediately available. | +| `connection_pool_max_life_time` | 1800000 | Sets the maximum lifetime of connections in the pool, in milliseconds. Connections exceeding this set time limit will be forcibly closed. Additionally, half of this value is used as the minimum evictable idle time for the pool. Connections reaching this idle time are considered for eviction, and the eviction task runs at intervals of one-tenth of the `connection_pool_max_life_time`. | +| `connection_pool_keep_alive` | false | Effective only on BE nodes, it controls whether to keep connections that have reached the minimum evictable idle time but not the maximum lifetime active. It is kept false by default to avoid unnecessary resource usage. | + +To prevent an accumulation of unused connection pool caches on the BE, the BE `jdbc_connection_pool_cache_clear_time_sec` parameter for the BE can be set to specify the interval for clearing the cache. With a default value of 28800 seconds (8 hours), the BE will forcibly clear all connection pool caches that have not been used beyond this interval. + ## Guide ### View the JDBC Catalog diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md b/docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md index 52aece0db2b..23e9f777a81 100644 --- a/docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md +++ b/docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md @@ -159,6 +159,22 @@ set enable_odbc_transcation = true; 事务保证了JDBC外表数据写入的原子性，但是一定程度上会降低数据写入的性能，可以考虑酌情开启该功能。 +## JDBC 连接池配置 + +在 Doris 中，每个 FE 和 BE 节点都会维护一个连接池，这样可以避免频繁地打开和关闭单独的数据源连接。连接池中的每个连接都可以用来与数据源建立连接并执行查询。任务完成后，这些连接会被归还到池中以便重复使用，这不仅提高了性能，还减少了建立连接时的系统开销，并帮助防止达到数据源的连接数上限。 + +下面列出了一些可用于调整连接池行为的 Catalog 配置属性： + +| 参数名称 | 默认值 | 描述和行为 | +|---------------------------------|---------|-----------------------------------------------------------------------------------------------------| +| `connection_pool_min_size` | 1 | 定义连接池的最小连接数，用于初始化连接池并保证在启用保活机制时至少有该数量的连接处于活跃状态。 | +| `connection_pool_max_size` | 10 | 定义连接池的最大连接数，每个 Catalog 对应的每个 FE 或 BE 节点最多可持有此数量的连接。 | +| `connection_pool_max_wait_time` | 5000 | 如果连接池中没有可用连接，定义客户端等待连接的最大毫秒数。 | +| `connection_pool_max_life_time` | 1800000 | 设置连接在连接池中保持活跃的最大时长（毫秒）。超时的连接将被回收。同时，此值的一半将作为连接池的最小逐出空闲时间，达到该时间的连接将成为逐出候选对象。执行回收操作的频率将是该最大生命周期的十分之一。 | +| `connection_pool_keep_alive` | false | 仅在 BE 节点上有效，用于决定是否保持达到最小逐出空闲时间但未到最大生命周期的连接活跃。默认关闭，以减少不必要的资源使用。 | + +为了避免在 BE 上累积过多的未使用的连接池缓存，可以通过设置 BE 的 `jdbc_connection_pool_cache_clear_time_sec` 参数来指定清理缓存的时间间隔。默认值为 28800 秒（8小时），此间隔过后，BE 将强制清理所有超过该时间未使用的连接池缓存。 + ## 使用指南 ### 查看 JDBC Catalog --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
