CritasWang opened a new issue, #61:
URL: https://github.com/apache/iotdb-client-csharp/issues/61
## Summary / 概述
After the IoTDB server becomes unreachable, `SessionPool` stops raising
errors and blocks callers
indefinitely. A `SessionPool` that survived an outage cannot be used again,
and it does not recover even
after the server comes back — the process has to be restarted.
IoTDB 服务端不可达后,`SessionPool` 会停止抛出异常并让调用方永久阻塞。经历过一次断连的连接池无法继续
使用,且服务端恢复后也无法自愈,只能重启进程。
Reported by a user on Windows with IoTDB 2.0.10.
由 Windows + IoTDB 2.0.10 环境的用户反馈。
## Reproduce / 复现步骤
1. Open a `SessionPool` against a running IoTDB (default `PoolSize = 8`).
连接一个正常运行的 IoTDB 并 `Open()`(默认 `PoolSize = 8`)。
2. Stop the DataNode service to simulate an outage.
停止 DataNode 服务以模拟异常。
3. Call `InsertAlignedRecordAsync` (or any write) in a loop.
循环调用 `InsertAlignedRecordAsync`(或任意写入接口)。
Observed / 实际现象:
- The first ~8 calls throw `SessionPoolDepletedException` as expected.
前约 8 次调用会按预期抛出 `SessionPoolDepletedException`。
- From the 9th call on, the await never completes. No exception, no `catch`,
no timeout.
从第 9 次调用起 await 永不返回:不抛异常、不进 `catch`、也不超时。
- `IsOpen()` keeps returning `true` throughout.
全程 `IsOpen()` 始终返回 `true`。
## Root cause / 根因
### 1. Pool wait timeout: milliseconds written, seconds read / 池等待超时的单位错误
`SessionPool.Open()` assigns a millisecond value:
```csharp
_clients.Timeout = _timeout * 5; // _timeout is SetConnectionTimeoutInMs,
default 500
```
but `ConcurrentClientQueue.Take()` reads it back as seconds:
```csharp
timeout = !Monitor.Wait(ClientQueue, TimeSpan.FromSeconds(Timeout));
```
`500 * 5 = 2500` is therefore interpreted as **2500 seconds ≈ 41 minutes**.
Callers using the legacy
positional constructor with `timeout: 10000` get `50000` seconds ≈ **13.9
hours**. That is why the block
looks permanent.
`Open()` 按毫秒赋值,`Take()` 却按秒解释,`500 * 5 = 2500` 被当作 **2500 秒 ≈ 41 分钟**;旧的位置参数
构造函数传 `timeout: 10000` 时更是 **13.9 小时**——所以看起来像永久卡死。
### 2. Pool slots are lost on every failed reconnection / 重连失败导致槽位永久丢失
In `ExecuteClientOperationAsync`, when reconnection fails the dead client is
discarded and never returned:
```csharp
catch (ReconnectionFailedException reconnectEx)
{
shouldReturnClient = false; // slot is gone for good
throw new SessionPoolDepletedException(...);
}
```
Nothing replenishes the pool, so capacity shrinks by one per failure. After
`PoolSize` failures the queue
is empty and every subsequent `Take()` waits on a queue that nobody will
ever feed — combined with defect
(1), that wait is effectively unbounded. Because all connections have been
discarded, the pool cannot
recover even after the server is reachable again.
重连失败时失效连接被丢弃且不归还,池也没有补充机制,因此每失败一次容量就减一。`PoolSize` 次失败后队列
为空,之后每次 `Take()` 都在等一个永远不会有人投喂的队列——叠加缺陷 (1) 后等待时间实际上是无限的。
由于所有连接都已被丢弃,服务端恢复后连接池也无法自愈。
### 3. `IsOpen()` is widely mistaken for a health check / `IsOpen()`
被普遍误解为健康检查
`IsOpen()` is `!_isClose`, and `_isClose` only flips on explicit `Open()` /
`Close()`. There is no
heartbeat, so a server going down never changes it. This is working as
designed and matches the Java
client, but the name invites the following guard, which silently prevents
the pool from ever being
rebuilt:
```csharp
if (_pool != null && _pool.IsOpen()) return; // short-circuits forever
```
`IsOpen()` 即 `!_isClose`,只有显式 `Open()` / `Close()` 会改变它;客户端无心跳,服务端断开不会回写。
这是符合设计的(与 Java 客户端一致),但命名容易引起误用,上面这种守卫会导致连接池永远无法被重建。
## Expected / 期望行为
- An operation against a dead server fails within a bounded, configurable
time.
对已断开服务端的操作应在有界且可配置的时间内失败。
- Pool capacity stays at `PoolSize`; failed connections are rebuilt rather
than leaked.
池容量应保持为 `PoolSize`,失效连接应被重建而不是泄漏。
- Once the server is reachable again, the pool recovers without `Close()` +
`Open()`.
服务端恢复后,连接池应无需 `Close()` + `Open()` 即可自行恢复。
- The lifecycle-only semantics of `IsOpen()` are documented.
`IsOpen()` 仅表示生命周期的语义应在文档中说明。
## Workaround before the fix / 修复前的规避方案
- Do not use `IsOpen()` as a connectivity check.
不要用 `IsOpen()` 判断连通性。
- Wrap operations in your own timeout (`Task.WhenAny` + `Task.Delay`) so
business threads are not pinned.
用 `Task.WhenAny` + `Task.Delay` 自行加超时,避免业务线程被挂死。
- On `SessionPoolDepletedException`, explicitly `Close()` and re-`Open()`
the pool.
捕获 `SessionPoolDepletedException` 后显式 `Close()` 并重新 `Open()`。
## Environment / 环境
- Client: `Apache.IoTDB` C# SDK, `main` @ 8214361
- Server: IoTDB 2.0.10, Windows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]