yamass opened a new pull request, #19092:
URL: https://github.com/apache/iotdb/pull/19092
## Description
### TL;DR
SessionPool only tracks sessions held by open result sets (the `occupied`
map), but treats that map as "all sessions in use".
As a result, `currentOccupiedSize()`, six setters and `close()` ignore
sessions executing a request.
`closeResultSet()` can also release a session that a later query already has
taken, leaks the session when closing fails,
and throws an NPE for a wrapper without a session.
*All these bugs stem from the same root cause.* Each bug has one or more
tests in `SessionPoolTest` that fails on master. See table below.
<details>
<summary>Longer Description</summary>
### Problem
`SessionPool` keeps idle sessions in `queue`. Apart from that, it has an
additional collection: the `occupied` map.
It contains only sessions whose query result set is still open, until the
result set is closed.
A session *executing a request* – an insert, a non-query statement, or a
query whose result set has not been returned yet – is in neither `queue` nor
`occupied`.
Note that in the following, I will just refer sessions *executing requests*
instead of spelling out that these requests are inserts, non-query statements,
or queries whose result set has not been returned yet.
However, several parts of the class treat `occupied` as if it held *every*
session taken from the pool,
which it doesn't:
1. **`currentOccupiedSize()`** ignores sessions executing a request. On a
write-heavy pool whose sessions are all
writing, it reports 0 occupied and 0 available. We saw exactly this in
production: "occupied" stayed at 0–1 and "available" was 0,
all while writes queued up.
2. **`setFetchSize`, `setTimeZone`, `setEnableRedirection`,
`setEnableQueryRedirection`,
`setVersion` and `setQueryTimeout`** apply the new value to `queue` and
`occupied` only. A session executing a request
misses the change and keeps the old value for its whole life after being
put back, so the pool ends up with mixed settings.
3. **`close()`** closes `queue` and `occupied` only. A session executing a
request at that moment is later put
back into the already cleared queue, and its connection is never closed.
`closeResultSet()` has problems of its own:
4. **A late second release can steal another thread's session.** A result
set is usually released
twice: by `SessionDataSetWrapper.hasNext()` once it is exhausted, and
again by the caller's
`closeResultSet()`/`close()` in a `finally` block. The second call only
checks whether the
*session* is still in `occupied`. If another thread took that session in
between and opened a
new result set with it, the late call puts the session back while the
other thread is still
reading, and two threads end up sharing one connection.
5. **A failed close leaks the session.** When closing the operation handle
fails, a replacement
session is opened, but the failed session is neither put back nor closed,
so its connection
leaks. The failed close also leaves the data set open, so the second
release (see 4) sends the
close request again over the broken session and opens yet another
replacement for it.
`tryConstructNewSession()` does not count replacements in `size`, so the
pool can end up with
more sessions than `maxSize`.
6. A wrapper without a session throws a `NullPointerException`
(`ConcurrentHashMap.remove(null)`).
### Fix
Only one implementation file was changed: `SessionPool`.
- `occupied` is now a concurrent set of all sessions handed out by
`getSession()`. A session leaves
it when it is put back (`putBack`), closed after a connection error
(`closeSession`, used by both
`cleanSessionAndMayThrowConnectionException` overloads), or released by
`closeResultSet()`.
- `currentOccupiedSize()` and the timeout warning report `occupied.size()`
(1).
- The six setters and `close()` iterate `occupied` instead of the result-set
sessions (2, 3).
The setters only change client-side fields of the session, as they already
did for sessions
held by result sets. A session closed by `close()` makes its running
request fail with a
connection error, the same as reading a result set of a closed pool
already did. The retry then
fails with "session pool is closed". `Session.close()` returns early when
already closed, so a
second close by the request's error handling is harmless.
- The former `occupied` map is renamed to `occupiedByResultSet` and maps
each session to the
`SessionDataSetWrapper` holding it; `occupy(session, wrapper)` stores it.
- `closeResultSet()` first claims the session with
`occupiedByResultSet.remove(session, wrapper)`
and returns if that fail. So only the first release of *this* result set
closes its operation
handle and hands the session back (4), even if that close failed (5). A
failed close goes through
`cleanSessionAndMayThrowConnectionException(session)`, which closes the
session and opens a
replacement (5). A wrapper without a session is ignored (6).
No public API changes.
</details>
### Tests
New tests in `SessionPoolTest`. Eight of them fail on master, at least one
per bug above; the other
four cover every way a session goes back, so the count cannot drift.
| Test
| Covers | On master |
|---------------------------------------------------------------------------------------|-----------------------------|-----------|
| `test_CurrentOccupiedSize_WhenInsertIsRunning_CountsItsSession`
| (1) | fails |
| `test_CurrentOccupiedSize_WhenNonQueryIsRunning_CountsItsSession`
| (1) | fails |
| `test_Setters_WhenCalledWhileInsertIsRunning_ReachSessionExecutingInsert`
| (2) | fails |
| `test_Close_WhenInsertIsRunning_ClosesSessionExecutingInsert`
| (3) | fails |
|
`test_CloseResultSet_WhenCalledAgainAfterNextQueryTookSession_LeavesItToNextQuery`
| (4) | fails |
|
`test_CloseResultSet_WhenClosingOperationHandleFails_ClosesSessionAndStopsCountingIt`
| (5) | fails |
| `test_CloseResultSet_WhenCalledAgainAfterFailedClose_DoesNotRetryClose`
| (5) | fails |
| `test_CloseResultSet_WhenWrapperHasNoSession_DoesNothing`
| (6) | fails |
| `test_CurrentOccupiedSize_WhenInsertWasRejected_IsZero`
| return via `putBack` | passes |
| `test_CurrentOccupiedSize_WhenBrokenSessionWasCleanedUp_IsZero`
| return via cleanup | passes |
| `test_CurrentOccupiedSize_WhenResultSetIsOpen_CountsItsSessionUntilClosed`
| return via `closeResultSet` | passes |
| `test_CloseResultSet_WhenCalledTwice_PutsSessionBackIntoPoolOnlyOnce`
| double release | passes |
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious
for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold
for code coverage.
- [ ] been tested in a test IoTDB cluster.
<!-- Check the items by putting "x" in the brackets for the done things. Not
all of these items
apply to every PR. Remove the items which are not done or not relevant to
the PR. None of the items
from the checklist above are strictly necessary, but it would be very
helpful if you at least
self-review the PR. -->
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
* `SessionPool.java` + tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]