JackieTien97 commented on code in PR #19247:
URL: https://github.com/apache/iotdb/pull/19247#discussion_r4570445716
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -804,7 +810,15 @@ private ISession getSession() throws
IoTDBConnectionException {
}
protected ITableSession getPooledTableSession() throws
IoTDBConnectionException {
- return new TableSessionWrapper((Session) getSession(), this);
+ Session session = (Session) getSession();
+ synchronized (this) {
+ if (closed) {
+ closeSession(session);
+ throw new IoTDBConnectionException(SESSION_POOL_IS_CLOSED);
+ }
+ occupied.put(session, session);
Review Comment:
This fixes the borrow/close race for table sessions, but the tree-model
query path still has it. The 12 query methods call `occupy(session)` after the
RPC returns, and `occupy()` does not check `closed`. If `close()` runs while
the query RPC is in flight, the session goes into `occupied` after `close()`
has already cleared it.
Probe: `executeQueryStatement` blocked inside the RPC, then `pool.close()`,
then the RPC returns. The wrapper is still returned, `session.close()` was
called 0 times, and `currentOccupiedSize()` is 1 on the closed pool. The new
`releaseSession()` does close the session once the caller closes the result
set. If the caller relies on `pool.close()` for cleanup, for example by
abandoning result sets at shutdown, the connection stays open.
Could `occupy()` do the same check under the lock as this code, closing the
session and throwing if the pool is closed? Another option is to register every
borrowed session in `getSession()` itself, under the lock that already checks
`closed`. That would also remove the window between the two lock acquisitions
here.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -941,11 +997,6 @@ private void cleanSessionAndMayThrowConnectionException(
}
}
- protected void cleanSessionAndMayThrowConnectionException(ISession session) {
Review Comment:
`SessionPool` is public and not final, and this protected overload has
existed since 2.0.1-beta. Removing it breaks any subclass that calls it.
Recompiling against the new jar fails, and running an old build against the new
jar throws `NoSuchMethodError`. If compatibility matters here, a `@Deprecated`
overload that delegates to `discardSession(session)` would keep those
subclasses working. Also note that `releaseSession` and `isPoolClosed` become
new protected API.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -819,15 +833,58 @@ public int currentOccupiedSize() {
@SuppressWarnings({"squid:S2446"})
protected void putBack(ISession session) {
- queue.push(session);
synchronized (this) {
- // we do not need to notifyAll as any waited thread can continue to work
after waked up.
- this.notify();
- // comment the following codes as putBack is too frequently called.
- // if (logger.isTraceEnabled()) {
- // logger.trace("put a session back and notify others...,
queue.size = {}",
- // queue.size());
- // }
+ if (!closed) {
+ queue.push(session);
+ // we do not need to notifyAll as any waited thread can continue to
work after waked up.
+ this.notify();
+ return;
+ }
+ }
+ closeSession(session);
+ }
+
+ /**
+ * Releases a session borrowed from this pool. A session that cannot be
reused is closed and
+ * replaced while the pool remains open.
+ *
+ * @return whether the session was still owned by the pool and has been
released
+ */
+ protected boolean releaseSession(ISession session, boolean reusable) {
+ boolean replenish = false;
+ synchronized (this) {
+ if (occupied.remove(session) == null) {
+ return false;
+ }
+ if (reusable && !closed) {
+ queue.push(session);
+ this.notifyAll();
Review Comment:
nit: only one session is pushed here, but `notifyAll()` wakes every waiting
borrower. `putBack()` and `tryConstructNewSession()` use `notify()`. With many
waiters, each release wakes all of them; all but one get `null` from `poll()`,
and each of those calls `notify()` again. `notify()` is enough here and matches
`putBack()`.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -838,22 +895,21 @@ private void occupy(ISession session) {
/** close all connections in the pool */
@Override
public synchronized void close() {
+ if (closed) {
+ return;
+ }
+ closed = true;
+
for (ISession session : queue) {
- try {
- session.close();
- } catch (IoTDBConnectionException e) {
- // do nothing
- LOGGER.warn(CLOSE_THE_SESSION_FAILED, e);
- }
+ closeSession(session);
}
for (ISession session : occupied.keySet()) {
- try {
- session.close();
- } catch (IoTDBConnectionException e) {
- // do nothing
- LOGGER.warn(CLOSE_THE_SESSION_FAILED, e);
- }
+ closeSession(session);
Review Comment:
Borrowed table sessions are now tracked in `occupied`, so this loop closes
sessions that other threads may be using at that moment, and it holds the pool
monitor while doing so. `Session.close()` calls `SessionConnection.close()`,
which sends `closeSession` through the synchronized client proxy
(`SynchronizedHandler` locks on the client). The call therefore waits until the
other thread's in-flight RPC returns.
Probe: a fake server, a table pool with `maxSize=2`, and one borrowed
session running a statement that the server holds for about 4.5 s.
- This PR: `close()` returned after 4528 ms. A concurrent `getSession()`
waited on the pool monitor for 1.5 s and then got `Session pool is closed`.
- Merge base: `close()` returned in 0 ms, because borrowed table sessions
were not touched here.
`connectionTimeoutInMs` defaults to 0, which means no socket read timeout.
One stuck RPC can therefore hang `close()` indefinitely. Everything that needs
the monitor hangs with it: `getSession()`, `putBack()`, `releaseSession()`, and
`TableSessionWrapper.close()` (through `isPoolClosed()`).
Suggestion: under the lock, set `closed = true` and take and clear snapshots
of `queue` and `occupied`. Close the snapshots after leaving the monitor. For
sessions that are still in use, consider closing the transport directly instead
of queueing a `closeSession` RPC behind the in-flight call. `SessionConnection`
already has `forceCloseTransport()`.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/TableSessionWrapper.java:
##########
@@ -48,23 +47,20 @@ public class TableSessionWrapper implements ITableSession {
private final SessionPool sessionPool;
- private final AtomicBoolean closed;
+ private boolean closed;
protected TableSessionWrapper(Session session, SessionPool sessionPool) {
this.session = session;
this.sessionPool = sessionPool;
- this.closed = new AtomicBoolean(false);
+ this.closed = false;
}
@Override
public void insert(Tablet tablet) throws StatementExecutionException,
IoTDBConnectionException {
try {
session.insertRelationalTablet(tablet);
Review Comment:
`releaseOnConnectionFailure()` and `close()` both set `session` to `null`.
If the same wrapper is called again, this line dereferences `null` and throws
`NullPointerException` instead of an `IoTDBConnectionException`. Both
`executeQueryStatement` overloads and `executeNonQueryStatement` behave the
same way.
Probe: the first `executeNonQueryStatement` fails with
`IoTDBConnectionException`. The second throws `NullPointerException: Cannot
invoke "org.apache.iotdb.session.Session.executeNonQueryStatement(String)"
because "this.session" is null`. The merge base behaves the same, so this is
not a regression. Still, a caller that retries on the same `ITableSession`
after a connection error will hit it, and this PR already makes `close()` safe
after a connection failure. A guard at the top of these methods that throws
`IoTDBConnectionException`, with the message as an i18n constant, would make
the behavior consistent.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -928,8 +983,9 @@ private void closeSession(ISession session) {
private void cleanSessionAndMayThrowConnectionException(
ISession session, int times, IoTDBConnectionException e) throws
IoTDBConnectionException {
- closeSession(session);
- tryConstructNewSession();
+ if (!releaseSession(session, false)) {
Review Comment:
nit: every caller of this method is a tree-model operation, and its session
is not yet in `occupied` when the exception happens. `releaseSession(session,
false)` therefore always returns `false` here, and `discardSession()` does the
actual work. `discardSession()` also duplicates the close-then-replenish tail
of `releaseSession()`. A single private `discard(session)` that removes the
session from `occupied`, closes it, and replenishes if the pool is open could
serve both callers, so the two paths cannot drift apart.
##########
iotdb-client/session/src/test/java/org/apache/iotdb/session/pool/SessionPoolTest.java:
##########
@@ -98,6 +106,40 @@ public class SessionPoolTest {
@Mock private TSFetchMetadataResp fetchMetadataResp;
@Mock private TSFetchResultsResp fetchResultsResp;
+ private static class BlockingAfterPollDeque extends
ConcurrentLinkedDeque<ISession> {
+
+ private final SessionPool pool;
+ private final CountDownLatch polled = new CountDownLatch(1);
+ private final AtomicBoolean shouldBlock = new AtomicBoolean(true);
+
+ private BlockingAfterPollDeque(SessionPool pool) {
+ this.pool = pool;
+ }
+
+ @Override
+ public ISession poll() {
+ ISession session = super.poll();
+ if (session != null && shouldBlock.compareAndSet(true, false)) {
+ polled.countDown();
+ // getSession holds the pool monitor here. Waiting releases it so
close() can run before
+ // the borrowed session is registered in occupied.
+ while (!pool.isPoolClosed()) {
+ try {
+ pool.wait(5_000);
Review Comment:
nit: both race tests depend on implementation details, and if those change,
the failures hide the cause.
- This `pool.wait()` works only because `getSession()` calls `poll()` while
holding the pool monitor. If `poll()` moves out of the lock, `pool.wait()`
throws `IllegalMonitorStateException` inside the borrow. The test then fails on
`assertTrue(e.getCause() instanceof IoTDBConnectionException)` with no message,
so the real exception is never shown.
- `testTableSessionReleaseRacingWithPoolClose` depends on `getDatabase()`
being evaluated before `sessionPool.database != null`. If someone swaps that
condition, which is a natural micro-optimization, the latch never counts down.
The test then fails at `assertTrue(aboutToRelease.await(...))` after 5 s, again
with no message.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/TableSessionWrapper.java:
##########
@@ -100,48 +90,77 @@ public void executeNonQueryStatement(String sql)
try {
session.executeNonQueryStatement(sql);
} catch (IoTDBConnectionException e) {
- sessionPool.cleanSessionAndMayThrowConnectionException(session);
- closed.set(true);
- session = null;
- throw e;
+ throw releaseOnConnectionFailure(e);
}
}
@Override
public void close() throws IoTDBConnectionException {
- if (!Objects.equals(session.getSqlDialect(), sessionPool.sqlDialect)) {
- try {
- session.executeNonQueryStatement("set sql_dialect=" +
sessionPool.sqlDialect);
- } catch (StatementExecutionException e) {
- LOGGER.warn(
- SessionMessages
-
.LOG_FAILED_CHANGE_BACK_SQL_DIALECT_EXECUTING_SET_SQL_DIALECT_ARG_947F35E7,
- sessionPool.sqlDialect,
- e);
- session.close();
- session = null;
+ final Session sessionToRelease;
+ synchronized (this) {
+ if (closed) {
return;
}
+ closed = true;
+ sessionToRelease = session;
+ session = null;
+ }
+
+ if (sessionToRelease == null) {
Review Comment:
nit: `closed = true` is always set together with `session = null`.
`sessionToRelease` can only be `null` here if `closed` was already `true`, so
this branch is unreachable. Either use `session == null` as the closed state,
or keep `closed` and drop this branch. The `this.closed = false` in the
constructor is also redundant.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/TableSessionWrapper.java:
##########
@@ -100,48 +90,77 @@ public void executeNonQueryStatement(String sql)
try {
session.executeNonQueryStatement(sql);
} catch (IoTDBConnectionException e) {
- sessionPool.cleanSessionAndMayThrowConnectionException(session);
- closed.set(true);
- session = null;
- throw e;
+ throw releaseOnConnectionFailure(e);
}
}
@Override
public void close() throws IoTDBConnectionException {
- if (!Objects.equals(session.getSqlDialect(), sessionPool.sqlDialect)) {
- try {
- session.executeNonQueryStatement("set sql_dialect=" +
sessionPool.sqlDialect);
- } catch (StatementExecutionException e) {
- LOGGER.warn(
- SessionMessages
-
.LOG_FAILED_CHANGE_BACK_SQL_DIALECT_EXECUTING_SET_SQL_DIALECT_ARG_947F35E7,
- sessionPool.sqlDialect,
- e);
- session.close();
- session = null;
+ final Session sessionToRelease;
+ synchronized (this) {
+ if (closed) {
return;
}
+ closed = true;
+ sessionToRelease = session;
+ session = null;
+ }
+
+ if (sessionToRelease == null) {
+ return;
+ }
+
+ if (sessionPool.isPoolClosed()) {
+ sessionPool.releaseSession(sessionToRelease, false);
+ return;
}
- if (closed.compareAndSet(false, true)) {
- if (!Objects.equals(session.getDatabase(), sessionPool.database)
+ boolean reusable = true;
+ try {
+ if (!Objects.equals(sessionToRelease.getSqlDialect(),
sessionPool.sqlDialect)) {
+ try {
+ sessionToRelease.executeNonQueryStatement("set sql_dialect=" +
sessionPool.sqlDialect);
+ } catch (StatementExecutionException e) {
+ LOGGER.warn(
+ SessionMessages
+
.LOG_FAILED_CHANGE_BACK_SQL_DIALECT_EXECUTING_SET_SQL_DIALECT_ARG_947F35E7,
+ sessionPool.sqlDialect,
+ e);
+ reusable = false;
+ }
+ }
+
+ if (reusable
+ && !Objects.equals(sessionToRelease.getDatabase(),
sessionPool.database)
&& sessionPool.database != null) {
try {
- session.executeNonQueryStatement("use " + sessionPool.database);
+ sessionToRelease.executeNonQueryStatement("use " +
sessionPool.database);
} catch (StatementExecutionException e) {
LOGGER.warn(
SessionMessages.LOG_FAILED_CHANGE_BACK_DATABASE_EXECUTING_USE_ARG_274541CA,
sessionPool.database,
e);
- session.close();
- session = null;
- return;
+ reusable = false;
}
}
+ } catch (IoTDBConnectionException | RuntimeException e) {
+ sessionPool.releaseSession(sessionToRelease, false);
+ throw e;
+ }
- sessionPool.putBack(session);
+ sessionPool.releaseSession(sessionToRelease, reusable);
+ }
+
+ private IoTDBConnectionException
releaseOnConnectionFailure(IoTDBConnectionException exception) {
Review Comment:
nit (stale comments): the class Javadoc still says the wrapper calls
`cleanSessionAndMayThrowConnectionException` on connection errors, but that
overload is gone and this method releases through `releaseSession`. Likewise,
the comment on `occupied` in `SessionPool` (`// for session whose resultSet is
not released.`) is no longer accurate, because borrowed table sessions are
tracked there now. That also changes what `currentOccupiedSize()` reports.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/SessionPool.java:
##########
@@ -2971,8 +3022,11 @@ public List<String> showPathsTemplateUsingOn(String
templateName)
@Override
public void sortTablet(Tablet tablet) throws IoTDBConnectionException {
ISession session = getSession();
Review Comment:
nit: the `try/finally` fixes the leak. But `Session.sortTablet()` sorts in
memory and makes no RPC, so there is no need to borrow a pooled session.
Borrowing can open a new connection when the queue is empty, block for up to
`waitToGetSessionTimeoutInMs` when the pool is exhausted, or throw after the
pool is closed. Moving the sort into a static helper and calling it directly
would remove both the leak and the blocking.
##########
iotdb-client/session/src/main/java/org/apache/iotdb/session/pool/TableSessionWrapper.java:
##########
@@ -100,48 +90,77 @@ public void executeNonQueryStatement(String sql)
try {
session.executeNonQueryStatement(sql);
} catch (IoTDBConnectionException e) {
- sessionPool.cleanSessionAndMayThrowConnectionException(session);
- closed.set(true);
- session = null;
- throw e;
+ throw releaseOnConnectionFailure(e);
}
}
@Override
public void close() throws IoTDBConnectionException {
- if (!Objects.equals(session.getSqlDialect(), sessionPool.sqlDialect)) {
- try {
- session.executeNonQueryStatement("set sql_dialect=" +
sessionPool.sqlDialect);
- } catch (StatementExecutionException e) {
- LOGGER.warn(
- SessionMessages
-
.LOG_FAILED_CHANGE_BACK_SQL_DIALECT_EXECUTING_SET_SQL_DIALECT_ARG_947F35E7,
- sessionPool.sqlDialect,
- e);
- session.close();
- session = null;
+ final Session sessionToRelease;
+ synchronized (this) {
+ if (closed) {
return;
}
+ closed = true;
+ sessionToRelease = session;
+ session = null;
+ }
+
+ if (sessionToRelease == null) {
+ return;
+ }
+
+ if (sessionPool.isPoolClosed()) {
+ sessionPool.releaseSession(sessionToRelease, false);
+ return;
}
- if (closed.compareAndSet(false, true)) {
- if (!Objects.equals(session.getDatabase(), sessionPool.database)
+ boolean reusable = true;
+ try {
+ if (!Objects.equals(sessionToRelease.getSqlDialect(),
sessionPool.sqlDialect)) {
+ try {
+ sessionToRelease.executeNonQueryStatement("set sql_dialect=" +
sessionPool.sqlDialect);
+ } catch (StatementExecutionException e) {
+ LOGGER.warn(
+ SessionMessages
+
.LOG_FAILED_CHANGE_BACK_SQL_DIALECT_EXECUTING_SET_SQL_DIALECT_ARG_947F35E7,
+ sessionPool.sqlDialect,
+ e);
+ reusable = false;
+ }
+ }
+
+ if (reusable
+ && !Objects.equals(sessionToRelease.getDatabase(),
sessionPool.database)
&& sessionPool.database != null) {
try {
- session.executeNonQueryStatement("use " + sessionPool.database);
+ sessionToRelease.executeNonQueryStatement("use " +
sessionPool.database);
} catch (StatementExecutionException e) {
LOGGER.warn(
SessionMessages.LOG_FAILED_CHANGE_BACK_DATABASE_EXECUTING_USE_ARG_274541CA,
sessionPool.database,
e);
- session.close();
- session = null;
- return;
+ reusable = false;
}
}
+ } catch (IoTDBConnectionException | RuntimeException e) {
+ sessionPool.releaseSession(sessionToRelease, false);
+ throw e;
+ }
- sessionPool.putBack(session);
+ sessionPool.releaseSession(sessionToRelease, reusable);
Review Comment:
Two of the fixes listed in the description have no test:
1. Restoring `sql_dialect`/database fails: the session is closed and pool
capacity is kept (this line with `reusable = false`).
2. A connection failure in `insert`/`execute*`:
`releaseOnConnectionFailure()` closes the session and replenishes the pool.
If either path went back to a bare `session.close()`, every test in this PR
would still pass.
I checked (1) with a probe: `maxSize=1`, a `getSqlDialect()` that differs
from the pool's, and a restore that throws `StatementExecutionException`. The
merge base ends with `size=1` and an empty queue, so the pool is exhausted for
good. This PR ends with `size=0`: in the probe the replacement could not
connect, so the slot was freed. In `SessionPoolTest`, mocking a failing
`executeNonQueryStatement` and then asserting `size`, `currentOccupiedSize()`
and `currentAvailableSize()` would protect both paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]