JackieTien97 commented on code in PR #18643:
URL: https://github.com/apache/iotdb/pull/18643#discussion_r4071010102
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/ClientRPCServiceImpl.java:
##########
@@ -1683,8 +1695,22 @@ public TSStatus cancelOperation(TSCancelOperationReq
req) {
@Override
public TSStatus closeOperation(TSCloseOperationReq req) {
+ IClientSession clientSession = SESSION_MANAGER.getCurrSession();
+ if (req.isSetQueryId()
+ && clientSession != null
+ && clientSession.isLogin()
+ && COORDINATOR.getQueryExecution(req.queryId) != null
+ && !clientSession.containsQueryId(
+ req.isSetStatementId() ? req.getStatementId() : null,
req.queryId)) {
Review Comment:
[P2] Do not pass an unowned queryId to global cleanup
When this lookup returns null, the ownership check is skipped, but the
request still reaches SessionManager.closeDataset, which unconditionally
invokes Coordinator.cleanupQueryExecution(queryId). A query belonging to
another session can be registered between this lookup and the subsequent
map.remove: query IDs are allocated before the execution is published in
queryExecutionMap. The foreign close then removes and stops that query and
returns SUCCESS.
I reproduced this with a deterministic two-thread test using the actual
handler, SessionManager and Coordinator: pause closeOperation after the null
lookup, register the other session's execution, then resume. The result was
status=200, victimExecutionRemaining=false, while the owner's session binding
remained present. This leaves a concurrency gap in the new close protection.
Please ensure that a queryId not owned by the calling session never reaches
global cleanup. If it is absent, return the compatible no-op result directly
rather than falling through to cleanup, while preserving existing
malformed-request handling. Please also add a regression test for registration
between the existence check and cleanup.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/session/ClientSession.java:
##########
@@ -90,6 +90,25 @@ public void addQueryId(Long statementId, long queryId) {
queryIds.add(queryId);
}
+ @Override
+ public boolean containsQueryId(Long statementId, long queryId) {
+ return containsQueryId(statementIdToQueryId, statementId, queryId);
+ }
+
+ public static boolean containsQueryId(
+ Map<Long, Set<Long>> statementIdToQueryId, Long statementId, long
queryId) {
+ if (statementId == null) {
+ for (Set<Long> queryIds : statementIdToQueryId.values()) {
+ if (queryIds != null && queryIds.contains(queryId)) {
Review Comment:
[P2] Box queryId once before scanning the statement sets
queryId is a primitive long, so contains(queryId) boxes it on every loop
iteration. For IDs outside the Long cache, local JMH measurements against this
implementation show roughly 24 bytes allocated per visited statement set,
including empty sets whose statements remain open. This path is used by the
legacy Java V1 dataset and the current C++ V2 fetch implementation, both of
which omit statementId.
With the matching query in the last set, 1,000 statements added
approximately 8.6 us and 24 KB per fetch; 10,000 added approximately 73 us and
240 KB. These are local lookup/ownership microbenchmarks, not full RPC latency.
The allocations recur on every fetched page and can create avoidable GC
pressure on connections with many statements.
Please create a single Long boxedQueryId before the loop and reuse it for
contains. A separate comparison reduced the 1,000-statement scan from
approximately 24,056 B/op to 80 B/op without adding an index or changing
ownership semantics. The scan remains linear, but its temporary allocation no
longer needs to grow with the number of statements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]