JackieTien97 opened a new pull request, #19334:
URL: https://github.com/apache/iotdb/pull/19334

   ## Description
   
   Backport of #19330 to `dev/1.3`. It contains two fixes for query executions 
that stay registered in the `Coordinator` after nobody can use them any more.
   
   ### 1. Abandoned queries were never reclaimed
   
   A client runs a query and then neither fetches the rest of the result nor 
closes the statement, and no fragment instance of the query is left to fail it. 
`Coordinator.cleanUpStaleQueries` should reclaim such a query 60 s after its 
timeout, but it skipped every query for which `IQueryExecution.isActive()` 
returned true, and `isActive()` returned `startTimeOfCurrentRpc == -1`, which 
is true when **no** RPC is in progress. The cleanup therefore skipped exactly 
the abandoned queries.
   
   - `cleanUpStaleQueries` no longer depends on `isActive()`: every query that 
exceeded its timeout by more than 60 s is reclaimed.
   - `isActive()` returns true iff an RPC of the query is in progress, as 
documented. `QueryExecution.cleanUpCoordinatorContextMapIfNeeded` had been 
written against the inverted value, so its check is negated and its behaviour 
does not change.
   
   What was reclaimed before is still reclaimed, and the queries without an RPC 
in progress are reclaimed in addition. #19330 explains why the cleanup does not 
start to skip the queries with an RPC in progress, and shows the effect on a 
two-DataNode cluster: the query was still registered and fully readable after 
90 s, and is now removed 63.5 s after it started with `timeout=2000`.
   
   ### 2. Query statements of a batch were not run to their end, and not 
released
   
   A statement that is a query, such as `select s1 into root.db.dst(s1) from 
root.db.src`, is sent through `executeBatchStatement` (JDBC 
`Statement.executeBatch()`). The handler only started it and returned its 
status. A batch has no result set, so nobody fetched the result, and a query 
only makes progress while its result is consumed. The statement was not run to 
its end, the batch still reported success, and the execution stayed registered 
until the stale query cleanup removed it.
   
   The handler now consumes the result of such a statement and then releases 
its execution. `querySchemaTemplate` (e.g. 
`Session.showMeasurementsInTemplate`) returns its whole result in the response 
and now releases its execution when it returns.
   
   One consequence to be aware of: a plain `SELECT` in a batch is now executed 
to its end before the batch returns, and its result is discarded. The time this 
takes is bounded by `query_timeout_threshold`.
   
   ### Source and target
   
   - Source: #19330, merged into `master` as 
`90e3d967f9799086df50fd33084452b9268f25c0`. That commit is the squash of the 
two commits of the PR, `78bf43b042f34433cfa48fbc47a8707dd04586b2` and 
`9c3af0cfd3bcca347bba24a550a00a6a57372624`, and has the same tree as the second 
one. The two commits were cherry-picked with `-x`, so the two fixes stay 
separate here.
   - Target base: `dev/1.3` at `86f9335302`.
   
   ### Adaptations
   
   Main code:
   
   - `QueryExecution.cleanUpCoordinatorContextMapIfNeeded` keeps the 
`cleanupQueryExecution(queryId, null, t)` call of `dev/1.3`; only its condition 
is negated, as in the source.
   - `ClientRPCServiceImpl`: the surrounding code of both handlers differs on 
`dev/1.3` (no table model branch, the query id is requested at one place), so 
the changes were re-applied by hand on the `dev/1.3` file: the `consumeResult` 
helper, consuming the result and releasing the execution per statement in 
`executeBatchStatement`, and releasing the execution in 
`executeTemplateQueryStatement`.
   
   Tests:
   
   - `CoordinatorTest`, `QueryExecutionLifecycleTest` and `IoTDBShowQueriesIT` 
do not exist on `dev/1.3`. They are added with the tests of this change only, 
and `IoTDBShowQueriesIT` with the small fixture these tests need. They are 
adapted to the `Coordinator.execution` and `SessionInfo` signatures, the 
`QueryType` values and the `ColumnHeaderConstant` package of `dev/1.3`.
   - `IoTDBExecuteBatchIT` starts a cluster per test on `dev/1.3`, so the new 
test has no cleanup block.
   
   Not ported, because they depend on something `dev/1.3` does not have:
   
   - the assertions on the UDF query scope of an execution (`dev/1.3` has no 
UDF query scope);
   - the assertions on the `QUERY_TIMEOUT` status of a reclaimed query: 
`stopAndCleanup()` does not move the query state machine on `dev/1.3`, so that 
status stays `SUCCESS` there. The ported tests assert that the query is removed 
from the `Coordinator`. The next section shows what this means for a client;
   - `IoTDBInsertQueryIT.testInsertQueryInBatch` (table model).
   
   ### What a client sees when its query is reclaimed on `dev/1.3`
   
   `getStatus()` of a reclaimed execution keeps reporting `SUCCESS` on 
`dev/1.3`, because `QueryExecution.stopAndCleanup()` does not move the query 
state machine there (`master` does since #19224). Only the first RPC of a query 
reads that status. Every later RPC looks the query up in the `Coordinator` and 
reports `715` when it is gone.
   
   Measured on a cluster with two DataNodes, with `dev/1.3` at `86f9335302` and 
with this branch. The queries are sent to DataNode 1 with a timeout of 2 s:
   
   | Query | `dev/1.3` | This branch |
   | --- | --- | --- |
   | 20000 rows on DataNode 2, left alone after the first batch | not 
reclaimed: still registered 84 s later, and the client then reads all 20000 
rows | reclaimed after 63.5 s; the client reads the 4096 rows it had received 
and then gets `715: Query is not found, it may be killed by others, timeout or 
some other runtime errors, ...` |
   | 20000 rows on DataNode 1, left alone after the first batch | no longer 
registered when the client comes back; it reads 4096 rows and then gets `715` | 
the same |
   | first RPC still dispatching when the cleanup runs (DataNode 2 suspended 
for 75 s) | reclaimed after 63.0 s and 63.9 s (two runs); when DataNode 2 
continues, the RPC returns `SUCCESS` without a result set. 
`Statement.executeQuery` throws `SQLException: Cannot invoke 
"java.util.List.size()" because "columnNameList" is null`, 
`Session.executeQueryStatement` throws a `NullPointerException` with the same 
message | the same, reclaimed after 62.4 s |
   
   - The queries this PR starts to reclaim (first row) end with an explicit 
`715` on the next fetch. No result is cut short without an error.
   - The last row is where the `SUCCESS` status reaches a client. It is 
existing behaviour of `dev/1.3` and is not changed here: the old check skipped 
only the queries without an RPC in progress, so a query whose first RPC is in 
progress was reclaimed before this PR as well. On `master` the same run ends 
with `720: Current query is time out, ...` for both clients, because #19224 
made `stopAndCleanup()` fail the query state machine. That change is not part 
of #19330 and is not included in this backport. The run needed 
`dn_connection_timeout_ms=300000` to keep the dispatch RPC waiting for 75 s 
(default: 60 s).
   
   ### Validation on `dev/1.3`
   
   All commands were run offline on this branch with JDK 21.
   
   Unit tests:
   
   ```bash
   mvn -o test -pl iotdb-core/datanode -am \
     
-Dtest=CoordinatorTest,QueryExecutionLifecycleTest,ConfigExecutionTest,QueryExecutionRetryTest
 \
     -Dsurefire.failIfNoSpecifiedTests=false
   ```
   
   - With the four main files of the first commit as on `dev/1.3`: 15 tests, 3 
failures:
     - `CoordinatorTest.testStaleQueryWithoutRpcInProgressIsCleanedUp:97 
expected null, but was:<QueryExecution[...]>`
     - `ConfigExecutionTest.isActiveTellsWhetherAnRpcOfTheQueryIsInProgress:169`
     - 
`QueryExecutionLifecycleTest.isActiveTellsWhetherAnRpcOfTheQueryIsInProgress:48`
   - With this branch: 15 tests, 0 failures, with the spotless and checkstyle 
checks of the build enabled.
   
   Integration tests:
   
   ```bash
   mvn -o clean verify -DskipUTs 
'-Dit.test=IoTDBShowQueriesIT,IoTDBExecuteBatchIT' \
     -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false \
     -pl integration-test -am -P with-integration-tests
   ```
   
   - With `ClientRPCServiceImpl` as on `dev/1.3`, the three new tests fail:
     - `IoTDBExecuteBatchIT.testJDBCExecuteBatchWithSelectInto:105 expected:<2> 
but was:<0>`
     - `IoTDBShowQueriesIT.queryInBatchReleaseTest: expected:<[]> but 
was:<[20261119_125950_00007_1]>`
     - `IoTDBShowQueriesIT.schemaTemplateQueryReleaseTest: expected:<[]> but 
was:<[20261119_125950_00012_1]>`
   - With this branch: `IoTDBShowQueriesIT` (2 tests) and `IoTDBExecuteBatchIT` 
(3 tests) pass with the default profile (1C1D) and with `-PClusterIT` (1C3D).
   - `mvn spotless:check -pl iotdb-core/datanode,integration-test -P 
with-integration-tests` passes.
   - The table above comes from a real 1C2D cluster of each build 
(`query_timeout_threshold=5000`, `dn_connection_timeout_ms=300000`), driven 
through JDBC and the Session API. The batch measurements of #19330 on two 
DataNodes were not repeated on `dev/1.3`; the integration tests above cover 
that fix.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added comments explaining the "why" and the intent of the code 
wherever would not be obvious
     for an unfamiliar reader.
   - [x] added unit tests or modified existing tests to cover new code paths, 
ensuring the threshold
     for code coverage.
   - [x] added integration tests.
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) 
in this PR
   
   - `iotdb-core/datanode`: `org.apache.iotdb.db.queryengine.plan.Coordinator`, 
`org.apache.iotdb.db.queryengine.plan.execution.QueryExecution`, 
`org.apache.iotdb.db.queryengine.plan.execution.config.ConfigExecution`, 
`org.apache.iotdb.db.protocol.thrift.impl.ClientRPCServiceImpl`
   - `iotdb-core/datanode` (test): `CoordinatorTest`, `ConfigExecutionTest`, 
`QueryExecutionLifecycleTest`
   - `integration-test`: `IoTDBShowQueriesIT`, `IoTDBExecuteBatchIT`
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to