JackieTien97 opened a new pull request, #19240:
URL: https://github.com/apache/iotdb/pull/19240
## Description
Follow-up to #19107. When `IoTDBLocal.query()` rejects a statement that is
not a query, the `QueryExecution` built for that statement never finalizes its
UDF query scope. The class loaders pinned for the UDFs used by the statement
therefore stay open after `CREATE FUNCTION` deprecates them.
### Scenario
A table scalar UDF `localq` calls `IoTDBLocal.query()` in `beforeStart` with
a write statement that uses the external UDF `lazy_f`:
```java
local.query("INSERT INTO db.t2(time, device_id, v) SELECT time, device_id,
lazy_f(v) FROM db.t");
```
The statement is rejected as expected with `Only query is supported for
IoTDBLocal query interface`. After `SELECT localq(v) FROM one` runs 5 times, a
`CREATE FUNCTION` deprecates the class loader that defined `lazy_f`:
| | Class loader that defined `lazy_f` |
|---|---|
| Before | `activeQueriesCount=5`, deprecated, never closed |
| After | `activeQueriesCount=0`, deprecated, closed |
Read-only internal queries such as `SELECT lazy_f(v) FROM db.t` already left
`activeQueriesCount=0`, whether the UDF closes the result set or leaves it to
the framework. They are unchanged.
### Cause
`Coordinator.execution()` constructs the `QueryExecution` first. Its
analysis runs inside the execution's UDF query scope, where `reflect()` pins
the class loader that defined each external UDF. Only then does
`InternalQueryExecutor.validateReadOnlyQuery()` throw `SemanticException` for a
statement that is not a query. At that point the execution is not in
`queryExecutionMap` and has not been started, so `cleanupQueryExecution()`
cannot find it and nothing calls `stopAndCleanup()`. The scope is never
finalized.
### Fix
`Coordinator.execution()` now calls `stopAndCleanup()` on the rejected
execution and rethrows the same exception, so the message and status code
returned to the caller do not change.
- For a `QueryExecution` that was never started, `stopAndCleanup()` releases
what the analysis acquired and finalizes the UDF query scope.
`ConfigExecution.stopAndCleanup()` does nothing.
- Nothing else between construction and `queryExecutionMap.put()` /
`start()` can throw. Construction failures are already handled by
`analyzeWithCleanupOnFailure()`, and `start()` finalizes the scope itself when
it fails.
### Validation
- The new
`CoordinatorTest#testRejectedReadOnlyInternalQueryClosesUDFQueryScope` calls
`Coordinator.execution(..., readOnlyInternalQuery = true, ...)` with a real
`QueryExecution` whose analysis marks the statement as `WRITE`. It checks that:
- the same `SemanticException` is thrown;
- the execution is not registered;
- its UDF query scope is closed.
On master (54bff3d9a4) it fails at the scope check, and with the fix it
passes.
- With the fix on master, these 7 classes pass (52 tests) with Spotless and
Checkstyle enabled: `CoordinatorTest`, `QueryExecutionRetryTest`,
`QueryOwnershipTest`, `LoadTsFileAnalyzerDiagnosticsTest`,
`LoadTsFileDataTypeConverterSchemaTest`, `CteMaterializerTest` and
`CteSubqueryTest`.
- The scenario above ran on a 1C1D cluster built from 449a2b67bc, the head
merged as #19107, once without and once with the fix. With the fix on that
head, `IoTDBLocalIT` (TableSimpleIT) passed: 22 tests passed, including
`testRejectMutatingStatementsViaIoTDBLocal`, and 1 was skipped because it
requires cluster execution. The query execution, IoTDBLocal and UDF class
loader code is unchanged on master since that head.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious
for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold
for code coverage.
- [x] been tested in a test IoTDB cluster.
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
- `Coordinator`
- `CoordinatorTest` (new)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]