JackieTien97 commented on code in PR #19107:
URL: https://github.com/apache/iotdb/pull/19107#discussion_r4521648217


##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/service/UDFClassLoaderManager.java:
##########
@@ -50,30 +55,135 @@ public class UDFClassLoaderManager implements IService {
    */
   private final AtomicReference<UDFClassLoader> activeClassLoader = new 
AtomicReference<>();
 
+  public static final class QueryScope {
+    private final String queryId;
+    private final Set<UDFClassLoader> pinnedClassLoaders =
+        Collections.newSetFromMap(new IdentityHashMap<>());
+    private int activeActions;
+    private boolean finalizeRequested;
+    private boolean finalized;
+
+    private QueryScope(String queryId) {
+      this.queryId = queryId;
+    }
+  }
+
   private UDFClassLoaderManager(String libRoot) {
     this.libRoot = libRoot;
     LOGGER.info(CommonMessages.UDF_LIB_ROOT, libRoot);
-    queryIdToUDFClassLoaderMap = new ConcurrentHashMap<>();
+    queryScopeStack = ThreadLocal.withInitial(ArrayDeque::new);
   }
 
   @TestOnly
   private UDFClassLoaderManager() throws IOException {
     this.libRoot = null;
-    queryIdToUDFClassLoaderMap = new ConcurrentHashMap<>();
+    queryScopeStack = ThreadLocal.withInitial(ArrayDeque::new);
     activeClassLoader.set(new UDFClassLoader(""));
   }
 
-  public void initializeUDFQuery(String queryId) {
-    activeClassLoader.get().acquire();
-    queryIdToUDFClassLoaderMap.put(queryId, activeClassLoader.get());
+  public QueryScope openQueryScope(String queryId) {
+    return new QueryScope(Objects.requireNonNull(queryId));
   }
 
-  public void finalizeUDFQuery(String queryId) {
-    UDFClassLoader classLoader = queryIdToUDFClassLoaderMap.remove(queryId);
+  /**
+   * Bind an existing logical query scope to the current thread. Pins acquired 
by reflect() remain
+   * owned by the scope after this method returns and must be released by 
finalizeUDFQuery().
+   */
+  public <T> T withQueryScope(QueryScope queryScope, Supplier<T> action) {
+    Objects.requireNonNull(queryScope);
+    Objects.requireNonNull(action);
+    synchronized (this) {
+      if (queryScope.finalizeRequested || queryScope.finalized) {
+        throw new IllegalStateException(
+            String.format(
+                QueryMessages.EXCEPTION_UDF_QUERY_SCOPE_IS_CLOSED_ARG_7381D949,
+                queryScope.queryId));
+      }
+      queryScope.activeActions++;
+    }
+
+    Deque<QueryScope> scopeStack = queryScopeStack.get();
+    scopeStack.push(queryScope);
     try {
-      if (classLoader != null) {
-        classLoader.release();
+      return action.get();
+    } finally {
+      scopeStack.pop();
+      if (scopeStack.isEmpty()) {
+        queryScopeStack.remove();
       }
+      synchronized (this) {
+        queryScope.activeActions--;
+        closeScopeIfPossible(queryScope);
+      }
+    }
+  }
+
+  /**
+   * Pin the exact class loader that defined the reflected UDF class. Built-in 
functions loaded by
+   * the parent class loader do not create a pin.
+   */
+  public void pinClassLoaderForCurrentQuery(Class<?> udfClass) {
+    ClassLoader classLoader = udfClass.getClassLoader();
+    if (!(classLoader instanceof UDFClassLoader)) {
+      return;
+    }
+
+    Deque<QueryScope> scopeStack = queryScopeStack.get();
+    QueryScope queryScope = scopeStack.peek();
+    if (queryScope == null) {
+      throw new IllegalStateException(

Review Comment:
   Turning a missing scope into a hard error is a good way to surface unbound 
paths, but some callers of `reflect()` still have no scope.
   
   **`CREATE CQ` (regression).** 
`ClusterConfigTaskExecutor#createContinuousQuery` calls 
`Analyzer.analyze(createContinuousQueryStatement.getQueryBodyStatement(), 
context)` on the `ConfigExecution` thread. Type inference for an external 
function goes through `UDTFInformationInferrer` / `UDAFInformationInferrer` → 
`reflect()` → this throw.
   
   ```sql
   CREATE CQ cq_udf RESAMPLE EVERY 30s
   BEGIN SELECT tree_f(s1) INTO root.tdb.out(tf) FROM root.tdb.d END
   ```
   
   - On e6b44b5 this creates the CQ, and `SHOW CQS` lists it as ACTIVE.
   - On 3f6dd87 it fails with `701: Error occurred during inferring UDF data 
type: java.lang.IllegalStateException: Cannot pin UDF ClassLoader for class 
lazy.TreeF because no UDF query scope is active.`
   - The same statement with `abs(s1)` still works.
   
   **Logical views.** Two more paths follow the same pattern. They matter for 
builds that provide a logical-view MNode implementation: the open-source 
`MemMNodeFactory` rejects `CREATE VIEW`, so I could only confirm that `ALTER 
VIEW` now reports this scope error.
   
   - `ClusterConfigTaskExecutor#alterLogicalView` also runs `Analyzer.analyze` 
without a scope.
   - `TimeseriesReaderWithViewFetch#fetchViewTimeSeriesSchemaInfo` runs on the 
into-operation executor and calls `ExpressionTypeAnalyzer` for view 
expressions. The exception there is swallowed by `catch (Exception e)`. A view 
built on an external UDF would therefore get data type `UNKNOWN` and drop out 
of `SHOW TIMESERIES ... WHERE DATATYPE = ...`.
   
   None of these analyses keeps the UDF instance afterwards, so a short-lived 
scope around them should be enough: open, run inside `withQueryScope`, finalize 
in `finally`. An IT that creates a CQ with an external UDF would keep this 
covered.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/QueryExecution.java:
##########
@@ -434,6 +476,11 @@ public void stopAndCleanup(Throwable t) {
     releaseResource(t);

Review Comment:
   For READ queries, the scope is now released only when the state machine 
reaches a terminal state: the listener above, or `start()` failing. But 
`Coordinator#cleanupQueryExecution` ends a query through `stopAndCleanup()`, 
and at that point the query is often still `RUNNING`:
   
   - `stop()` aborts the state tracker, which only polls every 500 ms;
   - `getResult()` returns `Optional.empty()` without `transitionToFinished()` 
when the handle finishes while it is blocked;
   - a client that closes the result set early never drives the state to a 
terminal state at all.
   
   In those cases the scope is never finalized, and the loader it pinned keeps 
`activeQueriesCount > 0` after deprecation.
   
   I measured this on a 1C1D build of 3f6dd87. Each scenario ran on a fresh 
loader; afterwards I rotated the loader and read the counter.
   
   | scenario | runs | pins left |
   |---|---|---|
   | table `SELECT lazy_f(v) FROM t`, read fully | 20 | 5 (7 in another run) |
   | table, 30,000 rows, fetchSize 1000, read fully | 5 | 5 |
   | same, closed after the first row | 5 | 5 |
   | tree `SELECT tree_f(s1) FROM root.tdb.d`, read fully | 20 | 1 (0 in 
another run) |
   | tree, fetchSize 1, closed after the first row | 5 | 5 |
   
   Every loader with pins left stays `deprecated=true, closed=false`. Query 
results are fine, but the loader is no longer closed deterministically.
   
   With the one-line change below, every scenario ends with `active=0, 
closed=true`. `retry()` already goes through `cleanupForRetry()`, and the only 
caller of `stopAndCleanup()` is `Coordinator#cleanupQueryExecutionInternal`, so 
retries are unaffected. On the base build the counter is 0 in all scenarios 
(table queries don't pin there at all).
   
   ```suggestion
       releaseResource(t);
       finalizeUDFClassLoader();
   ```
   
   `QueryExecutionRetryTest` builds every context with `QueryType.OTHER`, so it 
never reaches this path. A test with a `READ` context that reaches `RUNNING` 
and then calls `stopAndCleanup()` would catch it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to