JackieTien97 opened a new pull request, #19224:
URL: https://github.com/apache/iotdb/pull/19224

   ## Description
   
   Fix two cleanup failures after partial query fragment dispatch.
   
   Planners previously started a scheduler before returning it to 
`QueryExecution`. A terminal-state listener could consume the stop flag while 
the scheduler reference was still null or referred to the previous attempt, 
leaving the current scheduler unstopped. Separately, `DataNodeQueryContext` 
removal depended on the planned FI count reaching zero, so an FI that was never 
dispatched could retain the context indefinitely.
   
   ### Example
   
   Query `Q` plans three FIs on DN1: FI1 starts, FI2 fails with status 719, and 
FI3 is never dispatched. Previously, termination could be lost and the shared 
context would continue waiting for FI3. With this change, closing `(Q, 0)` 
cancels the actual registered FIs and releases their shared context once its 
users finish cleanup. A retry uses `(Q, 1)`, so delayed cancellation and 
resource-release callbacks from the first attempt cannot remove the retry's 
context.
   
   ### Scheduler and attempt lifecycle
   
   - Create schedulers without starting them, publish them through a 
per-attempt handoff, and retain stop requests that arrive before publication.
   - Make cluster scheduler stop idempotent, abort further dispatch, and wake 
topology dispatch waits. Preserve the normal completion grace period and 
terminate immediately when stopping during dispatch or after a dispatch failure.
   - Adapt LOAD scheduler cleanup so stopping before start also releases 
constructor-owned memory exactly once.
   - Register shared DataNode state by `(queryId, attemptId)`. Keep it across 
gaps between sequential FI arrivals, and close a failed/cancelled attempt 
without waiting for never-dispatched FIs. Planning failures and execution 
cleanup release registrations idempotently.
   
   ### Late requests and compatibility
   
   After its actual users release their resources, a closed attempt retains 
only a lightweight admission record, without LAST caches or other shared query 
data. Admission deadlines are separate from execution deadlines: the admission 
window follows the internal DataNode RPC timeout (60 seconds when the socket 
timeout is unlimited), capped by the query deadline. Transport retries reuse 
the original FI deadline; stopping dispatch freezes the latest issued cutoff. 
Expired requests remain rejected after the corresponding record is removed, 
while an already admitted long-running query keeps its execution timeout. 
Deadline comparisons use coordinator-provided wall-clock timestamps.
   
   Attempt metadata is carried in optional `TSendFragmentInstanceReq` and 
`TCancelQueryReq` fields; the binary FI body is unchanged. Legacy requests 
retain their fallback behavior, and older receivers can ignore the new fields. 
Complete attempt isolation requires both endpoints to support the metadata.
   
   ### Validation
   
   - JDK 17 targeted reactor `clean test`: **79 tests passed across 14 
classes**, with zero failures, errors, or skips. Coverage includes scheduler 
publication/stop races, partial dispatch, retry isolation, late requests, 
bounded admission records, RPC retries, legacy wire compatibility, LOAD 
cleanup, LAST cache behavior, and execution memory budgets.
   - `mvn test-compile -DskipTests`: passed for the full reactor.
   - `mvn test-compile -P with-zh-locale -DskipTests`: passed for the full 
reactor.
   - Formatting and Checkstyle passed in the reactor builds; English/Chinese 
message keys and the new message's placeholders match.
   
   Multi-node cluster and rolling-upgrade integration tests have not been run.
   
   This PR has:
   - [x] been self-reviewed.
   - [x] added comments explaining lifecycle ownership and concurrency 
decisions.
   - [x] added or updated unit tests for the changed code paths.
   
   ##### Key changed/added classes
   
   `QueryExecution`, `ClusterScheduler`, `FragmentInstanceDispatcherImpl`, 
`SimpleQueryTerminator`, `QueryAttemptId`, `QueryAttemptRegistry`, 
`FragmentInstanceManager`, and `FragmentInstanceContext`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to