Doris-Breakwater commented on issue #70632:
URL: https://github.com/apache/doris/issues/70632#issuecomment-6914884798

   Breakwater-GitHub-Analysis-Slot: slot_a4f61110efc2
   
   **Assessment:** The reported path is supported by source inspection: a 
normal OLAP insert can register an `InsertLoadJob` and then return successfully 
for a planner-proven empty source without completing that job. This is a 
remaining lifecycle gap after 
[#62282](https://github.com/apache/doris/pull/62282) / 
[#62889](https://github.com/apache/doris/pull/62889). Suggested triage: FE / 
INSERT / memory leak, with high priority given the reported master-FE 
availability impact. The issue currently has no labels or assignee.
   
   **Verified in code**
   
   I checked the configured local repository at `4.1.3` (commit `7126cf65d96e`) 
and `4.1.5-release` (`917b20be8387`), and fetched the relevant upstream master 
files at `13cff7ff484596f29aa1381750e906db3c8bb704` (2026-11-16). This was 
static investigation; I did not run the SQL against a cluster or modify source 
code.
   
   - In 4.1.3, ordinary OLAP INSERT SELECT passes `needRegister = jobId != -1`; 
the base constructor immediately registers the job. `LoadJob.state` initially 
equals `PENDING`. Registration retains it in both the ID map and the 
database/label index. See [executor 
selection](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java#L430-L448),
 
[constructor](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/AbstractInsertExecutor.java#L106-L120),
 and 
[registration/completion](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java#L190-L235).
   - For the empty OLAP path, `initPlan` skips transaction creation, and 
`runInternal` returns before either execution method. OLAP inherits 
`shouldExecuteEmptyInsert() == false`. Consequently, neither successful 
completion bookkeeping nor failure bookkeeping runs. The 4.1.5-release path has 
the same gap. See [4.1.3 transaction 
gate](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java#L306-L314)
 and [4.1.5 early 
return](https://github.com/apache/doris/blob/917b20be83878e98f94e3ebf7efccc80346e3e50/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java#L660-L677).
   - Current master still registers before [the `requiresTransaction()` 
return](https://github.com/apache/doris/blob/13cff7ff484596f29aa1381750e906db3c8bb704/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertIntoTableCommand.java#L771-L786).
 For an ordinary empty OLAP source without stream offset updates, this 
predicate is false.
   - Normal retention cannot reclaim these entries: 
[`isExpired`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadJob.java#L1137-L1151)
 rejects unfinished jobs, while [count-based 
cleanup](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/LoadManager.java#L417-L434)
 selects only completed jobs. If stuck entries alone exceed the threshold, that 
loop exhausts all completed jobs present in this manager. Metrics aggregation 
also traverses the retained jobs. Registration does not journal the job; 
completion does, which is consistent with the reported master-only growth and 
restart reset.
   
   The production growth rate, retained bytes per job, GC pauses, and audit 
correlation remain reporter-provided observations, not independently measured 
results. The mechanism is sufficient for triage, but does not prove that every 
pending job or every pause has this cause. The problematic boundary is the 
physical empty plan, rather than LIST partitioning specifically. An executed 
query that happens to return zero rows follows a different lifecycle.
   
   **Recommended fix and validation**
   
   1. Prefer an OLAP-specific completion path for the successful no-op that 
finalizes an already registered job as `FINISHED`, with zero-row statistics and 
normal terminal timestamps/persistence. Alternatively, suppress registration 
for inserts that will take the no-transaction return, if omitting their SHOW 
LOAD history is accepted.
   2. Do not simply invoke the existing OLAP `onComplete()` or `afterExec()` 
unchanged: no transaction was opened (`txnId == -1`), `onComplete()` expects a 
real transaction, and the executor's initial transaction status is `ABORTED`. 
Preserve a correct successful client response without fabricating a committed 
transaction. Keep intentional non-registration for VALUES, internal group 
commit, and external executors.
   3. Preserve empty-source side effects: 4.1.x has external overwrite 
handling, and master [requires a transaction for stream offset updates even 
with no output 
rows](https://github.com/apache/doris/blob/13cff7ff484596f29aa1381750e906db3c8bb704/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/AbstractInsertExecutor.java#L350-L357).
 A blanket empty-source skip would break those semantics.
   4. Extend 
`regression-test/suites/load_p0/insert/test_insert_statistic.groovy` with 
repeated missing-LIST-partition inserts and a constant-false source. Assert the 
INSERT plan is empty and each uniquely labeled job becomes terminal (or is 
absent under the alternate design), with no residual PENDING growth. Cover an 
executed zero-row query, ordinary success/failure, and the existing 
empty-source side-effect cases. Validate cleanup and, if recording finished 
jobs, replay after restart. Backport after testing each affected branch.
   
   Treat an age-based PENDING cleaner as a separate change. Live INSERT jobs 
also remain PENDING until completion, so age alone does not establish that an 
entry is orphaned. Lowering retention or increasing the count threshold will 
not remove the stuck entries.
   
   **Remaining evidence / next steps**
   
   The supplied reproducer is enough to start a regression and PR; a full 
production dump is not a prerequisite. Please add the exact FE build hash, 
relevant session settings (especially group commit), the full `EXPLAIN INSERT 
INTO ... SELECT ...` output, and uniquely labeled SHOW LOAD results plus 
before/after master PENDING counts from an otherwise quiet reproduction. These 
confirm that the actual INSERT selects the claimed executor and physical plan. 
A redacted heap dominator/class-histogram comparison and time-aligned GC logs 
would help quantify retained memory and pause attribution if needed.
   
   For temporary prevention, skipping a plain INSERT whose source is 
authoritatively known to be empty avoids this path. A separate EXPLAIN can 
become stale when source data or partitions change, so it should not become a 
general correctness rule. It also does not reclaim existing entries. The 
reporter's willingness to submit a PR is appropriate; the missing regression 
should accompany it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to