u70b3 opened a new pull request, #68669:
URL: https://github.com/apache/doris/pull/68669
### What problem does this PR solve?
Issue Number: #66497
Related PR: #67978 (PR3D), and the stacked worker PR (PR4A) — this PR stacks
on PR4A's head; only the top commits belong to this PR
Problem Summary: PR4A delivers the isolated worker and supervisor; this
stacked PR delivers the slice-4 acceptance evidence — the IVF_PQ tracer bullet
through the real isolated path and the worker-fault/UNKNOWN-convergence suites
(v5.1 §9.4 G3), plus the BE debug points they need. The
`enable_lance_index_mutation` gate stays off.
What lands:
1. **Worker fault-injection debug points** (gated by `enable_debug_points`,
default off):
- `LanceIndexWorker.hang` — blocks after dispatch validation, before the
first FFI call (wall-clock escalation → UNKNOWN);
- `LanceIndexWorker.skip_report` — executes the FFI, exits 0 without a
result frame (silent-worker → UNKNOWN);
- `LanceIndexSupervisor.reject_after_enqueue` — the supervisor rejects
after dequeue, before any fork, through the production async-rejection path
(full `PRE_INVOCATION_RESOURCE_REJECTED` + `NEVER_LAUNCHED` envelope).
The exec'd worker starts with an empty debug-point registry, so the
supervisor hands the two worker-side point names across the exec boundary
through the controlled environment allowlist
(`DORIS_LANCE_WORKER_DEBUG_POINTS`) — names only, derived from the supervisor's
own registry, never operator-inherited.
2. **Tracer suite** (`test_lance_index_worker_tracer.groovy`) — CREATE INDEX
(4 partitions / 8 bits / 4 subvectors on the preinstalled 16-dim float32 table)
→ COMMITTED with PossibleLive=NO → refresh REQUIRED→DONE →
`lance_index_entries` shows the new index with an advanced dataset version →
`vector_search` consumption proof (exact flat ladder baseline, indexed top-1
hit, recall floor, nprobes discriminator) → DROP INDEX second job closes the
loop. The suite derives the runner's delegation profile from the first job's
terminal state and loudly skips the positive remainder on a delegation-less
runner (that profile is covered by the negative suite).
3. **Faults suite** (`test_lance_index_worker_faults.groovy`) — exact-PID
kill of a hung worker (attributed via `/proc/<pid>/cgroup` matching the
invocation group; an unattributed PID is never killed — the case degrades to
the supervisor wall-clock path) → UNKNOWN convergence with the termination
proof releasing the possible-live slot and the BE surviving; hang + shortened
deadline; skip_report on a DROP job → UNKNOWN at the deadline sweep;
reject_after_enqueue → NOT_COMMITTED, never UNKNOWN.
Existing 3C/3D negative suites are untouched. The dispatcher-promptness need
this PR originally carried its own fix for was solved upstream in PR3D's sleep
slicing; nothing here duplicates that.
Positive-path acceptance still requires a CI runner with a writable cgroup
v2 delegation (v5.1 §5.2 has no soft fallback — a delegation-less runner
correctly rejects, and the negative suite covers exactly that). The delegation
evidence is recorded by the suites themselves from the observed terminal states.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [x] Other reason <!-- Add your reason? -->
The debug points carry unit coverage (delegated runner: 72/72); the two
groovy suites need the external docker fixture and run in the
`external`/`nonConcurrent` pipelines.
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
