shuke987 opened a new pull request, #67624: URL: https://github.com/apache/doris/pull/67624
### What problem does this PR solve? Problem Summary: A transient Codex capacity error currently ends an entire automated review: `Selected model is at capacity. Please try a different model.` The review runner already persists the main session and the shared review ledger. This PR adds a runner-side recovery helper so a capacity failure can wait and resume that same goal session, without rebuilding Codex, rotating accounts, or rerunning the workflow. - Resume the exact main UUID using `codex exec --goal ... resume <UUID> <recovery objective>`; never use `--last` or a thread name. - Allow at most three resumes, with 30 / 60 / 120 second backoff. - Share one monotonic 89-minute helper budget across all attempts and waits. The review step retains its existing 90-minute timeout and subsequent GitHub verification. - Require a persisted rollout for that UUID and checkout, and check the installed CLI accepts goal-resume arguments. - Before resuming, recheck that the PR is open with the expected base/head. Refuse automatic resume if a bot review for this head was already submitted during this run; keep that failure visible for manual verification instead of potentially duplicating the final review. - Preserve the existing CODEX_HOME, checkout, ledger, model and reasoning effort. Recovery instructions retain completed work and the original review-round limit, account for interrupted child agents, and require fresh GitHub review/comment checks before writes. - Keep raw per-attempt events/stderr/final messages in the run's temporary context directory. Merge parseable events for the existing Litefuse uploader, retaining failed-attempt items and child-thread IDs. Do not reuse an earlier attempt's partial final message. - Authentication errors, usage limits, unknown errors, cancellation and timeout are not retried. Terminal errors still reach the existing failure and usage-limit handling. The helper is fetched from the workflow's pinned ref, following the existing helper-loading pattern, not executed from the PR checkout. ### Validation and Draft status **This is deliberately a Draft pending validation with the actual deployed Linux Codex goal binary.** The source linked by [Doris #64658](https://github.com/apache/doris/pull/64658), [zclllyybb/codex at 8072a457fad9](https://github.com/zclllyybb/codex/blob/8072a457fad974d6e856b319e225541ee1bde59d/codex-rs/exec/src/lib.rs#L769), explicitly supports goal + resume. Its UUID resolution calls thread/resume directly rather than falling back to a new thread. However, the mutable OSS binary reports only `0.0.0`; its exact source SHA has not been established. Local validation completed: - `python3 -m unittest discover -s .github/scripts -p 'test_*.py'`: **87 tests passed**, including 28 new tests. - A local fake-Codex process test fails the first invocation with capacity, then succeeds in a new process using the original UUID and preserved ledger. - Actual subprocess timeout and CLI SIGTERM tests confirm cleanup and no automatic restart. - Error classification, retry exhaustion, shared deadline, missing/wrong session evidence, unsupported arguments, stale PR, paginated already-submitted reviews, malformed/truncated output, and Litefuse event/child-ID preservation are covered. - Ruff formatting/check passed. - Workflow YAML parsed; all 21 shell blocks passed `bash -n`. - `git diff --check` passed. Before marking ready: - [ ] Use the actual deployed Linux binary in a controlled capacity-failure/recovery smoke test. - [ ] Verify the main UUID remains unchanged and goal execution progresses after resume. - [ ] Verify unfinished subagent work can be recovered without repeating completed review work or submitting duplicate GitHub reviews. - [ ] Verify final main/subagent trace linkage in Litefuse. No production review, OSS replacement, credential operation, or real model request was performed for these tests. Process-level session recovery is not the same as retrying an in-flight model request inside the original Codex process. ### Release note None ### Check List (For Author) - Test - [x] Unit Test - [x] Manual test (local subprocess fault injection and workflow syntax checks described above) - Behavior changed: - [x] Yes. Bounded recovery of transient capacity failures inside the same review job. - Does this need documentation? - [x] No. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases and deployed-binary smoke validation - [ ] Confirm document - [ ] Add branch pick label -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
