shuke987 opened a new pull request, #65651:
URL: https://github.com/apache/doris/pull/65651

   ### What problem does this PR solve?
   
   Issue Number: close #xxx
   
   Related TeamCity build: 
http://43.132.222.7:8111/buildConfiguration/Doris_DorisRegression_NonConcurrentRegression/993633
   
   Problem Summary:
   
   During graceful BE shutdown, `doris_main` stopped and joined the HTTP 
service before `ExecEnv::destroy()` stopped `StreamLoadRecorderManager`. The 
recorder worker sends audit records through the local BE HTTP stream-load 
endpoint and uses a 600-second request timeout.
   
   In build 993633, shutdown reached `Http service stopped`, then stopped 
`GroupCommitMgr`, and remained blocked until the external ten-minute grace 
timeout sent SIGABRT. The captured main-thread stack is:
   
   ```text
   std::thread::join()
   doris::ExecEnv::destroy()
   main
   ```
   
   In the corresponding destroy order, `StreamLoadRecorderManager::stop()` is 
the next direct `std::thread` join after GroupCommitMgr/routine-load shutdown. 
The triggering PR #65463 changes only regression Groovy code and is 
path-unrelated to this lifecycle issue.
   
   This PR adds an explicit, idempotent recorder-manager stop entrypoint and 
invokes it before any server teardown. The existing call from 
`ExecEnv::destroy()` remains as a fallback. This keeps the local HTTP endpoint 
and the rest of the load path available while the recorder worker finishes.
   
   The exact worker-side curl frame was not captured because the failure 
artifact contains only the signal-thread stack; the conclusion is based on the 
direct join site, shutdown order, local HTTP dependency, and matching timeout.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Test
       - [ ] Regression test
       - [ ] Unit Test
       - [x] Manual test
           - `git diff --check`
           - Verified current `origin/master` lifecycle order and build 993633 
BE logs/stack.
           - TeamCity buildall and graceful-stop runtime validation are pending.
       - [ ] No need to test or manual test. Explain why:
   
   - Behavior changed:
       - [ ] No.
       - [x] Yes.
           - Previous behavior: full graceful teardown stopped HTTP before 
joining the audit stream-load recorder worker.
           - New behavior: the recorder worker is stopped and joined while HTTP 
and its load dependencies are still available.
           - Impact: only the full graceful-exit path; normal audit batching 
and stream-load timeouts are unchanged.
   
   - Does this need documentation?
       - [x] No.
       - [ ] Yes.
   
   ### Check List (For Reviewer who merge this PR)
   
   - [ ] Confirm the release note
   - [ ] Confirm test cases
   - [ ] Confirm document
   - [ ] Add branch pick label
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to