wenzhenghu opened a new pull request, #68740:
URL: https://github.com/apache/doris/pull/68740
### What problem does this PR solve?
Problem Summary:
The disk monitor performs health checks synchronously. If a check or its
error-handling path blocks indefinitely, the monitor cannot continue checking
disks or complete the existing failure handling.
Add an independent watchdog around startup data/spill directory checks and
local StorageEngine disk health checks, including broken-path persistence,
subsequent disk-state/failure handling, and warnings for invalid monitor
intervals. Each operation is timed separately; normal monitor intervals are
excluded. Supervision remains active while shutdown waits for the disk monitor.
On timeout, the watchdog calls `_exit(74)` without logging, flushing,
running fatal handlers, or joining threads. Existing handling of completed IO
errors remains unchanged. This protects the existing probe paths; it does not
provide per-disk isolation, cover every stalled business IO, or act as a
general crash watchdog.
The new translation unit is excluded from Storage unity batches to preserve
their existing grouping and avoid exposing unrelated implicit include
dependencies.
### Release note
Add the static BE configuration `disk_health_check_timeout_seconds`,
defaulting to `0` (disabled). A positive value enables supervision and makes a
timeout in a protected operation terminate BE with exit status `74`.
Configuration changes require a BE restart.
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test
- [ ] No need to test or manual test.
Targeted ASAN validation and clang-tidy are being rerun on this PR head.
Results will be updated before this PR is marked ready for review. The
validation uses the standard test entry point with a temporary selection of the
relevant test sources and required helpers; this is not a full BE unit-test
build.
Full BE executable build/startup, startup fault injection, Linux kernel IO
hangs, and cluster failover have not been validated.
- Behavior changed:
- [ ] No.
- [x] Yes. A timeout terminates BE when the new configuration is
enabled; the default is disabled.
- Does this need documentation?
- [ ] No.
- [x] Yes. The new BE configuration needs a configuration-reference
entry; no documentation PR is included yet.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
