smengcl opened a new pull request, #11611: URL: https://github.com/apache/ozone/pull/11611
Generated-by: Claude Code (Opus 5.5) ## What changes were proposed in this pull request? A datanode with many containers can take a long time to start. Until then, operators can only follow the logs to tell which phase it is in and how much longer it will take. This PR adds a "Startup Progress" page to the Datanode web UI. ### `StartupProgress` (hdds-server-framework) A small tracker that can be reused for OM and SCM later. - Phases are the constants of an enum, in execution order. Beginning or ending a phase also ends all earlier phases, so a service only needs to mark phase boundaries. - A phase can have concurrent steps (e.g. one per volume), each counting done out of total work items. - The remaining time of a step is estimated from its rate so far. A phase reports the remaining time of its slowest running step, or unknown when none of its steps is running. - Phases and steps are `PENDING`, `RUNNING`, `COMPLETE` or `FAILED`. - Exposed through `StartupProgressMXBean`, so the data is also available via `/jmx`. ### Datanode Phases (`DatanodeStartupPhase`): 1. Initializing 2. Waiting for SCM (until the version response is received) 3. Preparing volumes (`VersionEndpointTask#checkVolumeSet`: volume format and DB load) 4. Loading containers (one step per volume, with done / total containers) 5. Checking volumes 6. Starting container services (includes Ratis pipeline recovery) Other changes: - `ContainerReader` counts the container directories of the volume with `File#list()` up front to get the total, then loads containers with the existing loop. A volume that fails to load is shown as a `FAILED` step. - If container services fail to start, the running phases are marked `FAILED`. - The bean is registered as `Hadoop:service=HddsDatanode,name=StartupProgress`. - New "Startup Progress" navbar entry, using the existing `extraNavLinks` of the navmenu. The page shows each phase with its per volume steps (status, elapsed time, done / total, estimated remaining time), and refreshes every 2 seconds until startup completes or fails. The Datanode HTTP server already starts before the slow phases, so their progress is visible while the datanode starts. OM and SCM start their HTTP server near the end of startup, so they are left for a follow-up. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17107 ## How was this patch tested? - New `TestStartupProgress`: phase transitions, remaining time estimate for concurrent steps, failure marking, and a JMX round trip that checks the attribute names the UI reads. - New `TestContainerReader#testContainerReaderProgress`. - `TestRunningDatanodeState` now checks that a failed container start marks startup as failed, and a successful one does not. - The page was rendered in headless Chromium against mocked `/jmx` responses. - [CI on this branch](https://github.com/smengcl/hadoop-ozone/actions/runs/39121031207) passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
