[
https://issues.apache.org/jira/browse/HDDS-17236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17236:
----------------------------------
Labels: pull-request-available (was: )
> Intermittent failure in ozonesecure public key and certificate recovery test
> ----------------------------------------------------------------------------
>
> Key: HDDS-17236
> URL: https://issues.apache.org/jira/browse/HDDS-17236
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: docker-compose, test
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
>
> {code}
> Port 9888 is available on recon
> find /data/metadata/recon/keys/public.pem -delete && find
> /data/metadata/recon/certs/*.crt -delete succeed
> Ncat: Could not resolve hostname "recon": Name or service not known. QUITTING.
> Port 9888 is not available on recon yet
> ...
> Timed out waiting on recon 9888 to become available
> ERROR: Test execution of ozonesecure/test.sh is FAILED!!!!
> {code}
> h3. Root cause
> {{ozonesecure/public-key-cert-recovery-test.sh}} restarts Recon (and later
> the datanodes) with {{docker-compose stop recon}} followed by
> {{docker-compose start recon}}. The Recon container log shows the SIGTERM and
> a clean shutdown, but no second {{STARTUP_MSG}}, and the hostname stays
> unresolvable for the whole 120s wait. So the container was never started
> again, yet {{docker-compose start}} exited 0 (the script runs with {{set -e}}
> and kept going).
> {{docker compose start}} (v2.38.2 on the runner) lists the project containers
> once and skips any whose state is {{running}}. The Docker daemon wakes up the
> {{docker stop}} caller in {{handleContainerExit}} via {{SetStopped}} before
> it writes the new state to the replica that {{ContainerList}} reads
> ({{CheckpointTo(containersReplica)}}). If {{start}} lists containers in that
> window, it still sees Recon as running and silently does nothing.
> The same failure appears on master on 2026-07-13, 2026-07-15 and 2026-08-31
> (ozone-build-results), always with the identical log pattern.
> h3. Fix
> Use {{docker-compose restart}} instead of {{stop}} + {{start}}. It calls
> {{ContainerRestart}} for each container unconditionally, so the stop and
> start happen inside the daemon with no stale state check in between.
> - https://github.com/apache/ozone/actions/runs/39561350318/job/119114972849
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
