Siyao Meng created HDDS-16881:
---------------------------------
Summary: Intermittent failure in ozone-ha/test-haproxy-s3g.sh
Key: HDDS-16881
URL: https://issues.apache.org/jira/browse/HDDS-16881
Project: Apache Ozone
Issue Type: Sub-task
Components: docker-compose, test
Reporter: Siyao Meng
Assignee: Siyao Meng
{code}
Executing test ozone-ha/test-haproxy-s3g.sh
...
SCM is out of safe mode.
No OM HA service, no need to wait
==============================================================================
s3-generated.Awss3 :: S3 gateway test with aws cli
==============================================================================
File upload and directory list | FAIL |
1 != 0
{code}
h3. Root cause
HDDS-16360 made HAProxy gate S3 traffic on {{/health/ready}} and added
{{wait_for_s3g_ready}} to {{testlib.sh}}, so tests start only after s3g1
reports ready. The wait is guarded by:
{code}
if ! docker-compose config --services 2>/dev/null | grep -q "^s3g1$"; then
return 0
fi
{code}
{{testlib.sh}} runs with {{set -o pipefail}}. {{grep -q}} exits on the first
match, and {{docker-compose}} writes the remaining service names ({{s3g2}},
{{s3g3}}, {{scm1}}, ...) to a closed pipe, gets SIGPIPE, and the pipeline
returns 141. The guard then skips the wait, the first S3 tests run while all
HAProxy backends are still DOWN, and the aws cli gets 503.
Passing runs log {{Waiting for S3 Gateway (s3g1) to report ready...}}; both
failing runs below do not.
h3. Fix
Use {{grep ... > /dev/null}} instead of {{grep -q}}, so grep consumes the whole
output.
- https://github.com/apache/ozone/actions/runs/38102729160/job/114364422083
- https://github.com/apache/ozone/actions/runs/37874990667/job/113645569605
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
