adoroszlai opened a new pull request, #6952:
URL: https://github.com/apache/ozone/pull/6952
## What changes were proposed in this pull request?
Logs for one of the containers for each run is missing from acceptance test
artifacts.
For example, `docker-ozone-datanode-1.log` and
`docker-ozonesecure-ha-datanode1-1.log` here:
```
$ find 2024/07/12/32233/acceptance-s3a -name 'docker*' | sort
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-datanode-2.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-datanode-3.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-httpfs-1.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-om-1.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-recon-1.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-s3g-1.log
2024/07/12/32233/acceptance-s3a/ozone/s3a/docker-ozone-scm-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-datanode2-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-datanode3-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-httpfs-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-kdc-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-kms-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-om1-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-om2-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-om3-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-recon-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-s3g-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-scm1.org-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-scm2.org-1.log
2024/07/12/32233/acceptance-s3a/ozonesecure-ha/s3a/docker-ozonesecure-ha-scm3.org-1.log
```
This happens due to difference in output of Docker Compose v1 and v2:
- v1 prints two lines of header,
- v2 prints only one.
```
$ docker-compose ps
Name Command State
Ports
----------------------------------------------------------------------------------------------------------------------------------------------------
ozone-datanode-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:33548->19864/tcp,:::33548->19864/tcp,
0.0.0.0:33549->9882/tcp,:::33549->9882/tcp
ozone-datanode-2 /usr/local/bin/dumb-init - ... Up
0.0.0.0:33550->19864/tcp,:::33550->19864/tcp,
0.0.0.0:33551->9882/tcp,:::33551->9882/tcp
ozone-datanode-3 /usr/local/bin/dumb-init - ... Up
0.0.0.0:33546->19864/tcp,:::33546->19864/tcp,
0.0.0.0:33547->9882/tcp,:::33547->9882/tcp
ozone-httpfs-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:14000->14000/tcp,:::14000->14000/tcp
ozone-om-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:9862->9862/tcp,:::9862->9862/tcp,
0.0.0.0:9874->9874/tcp,:::9874->9874/tcp
ozone-recon-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:9888->9888/tcp,:::9888->9888/tcp
ozone-s3g-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:9878->9878/tcp,:::9878->9878/tcp
ozone-scm-1 /usr/local/bin/dumb-init - ... Up
0.0.0.0:9860->9860/tcp,:::9860->9860/tcp,
0.0.0.0:9876->9876/tcp,:::9876->9876/tcp
```
```
$ docker compose ps
NAME IMAGE COMMAND
SERVICE CREATED STATUS PORTS
ozone-datanode-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" datanode 25 seconds ago Up 23 seconds
0.0.0.0:33549->9882/tcp, :::33549->9882/tcp, 0.0.0.0:33548->19864/tcp,
:::33548->19864/tcp
ozone-datanode-2 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" datanode 25 seconds ago Up 22 seconds
0.0.0.0:33551->9882/tcp, :::33551->9882/tcp, 0.0.0.0:33550->19864/tcp,
:::33550->19864/tcp
ozone-datanode-3 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" datanode 25 seconds ago Up 23 seconds
0.0.0.0:33547->9882/tcp, :::33547->9882/tcp, 0.0.0.0:33546->19864/tcp,
:::33546->19864/tcp
ozone-httpfs-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" httpfs 25 seconds ago Up 23 seconds
0.0.0.0:14000->14000/tcp, :::14000->14000/tcp
ozone-om-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" om 25 seconds ago Up 23 seconds
0.0.0.0:9862->9862/tcp, :::9862->9862/tcp, 0.0.0.0:9874->9874/tcp,
:::9874->9874/tcp
ozone-recon-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" recon 25 seconds ago Up 23 seconds
0.0.0.0:9888->9888/tcp, :::9888->9888/tcp
ozone-s3g-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" s3g 25 seconds ago Up 23 seconds
0.0.0.0:9878->9878/tcp, :::9878->9878/tcp
ozone-scm-1 apache/ozone-runner:20240316-jdk17-1
"/usr/local/bin/dumb…" scm 25 seconds ago Up 23 seconds
0.0.0.0:9860->9860/tcp, :::9860->9860/tcp, 0.0.0.0:9876->9876/tcp,
:::9876->9876/tcp
```
and the code that collects logs only starts at the third line (`tail -n +3`):
https://github.com/apache/ozone/blob/63a232b798c2eba71f19bafc607c20c8144a41af/hadoop-ozone/dist/src/main/compose/testlib.sh#L305-L311
https://issues.apache.org/jira/browse/HDDS-11186
## How was this patch tested?
Verified that datanode1 log is also saved (in addition to all other
container logs):
```
renamed 'ozone-balancer/result/docker-ozone-balancer-datanode1-1.log' ->
'/home/runner/work/ozone/ozone/hadoop-ozone/dist/target/ozone-1.5.0-SNAPSHOT/compose/result/ozone-balancer/docker-ozone-balancer-datanode1-1.log'
```
https://github.com/adoroszlai/ozone/actions/runs/9954339826/job/27500597426#step:5:124
whereas previously datanode2 was the first item, datanode1 was missing:
```
renamed 'ozone-balancer/result/docker-ozone-balancer-datanode2-1.log' ->
'/home/runner/work/ozone/ozone/hadoop-ozone/dist/target/ozone-1.5.0-SNAPSHOT/compose/result/ozone-balancer/docker-ozone-balancer-datanode2-1.log'
```
https://github.com/apache/ozone/actions/runs/9936569749/job/27445931351#step:5:123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
