[
https://issues.apache.org/jira/browse/HDDS-13487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bablu Raul updated HDDS-13487:
------------------------------
Environment:
{{[[email protected] ~]# for cid in 5030 5027 5028 5026
5019; do> echo "Checksums for container $cid:"
> ozone admin container info --json $cid | jq -r '.replicas[] |
> "\(.datanodeDetails.hostName) \(.dataChecksum)"'
> echo "-----"
> done
Checksums for container 5030:
ccycloud-6.baraul.root.comops.site a0b5e024
ccycloud-5.baraul.root.comops.site a0b5e024
ccycloud-3.baraul.root.comops.site 572ab97a
-----
Checksums for container 5027:
ccycloud-4.baraul.root.comops.site 20120d7e
ccycloud-8.baraul.root.comops.site 20120d7e
ccycloud-9.baraul.root.comops.site 904f5997
-----
Checksums for container 5028:
ccycloud-6.baraul.root.comops.site 1e8c0039
ccycloud-5.baraul.root.comops.site 1e8c0039
ccycloud-3.baraul.root.comops.site 1e8c0039
-----
Checksums for container 5026:
ccycloud-9.baraul.root.comops.site e7c3f2cf
ccycloud-4.baraul.root.comops.site e7c3f2cf
ccycloud-8.baraul.root.comops.site e7c3f2cf
-----
Checksums for container 5019:
ccycloud-6.baraul.root.comops.site 38e8d3d8
ccycloud-5.baraul.root.comops.site 38e8d3d8
ccycloud-3.baraul.root.comops.site 4e9c35b1
-----}}
This only involves multiple closed container checksums
{{[[email protected] hadoop-ozone]# grep 5028
/var/log/hadoop-ozone/dn-container.log2025-06-12 17:18:17,268 | INFO | ID=5028
| Index=0 | BCSID=0 | State=OPEN | DataChecksum=0 |
2025-06-12 19:32:50,015 | WARN | ID=5028 | Index=0 | BCSID=991 | State=CLOSING
| DataChecksum=1e8c0039 | Container data checksum updated from 0 to 1e8c0039 |
2025-06-12 19:32:50,015 | INFO | ID=5028 | Index=0 | BCSID=991 | State=CLOSING
| DataChecksum=1e8c0039 |
2025-06-12 19:32:50,017 | INFO | ID=5028 | Index=0 | BCSID=991 | State=CLOSING
| DataChecksum=1e8c0039 |
2025-06-12 19:32:50,018 | INFO | ID=5028 | Index=0 | BCSID=991 | State=CLOSED
| DataChecksum=1e8c0039 |
2025-06-12 19:50:04,341 | WARN | ID=5028 | Index=0 | BCSID=991 | State=CLOSED
| DataChecksum=2571a4df | Container data checksum updated from 1e8c0039 to
2571a4df |
[[email protected] hadoop-ozone]#
[[email protected] hadoop-ozone]#
[[email protected] hadoop-ozone]#
[[email protected] hadoop-ozone]# grep 5030
/var/log/hadoop-ozone/dn-container.log
2025-06-12 17:23:27,289 | INFO | ID=5030 | Index=0 | BCSID=0 | State=OPEN |
DataChecksum=0 |
2025-06-12 19:32:49,561 | WARN | ID=5030 | Index=0 | BCSID=1091 |
State=CLOSING | DataChecksum=a0b5e024 | Container data checksum updated from 0
to a0b5e024 |
2025-06-12 19:32:49,562 | INFO | ID=5030 | Index=0 | BCSID=1091 |
State=CLOSING | DataChecksum=a0b5e024 |
2025-06-12 19:32:49,563 | INFO | ID=5030 | Index=0 | BCSID=1091 | State=CLOSED
| DataChecksum=a0b5e024 |
2025-06-12 19:34:56,818 | WARN | ID=5030 | Index=0 | BCSID=1091 | State=CLOSED
| DataChecksum=572ab97a | Container data checksum updated from a0b5e024 to
572ab97a |
2025-06-12 19:50:55,837 | WARN | ID=5030 | Index=0 | BCSID=1091 | State=CLOSED
| DataChecksum=7cd417ef | Container data checksum updated from 572ab97a to
7cd417ef | }}
In the case of container ID 4028, the checksum is updated when transitioning
from the CLOSING to the CLOSED state, which is the expected behavior. However,
for container ID 4030, once the container reaches the CLOSED state, the
checksum should remain unchanged. The correct behavior is that the checksum
must not be updated after the container is in the CLOSED state. It is therefore
unexpected and potentially problematic that the checksum for container ID 4030
is being updated after achieving the CLOSED state
> Checksum is updated after reaching CLOSED state (should remain immutable)
> -------------------------------------------------------------------------
>
> Key: HDDS-13487
> URL: https://issues.apache.org/jira/browse/HDDS-13487
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Environment:
> {{[[email protected] ~]# for cid in 5030 5027 5028 5026
> 5019; do> echo "Checksums for container $cid:"
> > ozone admin container info --json $cid | jq -r '.replicas[] |
> > "\(.datanodeDetails.hostName) \(.dataChecksum)"'
> > echo "-----"
> > done
> Checksums for container 5030:
> ccycloud-6.baraul.root.comops.site a0b5e024
> ccycloud-5.baraul.root.comops.site a0b5e024
> ccycloud-3.baraul.root.comops.site 572ab97a
> -----
> Checksums for container 5027:
> ccycloud-4.baraul.root.comops.site 20120d7e
> ccycloud-8.baraul.root.comops.site 20120d7e
> ccycloud-9.baraul.root.comops.site 904f5997
> -----
> Checksums for container 5028:
> ccycloud-6.baraul.root.comops.site 1e8c0039
> ccycloud-5.baraul.root.comops.site 1e8c0039
> ccycloud-3.baraul.root.comops.site 1e8c0039
> -----
> Checksums for container 5026:
> ccycloud-9.baraul.root.comops.site e7c3f2cf
> ccycloud-4.baraul.root.comops.site e7c3f2cf
> ccycloud-8.baraul.root.comops.site e7c3f2cf
> -----
> Checksums for container 5019:
> ccycloud-6.baraul.root.comops.site 38e8d3d8
> ccycloud-5.baraul.root.comops.site 38e8d3d8
> ccycloud-3.baraul.root.comops.site 4e9c35b1
> -----}}
> This only involves multiple closed container checksums
>
> {{[[email protected] hadoop-ozone]# grep 5028
> /var/log/hadoop-ozone/dn-container.log2025-06-12 17:18:17,268 | INFO |
> ID=5028 | Index=0 | BCSID=0 | State=OPEN | DataChecksum=0 |
> 2025-06-12 19:32:50,015 | WARN | ID=5028 | Index=0 | BCSID=991 |
> State=CLOSING | DataChecksum=1e8c0039 | Container data checksum updated from
> 0 to 1e8c0039 |
> 2025-06-12 19:32:50,015 | INFO | ID=5028 | Index=0 | BCSID=991 |
> State=CLOSING | DataChecksum=1e8c0039 |
> 2025-06-12 19:32:50,017 | INFO | ID=5028 | Index=0 | BCSID=991 |
> State=CLOSING | DataChecksum=1e8c0039 |
> 2025-06-12 19:32:50,018 | INFO | ID=5028 | Index=0 | BCSID=991 |
> State=CLOSED | DataChecksum=1e8c0039 |
> 2025-06-12 19:50:04,341 | WARN | ID=5028 | Index=0 | BCSID=991 |
> State=CLOSED | DataChecksum=2571a4df | Container data checksum updated from
> 1e8c0039 to 2571a4df |
> [[email protected] hadoop-ozone]#
> [[email protected] hadoop-ozone]#
> [[email protected] hadoop-ozone]#
> [[email protected] hadoop-ozone]# grep 5030
> /var/log/hadoop-ozone/dn-container.log
> 2025-06-12 17:23:27,289 | INFO | ID=5030 | Index=0 | BCSID=0 | State=OPEN |
> DataChecksum=0 |
> 2025-06-12 19:32:49,561 | WARN | ID=5030 | Index=0 | BCSID=1091 |
> State=CLOSING | DataChecksum=a0b5e024 | Container data checksum updated from
> 0 to a0b5e024 |
> 2025-06-12 19:32:49,562 | INFO | ID=5030 | Index=0 | BCSID=1091 |
> State=CLOSING | DataChecksum=a0b5e024 |
> 2025-06-12 19:32:49,563 | INFO | ID=5030 | Index=0 | BCSID=1091 |
> State=CLOSED | DataChecksum=a0b5e024 |
> 2025-06-12 19:34:56,818 | WARN | ID=5030 | Index=0 | BCSID=1091 |
> State=CLOSED | DataChecksum=572ab97a | Container data checksum updated from
> a0b5e024 to 572ab97a |
> 2025-06-12 19:50:55,837 | WARN | ID=5030 | Index=0 | BCSID=1091 |
> State=CLOSED | DataChecksum=7cd417ef | Container data checksum updated from
> 572ab97a to 7cd417ef | }}
> In the case of container ID 4028, the checksum is updated when transitioning
> from the CLOSING to the CLOSED state, which is the expected behavior.
> However, for container ID 4030, once the container reaches the CLOSED state,
> the checksum should remain unchanged. The correct behavior is that the
> checksum must not be updated after the container is in the CLOSED state. It
> is therefore unexpected and potentially problematic that the checksum for
> container ID 4030 is being updated after achieving the CLOSED state
>
> Reporter: Bablu Raul
> Priority: Major
>
> Multiple containers show unexpected checksum changes after transitioning to
> the *CLOSED* state, which violates expected behavior. Once a container
> reaches the *CLOSED* state, its checksum should remain stable.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
