errose28 opened a new pull request, #7944:
URL: https://github.com/apache/ozone/pull/7944
## What changes were proposed in this pull request?
### Summary
Currently ozone admin container reconcile is used to trigger reconciliation
for a specific container, but the resulting checksums are only shown in ozone
admin container info --json which can be verbose and usually requires jq
filtering to get the desired information.
This PR adds new functionality to the `ozone admin container reconcile
command`:
- Multiple container IDs can be passed: `ozone admin container reconcile 1 2
3 4`
- IDs can be read from stdin, similar to `ozone admin container info`: `cat
<id-file> | ozone admin container reconcile -`
- Either of the above options work with a `--status` flag that prints
replica and checksum information instead of sending reconcile commands: `ozone
admin container reconcile --status 1 2 3 4`
- In the status output, a `replicasMatch` field is set to `true` or
`false` on the client side, which can be used to quickly filter out containers
with mismatched replicas.
No proto changes are made. The same `ScmClient#getContainer` and
`ScmClient#getContainerReplicas` methods used by `ozone admin container info`
are re-used for `ozone admin container reconcile --status`. This command just
provides a more filtered view of the results specific to reconciliation and
adds the `replicasMatch` field.
### Examples
Querying status of one container:
```
$ ozone admin container reconcile --status 1
[ {
"containerID" : 1,
"state" : "CLOSING",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"minimumNodes" : 1,
"replicationType" : "RATIS"
},
"replicasMatch" : true,
"replicas" : [ {
"datanode" : {
"hostname" : "ha-dn4-1.ha_net",
"uuid" : "cf484945-b60f-4906-9c4c-024ba429783a"
},
"state" : "OPEN",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn5-1.ha_net",
"uuid" : "927a2620-feb9-43f0-bf0c-3be63eae8f46"
},
"state" : "OPEN",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn3-1.ha_net",
"uuid" : "e39fb4dc-c676-4ace-a3f0-d52c8a3a21c8"
},
"state" : "OPEN",
"dataChecksum" : "0"
} ]
} ]
```
Querying the status of multiple containers (json list output):
```
sh-4.4$ ozone admin container reconcile --status 1 3
[ {
"containerID" : 1,
"state" : "CLOSED",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"minimumNodes" : 1,
"replicationType" : "RATIS"
},
"replicasMatch" : true,
"replicas" : [ {
"datanode" : {
"hostname" : "ha-dn4-1.ha_net",
"uuid" : "cf484945-b60f-4906-9c4c-024ba429783a"
},
"state" : "CLOSED",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn5-1.ha_net",
"uuid" : "927a2620-feb9-43f0-bf0c-3be63eae8f46"
},
"state" : "CLOSED",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn3-1.ha_net",
"uuid" : "e39fb4dc-c676-4ace-a3f0-d52c8a3a21c8"
},
"state" : "CLOSED",
"dataChecksum" : "0"
} ]
}, {
"containerID" : 3,
"state" : "OPEN",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"minimumNodes" : 1,
"replicationType" : "RATIS"
},
"replicasMatch" : true,
"replicas" : [ {
"datanode" : {
"hostname" : "ha-dn5-1.ha_net",
"uuid" : "927a2620-feb9-43f0-bf0c-3be63eae8f46"
},
"state" : "OPEN",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn3-1.ha_net",
"uuid" : "e39fb4dc-c676-4ace-a3f0-d52c8a3a21c8"
},
"state" : "OPEN",
"dataChecksum" : "0"
}, {
"datanode" : {
"hostname" : "ha-dn4-1.ha_net",
"uuid" : "cf484945-b60f-4906-9c4c-024ba429783a"
},
"state" : "OPEN",
"dataChecksum" : "0"
} ]
} ]
```
Given a list of container IDs in a file, output the IDs of all the ones with
mismatched replicas:
```
$ cat ids.txt | ozone admin container reconcile --status - | jq '.[] |
select(.replicasMatch = false) | .containerID'
1
3
```
Same as above, but now send the IDs of mismatched containers back in for
reconciliation (TODO clean up duplicate output):
```
cat ids.txt | ozone admin container reconcile --status - | jq '.[] |
select(.replicasMatch = true) | .containerID' | ozone admin container reconcile
-
Reconciliation has been triggered for container 1
Use "ozone admin container reconcile --status 1" to see the checksums of
each container replica
Reconciliation has been triggered for container 3
Use "ozone admin container reconcile --status 3" to see the checksums of
each container replica
```
## What is the link to the Apache JIRA
HDDS-12078
## How was this patch tested?
- [x] Tested manually in docker
- [ ] Unit tests (WIP)
- [ ] Update existing acceptance tests (WIP)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
