Pratyush Bhatt created HDDS-9542:
------------------------------------
Summary: ozone debug chunkinfo command shows incorrect number of
entries
Key: HDDS-9542
URL: https://issues.apache.org/jira/browse/HDDS-9542
Project: Apache Ozone
Issue Type: Bug
Reporter: Pratyush Bhatt
*Scenario:* Stop a DN from the pipeline and check re-replication.
*Steps:*
1. Do a key put.
2. List all the replica DNs.
3. Stop one of the DN.
4. Do "ozone debug chuninfo" command.
*Configs:*
{code:java}
"hdds.scm.replication.thread.interval": "5m",
"ozone.scm.stale.node.interval": "2m",
"ozone.scm.dead.node.interval": "4m", {code}
*Observed behavior:*
The command is throwing _ERROR scm.XceiverClientGrpc: Failed to execute command
GetBlock_ and it lists only 1 Keylocation(sometimes 0).
{code:java}
[root@ozn-decom75-1 ~]# ozone debug chunkinfo /testv1/testb1/passwd
23/10/26 03:30:34 WARN impl.MetricsConfig: Cannot locate configuration: tried
hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
23/10/26 03:30:34 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period
at 10 second(s).
23/10/26 03:30:34 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
system started
23/10/26 03:30:35 ERROR scm.XceiverClientGrpc: Failed to execute command
GetBlock. Exception Class: java.util.concurrent.ExecutionException, Exception
Message: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
UNAVAILABLE: io exception
{
"KeyLocations": [
[
{
"Datanode-HostName": "ozn-decom75-9.ozn-decom75.xyz",
"Datanode-IP": "172.27.139.192",
"Container-ID": 12002,
"Block-ID": 111677748019344024,
"Locations": {
"files": [
"/hadoop-ozone/datanode/data184295/hdds/CID-d16eaa18-863d-4914-8769-bc3763e7e83c/current/containerDir23/12002/chunks/111677748019344024.block"
],
"pipelineID": "9cfb4d55-a00d-4231-aeb6-cb63b8e26b3c"
}
}
]
]
} {code}
This happens till the dead node interval is met. After that it shows 2
Keylocations(The DNs that are up from that pipeline)
{code:java}
[root@ozn-decom75-1 ~]# ozone debug chunkinfo /testv1/testb1/passwd
23/10/26 03:34:22 WARN impl.MetricsConfig: Cannot locate configuration: tried
hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
23/10/26 03:34:22 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period
at 10 second(s).
23/10/26 03:34:22 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
system started
{
"KeyLocations": [
[
{
"Datanode-HostName": "ozn-decom75-9.ozn-decom75.xyz",
"Datanode-IP": "172.27.139.192",
"Container-ID": 12002,
"Block-ID": 111677748019344024,
"Locations": {
"files": [
"/hadoop-ozone/datanode/data184295/hdds/CID-d16eaa18-863d-4914-8769-bc3763e7e83c/current/containerDir23/12002/chunks/111677748019344024.block"
],
"pipelineID": "7b36169b-6f94-4e86-a8f7-4c9091cb3f7a"
}
},
{
"Datanode-HostName": "ozn-decom75-3.ozn-decom75.xyz",
"Datanode-IP": "172.27.198.65",
"Container-ID": 12002,
"Block-ID": 111677748019344024,
"Locations": {
"files": [
"/hadoop-ozone/datanode/data184295/hdds/CID-d16eaa18-863d-4914-8769-bc3763e7e83c/current/containerDir23/12002/chunks/111677748019344024.block"
],
"pipelineID": "7b36169b-6f94-4e86-a8f7-4c9091cb3f7a"
}
}
]
]
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
