sodonnel commented on PR #6613:
URL: https://github.com/apache/ozone/pull/6613#issuecomment-3311663690
These are the steps I used to test the streaming block code.
1. Download this PR.
2. Merge in master - there was one small compile error I fixed by passing a
string to the container scanner
3. Create a single node docker cluster with the build, but adjust the config
to make the blocksize 128MB rather than 1MB (docker_config file in the
compose/ozone folder)
Then create a 1GB file:
```
dd if=/dev/urandom of=1gb bs=1M count=1024
```
Create a bucket, volume and the key:
```
ozone sh volume create sodonnell
ozone sh bucket create sodonnell/bucket
ozone sh key put sodonnell/bucket/1bg
export OZONE_ROOT_LOGGER=INFO,console
ozone sh key cat sodonnell/bucket/1gb > /dev/null
```
The cat command fails with this checksum error:
```
2025-09-19 10:17:17,474 [main] INFO impl.MetricsConfig: Loaded properties
from hadoop-metrics2.properties
2025-09-19 10:17:17,531 [main] INFO impl.MetricsSystemImpl: Scheduled Metric
snapshot period at 10 second(s).
2025-09-19 10:17:17,531 [main] INFO impl.MetricsSystemImpl:
XceiverClientMetrics metrics system started
2025-09-19 10:17:18,249 [main] WARN scm.XceiverClientGrpc: Failed to execute
command ReadBlock on the pipeline Pipeline{ Id:
dbc8f351-a151-432f-83de-ece94494810e, Nodes: [
{7fb912ed-eec9-4788-8b35-15f243b514ee(ozone-datanode-1.ozone_default/172.20.0.5),
ReplicaIndex: 0},], ReplicationConfig: STANDALONE/ONE, State:ALLOCATED,
leaderId:, CreationTimestamp2025-09-19T10:13:08.181Z[UTC]}.
2025-09-19 10:17:18,251 [main] WARN scm.XceiverClientGrpc: Failed to execute
command ReadBlock on the pipeline Pipeline{ Id:
dbc8f351-a151-432f-83de-ece94494810e, Nodes: [
{7fb912ed-eec9-4788-8b35-15f243b514ee(ozone-datanode-1.ozone_default/172.20.0.5),
ReplicaIndex: 0},], ReplicationConfig: STANDALONE/ONE, State:ALLOCATED,
leaderId:, CreationTimestamp2025-09-19T10:13:08.181Z[UTC]}.
2025-09-19 10:17:18,253 [main] WARN scm.XceiverClientGrpc: Failed to execute
command ReadBlock on the pipeline Pipeline{ Id:
dbc8f351-a151-432f-83de-ece94494810e, Nodes: [
{7fb912ed-eec9-4788-8b35-15f243b514ee(ozone-datanode-1.ozone_default/172.20.0.5),
ReplicaIndex: 0},], ReplicationConfig: STANDALONE/ONE, State:ALLOCATED,
leaderId:, CreationTimestamp2025-09-19T10:13:08.181Z[UTC]}.
Checksum count mismatched: thatChecksumsCount=1 > thisChecksumsCount (=256 )
- thisStartIndex (=256)
```
I captured debug logs before this error occurred, and I can see it performed
259 `getBlock` calls against the same block (4kb at a time)
[debug.log.gz](https://github.com/user-attachments/files/22425454/debug.log.gz)
```
grep "Executing command cmdType: ReadBlock" debug.log | wc -l
259
```
So the code as it stands in this test doesn't seem to be making a single
getBlock call to read the entire block. Its making a call for every 4kb, which
is even more strange as the checksum size is set to 16kb here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
