Attila Doroszlai created HDDS-8603:
--------------------------------------
Summary: Push replication may fail for concurrent streams to same
target
Key: HDDS-8603
URL: https://issues.apache.org/jira/browse/HDDS-8603
Project: Apache Ozone
Issue Type: Sub-task
Components: Ozone Datanode
Affects Versions: 1.4.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
{code:title=SCM}
2023-05-12 06:37:25,691 [Under Replicated Processor] INFO
replication.ReplicationManager
(ReplicationManager.java:sendDatanodeCommand(684)) - Sending command
[replicateContainerCommand: containerId=11, replicaIndex=1,
targetNode=dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47),
priority=NORMAL] for container ContainerInfo{id=#11, state=CLOSED,
pipelineID=PipelineID=1fdad600-ca58-422a-8cbc-f3f0ea92deb5,
stateEnterTime=2023-05-12T06:37:20.327Z, owner=om1} to
1aeadef1-6c90-4225-addb-3772c43da32e(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47)
with datanode deadline 1683875215691 and scm deadline 1683875245691
2023-05-12 06:37:25,691 [Under Replicated Processor] INFO
replication.ReplicationManager
(ReplicationManager.java:sendDatanodeCommand(684)) - Sending command
[replicateContainerCommand: containerId=10, replicaIndex=4,
targetNode=dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47),
priority=NORMAL] for container ContainerInfo{id=#10, state=CLOSED,
pipelineID=PipelineID=f8d093fe-a02d-4415-b965-bb0272b84054,
stateEnterTime=2023-05-12T06:37:19.680Z, owner=om1} to
1aeadef1-6c90-4225-addb-3772c43da32e(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47)
with datanode deadline 1683875215691 and scm deadline 1683875245691
{code}
{code:title=source datanode}
2023-05-12 06:37:27,364 [ContainerReplicationThread-1] INFO
replication.PushReplicator (PushReplicator.java:replicate(58)) - Starting
replication of container 11 to
dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47)
using NO_COMPRESSION
2023-05-12 06:37:27,367 [ContainerReplicationThread-2] INFO
replication.PushReplicator (PushReplicator.java:replicate(58)) - Starting
replication of container 10 to
dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47)
using NO_COMPRESSION
2023-05-12 06:37:27,384 [ContainerReplicationThread-1] INFO
replication.GrpcOutputStream (GrpcOutputStream.java:close(111)) - Sent 13824
bytes for container 11
2023-05-12 06:37:27,407 [grpc-default-executor-7] WARN
replication.GrpcContainerUploader (GrpcContainerUploader.java:onError(113)) -
Failed to upload container 11 to
dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47)
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE:
Channel shutdown invoked
...
2023-05-12 06:37:27,407 [ContainerReplicationThread-1] WARN
replication.PushReplicator (PushReplicator.java:replicate(73)) - Container 11
replication was unsuccessful.
2023-05-12 06:37:27,408 [ContainerReplicationThread-1] WARN
replication.ReplicationSupervisor (ReplicationSupervisor.java:run(350)) -
Failed FAILED replicateContainerCommand: containerId=11, replicaIndex=1,
targetNode=dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47),
priority=NORMAL, transferred 13824 bytes
...
2023-05-12 06:37:27,411 [ContainerReplicationThread-2] INFO
replication.GrpcOutputStream (GrpcOutputStream.java:close(111)) - Sent 12800
bytes for container 10
2023-05-12 06:37:27,439 [ContainerReplicationThread-2] INFO
replication.ReplicationSupervisor (ReplicationSupervisor.java:run(353)) -
Successful DONE replicateContainerCommand: containerId=10, replicaIndex=4,
targetNode=dacb7084-9cbe-4e43-829c-fba4bce0d9b7(fv-az173-42.h4pcb1qgckmubfchnqyq5btvmd.gx.internal.cloudapp.net/10.1.0.47),
priority=NORMAL, transferred 12800 bytes
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
