Stephen O'Donnell created HDDS-6434:
---------------------------------------
Summary: Overwriting an EC key with a Ratis key fails
Key: HDDS-6434
URL: https://issues.apache.org/jira/browse/HDDS-6434
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Stephen O'Donnell
Creating a key with Replication Config EC, and then overwiting the key with a
Ratis key fails with the error:
{code}
bash-4.2$ ozone sh key put -t EC -r rs-3-2-1024k /vol1/bucket1/key11 /etc/hosts
bash-4.2$ ozone sh key put -t RATIS -r THREE /vol1/bucket1/key11 /etc/hosts
Required instance of class org.apache.hadoop.hdds.client.ECReplicationConfig
but object.getClass() is class
org.apache.hadoop.hdds.client.RatisReplicationConfig
{code}
If you repeat this test with a RATIS/ONE key and overwrite with RATIS/THREE,
the key info continues to have a replication config of ONE, but the first
version goes into a ONE container. Then the next version goes into a THREE
container.
{code}
bash-4.2$ ozone sh key put -t RATIS -r ONE /vol1/bucket1/key40 3mb
bash-4.2$ ozone sh key put -t RATIS -r THREE /vol1/bucket1/key40 3mb
bash-4.2$ ozone sh key info /vol1/bucket1/key40
{
"volumeName" : "vol1",
"bucketName" : "bucket1",
"name" : "key40",
"dataSize" : 3145728,
"creationTime" : "2022-03-09T21:53:29.492Z",
"modificationTime" : "2022-03-09T21:53:39.109Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"ozoneKeyLocations" : [ {
"containerID" : 20,
"localID" : 109611004723200035,
"length" : 3145728,
"offset" : 0,
"keyOffset" : 0
} ],
"metadata" : { }
}
{code}
Current container is 20:
{code}
bash-4.2$ ozone admin container info 20
Container id: 20
Pipeline id: 35b541ec-b50f-49ac-ae93-c8bdac3dd659
Container State: OPEN
Datanodes: [7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09/ozone_datanode_4.ozone_default,
71c5c10f-6d38-428e-85aa-7202028e3c1c/ozone_datanode_2.ozone_default,
25c43ccd-b027-43f0-b3cc-88e415390e86/ozone_datanode_5.ozone_default]
Replicas: [State: OPEN; ReplicaIndex: 0; Origin:
25c43ccd-b027-43f0-b3cc-88e415390e86; Location:
25c43ccd-b027-43f0-b3cc-88e415390e86/ozone_datanode_5.ozone_default,
State: OPEN; ReplicaIndex: 0; Origin: 71c5c10f-6d38-428e-85aa-7202028e3c1c;
Location: 71c5c10f-6d38-428e-85aa-7202028e3c1c/ozone_datanode_2.ozone_default,
State: OPEN; ReplicaIndex: 0; Origin: 7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09;
Location: 7465dd7d-6ee7-4c5b-b9f0-5d63b3bdbf09/ozone_datanode_4.ozone_default]
{code}
Write an EC key - note container changes to 22, but repConfig is still
RATIS/ONE in the key info, and container 22 is RATIS/ONE
{code}
ozone sh key put -t EC -r rs-3-2-1024k /vol1/bucket1/key40 3mb
bash-4.2$ ozone sh key info /vol1/bucket1/key40
{
"volumeName" : "vol1",
"bucketName" : "bucket1",
"name" : "key40",
"dataSize" : 3145728,
"creationTime" : "2022-03-09T21:53:29.492Z",
"modificationTime" : "2022-03-09T21:57:10.909Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"ozoneKeyLocations" : [ {
"containerID" : 22,
"localID" : 109611004723200037,
"length" : 3145728,
"offset" : 0,
"keyOffset" : 0
} ],
"metadata" : { }
}
{code}
Even more strangely, there is a container 21 created, which is EC and has 3MB
written to the first replica, and hence not correctly stripped across the
blocks. Its also not stored in the container:
{code}
bash-4.2$ ozone admin container info 21 --json
...
"placeOfBirth" : "25c43ccd-b027-43f0-b3cc-88e415390e86",
"sequenceId" : 0,
"keyCount" : 0,
"bytesUsed" : 3145728,
"replicaIndex" : 1
{code}
Its like it writes a RATIS/ONE and also writes the data to an EC container too
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
