Michael Smith created HDDS-8543:
-----------------------------------
Summary: New key prefix in EC bucket with FSO reports RATIS
replication
Key: HDDS-8543
URL: https://issues.apache.org/jira/browse/HDDS-8543
Project: Apache Ozone
Issue Type: Task
Components: EC
Affects Versions: 1.3.0
Reporter: Michael Smith
This appears to be a regression on the OZONE-parcel branch.
I have a bucket with erasure coding enabled
{code}
$ ozone sh bucket info impala/test-warehouse
{
"metadata" : { },
"volumeName" : "impala",
"name" : "test-warehouse",
"storageType" : "DISK",
"versioning" : false,
"usedBytes" : 16966237353,
"usedNamespace" : 28550,
"creationTime" : "2023-04-11T16:59:38.827Z",
"modificationTime" : "2023-04-11T17:22:22.328Z",
"encryptionKeyName" : "testkey",
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"bucketLayout" : "FILE_SYSTEM_OPTIMIZED",
"owner" : "michael",
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"replicationType" : "EC",
"requiredNodes" : 5
},
"link" : false
}
{code}
I’m going to create a key at foo/bar in that bucket. It doesn’t exist yet.
{code}
$ ozone sh key info impala/test-warehouse/foo
KEY_NOT_FOUND Key:foo not found
{code}
I create the key in that bucket
{code}
$ ozone sh key cp impala/test-warehouse
alltypes_avro_snap/year=2009/month=1/000000_0 foo/bar
23/04/11 11:13:41 WARN
erasurecode.ErasureCodeNative: ISA-L support is not available in your
platform... using builtin-java codec where applicable
23/04/11 11:13:41 WARN impl.MetricsConfig: Cannot locate configuration: tried
hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
23/04/11 11:13:41 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period
at 10 second(s).
23/04/11 11:13:41 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics
system started
{code}
The key reports EC replication
{code}
$ ozone sh key info impala/test-warehouse/foo/bar
{
"volumeName" : "impala",
"bucketName" : "test-warehouse",
"name" : "foo/bar",
"dataSize" : 6937,
"creationTime" : "2023-04-11T18:13:41.081Z",
"modificationTime" : "2023-04-11T18:13:41.660Z",
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"replicationType" : "EC",
"requiredNodes" : 5
},
"metadata" : { },
"ozoneKeyLocations" : [ {
"containerID" : 5,
"localID" : 111677748019216002,
"length" : 6937,
"offset" : 0,
"keyOffset" : 0
} ],
"fileEncryptionInfo" : {
"cipherSuite" : "AES_CTR_NOPADDING",
"iv" : "7y6R8DxVj9q0+1yWbl4ZRw==",
"keyName" : "testkey",
"ezKeyVersionName" : "testkey@0",
"encryptedDataEncryptionKey" : "yn/4CXq5dP4Dnwif/oxf0w==",
"cryptoProtocolVersion" : "ENCRYPTION_ZONES"
}
}
However the prefix reports RATIS
$ ozone sh key info impala/test-warehouse/foo
{
"volumeName" : "impala",
"bucketName" : "test-warehouse",
"name" : "foo/",
"dataSize" : 0,
"creationTime" : "2023-04-11T18:13:41.081Z",
"modificationTime" : "2023-04-11T18:13:41.081Z",
"replicationConfig" : {
"replicationFactor" : "ONE",
"requiredNodes" : 1,
"replicationType" : "RATIS"
},
"metadata" : { },
"ozoneKeyLocations" : [ ]
}
{code}
It used to report EC for the prefix (maybe with LEGACY layout), not RATIS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
