Marton Elek created HDDS-5129:
---------------------------------
Summary: Support new field in datanode container descriptors
Key: HDDS-5129
URL: https://issues.apache.org/jira/browse/HDDS-5129
Project: Apache Ozone
Issue Type: Improvement
Reporter: Marton Elek
Assignee: Marton Elek
Each of the containers have a container descriptor in datanode which contains
metadata (containerId, path to the metadata/chunks, schema version...).
These fields are checksummed in the same file one specific field should contain
the checksum of the other (pre-defined) fields.
The checksum uses the pre-defined list of the properties to calculate the
cheksum (KeyValueContainerData)
{code}
static {
// Initialize YAML fields
KV_YAML_FIELDS = Lists.newArrayList();
KV_YAML_FIELDS.addAll(YAML_FIELDS);
KV_YAML_FIELDS.add(METADATA_PATH);
KV_YAML_FIELDS.add(CHUNKS_PATH);
KV_YAML_FIELDS.add(CONTAINER_DB_TYPE);
KV_YAML_FIELDS.add(SCHEMA_VERSION);
KV_YAML_FIELDS.add(REPLICA_INDEX);
}
{code}
This approach makes it impossible to extend the fields with a backward
compatible way.
If a new field is added, the checksum will not match:
new code will calculate checksum = (old fields + new field), the old code will
calculate checksum = (old fields)
Therefore, if the new code starts to create metadata (with checksum), old code
will fail to read the descriptors because the checksum violation.
It would be better to use ALL the fields (except the checksum itself) for
calculating the checksum. With this approach the old code can read the
descriptor even if there are unknown fields included.
The only question: how can we switch to the new type of calculation.
1. Option1: Using upgrade framework
Using upgrade framework we can start to use any new typo of calculation after
finalize step
2. Option2: Using new field
(Thanks to Uma, for the suggestion).
It is also possible to use a new field to store the new type of checkum.
"checksum" field can be calculated as earlier, but we can calculate
"fullchecksum" property based on all the properties.
This is a fully backward compatible way, but we should calculate both the
checksum until the next major version where we can allow doing incompatible
changes (possible with using upgrade framework).
Personally I would suggest to use option1 as we need an upgrade framework
anyway, it may be better to avoid duplicated information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
