[
https://issues.apache.org/jira/browse/HDDS-17681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-17681:
-----------------------------------
Description:
ReconLayoutVersionManager accepts a stored metadata layout version (MLV)
greater than its software layout version (SLV). Older Recon can consequently
start and serve incorrect derived metadata after a direct downgrade.
Reproduced with Apache release images:
1. Start Ozone 2.0.0 with an FSO bucket containing nested keys; Recon metadata
version is 1.
2. Stop services and back up Recon, then upgrade to 2.2.2 without finalizing OM
or SCM. Recon automatically migrates its metadata to version 5. Let Recon
synchronize a third key written while pre-finalized.
3. Stop services and downgrade to 2.0.0. On a separate copy of the migrated
Recon data, do not restore the backup.
4. Recon logs Current MLV: 5. SLV: 1 and initializes successfully, but
namespace summary reports zero directories/keys and disk usage zero. The bucket
actually has three keys. The cluster API reports a stale count of two.
5. Another key can be written/read through OM. Recon catches up to OM with zero
sequence-number lag, yet its namespace remains empty and its cluster count
remains stale. Offline SQL still reports metadata version 5.
Control: restoring the stopped 2.0.0 Recon backup before downgrade succeeds.
Recon metadata returns to version 1 and resynchronizes all three keys with
correct counts and usage. This is incorrect Recon reporting, not evidence of
lost Ozone keys.
Regression:
TestReconLayoutVersionManager#testInitializationRejectsNewerMetadataLayoutVersion
supplies MLV 3 with SLV 2. It fails because initialization throws no
exception. A matching-version control passes. The constructor has no MLV > SLV
compatibility check.
Expected: reject an unsupported newer MLV before schema repair or upgrade
actions, report both versions and explain that compatible binaries or the
pre-upgrade Recon backup are required. Preserve startup support when MLV equals
or is older than SLV.
This is a concrete product follow-up to HDDS-17625, separate from Recon
intentionally migrating at startup. Documentation is tracked in HDDS-17624.
Branch scope: current master uses ReconVersionManager and already rejects
unknown persisted versions. The missing guard remains in
ReconLayoutVersionManager on the ozone-2.2 branch; the proposed fix targets
that branch. It cannot make unpatched historical 2.0.0 binaries reject migrated
data, so the documented backup/restore requirement still applies.
was:
ReconLayoutVersionManager accepts a stored metadata layout version (MLV)
greater than its software layout version (SLV). Older Recon can consequently
start and serve incorrect derived metadata after a direct downgrade.
Reproduced with Apache release images:
1. Start Ozone 2.0.0 with an FSO bucket containing nested keys; Recon metadata
version is 1.
2. Stop services and back up Recon, then upgrade to 2.2.2 without finalizing OM
or SCM. Recon automatically migrates its metadata to version 5. Let Recon
synchronize a third key written while pre-finalized.
3. Stop services and downgrade to 2.0.0. On a separate copy of the migrated
Recon data, do not restore the backup.
4. Recon logs Current MLV: 5. SLV: 1 and initializes successfully, but
namespace summary reports zero directories/keys and disk usage zero. The bucket
actually has three keys. The cluster API reports a stale count of two.
5. Another key can be written/read through OM. Recon catches up to OM with zero
sequence-number lag, yet its namespace remains empty and its cluster count
remains stale. Offline SQL still reports metadata version 5.
Control: restoring the stopped 2.0.0 Recon backup before downgrade succeeds.
Recon metadata returns to version 1 and resynchronizes all three keys with
correct counts and usage. This is incorrect Recon reporting, not evidence of
lost Ozone keys.
Regression:
TestReconLayoutVersionManager#testInitializationRejectsNewerMetadataLayoutVersion
supplies MLV 3 with SLV 2. It fails because initialization throws no
exception. A matching-version control passes. The constructor has no MLV > SLV
compatibility check.
Expected: reject an unsupported newer MLV before schema repair or upgrade
actions, report both versions and explain that compatible binaries or the
pre-upgrade Recon backup are required. Preserve startup support when MLV equals
or is older than SLV.
This is a concrete product follow-up to HDDS-17625, separate from Recon
intentionally migrating at startup. Documentation is tracked in HDDS-17624.
> Recon must reject metadata layout versions newer than its software supports
> ---------------------------------------------------------------------------
>
> Key: HDDS-17681
> URL: https://issues.apache.org/jira/browse/HDDS-17681
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Recon
> Affects Versions: 2.0.0
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> ReconLayoutVersionManager accepts a stored metadata layout version (MLV)
> greater than its software layout version (SLV). Older Recon can consequently
> start and serve incorrect derived metadata after a direct downgrade.
> Reproduced with Apache release images:
> 1. Start Ozone 2.0.0 with an FSO bucket containing nested keys; Recon
> metadata version is 1.
> 2. Stop services and back up Recon, then upgrade to 2.2.2 without finalizing
> OM or SCM. Recon automatically migrates its metadata to version 5. Let Recon
> synchronize a third key written while pre-finalized.
> 3. Stop services and downgrade to 2.0.0. On a separate copy of the migrated
> Recon data, do not restore the backup.
> 4. Recon logs Current MLV: 5. SLV: 1 and initializes successfully, but
> namespace summary reports zero directories/keys and disk usage zero. The
> bucket actually has three keys. The cluster API reports a stale count of two.
> 5. Another key can be written/read through OM. Recon catches up to OM with
> zero sequence-number lag, yet its namespace remains empty and its cluster
> count remains stale. Offline SQL still reports metadata version 5.
> Control: restoring the stopped 2.0.0 Recon backup before downgrade succeeds.
> Recon metadata returns to version 1 and resynchronizes all three keys with
> correct counts and usage. This is incorrect Recon reporting, not evidence of
> lost Ozone keys.
> Regression:
> TestReconLayoutVersionManager#testInitializationRejectsNewerMetadataLayoutVersion
> supplies MLV 3 with SLV 2. It fails because initialization throws no
> exception. A matching-version control passes. The constructor has no MLV >
> SLV compatibility check.
> Expected: reject an unsupported newer MLV before schema repair or upgrade
> actions, report both versions and explain that compatible binaries or the
> pre-upgrade Recon backup are required. Preserve startup support when MLV
> equals or is older than SLV.
> This is a concrete product follow-up to HDDS-17625, separate from Recon
> intentionally migrating at startup. Documentation is tracked in HDDS-17624.
> Branch scope: current master uses ReconVersionManager and already rejects
> unknown persisted versions. The missing guard remains in
> ReconLayoutVersionManager on the ozone-2.2 branch; the proposed fix targets
> that branch. It cannot make unpatched historical 2.0.0 binaries reject
> migrated data, so the documented backup/restore requirement still applies.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
