[
https://issues.apache.org/jira/browse/HDDS-17702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17702:
------------------------------
Description:
The xcompat test exercises only the legacy snapshot diff path on the current
cluster, because {{ozone.om.snapshot.diff.optimized.enabled}} defaults to
false. Old clients never read a report produced by the optimized path,
including the {{--ordered}} (dependency ordered) report written by 2.3.0
clients.
Also, the check in {{compatibility/read.robot}} that legacy clients do not see
"Report is dependency" cannot fail: clients before 2.3.0 print the reason of a
snapshot diff response only for FAILED jobs, never for DONE reports.
Proposed change:
* Enable the optimized snapshot diff in xcompat {{docker-config}}, together
with {{ozone.om.snapshot.force.full.diff=true}}, so OM does not fall back to
the legacy diff when the {{ozone_rocksdb_tools}} native library is not
available. Old OMs ignore both keys.
* Let legacy clients (before 2.2.0) read the {{--ordered}} diff of
{{snapshot-1}} to {{snapshot-3}} written by 2.3.0 clients.
* Check that 2.3.0 clients on a 2.3.0 cluster get "Report is dependency
ordered." for that diff, so the test fails if xcompat silently falls back to
the legacy diff.
* Remove the legacy "Report is dependency" check.
was:
The xcompat test checks that a legacy client (older than 2.2.0) does not see
the dependency metadata of a snapshot diff report:
{code}
${output} = Wait For Snapshot diff ${FSO_BUCKET} ${SNAPSHOT1}
${SNAPSHOT2} ${SNAPSHOT_KEY1}
# Legacy Snapshot Diff Report Omits Dependency Metadata
Should Not Contain ${output} Report is dependency
{code}
This assertion cannot fail currently:
* The "Report is dependency ordered/unordered." reason is set only by the
optimized snapshot diff path, which is disabled by default
({{ozone.om.snapshot.diff.optimized.enabled=false}}) and not enabled in xcompat
{{docker-config}}.
* Enabling it alone is not enough. Without the {{ozone_rocksdb_tools}} native
library, OM logs {{Falling back to legacy diff}} unless full diff is forced.
* The legacy client only reads the unordered {{snapshot-1}} to {{snapshot-2}}
diff, never the {{--ordered}} {{snapshot-1}} to {{snapshot-3}} diff written by
2.3.0 clients.
Proposed change:
# Enable the optimized snapshot diff in {{compose/xcompat/docker-config}} (old
OMs ignore the keys):
{code}
OZONE-SITE.XML_ozone.om.snapshot.diff.optimized.enabled=true
OZONE-SITE.XML_ozone.om.snapshot.force.full.diff=true
{code}
# Let the legacy client read the {{--ordered}} diff in
{{compatibility/read.robot}}:
{code}
Legacy Client Can Read --ordered Snapshot Diff
Pass Execution If '${CLUSTER_VERSION}' < '${SNAPSHOT_VERSION}'
Cluster does not support snapshots
Pass Execution If '${CLIENT_VERSION}' < '${SNAPSHOT_VERSION}' Client
does not support snapshots
Pass Execution If '${CLIENT_VERSION}' >=
'${SNAPSHOT_DIFF_SUBMIT_VERSION}' Tested elsewhere with --get-report
Pass Execution If '${DATA_VERSION}' < '${SNAPSHOT_DIFF_ORDERED_VERSION}'
Skipped write test case
${output} = Wait For Snapshot diff ${FSO_BUCKET} ${SNAPSHOT1}
${SNAPSHOT3} ${SNAPSHOT_KEY1} ${SNAPSHOT_KEY2}
Should Not Contain ${output} Report is dependency
{code}
Tried locally with xcompat (old clients 1.4.1 and 2.1.2 on the current cluster,
and old clusters 1.4.1 and 2.1.2): all tests passed. With both keys set, the
2.3.0 client's {{--get-report}} shows "Report is dependency ordered." for
{{snapshot-1}} to {{snapshot-3}} and "Report is dependency unordered." for
{{snapshot-1}} to {{snapshot-2}}, while 1.4.1 and 2.1.2 clients reading the
same diffs do not.
Summary: Run optimized snapshot diff in xcompat test (was: Exercise
ordered snapshot diff metadata with legacy clients in xcompat test)
> Run optimized snapshot diff in xcompat test
> -------------------------------------------
>
> Key: HDDS-17702
> URL: https://issues.apache.org/jira/browse/HDDS-17702
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Snapshot, test
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> The xcompat test exercises only the legacy snapshot diff path on the current
> cluster, because {{ozone.om.snapshot.diff.optimized.enabled}} defaults to
> false. Old clients never read a report produced by the optimized path,
> including the {{--ordered}} (dependency ordered) report written by 2.3.0
> clients.
> Also, the check in {{compatibility/read.robot}} that legacy clients do not
> see "Report is dependency" cannot fail: clients before 2.3.0 print the reason
> of a snapshot diff response only for FAILED jobs, never for DONE reports.
> Proposed change:
> * Enable the optimized snapshot diff in xcompat {{docker-config}}, together
> with {{ozone.om.snapshot.force.full.diff=true}}, so OM does not fall back to
> the legacy diff when the {{ozone_rocksdb_tools}} native library is not
> available. Old OMs ignore both keys.
> * Let legacy clients (before 2.2.0) read the {{--ordered}} diff of
> {{snapshot-1}} to {{snapshot-3}} written by 2.3.0 clients.
> * Check that 2.3.0 clients on a 2.3.0 cluster get "Report is dependency
> ordered." for that diff, so the test fails if xcompat silently falls back to
> the legacy diff.
> * Remove the legacy "Report is dependency" check.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
