[
https://issues.apache.org/jira/browse/HDDS-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-14939:
--------------------------
Description:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reached the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths using iceberg's
RewriteTablePathUtil from sourcePrefix -> targetPrefix and then add the
rewritten metadata file to the staging location.
We should also update copy plan with *staging -> final path mappings* for each
version file rewritten.
was:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reached the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths (using iceberg's
RewriteTablePathUtil) sourcePrefix -> targetPrefix and then add the rewritten
metadata file in the staging location.
We should also update copy plan with *staging -> final path mappings* for each
version file rewritten.
> Implement version file rewrite logic for path migration across metadata
> history
> -------------------------------------------------------------------------------
>
> Key: HDDS-14939
> URL: https://issues.apache.org/jira/browse/HDDS-14939
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> This change implements the logic to rewrite table metadata version files
> across the metadata history, enabling path migration from a source prefix to
> a target prefix.
> Traversal should start from the provided endMetadata (current or specified
> version) and iterate backwards through previous files until reached the
> startVersion provided if any(if not then till beginning). We also need to
> collect all the snapshots from each traversed historical metadata version.
> For each metadata version we have to rewrite file paths using iceberg's
> RewriteTablePathUtil from sourcePrefix -> targetPrefix and then add the
> rewritten metadata file to the staging location.
> We should also update copy plan with *staging -> final path mappings* for
> each version file rewritten.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
