[
https://issues.apache.org/jira/browse/HDDS-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-14939:
--------------------------
Description:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reaching the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths sourcePrefix ->
targetPrefix and generate a new metadata file in the staging location (using
iceberg's RewriteTablePathUtil).
Update copy plan with *staging -> final path mappings* for each version file
rewritten.
was:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reaching the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths sourcePrefix ->
targetPrefix and generate a new metadata file in the staging location.
Update copy plan with *staging -> final path mappings* for each version file
rewritten.
> Implement version file rewrite logic for path migration across metadata
> history
> -------------------------------------------------------------------------------
>
> Key: HDDS-14939
> URL: https://issues.apache.org/jira/browse/HDDS-14939
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> This change implements the logic to rewrite table metadata version files
> across the metadata history, enabling path migration from a source prefix to
> a target prefix.
> Traversal should start from the provided endMetadata (current or specified
> version) and iterate backwards through previous files until reaching the
> startVersion provided if any(if not then till beginning). We also need to
> collect all the snapshots from each traversed historical metadata version.
> For each metadata version we have to rewrite file paths sourcePrefix ->
> targetPrefix and generate a new metadata file in the staging location (using
> iceberg's RewriteTablePathUtil).
> Update copy plan with *staging -> final path mappings* for each version file
> rewritten.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
