[
https://issues.apache.org/jira/browse/HDDS-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-14939:
--------------------------
Description:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous metadata files until reached
the startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata file we can take help of iceberg's RewriteTablePathUtil to
rewrite absolute file paths in it from sourcePrefix -> targetPrefix and then
we have to add the rewritten metadata file to the staging location.
We should also update copy plan with staging -> final path mappings for each
version file rewritten.
was:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous metadata files until reached
the startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we can take help of iceberg's RewriteTablePathUtil to
rewrite file paths from sourcePrefix -> targetPrefix and then we have to add
the rewritten metadata file to the staging location.
We should also update copy plan with staging -> final path mappings for each
version file rewritten.
> Implement version file rewrite logic for path migration across metadata
> history
> -------------------------------------------------------------------------------
>
> Key: HDDS-14939
> URL: https://issues.apache.org/jira/browse/HDDS-14939
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> This change implements the logic to rewrite table metadata version files
> across the metadata history, enabling path migration from a source prefix to
> a target prefix.
> Traversal should start from the provided endMetadata (current or specified
> version) and iterate backwards through previous metadata files until reached
> the startVersion provided if any(if not then till beginning). We also need to
> collect all the snapshots from each traversed historical metadata version.
> For each metadata file we can take help of iceberg's RewriteTablePathUtil to
> rewrite absolute file paths in it from sourcePrefix -> targetPrefix and then
> we have to add the rewritten metadata file to the staging location.
> We should also update copy plan with staging -> final path mappings for each
> version file rewritten.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
