[
https://issues.apache.org/jira/browse/HDDS-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-14939:
--------------------------
Description:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reached the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths sourcePrefix ->
targetPrefix (using iceberg's RewriteTablePathUtil) and generate a new metadata
file in the staging location.
We should also update copy plan with *staging -> final path mappings* for each
version file rewritten.
was:
This change implements the logic to rewrite table metadata version files across
the metadata history, enabling path migration from a source prefix to a target
prefix.
Traversal should start from the provided endMetadata (current or specified
version) and iterate backwards through previous files until reached the
startVersion provided if any(if not then till beginning). We also need to
collect all the snapshots from each traversed historical metadata version.
For each metadata version we have to rewrite file paths sourcePrefix ->
targetPrefix and generate a new metadata file in the staging location which can
be done using iceberg's RewriteTablePathUtil methods.
We should also update copy plan with *staging -> final path mappings* for each
version file rewritten.
> Implement version file rewrite logic for path migration across metadata
> history
> -------------------------------------------------------------------------------
>
> Key: HDDS-14939
> URL: https://issues.apache.org/jira/browse/HDDS-14939
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> This change implements the logic to rewrite table metadata version files
> across the metadata history, enabling path migration from a source prefix to
> a target prefix.
> Traversal should start from the provided endMetadata (current or specified
> version) and iterate backwards through previous files until reached the
> startVersion provided if any(if not then till beginning). We also need to
> collect all the snapshots from each traversed historical metadata version.
> For each metadata version we have to rewrite file paths sourcePrefix ->
> targetPrefix (using iceberg's RewriteTablePathUtil) and generate a new
> metadata file in the staging location.
> We should also update copy plan with *staging -> final path mappings* for
> each version file rewritten.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
