[
https://issues.apache.org/jira/browse/HDDS-14945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sreeja updated HDDS-14945:
--------------------------
Description:
Position delete files of an Iceberg table contain the absolute path to the data
files which contains the rows deleted.
We need to implement a logic to rewrite these position delete files as part of
path migration. For each selected position delete file, we can take help of
Iceberg's RewriteTablePathUtil to change sourcePrefix to targetPrefix for each
data file absolute path mentioned in it, and add the rewritten position delete
file to a staging location.
Introduce OzonePositionDeleteReaderWriter, which implements Iceberg’s
PositionDeleteReaderWriter, to perform format-specific reads and writes for
Avro, Parquet, and ORC.
was:
Implement logic to rewrite position delete files as part of path migration. For
each selected position delete file, we can use Iceberg's RewriteTablePathUtil
to do path change from sourcePrefix to targetPrefix, and add the rewritten file
to a staging location.
Introduce OzonePositionDeleteReaderWriter, which implements Iceberg’s
PositionDeleteReaderWriter, to perform format-specific reads and writes for
Avro, Parquet, and ORC.
> Implement position delete file rewrite for path migration
> ---------------------------------------------------------
>
> Key: HDDS-14945
> URL: https://issues.apache.org/jira/browse/HDDS-14945
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Sreeja
> Assignee: Sreeja
> Priority: Major
>
> Position delete files of an Iceberg table contain the absolute path to the
> data files which contains the rows deleted.
> We need to implement a logic to rewrite these position delete files as part
> of path migration. For each selected position delete file, we can take help
> of Iceberg's RewriteTablePathUtil to change sourcePrefix to targetPrefix for
> each data file absolute path mentioned in it, and add the rewritten position
> delete file to a staging location.
> Introduce OzonePositionDeleteReaderWriter, which implements Iceberg’s
> PositionDeleteReaderWriter, to perform format-specific reads and writes for
> Avro, Parquet, and ORC.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
