Akash3121 commented on code in PR #938: URL: https://github.com/apache/incubator-xtable/pull/938#discussion_r4052414475
########## website/docs/features-and-limitations.md: ########## @@ -42,3 +42,4 @@ HMS and AWS Glue are the two catalogs supported right now, support for other cat ### Delta - When using Delta as the source for an Iceberg target, you may require field IDs set in the parquet schema. To enable that, follow the instructions for enabling column mapping [here](https://docs.delta.io/latest/delta-column-mapping.html). - When Delta is the source, Generated Columns are not synced to the target schema. For tables that are partitioned on Generated Columns, there is limited support. For example, we support date functions like transforming a timestamp to `yyyy-MM-dd` format. Please file a GitHub issue or pull-request for any cases that you think should be supported. +- Delta deletion vectors are not supported by conversion targets. Sync fails by default when a Delta source contains an active deletion vector. The bundled utility can set `allowUnsupportedDeletionVectors: "true"` for a dataset to log a warning and continue, but the target may then expose rows that were deleted from the source. Before disabling the override, rebuild affected targets with a full sync so previously ignored deletes are not retained. Review Comment: Updated ########## README.md: ########## @@ -81,6 +81,7 @@ datasets: - `tableBasePath` is the basePath of the table - `tableDataPath` is an optional field specifying the path to the data files. If not specified, the tableBasePath will be used. For Iceberg source tables, you will need to specify the `/data` path. - `namespace` is an optional field specifying the namespace of the table and will be used when syncing to a catalog. +- `allowUnsupportedDeletionVectors` is an optional, temporary Delta-source safety override. Delta deletion vectors are rejected by default because XTable cannot translate them to target table metadata. Setting this field to `"true"` logs a warning and continues while ignoring the deletion vectors, which may expose rows that were deleted from the source. Before disabling the override, rebuild affected targets with a full sync so previously ignored deletes are not retained. Review Comment: updated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
