vaultah commented on code in PR #16910:
URL: https://github.com/apache/iceberg/pull/16910#discussion_r3509271140
##########
core/src/main/java/org/apache/iceberg/RewriteTablePathUtil.java:
##########
@@ -301,6 +334,8 @@ public static RewriteResult<ManifestFile>
rewriteManifestList(
for (ManifestFile file : manifestFiles) {
ManifestFile newFile = file.copy();
((StructLike) newFile).set(0, newPath(newFile.path(), sourcePrefix,
targetPrefix));
+ ((StructLike) newFile)
+ .set(1, rewrittenManifestLengths.getOrDefault(file.path(),
file.length()));
Review Comment:
This seems like an intentional decision, based on
> Manifests not rewritten in an incremental run keep their original length.
but it does still mean that this procedure will silently produce incorrect
manifest lists in incremental mode, if rewritten manifest lists referenced
manifests that were already rewritten in a previous run.
If the rewritten manifest size changed after the prefix rewrite, which it
typically does, the new manifest list is still inconsistent with the file it
references.
This is a known incorrect behaviour described in
https://github.com/apache/iceberg/pull/13720#issuecomment-3235133985 and
acknowledged there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
