vaultah commented on code in PR #13720:
URL: https://github.com/apache/iceberg/pull/13720#discussion_r2286621701
##########
spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewriteTablePathsAction.java:
##########
@@ -1209,6 +1213,191 @@ public void testNestedDirectoryStructurePreservation()
throws Exception {
assertThat(targetPath2).startsWith(targetTableLocation());
}
+ @Test
+ public void testFullRewriteUpdatesAllManifestLengthsInManifestList(
+ @TempDir Path rootTargetLocation) throws Exception {
+ String location = newTableLocation();
+ Table sourceTable = createTableWithSnapshots(location, 10);
+
+ addAnyPositionDelete(
+ sourceTable, removePrefix(sourceTable.location() +
"/data/deeply/nested/deletes.parquet"));
+
+ Map<String, Long> manifestSizesBeforeRewrite =
+ sourceTable.currentSnapshot().allManifests(sourceTable.io()).stream()
+ .collect(Collectors.toMap(m -> fileName(m.path()), m ->
m.length()));
+
+ // Rewrite table metadata to a location that's much longer than the
original in order
+ // to make manifests larger
+ String targetLocation = toAbsolute(rootTargetLocation) +
generateLongNestedPath(25);
Review Comment:
Because this extra nesting is potentially inefficient and only needed for
these three tests, I wanted to avoid applying it globally, and having a new
location unrelated to `targetTableDir` would arguably be safer (e.g., if
`targetTableDir` ends up being reused in the same test).
I added
```
protected String longTargetTableLocation() throws IOException {
return toAbsolute(targetTableDir) + generateLongNestedPath(5);
}
```
to highlight the difference and purpose. Let me know if you want me to merge
it into `targetTableLocation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
