wombatu-kun commented on code in PR #16910:
URL: https://github.com/apache/iceberg/pull/16910#discussion_r3452736662
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java:
##########
@@ -321,6 +321,24 @@ private Result rebuildMetadata() {
.collect(Collectors.toCollection(DeleteFileSet::create));
rewritePositionDeletes(deleteFiles);
+ // rebuild manifest-list files last, stamping manifest_length with the
rewritten manifest sizes
+ Set<RewriteResult<ManifestFile>> manifestListResults =
Sets.newConcurrentHashSet();
+ Tasks.foreach(validSnapshots)
Review Comment:
The threading here is unchanged by this PR - both the new discovery loop and
the manifest-list loop reuse the action's existing `executorService`, the same
as the original single loop did. Making that executor parallel by default is
orthogonal to the `manifest_length` fix, so #16752 is the right place to
address it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
