stevenzwu commented on code in PR #16730:
URL: https://github.com/apache/iceberg/pull/16730#discussion_r3406184878
##########
core/src/main/java/org/apache/iceberg/ManifestFiles.java:
##########
@@ -602,4 +612,54 @@ static long cacheMaxContentLength(FileIO io) {
CatalogProperties.IO_MANIFEST_CACHE_MAX_CONTENT_LENGTH,
CatalogProperties.IO_MANIFEST_CACHE_MAX_CONTENT_LENGTH_DEFAULT);
}
+
+ /**
+ * Writes the given files into manifests in parallel, splitting them into
the given number of
+ * groups and submitting each group to the provided executor.
+ *
+ * @param files content files to write
+ * @param parallelism number of parallel groups; the caller decides this
based on its own
+ * parallelism and minimum-group-size policy
Review Comment:
nit: the Javadoc says "splitting them into the given number of groups," but
`divide` rounds the group size up, so the produced count can be less than the
requested value (e.g. files=10, parallelism=7 → groupSize=2 → 5 groups). Worth
wording it as "target number of groups" or "upper bound on parallelism."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
