rdblue commented on a change in pull request #3073:
URL: https://github.com/apache/iceberg/pull/3073#discussion_r706488919
##########
File path: core/src/main/java/org/apache/iceberg/actions/BinPackStrategy.java
##########
@@ -211,7 +220,13 @@ protected long writeMaxFileSize() {
}
private long sizeOfInputFiles(List<FileScanTask> group) {
- return group.stream().mapToLong(FileScanTask::length).sum();
+ return group.stream().mapToLong(this::sizeOfInputFile).sum();
+ }
+
+ private long sizeOfInputFile(FileScanTask file) {
Review comment:
@RussellSpitzer, what do you think about this change to
`BinPackStrategy`?
I'm not sure that we want to make this change to the file rewrite. While
delete files are relevant to how this will run, the bin packing here is not to
balance work in this job, it is to pack data into files so that work is
balanced in future jobs.
I think it is probably a mistake to change this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
