wForget commented on issue #6669: URL: https://github.com/apache/iceberg/issues/6669#issuecomment-1720880384
We also encountered the same problem, I think it is because `defaultMinFileSize` = `targetFileSize` * `MIN_FILE_SIZE_DEFAULT_RATIO (0.75)`, and we cannot pack `0.5~0.75 * targetFileSize` files into a bin. So we can set `minFileSize` = `0.5 * targetFileSize` to avoid this problem, but it doesn't seem like a good way. @RussellSpitzer Can we split the task smaller before BinPacking? Currently, it always prefers to split tasks to targetSplitSize or parquetBlockSize and then do BinPacking. I tried the following and it seems to have some effect:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
