JingsongLi commented on a change in pull request #24:
URL: https://github.com/apache/flink-table-store/pull/24#discussion_r812783109
##########
File path:
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/manifest/ManifestFileMeta.java
##########
@@ -135,19 +135,22 @@ public String toString() {
List<ManifestFileMeta> metas,
List<ManifestEntry> entries,
ManifestFile manifestFile,
- long suggestedMetaSize) {
+ long suggestedMetaSize,
+ int suggestedMinMetaCount) {
List<ManifestFileMeta> result = new ArrayList<>();
// these are the newly created manifest files, clean them up if
exception occurs
List<ManifestFileMeta> newMetas = new ArrayList<>();
List<ManifestFileMeta> candidate = new ArrayList<>();
long totalSize = 0;
+ int metaCount = 0;
try {
// merge existing manifests first
for (ManifestFileMeta manifest : metas) {
totalSize += manifest.fileSize;
+ metaCount += 1;
candidate.add(manifest);
- if (totalSize >= suggestedMetaSize) {
+ if (totalSize >= suggestedMetaSize && metaCount >=
suggestedMinMetaCount) {
Review comment:
We should choose not to merge when the totalSize is not satisfied and
the min count is not satisfied.
So we need check before below `mergeIntoOneFile`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
