JingsongLi commented on code in PR #131:
URL: https://github.com/apache/flink-table-store/pull/131#discussion_r878813019
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/FileStoreOptions.java:
##########
@@ -84,6 +84,13 @@ public class FileStoreOptions implements Serializable {
"To avoid frequent manifest merges, this parameter
specifies the minimum number "
+ "of ManifestFileMeta to merge.");
+ public static final ConfigOption<MemorySize> TARGET_TASK_SPLIT_SIZE =
+ ConfigOptions.key("target-task-split-size")
Review Comment:
target-source-split-size?
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/FileStoreOptions.java:
##########
@@ -84,6 +84,13 @@ public class FileStoreOptions implements Serializable {
"To avoid frequent manifest merges, this parameter
specifies the minimum number "
+ "of ManifestFileMeta to merge.");
+ public static final ConfigOption<MemorySize> TARGET_TASK_SPLIT_SIZE =
+ ConfigOptions.key("target-task-split-size")
+ .memoryType()
+ .defaultValue(MemorySize.ofMebiBytes(256))
Review Comment:
maybe 128 MB？
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/FileStoreOptions.java:
##########
@@ -84,6 +84,13 @@ public class FileStoreOptions implements Serializable {
"To avoid frequent manifest merges, this parameter
specifies the minimum number "
+ "of ManifestFileMeta to merge.");
+ public static final ConfigOption<MemorySize> TARGET_TASK_SPLIT_SIZE =
+ ConfigOptions.key("target-task-split-size")
+ .memoryType()
+ .defaultValue(MemorySize.ofMebiBytes(256))
+ .withDescription(
+ "Target size of a split task when scanning a
bucket in the append-only table.");
Review Comment:
We can use this for changelog table too.
We can use `IntervalPartition` for files. The files can be split into
multiple sets without intersections, i.e. multiple splits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
