nsivabalan commented on code in PR #8758:
URL: https://github.com/apache/hudi/pull/8758#discussion_r1225582669
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -234,6 +235,55 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.withDocumentation("When there is a pending instant in data table, this
config limits the allowed number of deltacommits in metadata table to "
+ "prevent the metadata table's timeline from growing unboundedly as
compaction won't be triggered due to the pending data table instant.");
+ public static final ConfigProperty<Boolean> RECORD_INDEX_CREATE_PROP =
ConfigProperty
+ .key(METADATA_PREFIX + ".record.index.create")
+ .defaultValue(false)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Create the HUDI Record Index within the Metadata
Table");
+
+ public static final ConfigProperty<Integer>
RECORD_INDEX_MIN_FILE_GROUP_COUNT_PROP = ConfigProperty
+ .key(METADATA_PREFIX + ".record.index.min.filegroup.count")
+ .defaultValue(10)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Minimum number of file groups to use for Record
Index.");
+
+ public static final ConfigProperty<Integer>
RECORD_INDEX_MAX_FILE_GROUP_COUNT_PROP = ConfigProperty
+ .key(METADATA_PREFIX + ".record.index.max.filegroup.count")
+ .defaultValue(1000)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Maximum number of file groups to use for Record
Index.");
+
+ public static final ConfigProperty<Integer>
RECORD_INDEX_MAX_FILE_GROUP_SIZE_BYTES_PROP = ConfigProperty
+ .key(METADATA_PREFIX + ".record.index.max.filegroup.size")
+ .defaultValue(1024 * 1024 * 1024)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Maximum size in bytes of a single file group. Large
file group takes longer to compact.");
+
+ public static final ConfigProperty<Float> RECORD_INDEX_GROWTH_FACTOR_PROP =
ConfigProperty
+ .key(METADATA_PREFIX + ".record.index.growth.factor")
+ .defaultValue(2.0f)
+ .sinceVersion("0.14.0")
+ .withDocumentation("The current number of records are multiplied by this
number when estimating the number of "
+ + "file groups to create automatically. This helps account for
growth in the number of records in the dataset.");
+
+ public static final ConfigProperty<Long> MAX_READER_MEMORY_PROP =
ConfigProperty
+ .key(METADATA_PREFIX + ".max.reader.memory")
+ .defaultValue(1024 * 1024 * 1024L)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Max memory to use for the reader to read from
metadata");
+
+ public static final ConfigProperty<Integer> MAX_READER_BUFFER_SIZE_PROP =
ConfigProperty
+ .key(METADATA_PREFIX + ".max.reader.buffer.size")
+ .defaultValue(10 * 1024 * 1024)
+ .sinceVersion("0.14.0")
+ .withDocumentation("Max memory to use for the reader buffer while
merging log blocks");
+
+ public static final ConfigProperty<String> SPILLABLE_MAP_DIR_PROP =
ConfigProperty
+ .key(METADATA_PREFIX + ".spillable.dir")
+ .defaultValue(FileSystemViewStorageConfig.SPILLABLE_DIR.defaultValue())
Review Comment:
lets fix the since version and the docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
