vinothchandar commented on a change in pull request #3173:
URL: https://github.com/apache/hudi/pull/3173#discussion_r701503238
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
+ .key("hoodie.index.bucket.num")
+ .defaultValue(-1)
+ .withDocumentation("Only applies if index type is BUCKET_INDEX.
Determine the bucket num of the hudi table, "
+ + "and each partition is divided to N buckets.");
+
+ public static final ConfigProperty<String> INDEX_KEY_FILED_NAME =
ConfigProperty
Review comment:
lets name everything consistently using `BUCKET_INDEX_*` prefix? Also
the typo `FILED` -> `FIELD`
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
+ .key("hoodie.index.bucket.num")
Review comment:
`hoodie.bucket.index.num.buckets`?
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -1372,6 +1372,10 @@ public boolean getGlobalSimpleIndexUpdatePartitionPath()
{
return
getBoolean(HoodieIndexConfig.SIMPLE_INDEX_UPDATE_PARTITION_PATH_ENABLE);
}
+ public int getNumBuckets() {
Review comment:
lets make this more descriptive. `getBucketIndexNumBuckets()`?
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
+ .key("hoodie.index.bucket.num")
+ .defaultValue(-1)
+ .withDocumentation("Only applies if index type is BUCKET_INDEX.
Determine the bucket num of the hudi table, "
Review comment:
Just. `Determine the total number of buckets in each table partition.` ?
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
+ .key("hoodie.index.bucket.num")
+ .defaultValue(-1)
+ .withDocumentation("Only applies if index type is BUCKET_INDEX.
Determine the bucket num of the hudi table, "
+ + "and each partition is divided to N buckets.");
+
+ public static final ConfigProperty<String> INDEX_KEY_FILED_NAME =
ConfigProperty
+ .key("hoodie.index.key.field")
+ .noDefaultValue()
+ .withAlternatives("hoodie.datasource.write.indexkey.field")
+ .withDocumentation("Index key.It is used to index the record and find
its file group");
Review comment:
this is the field we hash by? if so, call this
`BUCKET_INDEX_HASH_FIELD` ? and also fix the property name?
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
Review comment:
`BUCKET_INDEX_NUM_BUCKETS`?
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieIndexConfig.java
##########
@@ -200,6 +203,19 @@
.defaultValue("true")
.withDocumentation("Similar to " +
BLOOM_INDEX_UPDATE_PARTITION_PATH_ENABLE + ", but for simple index.");
+ // ***** Bucket Index Configs *****
+ public static final ConfigProperty<Integer> BUCKET_INDEX_BUCKET_NUM =
ConfigProperty
+ .key("hoodie.index.bucket.num")
+ .defaultValue(-1)
Review comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
