yihua commented on code in PR #14180:
URL: https://github.com/apache/hudi/pull/14180#discussion_r2482161895
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndex.java:
##########
@@ -191,19 +191,28 @@ public enum IndexType {
@EnumFieldDescription("Internal Config for indexing based on Flink state.")
FLINK_STATE,
+ @Deprecated
@EnumFieldDescription("Index which saves the record key to location
mappings in the "
+ "HUDI Metadata Table. Record index is a global index, enforcing key
uniqueness across all "
+ "partitions in the table. Supports sharding to achieve very high
scale. For a table with "
- + "keys that are only unique inside each partition, use
`PARTITIONED_RECORD_INDEX` instead.")
+ + "keys that are only unique inside each partition, use
`PARTITIONED_RECORD_INDEX` instead. "
Review Comment:
```suggestion
+ "keys that are only unique inside each partition, use
`RECORD_LEVEL_INDEX` instead. "
```
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestGlobalRecordLevelIndexWithSQL.scala:
##########
@@ -52,7 +52,7 @@ class TestRecordLevelIndexWithSQL extends
RecordLevelIndexTestBase {
val hudiOpts = commonOpts ++ Map(
DataSourceWriteOptions.TABLE_TYPE.key ->
HoodieTableType.MERGE_ON_READ.name(),
"hoodie.metadata.index.column.stats.enable" -> "false",
- HoodieMetadataConfig.RECORD_INDEX_ENABLE_PROP.key -> "false")
+ HoodieMetadataConfig.GLOBAL_RECORD_LEVEL_INDEX_ENABLE_PROP.key ->
"false")
Review Comment:
Could we also add a test with the old config naming to validate it works and
prevents any breaking change in the future?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndex.java:
##########
@@ -191,19 +191,28 @@ public enum IndexType {
@EnumFieldDescription("Internal Config for indexing based on Flink state.")
FLINK_STATE,
+ @Deprecated
@EnumFieldDescription("Index which saves the record key to location
mappings in the "
+ "HUDI Metadata Table. Record index is a global index, enforcing key
uniqueness across all "
+ "partitions in the table. Supports sharding to achieve very high
scale. For a table with "
- + "keys that are only unique inside each partition, use
`PARTITIONED_RECORD_INDEX` instead.")
+ + "keys that are only unique inside each partition, use
`PARTITIONED_RECORD_INDEX` instead. "
+ + "This enum is deprecated. Use GLOBAL_RECORD_LEVEL_INDEX or
RECORD_LEVEL_INDEX "
+ + "based on global uniqueness constraints for indexing.")
Review Comment:
```suggestion
+ "This enum is deprecated. Use GLOBAL_RECORD_LEVEL_INDEX for global
uniqueness of record keys or RECORD_LEVEL_INDEX for partition-level uniqueness
of record keys.")
```
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -274,15 +274,16 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.withDocumentation("When there is a pending instant in data table, this
config limits the allowed number of deltacommits in metadata table to "
+ "prevent the metadata table's timeline from growing unboundedly as
compaction won't be triggered due to the pending data table instant.");
- public static final ConfigProperty<Boolean> RECORD_INDEX_ENABLE_PROP =
ConfigProperty
- .key(METADATA_PREFIX + ".record.index.enable")
+ public static final ConfigProperty<Boolean>
GLOBAL_RECORD_LEVEL_INDEX_ENABLE_PROP = ConfigProperty
+ .key(METADATA_PREFIX + ".global.record.level.index.enable")
Review Comment:
Have you audited the code that when fetching this config the alternative
config naming is always considered (i.e., no direct usage of
`GLOBAL_RECORD_LEVEL_INDEX_ENABLE_PROP.key()`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
