oglego commented on issue #375:
URL: https://github.com/apache/hudi-rs/issues/375#issuecomment-6686427875
Hey @yihua I am interested in trying to work on this but wanted to check on
the approach first!
For this I was thinking of the following:
- In `OptionResolver` found in crates/core/src/table/builder.rs, before the
persisted `hoodie.properties` values are merged over the user options, each
table config is compared. If the user supplied a different value, the builder
returns a `ConfigError` naming the key, the user value and the table value,
instead of silently overwriting.
- Values are compared after parsing, so equivalent spellings aren't flagged
- The classification lives in an exhaustive
`HudiTableConfig::is_immutable()`, so any new config would need to be added
there as well.
For the list of table configs to check would it just be the values found in
crates/core/src/config/table.rs:
```rust
impl AsRef<str> for HudiTableConfig {
fn as_ref(&self) -> &str {
match self {
Self::BaseFileFormat => "hoodie.table.base.file.format",
Self::BasePath => "hoodie.base.path",
Self::Checksum => "hoodie.table.checksum",
Self::CreateSchema => "hoodie.table.create.schema",
Self::DatabaseName => "hoodie.database.name",
Self::DropsPartitionFields =>
"hoodie.datasource.write.drop.partition.columns",
Self::IsHiveStylePartitioning =>
"hoodie.datasource.write.hive_style_partitioning",
Self::IsPartitionPathUrlencoded =>
"hoodie.datasource.write.partitionpath.urlencode",
Self::KeyGeneratorClass => "hoodie.table.keygenerator.class",
Self::KeyGeneratorType => "hoodie.table.keygenerator.type",
Self::PartitionFields => "hoodie.table.partition.fields",
Self::OrderingFields => "hoodie.table.ordering.fields",
Self::PopulatesMetaFields => "hoodie.populate.meta.fields",
Self::RecordKeyFields => "hoodie.table.recordkey.fields",
Self::RecordMergeStrategy =>
"hoodie.table.record.merge.strategy",
Self::TableName => "hoodie.table.name",
Self::TableType => "hoodie.table.type",
Self::TableVersion => "hoodie.table.version",
Self::TimelineLayoutVersion => "hoodie.timeline.layout.version",
Self::TimelineTimezone => "hoodie.table.timeline.timezone",
Self::ArchiveLogFolder => "hoodie.archivelog.folder",
Self::TimelinePath => "hoodie.timeline.path",
Self::TimelineHistoryPath => "hoodie.timeline.history.path",
Self::MetadataTableEnabled => "hoodie.metadata.enable",
Self::MetadataTablePartitions =>
"hoodie.table.metadata.partitions",
}
}
}
```
Thanks in advance for any help you can provide on this I really appreciate
it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
