alexeykudinkin commented on a change in pull request #5013:
URL: https://github.com/apache/hudi/pull/5013#discussion_r823993627
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -318,11 +337,13 @@ private static void modify(FileSystem fs, Path
metadataFolder, Properties modify
modifyFn.accept(props, modifyProps);
if (props.containsKey(TABLE_CHECKSUM.key()) &&
validateChecksum(props)) {
Review comment:
Let's extract this whole conditional as a `persist` method (for ex),
since it's pretty much replicated in 3 places
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -221,9 +220,12 @@ public HoodieTableConfig(FileSystem fs, String metaPath,
String payloadClassName
// FIXME(vc): wonder if this can be removed. Need to look into history.
try (FSDataOutputStream outputStream = fs.create(propertyPath)) {
if (!isValidChecksum()) {
- setValue(TABLE_CHECKSUM, String.valueOf(generateChecksum(props)));
+ Properties propsWithChecksum =
getOrderedPropertiesWithTableChecksum(props);
+ propsWithChecksum.store(outputStream, "Properties saved on " + new
Date(System.currentTimeMillis()));
Review comment:
Let's use `Instant.now` instead (Date is a legacy API)
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -233,6 +235,23 @@ public HoodieTableConfig(FileSystem fs, String metaPath,
String payloadClassName
"hoodie.properties file seems invalid. Please check for left over
`.updated` files if any, manually copy it to hoodie.properties and retry");
}
+ private static Properties getOrderedPropertiesWithTableChecksum(Properties
props) {
+ LinkedHashMap<String, String> propsMap = getOrderedPropertiesMap(props);
+ propsMap.put(TABLE_CHECKSUM.key(),
String.valueOf(generateChecksum(props)));
+ Properties orderedProps = new Properties();
Review comment:
This will destroy the ordering, right? You need to control the iteration
during `store`, you to create class that is similar to how this was handled in
`TypedProperties`
```
class OrderedProperties extends Properties {
LinkedHashSet hashSet;
Enumeration<> keys() { hashSet.keys() }
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
