xiarixiaoyao commented on a change in pull request #4786:
URL: https://github.com/apache/hudi/pull/4786#discussion_r816393746
##########
File path:
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java
##########
@@ -343,4 +350,44 @@ public void updateLastCommitTimeSynced(String tableName) {
}
}
}
+
+ public Schema getAvroSchemaWithoutMetadataFields() {
+ try {
+ return new
TableSchemaResolver(metaClient).getTableAvroSchemaWithoutMetadataFields();
+ } catch (Exception e) {
+ throw new HoodieSyncException("Failed to read avro schema", e);
+ }
+ }
+
+ public List<FieldSchema> getTableCommentUsingMetastoreClient(String
tableName) {
+ try {
+ List<FieldSchema> schema = client.getSchema(syncConfig.databaseName,
tableName);
+ return schema;
+ } catch (Exception e) {
+ throw new HoodieHiveSyncException("Failed to get table comments for : "
+ tableName, e);
+ }
+ }
+
+ public void updateTableComments(String tableName, List<FieldSchema>
oldSchema, List<Schema.Field> newSchema) {
+ Map<String,String> newComments =
newSchema.stream().collect(Collectors.toMap(field ->
field.name().toLowerCase(Locale.ROOT), field ->
StringUtils.isNullOrEmpty(field.doc()) ? "" : field.doc()));
+ updateTableComments(tableName,oldSchema,newComments);
+ }
+
+ public void updateTableComments(String tableName, List<FieldSchema>
oldSchema, Map<String,String> newComments) {
+ Map<String,String> oldComments =
oldSchema.stream().collect(Collectors.toMap(fieldSchema ->
fieldSchema.getName().toLowerCase(Locale.ROOT),
+ fieldSchema -> StringUtils.isNullOrEmpty(fieldSchema.getComment()) ?
"" : fieldSchema.getComment()));
+ Map<String,String> types =
oldSchema.stream().collect(Collectors.toMap(FieldSchema::getName,FieldSchema::getType));
+ Map<String, ImmutablePair<String,String>> alterComments = new HashMap<>();
+ oldComments.forEach((name,comment) -> {
+ String newComment = newComments.getOrDefault(name,"");
+ if (!newComment.equals(comment)) {
+ alterComments.put(name,new
ImmutablePair<>(types.get(name),newComment));
+ }
+ });
+ if (alterComments.size() > 0) {
+ ddlExecutor.updateTableComments(tableName, alterComments);
+ } else {
+ LOG.info(String.format("No comment difference of %s ",tableName));
+ }
+ }
Review comment:
code style. Please add a space between the two input parameters
eg： line379 Collectors.toMap(FieldSchema::getName,FieldSchema::getType)
->Collectors.toMap(FieldSchema::getName, FieldSchema::getType)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
