vinothchandar commented on a change in pull request #3808:
URL: https://github.com/apache/hudi/pull/3808#discussion_r839013128
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -591,6 +591,10 @@ private void revertCompleteToInflight(HoodieInstant
completed, HoodieInstant inf
}
}
+ private Path getInstantFileNamePath(String fileName) {
+ return new Path(fileName.contains(SAVE_SCHEMA_ACTION) ?
metaClient.getSchemaFolderName() : metaClient.getMetaPath(), fileName);
Review comment:
if we treat something as an instant, splitting this apart into folders
seem a bit odd to me
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java
##########
@@ -550,6 +551,9 @@ public void archive(HoodieEngineContext context,
List<HoodieInstant> instants) t
}
}
writeToFile(wrapperSchema, records);
+ // try to clean old history schema.
+ FileBasedInternalSchemaStorageManager fss = new
FileBasedInternalSchemaStorageManager(metaClient);
+ fss.cleanOldFiles(instants.stream().map(is ->
is.getTimestamp()).collect(Collectors.toList()));
Review comment:
should this be in cleaner?
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/model/WriteOperationType.java
##########
@@ -48,6 +48,8 @@
INSERT_OVERWRITE_TABLE("insert_overwrite_table"),
// compact
COMPACT("compact"),
+ // alter schema
+ ALTER_SCHEMA("alter_schema"),
// used for old version
Review comment:
Makes sense
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -493,4 +496,51 @@ public static MessageType readSchemaFromLogFile(FileSystem
fs, Path path) throws
}
return null;
}
+
+ /**
+ * Gets the InternalSchema for a hoodie table from the HoodieCommitMetadata
of the instant.
+ *
+ * @return InternalSchema for this table
+ */
+ public Option<InternalSchema> getTableInternalSchemaFromCommitMetadata() {
+ HoodieTimeline timeline =
metaClient.getActiveTimeline().getCommitsTimeline().filterCompletedInstants();
Review comment:
Save it as an indexed file (HFile) so you can just read the last record
or first record and be done? Having one source of truth would be good. We can
also do this as follow up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
