the-other-tim-brown commented on code in PR #14311:
URL: https://github.com/apache/hudi/pull/14311#discussion_r2578706927
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -1009,14 +998,12 @@ private static List<FileSlice>
filterFileSliceBasedOnInflightCleaning(List<FileS
private List<HoodieBaseFile>
filterBaseFileBasedOnInflightCleaning(List<HoodieBaseFile> sortedBaseFileList,
Set<String> baseDataFilesForCleaning) {
return sortedBaseFileList.stream()
- .filter(baseFile -> {
- return !baseDataFilesForCleaning.contains(baseFile.getFileName());
- }).collect(Collectors.toList());
+ .filter(baseFile ->
!baseDataFilesForCleaning.contains(baseFile.getFileName())).collect(Collectors.toList());
}
@SuppressWarnings("rawtypes")
private void validateAllColumnStats(HoodieMetadataValidationContext
metadataTableBasedContext, HoodieMetadataValidationContext fsBasedContext,
- String partitionPath, Set<String>
baseDataFilesForCleaning) throws Exception {
+ String partitionPath, Set<String> baseDataFilesForCleaning) throws
Exception {
Review Comment:
The indentation here looks off now. We typically align the method arguments
on the newline. There are multiple other changes in this file that have the
same issue.
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -1857,8 +1839,8 @@ public HoodieMetadataValidationContext(
}
private HoodieTableFileSystemView getFileSystemView(HoodieEngineContext
context,
- HoodieTableMetaClient
metaClient, HoodieMetadataConfig metadataConfig,
-
FileSystemViewStorageConfig viewConf, HoodieCommonConfig commonConfig) {
+ HoodieTableMetaClient metaClient, HoodieMetadataConfig metadataConfig,
Review Comment:
The indentation here looks off now. We typically align the method arguments
on the newline.
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -863,13 +856,9 @@ && compareTimestamps(
}
/**
- * Compare the file listing and index data between metadata table and
fileSystem.
- * For now, validate five kinds of apis:
- * 1. HoodieTableFileSystemView::getLatestFileSlices
- * 2. HoodieTableFileSystemView::getLatestBaseFiles
- * 3. HoodieTableFileSystemView::getAllFileGroups and
HoodieTableFileSystemView::getAllFileSlices
- * 4. HoodieTableFileSystemView::getColumnStats
- * 5. HoodieTableFileSystemView::getBloomFilters
+ * Compare the file listing and index data between metadata table and
fileSystem. For now, validate five kinds of apis: 1.
HoodieTableFileSystemView::getLatestFileSlices 2.
Review Comment:
Let's just leave the formatting as is?
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -247,16 +248,14 @@ public HoodieMetadataTableValidator(JavaSparkContext jsc,
Config cfg) {
}
/**
- * Returns list of Throwable which were encountered during validation. This
method is useful
- * when ignoreFailed parameter is set to true.
+ * Returns list of Throwable which were encountered during validation. This
method is useful when ignoreFailed parameter is set to true.
Review Comment:
Can you limit the changes to changes that are required for the schema? There
is a lot of formatting changes that will lead to longer review cycles.
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/procedures/ShowColumnStatsOverlapProcedure.scala:
##########
@@ -144,12 +144,12 @@ class ShowColumnStatsOverlapProcedure extends
BaseProcedure with ProcedureBuilde
}
}
- def getAvroSchema(metaClient: HoodieTableMetaClient): Schema = {
- new TableSchemaResolver(metaClient).getTableAvroSchema
+ def getHoodieSchema(metaClient: HoodieTableMetaClient): HoodieSchema = {
Review Comment:
nitpick: just name the method `getSchema`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
