[
https://issues.apache.org/jira/browse/HUDI-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461644#comment-17461644
]
Manoj Govindassamy commented on HUDI-2005:
------------------------------------------
I am looking at these FS calls from the Metadata table perspective and I don't
see metadata table active paths using them. Will explore more.
{code:java}
$ hdfindx "*.java" getFileStatus | grep -v Test | grep Meta
./hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMetrics.java:
totalBaseFileSizeInBytes +=
slice.getBaseFile().get().getFileStatus().getLen();
./hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java:
statuses = hoodieRecord.get().getData().getFileStatuses(hadoopConf.get(),
partitionPath);
./hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java:
result.put(partitionInfo.get(entry.getKey()).toString(),
entry.getValue().get().getData().getFileStatuses(hadoopConf.get(),
partitionInfo.get(entry.getKey())));
./hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java:
public FileStatus[] getFileStatuses(Configuration hadoopConf, Path
partitionPath) throws IOException {
$ hdfindx "*.java" listStatus | grep -v Test | grep Meta
./hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HoodieInputFormatUtils.java:
* super.listStatus() and gets back a FileStatus[] 2. Then it creates the
HoodieTableMetaClient for the paths listed.
./hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java:
FileStatus[] statuses = HoodieCLI.fs.listStatus(metadataPath);
./hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java:
FileStatus[] statuses = HoodieCLI.fs.listStatus(metadataPath);
./hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java:
HoodieCLI.fs.listStatus(metadataPath);
./hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
return fs.listStatus(metaPath, nameFilter);
./hudi-common/src/main/java/org/apache/hudi/metadata/FileSystemBackedTableMetadata.java:
return Pair.of(path, fileSystem.listStatus(path));
./hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
return new DirectoryInfo(relativeDirPath, fs.listStatus(path));
./hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieSnapshotExporter.java:
fileSystem.listStatus(new Path(cfg.sourceBasePath + "/" +
HoodieTableMetaClient.METAFOLDER_NAME), (commitFilePath) -> {
./hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieSnapshotCopier.java:
fs.listStatus(new Path(baseDir + "/" +
HoodieTableMetaClient.METAFOLDER_NAME), (commitFilePath) -> {
{code}
> Audit and remove references of fs.listStatus() and fs.getFileStatus() or
> fs.exists()
> ------------------------------------------------------------------------------------
>
> Key: HUDI-2005
> URL: https://issues.apache.org/jira/browse/HUDI-2005
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Nishith Agarwal
> Assignee: Manoj Govindassamy
> Priority: Critical
> Labels: oct18_2021, pull-request-available
> Fix For: 0.11.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
