[
https://issues.apache.org/jira/browse/HUDI-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-9681:
------------------------------
Description:
This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which is
an anti-pattern, as reader should not incur write operation:
protected FileStatus[] listPartition(Path partitionPath) throws IOException {
try {
return metaClient.getFs().listStatus(partitionPath);
} catch (IOException e) { // Create the path if it does not exist
alreadyif (!metaClient.getFs().exists(partitionPath)) {
metaClient.getFs().mkdirs(partitionPath);
return new FileStatus[0];
} else { // in case the partition path was created by another
caller return metaClient.getFs().listStatus(partitionPath);
}
}
}
Relevant PR: [https://github.com/apache/hudi/pull/3769]
Relevant issue: [https://github.com/apache/hudi/issues/8653]
was:
This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which is
an anti-pattern, as reader should not incur write operation:
protected FileStatus[] listPartition(Path partitionPath) throws IOException {
try {
return metaClient.getFs().listStatus(partitionPath);
} catch (IOException e) { // Create the path if it does not exist
alreadyif (!metaClient.getFs().exists(partitionPath)) {
metaClient.getFs().mkdirs(partitionPath);
return new FileStatus[0];
} else { // in case the partition path was created by another
caller return metaClient.getFs().listStatus(partitionPath);
}
}
}
Relevant PR: https://github.com/apache/hudi/pull/3769
Relevant issue: https://github.com/apache/hudi/issues/8653
> Remove mkdirs from the partition listing in the file system view
> ----------------------------------------------------------------
>
> Key: HUDI-9681
> URL: https://issues.apache.org/jira/browse/HUDI-9681
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Y Ethan Guo
> Assignee: Y Ethan Guo
> Priority: Major
> Fix For: 1.1.0
>
>
> This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which
> is an anti-pattern, as reader should not incur write operation:
> protected FileStatus[] listPartition(Path partitionPath) throws IOException {
> try {
> return metaClient.getFs().listStatus(partitionPath);
> } catch (IOException e) { // Create the path if it does not exist
> alreadyif (!metaClient.getFs().exists(partitionPath)) {
> metaClient.getFs().mkdirs(partitionPath);
> return new FileStatus[0];
> } else { // in case the partition path was created by another
> caller return metaClient.getFs().listStatus(partitionPath);
> }
> }
> }
>
> Relevant PR: [https://github.com/apache/hudi/pull/3769]
> Relevant issue: [https://github.com/apache/hudi/issues/8653]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
