[
https://issues.apache.org/jira/browse/HUDI-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-9681:
------------------------------
Description:
This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which is
an anti-pattern, as reader should not incur write operation:
protected FileStatus[] listPartition(Path partitionPath) throws IOException {
try
{ return metaClient.getFs().listStatus(partitionPath); }
catch (IOException e) { // Create the path if it does not exist alreadyif
(!metaClient.getFs().exists(partitionPath))
{ metaClient.getFs().mkdirs(partitionPath); return new FileStatus[0]; }
else
{ // in case the partition path was created by another caller return
metaClient.getFs().listStatus(partitionPath); }
}
}
Relevant PR introducing this change: [https://github.com/apache/hudi/pull/3769]
Relevant issue: [https://github.com/apache/hudi/issues/8653]
was:
This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which is
an anti-pattern, as reader should not incur write operation:
protected FileStatus[] listPartition(Path partitionPath) throws IOException {
try {
return metaClient.getFs().listStatus(partitionPath);
} catch (IOException e) { // Create the path if it does not exist
alreadyif (!metaClient.getFs().exists(partitionPath)) {
metaClient.getFs().mkdirs(partitionPath);
return new FileStatus[0];
} else { // in case the partition path was created by another
caller return metaClient.getFs().listStatus(partitionPath);
}
}
}
Relevant PR: [https://github.com/apache/hudi/pull/3769]
Relevant issue: [https://github.com/apache/hudi/issues/8653]
> Remove mkdirs from the partition listing in the file system view
> ----------------------------------------------------------------
>
> Key: HUDI-9681
> URL: https://issues.apache.org/jira/browse/HUDI-9681
> Project: Apache Hudi
> Issue Type: Bug
> Affects Versions: 0.12.0, 0.12.1, 0.12.2, 0.13.0, 0.13.1, 0.12.3, 0.14.0,
> 0.14.1, 0.15.0
> Reporter: Y Ethan Guo
> Assignee: Y Ethan Guo
> Priority: Major
> Fix For: 1.0.0-beta2, 1.0.0
>
>
> This code in {{AbstractTableFileSystemView}} incurs a mkdir operation which
> is an anti-pattern, as reader should not incur write operation:
> protected FileStatus[] listPartition(Path partitionPath) throws IOException {
> try
> { return metaClient.getFs().listStatus(partitionPath); }
> catch (IOException e) { // Create the path if it does not exist alreadyif
> (!metaClient.getFs().exists(partitionPath))
> { metaClient.getFs().mkdirs(partitionPath); return new FileStatus[0]; }
> else
> { // in case the partition path was created by another caller return
> metaClient.getFs().listStatus(partitionPath); }
> }
> }
>
> Relevant PR introducing this change:
> [https://github.com/apache/hudi/pull/3769]
> Relevant issue: [https://github.com/apache/hudi/issues/8653]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
