[
https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744293#comment-17744293
]
ASF GitHub Bot commented on HADOOP-18807:
-----------------------------------------
steveloughran commented on code in PR #5847:
URL: https://github.com/apache/hadoop/pull/5847#discussion_r1267028844
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java:
##########
@@ -17,26 +17,26 @@
*/
package org.apache.hadoop.fs.viewfs;
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
Review Comment:
can you put the static imports right down at the bottom, just above the
class declaration?
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java:
##########
@@ -1926,12 +1926,41 @@ enum RenameStrategy {
SAME_FILESYSTEM_ACROSS_MOUNTPOINT
}
+ private void closeChildFileSystems(FileSystem fs) throws IOException {
+ if (fs != null) {
+ FileSystem[] childFs = fs.getChildFileSystems();
+ for (FileSystem child : childFs) {
+ if (child != null) {
+ String disableCacheName = String.format("fs.%s.impl.disable.cache",
+ child.getUri().getScheme());
+ if (config.getBoolean(disableCacheName, false)) {
+ child.close();
Review Comment:
do failures need to be handled here? if so: what?
any fs set to delete files on close may raise exceptions here from
permissions...should they be ignored, fail fast (as this pr does) or should we
try best effort to close the others.
The simple patch here is better than today, so i'm not too worried -and that
delete on close stuff isn't something to use in production
> Close child file systems in ViewFileSystem when cache is disabled.
> ------------------------------------------------------------------
>
> Key: HADOOP-18807
> URL: https://issues.apache.org/jira/browse/HADOOP-18807
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Shuyan Zhang
> Priority: Major
> Labels: pull-request-available
>
> When the cache is configured to disabled (namely,
> `fs.viewfs.enable.inner.cache=false` and `fs.*.impl.disable.cache=true`),
> even if `FileSystem.close()` is called, the client cannot truly close the
> child file systems in a ViewFileSystem. This caused our long-running clients
> to constantly produce resource leaks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
