Hi,
I have a few questions regarding how VFS’s StandardFileSystemManager manages
SFTP sessions.
In my usage, I noticed that after a file transfer is completed, the session is
not actively closed, and corresponding sshd processes remain on the SFTP server.
I am using VFS in a service with 5 StandardFileSystemManager instances, each
handling a large number of concurrent upload tasks.
During local testing, when concurrency is high (e.g., 100), most server-side
processes are cleaned up, with only a few residual SSH-related processes.
However, in our production environment, we observe a large number of lingering
server-side processes.
Eventually, I found that SftpFileSystem#doCloseCommunicationLink is responsible
for closing the session.
I also noticed that explicitly calling FileSystemManager#closeFileSystem does
close the session, but it carries potential risks. Another alternative I found
is to call DefaultFileSystemManager#freeUnusedResources. And it worked.
Here are my questions:
Do we have to close the session manually now? If it is supposed to be
automatic, how does the mechanism work, and why aren’t the sessions being
closed?
If manual intervention is required, should we set
standardFileSystemManager.setFilesCache(new WeakRefFilesCache()) (to help with
quicker resource release), and then periodically call
DefaultFileSystemManager#freeUnusedResources()?
In the case of SoftRefFilesCache, I see a log that says “Close FileSystem” —
does this merely release the association, or does it actually close the
FileSystem?
The code is:
private StandardFileSystemManager fileSystemManager;
public FileClient() throws FileSystemException {
this.fileSystemManager = new StandardFileSystemManager();
this.fileSystemManager.init();
}
private <T> T execute(String path, Function<FileObject, T> function) throws
FileSystemException {
FileSystemOptions fileSystemOptions = this.genOptions();
try {
FileObject fileObject = fileSystemManager.resolveFile(path,
fileSystemOptions);
try {
return function.apply(fileObject);
} finally {
fileObject.close();
// manager.closeFileSystem(fileObject.getFileSystem());
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public boolean upload(String path, InputStream inputStream) throws
FileSystemException {
return execute(path, (fileObject) -> {
long startTime = YqgClock.now();
long total;
try (FileContent fileContent = fileObject.getContent();
OutputStream outputStream = fileContent.getOutputStream()) {
total = IOUtils.copy(inputStream, outputStream);
} catch (IOException e) {
throw new RuntimeException(e);
}
return true;
});
}
Thanks!
