[
https://issues.apache.org/jira/browse/VFS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16238470#comment-16238470
]
Bernd Eckenfels edited comment on VFS-647 at 11/3/17 10:11 PM:
---------------------------------------------------------------
It looks like your two threads both close the filesystems and the fsm
concurrently. You need to wait for all filesystems operations to be finished.
Fsm is a singleton and getFilesystem() is also shared if it has the same
parameters on resolve.
was (Author: b.eckenfels):
It looks like your two threads both close the filesystems and the fsm
concurrently. You need to wait for all filesystems operations to be finished
before using close.
> calling findFiles() causes copyFrom() to fail with a partially downloaded
> file.
> -------------------------------------------------------------------------------
>
> Key: VFS-647
> URL: https://issues.apache.org/jira/browse/VFS-647
> Project: Commons VFS
> Issue Type: Bug
> Affects Versions: 2.0, 2.2
> Environment: Windows 7 and Linux Red Hat.
> Reporter: Kenji
> Priority: Minor
>
> Edit: The problem isn't specific to findFiles. All I need to do is have 1
> thread downloads the file, and the other thread just log in, have a while
> loop for x seconds where x < download time (using Thread.sleep(x) causes
> interruption to happen immediately). Once x seconds passed, the download will
> get interupted with pipe closed. Using the same hostname but different
> username and password does not cause interruption.
> Original description:
> When using FileObject.copyFrom(remote, new AllFileSelector()) to download
> file from remote to local directory. If SftpFileObject.findFiles(new
> FileDepthSelector(1,1)) is called and finished, then the copyFrom will get
> interrupted with error pipe closed.
> Below are test codes and stack trace error. In real scenario, this all
> happens within 1 or 2 second time frame. However, with test scenario, I
> wasn't able to reproduce it easily therefore I had to choose a file that
> takes around 10 seconds to download and have Thread.sleep(5000) after
> findFiles() call. Only tested jsch-0.1.52, jsch-0.1.54, commons-vfs2-2.0 and
> commons-vfs2-2.2.
> {code:java}
> public class FtpClient {
>
> public static void main(String[] args) {
> if (args.length < 5) {
> throw new RuntimeException("args: host user pass local
> remote");
> }
> String hostname = args[0];
> String username = args[1];
> String password = args[2];
> int port = 22;
> String local = args[3];
> String remote = args[4];
> final String remoteDir = remote.substring(0, Math.max(0,
> remote.lastIndexOf("/")));
> Thread t0 = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.list(remoteDir);
> System.out.println("findFiles()
> completed.");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
>
> Thread ti = new Thread() {
> public void run() {
> try (Ftp ftp = new Ftp(hostname, port)) {
> ftp.login(username, password);
> ftp.download(local, remote);
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> };
> t0.start();
> ti.start();
> }
> }
> public class Ftp implements AutoCloseable {
> private int timeout = 0;
> private StaticUserAuthenticator userAuth;
> private FileSystemOptions fileSysOpts;
> private FileObject scr = null;
> private FileSystemManager fsm = null;
> private String hostName;
> private int port = 0;
> public Ftp(String remoteHost, int controlPort) throws
> FileSystemException {
> hostName = remoteHost;
> port = controlPort;
> fsm = VFS.getManager();
> }
> //login into a server with a valid account
> public void login(String user, String password) throws IOException {
> userAuth = new StaticUserAuthenticator(null, user, password);
> fileSysOpts = new FileSystemOptions();
>
> DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(fileSysOpts,
> userAuth);
>
> SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts,
> "no");
>
> SftpFileSystemConfigBuilder.getInstance().setTimeout(fileSysOpts, timeout);
>
> SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(fileSysOpts,
> false);
> scr = (FileObject)fsm.resolveFile("sftp://"; + hostName,
> fileSysOpts);
>
> System.out.println("login successfully.");
> }
> public void list(String dirName) throws Exception {
> SftpFileObject RemoteFo =
> (SftpFileObject)fsm.resolveFile("sftp://"; + hostName + dirName, fileSysOpts);
>
> FileObject[] afo = RemoteFo.findFiles(new
> FileDepthSelector(1,1));
> Thread.sleep(5000); //key. this must be here (or some other
> processing that takes time) for error to reproduce consistently.
> }
> public void download(String localPath, String remoteFile) throws
> FileSystemException {
>
> SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(fileSysOpts,
> "no");
> SftpFileObject RemoteFo =
> (SftpFileObject)fsm.resolveFile("sftp://"; + hostName + remoteFile
> ,fileSysOpts);
> scr = fsm.resolveFile("file:" + localPath);
> scr.copyFrom(RemoteFo, new AllFileSelector());
> }
>
> @Override
> public void close() {
> try{
> FileSystem fs = null;
> if(this.scr!=null){
> this.scr.close();
> fs = this.scr.getFileSystem();
> this.fsm.closeFileSystem(fs);
> }
> }catch(Exception e){
> System.out.println("unable to release ftp connection.");
> e.printStackTrace();
> }
> }
> }
> {code}
> StackTrace:
> {noformat}
> Oct 18, 2017 12:42:51 PM org.apache.commons.vfs2.VfsLog info
> INFO: Using "C:\Users\kenji\AppData\Local\Temp\vfs_cache" as temporary files
> stor
> e.
> login successfully.
> login successfully.
> findFiles() completed.
> org.apache.commons.vfs2.FileSystemException: Could not copy
> "sftp://hostname/path/to/file/FILE1"; to "file:///C:/test/ftp/FILE1".
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(Abstract
> FileObject.java:1062)
> at Ftp.download(Ftp.java:59)
> at FtpClient$2.run(FtpClient.java:36)
> Caused by: org.apache.commons.vfs2.FileSystemException: Could not close the
> inpu
> t stream for file "sftp://hostname/path/to/file/FILE1";.
> at
> org.apache.commons.vfs2.provider.DefaultFileContent$FileContentInputS
> tream.close(DefaultFileContent.java:611)
> at org.apache.commons.vfs2.FileUtil.writeContent(FileUtil.java:95)
> at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:114)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(Abstract
> FileObject.java:1053)
> ... 2 more
> Caused by: java.io.IOException: Pipe closed
> at java.io.PipedInputStream.read(PipedInputStream.java:307)
> at java.io.PipedInputStream.read(PipedInputStream.java:377)
> at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2882)
> at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2908)
> at com.jcraft.jsch.ChannelSftp.access$500(ChannelSftp.java:36)
> at
> com.jcraft.jsch.ChannelSftp$RequestQueue.cancel(ChannelSftp.java:1238
> )
> at com.jcraft.jsch.ChannelSftp$2.close(ChannelSftp.java:1503)
> at java.io.BufferedInputStream.close(BufferedInputStream.java:483)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.close(MonitorInputStr
> eam.java:129)
> at java.io.BufferedInputStream.close(BufferedInputStream.java:483)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.close(MonitorInputStr
> eam.java:129)
> at
> org.apache.commons.vfs2.provider.DefaultFileContent$FileContentInputS
> tream.close(DefaultFileContent.java:607)
> ... 5 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
