Re: Problems with sftp and vfs
Coming back to this, when we investigated this a few weeks ago we found that this was caused by the sftp server implementation getting stuck over a command that was trying to get some information about the files (id -G), clearly not supported by that implementation. The workaround we found is to use a custom file object factory that can be used to tune the delegation of operations from the FTP file to the VFS file. I provided one implementation (ShallowReadOnlyVfsFtpFileFactory) that forces read-only behavior and that does not delegate methods that are not relevant in that context - which worked for Stefan. I already delivered the new binaries a while ago but I finally updated the documentation on the website (http://vfs-utils.sourceforge.net/ftpserver/configuration.html). Cheers, Frank VFS FTPServer - VFS Utils FTPServer Configuration<http://vfs-utils.sourceforge.net/ftpserver/configuration.html> VFS Utils FTPServer Configuration. To use this bridge you should just add the VFS libraries, the VFS Utils FTPServer library, the VFS Utils Utils library and, if needed, your custom VFS Provider and its dependencies to the common/lib directory of your Apache Mina FTP Server installation. vfs-utils.sourceforge.net From: Frank van der Kleij Sent: Wednesday, November 4, 2020 10:31 PM To: ftpserver-users@mina.apache.org Subject: Re: Problems with sftp and vfs Hi Stefan, Maybe we can take the discussion privately until we find a solution for your problem? For me it worked with small file that I downloaded from a local sftp server through an ftp server with vfs. Maybe for bigger files it behaves differently. It might not be very easy to reproduce for me since I don't have exactly the same environment as you, and it could be the problem is in the sftp provider of the VFS library, but I'll give it another shot. SFTP does not make the difference between active and passive modes, that exists only in the ftp/ftps protocols. Best regards, Frank From: Stefan Ziegler Sent: Wednesday, November 4, 2020 8:22 PM To: ftpserver-users@mina.apache.org Subject: Re: Problems with sftp and vfs When I use the ftp command in the terminal "ftp localhost 2221" I'm able to login but when try to list with "dir" it hangs: ftp> dir 200 Command PORT okay. 150 File status okay; about to open data connection. Logfile: 2020-11-04 20:18:38.130 INFO 18691 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: LIST 2020-11-04 20:18:38.154 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Opening active data connection 2020-11-04 20:18:38.155 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Binding active data connection to /127.0.0.1:0 2020-11-04 20:18:39.803 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.806 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u246727.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata It this a acitve/passive issue? Can I set the VFS to passive? Stefan On Tue, Nov 3, 2020 at 9:33 PM Frank van der Kleij wrote: > Hi Stefan, > > The authenticate method is always called and the user is passed to the > VFS. Then depending on how the VFS is initialised it will use some > information from the user; in your case it should just take the home > directory from the user. > > For me it is working - though I didn't use your code example, but I had to > add the jsch and jcifs libraries to the classpath of the ftp server because > they are needed for sftp (that is using ssh). > > > This is my configuration, but you can do it in code too: > > http://mina.apache.org/ftpserver/spring/v1; > xmlns:beans="http://www.springframework.org/schema/beans; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; > xsi:schemaLocation=" > http://mina.apache.org/ftpserver/spring/v1 > http://mina.apache.org/ftpserver/ftpserver-1.0.xsd > " > id="myServer"> > > > > > > > > > encrypt-passwords="true"/> > > > > class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory"> > value="sftp://myusr:mypwd@mysftpsrvr:22/"/&
Re: Problems with sftp and vfs
Hi Stefan, Maybe we can take the discussion privately until we find a solution for your problem? For me it worked with small file that I downloaded from a local sftp server through an ftp server with vfs. Maybe for bigger files it behaves differently. It might not be very easy to reproduce for me since I don't have exactly the same environment as you, and it could be the problem is in the sftp provider of the VFS library, but I'll give it another shot. SFTP does not make the difference between active and passive modes, that exists only in the ftp/ftps protocols. Best regards, Frank From: Stefan Ziegler Sent: Wednesday, November 4, 2020 8:22 PM To: ftpserver-users@mina.apache.org Subject: Re: Problems with sftp and vfs When I use the ftp command in the terminal "ftp localhost 2221" I'm able to login but when try to list with "dir" it hangs: ftp> dir 200 Command PORT okay. 150 File status okay; about to open data connection. Logfile: 2020-11-04 20:18:38.130 INFO 18691 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: LIST 2020-11-04 20:18:38.154 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Opening active data connection 2020-11-04 20:18:38.155 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Binding active data connection to /127.0.0.1:0 2020-11-04 20:18:39.803 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.806 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u246727.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata It this a acitve/passive issue? Can I set the VFS to passive? Stefan On Tue, Nov 3, 2020 at 9:33 PM Frank van der Kleij wrote: > Hi Stefan, > > The authenticate method is always called and the user is passed to the > VFS. Then depending on how the VFS is initialised it will use some > information from the user; in your case it should just take the home > directory from the user. > > For me it is working - though I didn't use your code example, but I had to > add the jsch and jcifs libraries to the classpath of the ftp server because > they are needed for sftp (that is using ssh). > > > This is my configuration, but you can do it in code too: > > http://mina.apache.org/ftpserver/spring/v1; > xmlns:beans="http://www.springframework.org/schema/beans; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; > xsi:schemaLocation=" > http://mina.apache.org/ftpserver/spring/v1 > http://mina.apache.org/ftpserver/ftpserver-1.0.xsd > " > id="myServer"> > > > > > > > > > encrypt-passwords="true"/> > > > > class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory"> > value="sftp://myusr:mypwd@mysftpsrvr:22/"/> > value="virtual"/> > > > > > > Hope this helps, > > Frank > > > From: Stefan Ziegler > Sent: Tuesday, November 3, 2020 5:21 PM > To: ftpserver-users@mina.apache.org > Subject: Re: Problems with sftp and vfs > > I tried to narrow it down but it's still strange: The exact same code works > when using "ftp://; but does not work with "sftp://; in the setVfsRoot() > methode. With "sftp://; I get an error like: > > org.apache.ftpserver.ftplet.FtpException: Error resolving home directory / > of user demo. > at > > org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64) > ~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na] > > The demo user is the login for the local ftp server. Why does it use the > local user for creating the virtual file system? For me these are > completely different things. And why does it work with plain "ftp://; vfs? > As far as I understand it should also use > VfsFileSystemFactory.createFileSystemView. > > Stefan > > On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij > wrote: > > > Hi Stefan, > > > > I haven't worked with this for a while now, but I can set up an > > environment to tes
Re: Problems with sftp and vfs
When I use the ftp command in the terminal "ftp localhost 2221" I'm able to login but when try to list with "dir" it hangs: ftp> dir 200 Command PORT okay. 150 File status okay; about to open data connection. Logfile: 2020-11-04 20:18:38.130 INFO 18691 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: LIST 2020-11-04 20:18:38.154 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Opening active data connection 2020-11-04 20:18:38.155 DEBUG 18691 --- [pool-3-thread-1] o.a.f.impl.IODataConnectionFactory : Binding active data connection to /127.0.0.1:0 2020-11-04 20:18:39.803 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.806 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u246727.your-storagebox.de/ch.so.agi.av.dm01avch24lv95d 2020-11-04 20:18:39.807 DEBUG 18691 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@ u00.your-storagebox.de/ch.so.agi.geodata It this a acitve/passive issue? Can I set the VFS to passive? Stefan On Tue, Nov 3, 2020 at 9:33 PM Frank van der Kleij wrote: > Hi Stefan, > > The authenticate method is always called and the user is passed to the > VFS. Then depending on how the VFS is initialised it will use some > information from the user; in your case it should just take the home > directory from the user. > > For me it is working - though I didn't use your code example, but I had to > add the jsch and jcifs libraries to the classpath of the ftp server because > they are needed for sftp (that is using ssh). > > > This is my configuration, but you can do it in code too: > > http://mina.apache.org/ftpserver/spring/v1; > xmlns:beans="http://www.springframework.org/schema/beans; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; > xsi:schemaLocation=" > http://mina.apache.org/ftpserver/spring/v1 > http://mina.apache.org/ftpserver/ftpserver-1.0.xsd > " > id="myServer"> > > > > > > > > > encrypt-passwords="true"/> > > > > class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory"> > value="sftp://myusr:mypwd@mysftpsrvr:22/"/> > value="virtual"/> > > > > > > Hope this helps, > > Frank > > > From: Stefan Ziegler > Sent: Tuesday, November 3, 2020 5:21 PM > To: ftpserver-users@mina.apache.org > Subject: Re: Problems with sftp and vfs > > I tried to narrow it down but it's still strange: The exact same code works > when using "ftp://; but does not work with "sftp://; in the setVfsRoot() > methode. With "sftp://; I get an error like: > > org.apache.ftpserver.ftplet.FtpException: Error resolving home directory / > of user demo. > at > > org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64) > ~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na] > > The demo user is the login for the local ftp server. Why does it use the > local user for creating the virtual file system? For me these are > completely different things. And why does it work with plain "ftp://; vfs? > As far as I understand it should also use > VfsFileSystemFactory.createFileSystemView. > > Stefan > > On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij > wrote: > > > Hi Stefan, > > > > I haven't worked with this for a while now, but I can set up an > > environment to test this one of these days. > > > > In the code you are using "ftp://; as virtual file system but in the > logs > > I see "sftp://;. > > > > So it was working when your client was using "ftp://; and your VFS was > > using "ftp://; too? And now you want your VFS to be "sftp://;? > > Have you tried with "ftps://" for the VFS? As sftp goes over SSH a > > different VFS provider is used. > > > > As for the approach, I am not 100% sure but I don't think you have to > call > > the createFileSystemView method, that should be done automatically; this > > part might be superfluous: > > > > BaseUser vfsUser = new BaseUser(); > > vfsUser.setName(ftpUserH
Re: Problems with sftp and vfs
I did a quick comparison with the logfile when using plain ftp. It hangs at
the PORT command:
2020-11-04 20:04:16.325 INFO 16425 --- [pool-3-thread-2]
o.a.f.listener.nio.FtpLoggingFilter : SENT: 227 Entering Passive Mode
(127,0,0,1,151,131)
2020-11-04 20:04:16.326 INFO 16425 --- [pool-3-thread-1]
o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: RETR
ch.so.arp.richtplan_dxf.zip
2020-11-04 20:04:16.326 DEBUG 16425 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: ftp://u00:***@
u00.your-storagebox.de/ch.so.agi.geodata/ch.so.arp.richtplan_dxf.zip
2020-11-04 20:04:16.326 DEBUG 16425 --- [pool-3-thread-1]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: ftp://u00:***@
u00.your-storagebox.de/ch.so.agi.geodata/ch.so.arp.richtplan_dxf.zip
2020-11-04 20:04:16.327 DEBUG 16425 --- [pool-3-thread-1]
.f.FtpClientFactory$FtpConnectionFactory : PORT 10,0,1,10,162,245
On Tue, Nov 3, 2020 at 9:33 PM Frank van der Kleij
wrote:
> Hi Stefan,
>
> The authenticate method is always called and the user is passed to the
> VFS. Then depending on how the VFS is initialised it will use some
> information from the user; in your case it should just take the home
> directory from the user.
>
> For me it is working - though I didn't use your code example, but I had to
> add the jsch and jcifs libraries to the classpath of the ftp server because
> they are needed for sftp (that is using ssh).
>
>
> This is my configuration, but you can do it in code too:
>
> http://mina.apache.org/ftpserver/spring/v1;
> xmlns:beans="http://www.springframework.org/schema/beans;
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
> xsi:schemaLocation="
> http://mina.apache.org/ftpserver/spring/v1
> http://mina.apache.org/ftpserver/ftpserver-1.0.xsd
> "
> id="myServer">
>
>
>
>
>
>
>
>
> encrypt-passwords="true"/>
>
>
>
> class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory">
> value="sftp://myusr:mypwd@mysftpsrvr:22/"/>
> value="virtual"/>
>
>
>
>
>
> Hope this helps,
>
> Frank
>
>
> From: Stefan Ziegler
> Sent: Tuesday, November 3, 2020 5:21 PM
> To: ftpserver-users@mina.apache.org
> Subject: Re: Problems with sftp and vfs
>
> I tried to narrow it down but it's still strange: The exact same code works
> when using "ftp://; but does not work with "sftp://; in the setVfsRoot()
> methode. With "sftp://; I get an error like:
>
> org.apache.ftpserver.ftplet.FtpException: Error resolving home directory /
> of user demo.
> at
>
> org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64)
> ~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na]
>
> The demo user is the login for the local ftp server. Why does it use the
> local user for creating the virtual file system? For me these are
> completely different things. And why does it work with plain "ftp://; vfs?
> As far as I understand it should also use
> VfsFileSystemFactory.createFileSystemView.
>
> Stefan
>
> On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij
> wrote:
>
> > Hi Stefan,
> >
> > I haven't worked with this for a while now, but I can set up an
> > environment to test this one of these days.
> >
> > In the code you are using "ftp://; as virtual file system but in the
> logs
> > I see "sftp://;.
> >
> > So it was working when your client was using "ftp://; and your VFS was
> > using "ftp://; too? And now you want your VFS to be "sftp://;?
> > Have you tried with "ftps://" for the VFS? As sftp goes over SSH a
> > different VFS provider is used.
> >
> > As for the approach, I am not 100% sure but I don't think you have to
> call
> > the createFileSystemView method, that should be done automatically; this
> > part might be superfluous:
> >
> > BaseUser vfsUser = new BaseUser();
> > vfsUser.setName(ftpUserHetzner);
> > vfsUser.setPassword(ftpPwdHetzner);
> > vfsUser.setHomeDirectory("/");
> >
> > vfsFileSystemFactory.createFileSystemView(vfsUser);
> >
> > Cheers,
> >
> > Frank
> >
> >
> >
> >
> >
> >
> > From: Stefan Ziegler
> > Sent: Sunday, November 1, 2020 8:20 PM
> > To: ftpserver-users@mina.apache.org
> > Subject: Problems with sftp and vfs
> >
Re: Problems with sftp and vfs
ns:xsi="http://www.w3.org/2001/XMLSchema-instance;
> xsi:schemaLocation="
> http://mina.apache.org/ftpserver/spring/v1
> http://mina.apache.org/ftpserver/ftpserver-1.0.xsd
> "
> id="myServer">
>
>
>
>
>
>
>
>
> encrypt-passwords="true"/>
>
>
>
> class="org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory">
> value="sftp://myusr:mypwd@mysftpsrvr:22/"/>
> value="virtual"/>
>
>
>
>
>
> Hope this helps,
>
> Frank
>
>
> From: Stefan Ziegler
> Sent: Tuesday, November 3, 2020 5:21 PM
> To: ftpserver-users@mina.apache.org
> Subject: Re: Problems with sftp and vfs
>
> I tried to narrow it down but it's still strange: The exact same code works
> when using "ftp://; but does not work with "sftp://; in the setVfsRoot()
> methode. With "sftp://; I get an error like:
>
> org.apache.ftpserver.ftplet.FtpException: Error resolving home directory /
> of user demo.
> at
>
> org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64)
> ~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na]
>
> The demo user is the login for the local ftp server. Why does it use the
> local user for creating the virtual file system? For me these are
> completely different things. And why does it work with plain "ftp://; vfs?
> As far as I understand it should also use
> VfsFileSystemFactory.createFileSystemView.
>
> Stefan
>
> On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij
> wrote:
>
> > Hi Stefan,
> >
> > I haven't worked with this for a while now, but I can set up an
> > environment to test this one of these days.
> >
> > In the code you are using "ftp://; as virtual file system but in the
> logs
> > I see "sftp://;.
> >
> > So it was working when your client was using "ftp://; and your VFS was
> > using "ftp://; too? And now you want your VFS to be "sftp://;?
> > Have you tried with "ftps://" for the VFS? As sftp goes over SSH a
> > different VFS provider is used.
> >
> > As for the approach, I am not 100% sure but I don't think you have to
> call
> > the createFileSystemView method, that should be done automatically; this
> > part might be superfluous:
> >
> > BaseUser vfsUser = new BaseUser();
> > vfsUser.setName(ftpUserHetzner);
> > vfsUser.setPassword(ftpPwdHetzner);
> > vfsUser.setHomeDirectory("/");
> >
> > vfsFileSystemFactory.createFileSystemView(vfsUser);
> >
> > Cheers,
> >
> > Frank
> >
> >
> >
> >
> >
> >
> > From: Stefan Ziegler
> > Sent: Sunday, November 1, 2020 8:20 PM
> > To: ftpserver-users@mina.apache.org
> > Subject: Problems with sftp and vfs
> >
> > Hi
> >
> > I'm setting up a plain ftp server in front of another ftp server as
> virtual
> > file system. This works great since the ftp server used in the virtual
> file
> > system does not allow anonymous logins. Switching from plain ftp to ftps
> in
> > the virtual file systems ftp server I'm not able to download files
> anymore.
> > It still shows the directories but when trying to download a file it
> seems
> > like it stalls somehow.
> >
> > According to the logs it seems that it tries to download as "anonymous"
> > user and not the user from the virtual file system ftp server (but I can
> > misinterpret this).
> >
> > Has anyone an idea or some hints? Do I mess up something in my code (with
> > the users)? Thanks for any help. See code and logs below.
> >
> > best regards
> > Stefan
> >
> > PropertiesUserManagerFactory userManagerFactory = new
> > PropertiesUserManagerFactory();
> > UserManager userManager = userManagerFactory.createUserManager();
> > BaseUser user = new BaseUser();
> > user.setName("anonymous");
> > userManager.save(user);
> >
> > ListenerFactory listenerFactory = new ListenerFactory();
> > listenerFactory.setPort(2221);
> >
> > ConnectionConfigFactory connectionConfigFactory = new
> > ConnectionConfigFactory();
> > connectionConfigFactory.setAnonymousLoginEnabled(true);
> > connectionConfigFactory.setMaxLogins(10);
> > connectionConfigFactory.setMaxThreads(10);
> >
> > FtpServerFactory factor
Re: Problems with sftp and vfs
Hi Stefan,
The authenticate method is always called and the user is passed to the VFS.
Then depending on how the VFS is initialised it will use some information from
the user; in your case it should just take the home directory from the user.
For me it is working - though I didn't use your code example, but I had to add
the jsch and jcifs libraries to the classpath of the ftp server because they
are needed for sftp (that is using ssh).
This is my configuration, but you can do it in code too:
http://mina.apache.org/ftpserver/spring/v1;
xmlns:beans="http://www.springframework.org/schema/beans;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="
http://mina.apache.org/ftpserver/spring/v1
http://mina.apache.org/ftpserver/ftpserver-1.0.xsd
"
id="myServer">
ftp://myusr:mypwd@mysftpsrvr:22/"/>
Hope this helps,
Frank
From: Stefan Ziegler
Sent: Tuesday, November 3, 2020 5:21 PM
To: ftpserver-users@mina.apache.org
Subject: Re: Problems with sftp and vfs
I tried to narrow it down but it's still strange: The exact same code works
when using "ftp://; but does not work with "sftp://; in the setVfsRoot()
methode. With "sftp://; I get an error like:
org.apache.ftpserver.ftplet.FtpException: Error resolving home directory /
of user demo.
at
org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64)
~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na]
The demo user is the login for the local ftp server. Why does it use the
local user for creating the virtual file system? For me these are
completely different things. And why does it work with plain "ftp://; vfs?
As far as I understand it should also use
VfsFileSystemFactory.createFileSystemView.
Stefan
On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij
wrote:
> Hi Stefan,
>
> I haven't worked with this for a while now, but I can set up an
> environment to test this one of these days.
>
> In the code you are using "ftp://; as virtual file system but in the logs
> I see "sftp://;.
>
> So it was working when your client was using "ftp://; and your VFS was
> using "ftp://; too? And now you want your VFS to be "sftp://;?
> Have you tried with "ftps://" for the VFS? As sftp goes over SSH a
> different VFS provider is used.
>
> As for the approach, I am not 100% sure but I don't think you have to call
> the createFileSystemView method, that should be done automatically; this
> part might be superfluous:
>
> BaseUser vfsUser = new BaseUser();
> vfsUser.setName(ftpUserHetzner);
> vfsUser.setPassword(ftpPwdHetzner);
> vfsUser.setHomeDirectory("/");
>
> vfsFileSystemFactory.createFileSystemView(vfsUser);
>
> Cheers,
>
> Frank
>
>
>
>
>
>
> From: Stefan Ziegler
> Sent: Sunday, November 1, 2020 8:20 PM
> To: ftpserver-users@mina.apache.org
> Subject: Problems with sftp and vfs
>
> Hi
>
> I'm setting up a plain ftp server in front of another ftp server as virtual
> file system. This works great since the ftp server used in the virtual file
> system does not allow anonymous logins. Switching from plain ftp to ftps in
> the virtual file systems ftp server I'm not able to download files anymore.
> It still shows the directories but when trying to download a file it seems
> like it stalls somehow.
>
> According to the logs it seems that it tries to download as "anonymous"
> user and not the user from the virtual file system ftp server (but I can
> misinterpret this).
>
> Has anyone an idea or some hints? Do I mess up something in my code (with
> the users)? Thanks for any help. See code and logs below.
>
> best regards
> Stefan
>
> PropertiesUserManagerFactory userManagerFactory = new
> PropertiesUserManagerFactory();
> UserManager userManager = userManagerFactory.createUserManager();
> BaseUser user = new BaseUser();
> user.setName("anonymous");
> userManager.save(user);
>
> ListenerFactory listenerFactory = new ListenerFactory();
> listenerFactory.setPort(2221);
>
> ConnectionConfigFactory connectionConfigFactory = new
> ConnectionConfigFactory();
> connectionConfigFactory.setAnonymousLoginEnabled(true);
> connectionConfigFactory.setMaxLogins(10);
> connectionConfigFactory.setMaxThreads(10);
>
> FtpServerFactory factory = new FtpServerFactory();
>
> factory.setConnectionConfig(connectionConfigFactory.createConnectionConfig());
> factory.setUserManager(userManager);
> factory.addListener("defau
Re: Problems with sftp and vfs
I tried to narrow it down but it's still strange: The exact same code works
when using "ftp://; but does not work with "sftp://; in the setVfsRoot()
methode. With "sftp://; I get an error like:
org.apache.ftpserver.ftplet.FtpException: Error resolving home directory /
of user demo.
at
org.vfsutils.ftpserver.filesystem.VfsFileSystemFactory.createFileSystemView(VfsFileSystemFactory.java:64)
~[vfsutils-ftpserver-0.2.0-SNAPSHOT.jar:na]
The demo user is the login for the local ftp server. Why does it use the
local user for creating the virtual file system? For me these are
completely different things. And why does it work with plain "ftp://; vfs?
As far as I understand it should also use
VfsFileSystemFactory.createFileSystemView.
Stefan
On Tue, Nov 3, 2020 at 12:49 PM Frank van der Kleij
wrote:
> Hi Stefan,
>
> I haven't worked with this for a while now, but I can set up an
> environment to test this one of these days.
>
> In the code you are using "ftp://; as virtual file system but in the logs
> I see "sftp://;.
>
> So it was working when your client was using "ftp://; and your VFS was
> using "ftp://; too? And now you want your VFS to be "sftp://;?
> Have you tried with "ftps://" for the VFS? As sftp goes over SSH a
> different VFS provider is used.
>
> As for the approach, I am not 100% sure but I don't think you have to call
> the createFileSystemView method, that should be done automatically; this
> part might be superfluous:
>
> BaseUser vfsUser = new BaseUser();
> vfsUser.setName(ftpUserHetzner);
> vfsUser.setPassword(ftpPwdHetzner);
> vfsUser.setHomeDirectory("/");
>
> vfsFileSystemFactory.createFileSystemView(vfsUser);
>
> Cheers,
>
> Frank
>
>
>
>
>
>
> From: Stefan Ziegler
> Sent: Sunday, November 1, 2020 8:20 PM
> To: ftpserver-users@mina.apache.org
> Subject: Problems with sftp and vfs
>
> Hi
>
> I'm setting up a plain ftp server in front of another ftp server as virtual
> file system. This works great since the ftp server used in the virtual file
> system does not allow anonymous logins. Switching from plain ftp to ftps in
> the virtual file systems ftp server I'm not able to download files anymore.
> It still shows the directories but when trying to download a file it seems
> like it stalls somehow.
>
> According to the logs it seems that it tries to download as "anonymous"
> user and not the user from the virtual file system ftp server (but I can
> misinterpret this).
>
> Has anyone an idea or some hints? Do I mess up something in my code (with
> the users)? Thanks for any help. See code and logs below.
>
> best regards
> Stefan
>
> PropertiesUserManagerFactory userManagerFactory = new
> PropertiesUserManagerFactory();
> UserManager userManager = userManagerFactory.createUserManager();
> BaseUser user = new BaseUser();
> user.setName("anonymous");
> userManager.save(user);
>
> ListenerFactory listenerFactory = new ListenerFactory();
> listenerFactory.setPort(2221);
>
> ConnectionConfigFactory connectionConfigFactory = new
> ConnectionConfigFactory();
> connectionConfigFactory.setAnonymousLoginEnabled(true);
> connectionConfigFactory.setMaxLogins(10);
> connectionConfigFactory.setMaxThreads(10);
>
> FtpServerFactory factory = new FtpServerFactory();
>
> factory.setConnectionConfig(connectionConfigFactory.createConnectionConfig());
> factory.setUserManager(userManager);
> factory.addListener("default", listenerFactory.createListener());
>
> VfsFileSystemFactory vfsFileSystemFactory = new VfsFileSystemFactory();
> VfsAuthenticator vfsAuthentificator = new VfsAuthenticator();
> vfsAuthentificator.setVfsRoot("ftp://
> "+ftpUserHetzner+":"+ftpPwdHetzner+"@"+ftpServerHetzner);
> vfsAuthentificator.setVfsType("virtual");
>
> vfsFileSystemFactory.setAuthenticator(vfsAuthentificator);
>
> BaseUser vfsUser = new BaseUser();
> vfsUser.setName(ftpUserHetzner);
> vfsUser.setPassword(ftpPwdHetzner);
> vfsUser.setHomeDirectory("/");
>
> vfsFileSystemFactory.createFileSystemView(vfsUser);
> factory.setFileSystem(vfsFileSystemFactory);
>
> FtpServer server = factory.createServer();
> server.start();
>
> 2020-11-01 19:50:31.043 INFO 117121 --- [ NioProcessor-4]
> o.a.f.listener.nio.FtpLoggingFilter : CREATED
> 2020-11-01 19:50:31.045 INFO 117121 --- [pool-3-thread-1]
> o.a.f.listener.nio.FtpLoggingFilter : OPENED
> 2020-11-01 19:50:31.047 INFO 117121 --- [pool-3-thread-2]
> o.a.f.listener.nio.FtpLoggingFilter : SENT: 220 Service ready for new
> user.
>
> 2020-11-01 19:50:31.060 INFO 117121 --- [pool-3-thread-1]
> o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: USER anonymous
> 2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
> org.apache.ftpserver.command.impl.USER : Currently 1 out of 10 anonymous
> users logged in
> 2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
> org.apache.ftpserver.command.impl.USER : Currently 1 out of 5 users
> logged in
> 2020-11-01 19:50:31.063 INFO 117121 --- [pool-3-thread-2]
>
Re: Problems with sftp and vfs
Hi Frank It's working when using "ftp://; in the VFS. When using "sftp://; _without_ the superflous vfsUser code I get a "Error resolving home directory / of user anonymous" error. When using "sftp://; _with_ the superfluous vfsUser I get the error "Error resolving home directory / of user u000". As far as I understand: In the first approach it tries to connect to the VFS with the user that was used to log in to the (frontend) ftp server. But why use a user anyway? The full uri for the VFS is set in the "setVfsRoot" method including username and password. In the second approach it uses the user for the VFS but struggles with setting the home dir. When using "ftps://" _without_ the vsfUser it shows the root dir of my local machine. So it simply ignores the VFS stuff. Then when setting the vfsUser in the "createFileSystemView" method it does return an empty list or something like that (see below). There's the line "Authenticated user anonymous based on filesystem org.apache.commons.vfs2.impl.VirtualFileSystem@4ec955ec" which makes me wonder. Why does it use the anonymous user in the VfsAuthenticator class? regards Stefan 2020-11-03 13:43:54.611 INFO 74492 --- [ NioProcessor-3] o.a.f.listener.nio.FtpLoggingFilter : CREATED 2020-11-03 13:43:54.615 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : OPENED 2020-11-03 13:43:54.619 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : SENT: 220 Service ready for new user. 2020-11-03 13:43:54.621 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: USER anonymous 2020-11-03 13:43:54.622 DEBUG 74492 --- [pool-3-thread-1] org.apache.ftpserver.command.impl.USER : Currently 0 out of 10 anonymous users logged in 2020-11-03 13:43:54.622 DEBUG 74492 --- [pool-3-thread-1] org.apache.ftpserver.command.impl.USER : Currently 0 out of 10 users logged in 2020-11-03 13:43:54.622 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : SENT: 331 Guest login okay, send your complete e-mail address as password. 2020-11-03 13:43:54.622 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: PASS * 2020-11-03 13:43:54.623 DEBUG 74492 --- [pool-3-thread-1] org.apache.ftpserver.command.impl.PASS : Currently 0 out of 10 anonymous users logged in 2020-11-03 13:43:54.623 DEBUG 74492 --- [pool-3-thread-1] org.apache.ftpserver.command.impl.PASS : Currently 0 out of 10 users logged in 2020-11-03 13:43:54.655 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : 220 ProFTPD Server (Hetzner Backup) [:::78.47.102.233] 2020-11-03 13:43:54.655 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : AUTH TLS 2020-11-03 13:43:54.668 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : 234 AUTH TLS successful 2020-11-03 13:43:54.815 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : USER u00 2020-11-03 13:43:54.828 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : 331 Password required for u00 2020-11-03 13:43:54.828 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : PASS *** 2020-11-03 13:43:54.860 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : 230 User u00 logged in 2020-11-03 13:43:54.861 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : TYPE I 2020-11-03 13:43:54.874 DEBUG 74492 --- [pool-3-thread-1] .FtpsClientFactory$FtpsConnectionFactory : 200 Type set to I 2020-11-03 13:43:54.874 DEBUG 74492 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: ftps://u00:***@ u00.your-storagebox.de/ 2020-11-03 13:43:54.874 DEBUG 74492 --- [pool-3-thread-1] o.a.c.vfs2.cache.SoftRefFilesCache : putFile: ftps://u00:***@ u00.your-storagebox.de/ 2020-11-03 13:43:54.874 INFO 74492 --- [pool-3-thread-1] o.v.f.filesystem.VfsAuthenticator: Authenticated user anonymous based on filesystem org.apache.commons.vfs2.impl.VirtualFileSystem@4ec955ec 2020-11-03 13:43:54.875 INFO 74492 --- [pool-3-thread-1] org.apache.ftpserver.command.impl.PASS : Anonymous login success - cyberd...@example.net 2020-11-03 13:43:54.875 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : SENT: 230 User logged in, proceed. 2020-11-03 13:43:54.875 INFO 74492 --- [pool-3-thread-2] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: FEAT 2020-11-03 13:43:54.876 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : SENT: 211-Extensions supported SIZE MDTM REST STREAM LANG en;zh-tw;ja;is MLST Size;Modify;Type;Perm AUTH SSL AUTH TLS MODE Z UTF8 TVFS MD5 MMD5 MFMT 211 End 2020-11-03 13:43:54.876 INFO 74492 --- [pool-3-thread-1] o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: OPTS UTF8 ON 2020-11-03 13:43:54.877 INFO 74492 ---
Re: Problems with sftp and vfs
Hi Stefan,
I haven't worked with this for a while now, but I can set up an environment to
test this one of these days.
In the code you are using "ftp://; as virtual file system but in the logs I see
"sftp://;.
So it was working when your client was using "ftp://; and your VFS was using
"ftp://; too? And now you want your VFS to be "sftp://;?
Have you tried with "ftps://" for the VFS? As sftp goes over SSH a different
VFS provider is used.
As for the approach, I am not 100% sure but I don't think you have to call the
createFileSystemView method, that should be done automatically; this part might
be superfluous:
BaseUser vfsUser = new BaseUser();
vfsUser.setName(ftpUserHetzner);
vfsUser.setPassword(ftpPwdHetzner);
vfsUser.setHomeDirectory("/");
vfsFileSystemFactory.createFileSystemView(vfsUser);
Cheers,
Frank
From: Stefan Ziegler
Sent: Sunday, November 1, 2020 8:20 PM
To: ftpserver-users@mina.apache.org
Subject: Problems with sftp and vfs
Hi
I'm setting up a plain ftp server in front of another ftp server as virtual
file system. This works great since the ftp server used in the virtual file
system does not allow anonymous logins. Switching from plain ftp to ftps in
the virtual file systems ftp server I'm not able to download files anymore.
It still shows the directories but when trying to download a file it seems
like it stalls somehow.
According to the logs it seems that it tries to download as "anonymous"
user and not the user from the virtual file system ftp server (but I can
misinterpret this).
Has anyone an idea or some hints? Do I mess up something in my code (with
the users)? Thanks for any help. See code and logs below.
best regards
Stefan
PropertiesUserManagerFactory userManagerFactory = new
PropertiesUserManagerFactory();
UserManager userManager = userManagerFactory.createUserManager();
BaseUser user = new BaseUser();
user.setName("anonymous");
userManager.save(user);
ListenerFactory listenerFactory = new ListenerFactory();
listenerFactory.setPort(2221);
ConnectionConfigFactory connectionConfigFactory = new
ConnectionConfigFactory();
connectionConfigFactory.setAnonymousLoginEnabled(true);
connectionConfigFactory.setMaxLogins(10);
connectionConfigFactory.setMaxThreads(10);
FtpServerFactory factory = new FtpServerFactory();
factory.setConnectionConfig(connectionConfigFactory.createConnectionConfig());
factory.setUserManager(userManager);
factory.addListener("default", listenerFactory.createListener());
VfsFileSystemFactory vfsFileSystemFactory = new VfsFileSystemFactory();
VfsAuthenticator vfsAuthentificator = new VfsAuthenticator();
vfsAuthentificator.setVfsRoot("ftp://
"+ftpUserHetzner+":"+ftpPwdHetzner+"@"+ftpServerHetzner);
vfsAuthentificator.setVfsType("virtual");
vfsFileSystemFactory.setAuthenticator(vfsAuthentificator);
BaseUser vfsUser = new BaseUser();
vfsUser.setName(ftpUserHetzner);
vfsUser.setPassword(ftpPwdHetzner);
vfsUser.setHomeDirectory("/");
vfsFileSystemFactory.createFileSystemView(vfsUser);
factory.setFileSystem(vfsFileSystemFactory);
FtpServer server = factory.createServer();
server.start();
2020-11-01 19:50:31.043 INFO 117121 --- [ NioProcessor-4]
o.a.f.listener.nio.FtpLoggingFilter : CREATED
2020-11-01 19:50:31.045 INFO 117121 --- [pool-3-thread-1]
o.a.f.listener.nio.FtpLoggingFilter : OPENED
2020-11-01 19:50:31.047 INFO 117121 --- [pool-3-thread-2]
o.a.f.listener.nio.FtpLoggingFilter : SENT: 220 Service ready for new
user.
2020-11-01 19:50:31.060 INFO 117121 --- [pool-3-thread-1]
o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: USER anonymous
2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
org.apache.ftpserver.command.impl.USER : Currently 1 out of 10 anonymous
users logged in
2020-11-01 19:50:31.061 DEBUG 117121 --- [pool-3-thread-1]
org.apache.ftpserver.command.impl.USER : Currently 1 out of 5 users
logged in
2020-11-01 19:50:31.063 INFO 117121 --- [pool-3-thread-2]
o.a.f.listener.nio.FtpLoggingFilter : SENT: 331 Guest login okay, send
your complete e-mail address as password.
2020-11-01 19:50:31.064 INFO 117121 --- [pool-3-thread-2]
o.a.f.listener.nio.FtpLoggingFilter : RECEIVED: PASS *
2020-11-01 19:50:31.064 DEBUG 117121 --- [pool-3-thread-2]
org.apache.ftpserver.command.impl.PASS : Currently 1 out of 10 anonymous
users logged in
2020-11-01 19:50:31.064 DEBUG 117121 --- [pool-3-thread-2]
org.apache.ftpserver.command.impl.PASS : Currently 1 out of 5 users
logged in
2020-11-01 19:50:31.066 DEBUG 117121 --- [pool-3-thread-2]
o.a.c.vfs2.cache.SoftRefFilesCache : putFile: sftp://u00:***@
u00.your-storagebox.de/
2020-11-01 19:50:31.089 INFO 117121 --- [pool-3-thread-2]
o.v.f.filesystem.VfsAuthenticator: Authenticated user anonymous
based on filesystem org.apache.commons.vfs2.impl.VirtualFileSystem@53de3b94
2020-11-01 19:50:31.090 INFO 117121 --- [pool-3-thread-2]
org.apache.ftpserver.command.impl.PASS : Anonymous login
