Hei, we use jsch through commons-vfs. We've made some load tests to investigate some issues. Our load test transfers 10 files simultaneously (using so many threads), in an infinite loop.
Things used to break down pretty fast (less than 20 seconds) including hangs and many transfer failures, but now the system is more reliable. There are still some failures once in a while, more on that later. Our issues where both in commons-vfs (e.g. https://issues.apache.org/jira/browse/VFS-186) and (we think) jsch. Attached is a patch that improved the reliability of jsch. Details: * compile jsch with debug mode by default * a known issue in jsch 0.1.36 * don't consider a read() that returns 0 to be a closed stream * add some logging to help investigating unexpected disconnections With these changes in place we are now facing at least one remaining issue: INFO - Caught an exception, leaving main loop due to com.jcraft.jsch.JSchException: SSH_MSG_DISCONNECT: 2 Could not create socket pairs: Too many open files at com.jcraft.jsch.Session.read(Session.java:858) at com.jcraft.jsch.Session.run(Session.java:1158) at java.lang.Thread.run(Thread.java:595) When this happens, the different SFTP threads end up all failing with various stack traces: Caused by: com.jcraft.jsch.JSchException: java.lang.NullPointerException at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:235) at com.jcraft.jsch.Channel.connect(Channel.java:192) at com.jcraft.jsch.Channel.connect(Channel.java:143) at org.apache.commons.vfs.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:147) ... 10 more Caused by: java.lang.NullPointerException at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2315) at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2339) at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:204) or Caused by: org.apache.commons.vfs.FileSystemException: Could not connect to SFTP server at ".......". at org.apache.commons.vfs.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:168) at org.apache.commons.vfs.provider.sftp.SftpFileObject.doGetOutputStream(SftpFileObject.java:445) at org.apache.commons.vfs.provider.AbstractFileObject.getOutputStream(AbstractFileObject.java:1259) ... 8 more Caused by: com.jcraft.jsch.JSchException: java.io.IOException: channel is broken at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:235) at com.jcraft.jsch.Channel.connect(Channel.java:192) at com.jcraft.jsch.Channel.connect(Channel.java:143) at org.apache.commons.vfs.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:147) ... 10 more Caused by: java.io.IOException: channel is broken at com.jcraft.jsch.Session.write(Session.java:1057) at com.jcraft.jsch.ChannelSftp.sendINIT(ChannelSftp.java:1946) at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:200) ... 13 more we hope to be able to improve the error handling as well as reduce the risks for this "too many open files" issue to occur. This of course could be an issue on our Unix SFTP host, or an issue in jsch/commons-vfs/our code where some resources are not properly released. More on that later. Thanks, Cheers, Jerome PS: is it possible to replace tabs with spaces in jsch code ? That would ease following through various libraries that use different tab to space indentation values.
jsch-0.1.36_pacth_1.diff
Description: Binary data
------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________ JSch-users mailing list JSch-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jsch-users
