Re: [VFS] jsch thread hanging while reading from SFTP
VFS is not very well suited to parallel usage, especially not if the meta data operations are relatively slow. One thing you can do (besides checking why the particular operation which blocks is slow) is to use multiple instances of the filesystems I think. (Won’t work with the default manager as this is a singleton). Gruss Bernd -- http://bernd.eckenfels.net Von: Shekhar B Gesendet: Friday, February 25, 2022 6:16:17 PM An: user@commons.apache.org Betreff: [VFS] jsch thread hanging while reading from SFTP I have noticed slowness in reading data from SFTP server, I have 10 parallel processes running in the different containers which are reading this data and slowness is usually observed in one or two containers randomly. I am using commons-vfs to read data from SFTP server and I have extended SFTP related classes to add some sort logic. I am using centos-7.9 to host my SFTP server. I have noticed the reader thread is in blocked state and waiting for monitor thread to release lock. Can someone please guide me on how to fix this issue. Thread dump: "Thread-135" #150 prio=5 os_prio=0 tid=0x7f1e3808d800 nid=0x3a2 waiting for monitor entry [0x7f1ea5eed000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:306) - waiting to lock <0x0005cedf23f8> (a org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:301) at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:76) at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:56) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) at org.apache.commons.vfs2.impl.DefaultVfsComponentContext.resolveFile(DefaultVfsComponentContext.java:46) at org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.findFile(AbstractLayeredFileProvider.java:54) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:677) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:632) at com.sqlstream.aspen.namespace.filevfs.FileVFSInputSource$FileReader.run(FileVFSInputSource.java:894) at java.lang.Thread.run(Thread.java:748) "Thread-134" #149 daemon prio=1 os_prio=0 tid=0x7f1e381e7800 nid=0x3a1 in Object.wait() [0x7f1ea6af9000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.io.PipedInputStream.read(PipedInputStream.java:326) - eliminated <0x0005d821ea90> (a com.jcraft.jsch.Channel$MyPipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:377) - locked <0x0005d821ea90> (a com.jcraft.jsch.Channel$MyPipedInputStream) at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2909) at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2935) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1643) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1553) at org.apache.commons.vfs2.provider.sftp.MySftpFileObject.doListChildrenResolved(MySftpFileObject.java:376) at org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1081) - locked <0x0005cedf23f8> (a org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.checkForNewChildren(MyFileMonitor.java:458) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.check(MyFileMonitor.java:543) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.access$200(MyFileMonitor.java:376) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor.run(MyFileMonitor.java:321) at java.lang.Thread.run(Thread.java:748)
Re: [VFS] jsch thread hanging while reading from SFTP
Hi, Sorry to hear you're have an issue with VFS. I'm not sure how to help. The usual route for support here is to provide a failing unit test or a whole standalone project we could debug. Short of that I would start by making sure you're using the latest versions of everything. Providing a reproducible bug in a program is key. Gary On Fri, Feb 25, 2022, 14:38 Shekhar B wrote: > I have noticed slowness in reading data from SFTP server, I have 10 > parallel processes running in the different containers which are reading > this data and slowness is usually observed in one or two containers > randomly. I am using commons-vfs to read data from SFTP server and I have > extended SFTP related classes to add some sort logic. I am using centos-7.9 > to host my SFTP server. I have noticed the reader thread is in blocked > state and waiting for monitor thread to release lock. > > Can someone please guide me on how to fix this issue. > > > Thread dump: > > "Thread-135" #150 prio=5 os_prio=0 tid=0x7f1e3808d800 nid=0x3a2 > waiting for monitor entry [0x7f1ea5eed000] >java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:306) > - waiting to lock <0x0005cedf23f8> (a > org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) > at > org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:301) > at > org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:76) > at > org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:56) > at > org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) > at > org.apache.commons.vfs2.impl.DefaultVfsComponentContext.resolveFile(DefaultVfsComponentContext.java:46) > at > org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.findFile(AbstractLayeredFileProvider.java:54) > at > org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) > at > org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:677) > at > org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:632) > at > com.sqlstream.aspen.namespace.filevfs.FileVFSInputSource$FileReader.run(FileVFSInputSource.java:894) > at java.lang.Thread.run(Thread.java:748) > "Thread-134" #149 daemon prio=1 os_prio=0 tid=0x7f1e381e7800 > nid=0x3a1 in Object.wait() [0x7f1ea6af9000] >java.lang.Thread.State: TIMED_WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > at java.io.PipedInputStream.read(PipedInputStream.java:326) > - eliminated <0x0005d821ea90> (a > com.jcraft.jsch.Channel$MyPipedInputStream) > at java.io.PipedInputStream.read(PipedInputStream.java:377) > - locked <0x0005d821ea90> (a > com.jcraft.jsch.Channel$MyPipedInputStream) > at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2909) > at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2935) > at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1643) > at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1553) > at > org.apache.commons.vfs2.provider.sftp.MySftpFileObject.doListChildrenResolved(MySftpFileObject.java:376) > at > org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1081) > - locked <0x0005cedf23f8> (a > org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) > at > com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.checkForNewChildren(MyFileMonitor.java:458) > at > com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.check(MyFileMonitor.java:543) > at > com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.access$200(MyFileMonitor.java:376) > at > com.sqlstream.aspen.namespace.filevfs.MyFileMonitor.run(MyFileMonitor.java:321) > at java.lang.Thread.run(Thread.java:748) >
[VFS] jsch thread hanging while reading from SFTP
I have noticed slowness in reading data from SFTP server, I have 10 parallel processes running in the different containers which are reading this data and slowness is usually observed in one or two containers randomly. I am using commons-vfs to read data from SFTP server and I have extended SFTP related classes to add some sort logic. I am using centos-7.9 to host my SFTP server. I have noticed the reader thread is in blocked state and waiting for monitor thread to release lock. Can someone please guide me on how to fix this issue. Thread dump: "Thread-135" #150 prio=5 os_prio=0 tid=0x7f1e3808d800 nid=0x3a2 waiting for monitor entry [0x7f1ea5eed000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:306) - waiting to lock <0x0005cedf23f8> (a org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:301) at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:76) at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:56) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) at org.apache.commons.vfs2.impl.DefaultVfsComponentContext.resolveFile(DefaultVfsComponentContext.java:46) at org.apache.commons.vfs2.provider.AbstractLayeredFileProvider.findFile(AbstractLayeredFileProvider.java:54) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:711) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:677) at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:632) at com.sqlstream.aspen.namespace.filevfs.FileVFSInputSource$FileReader.run(FileVFSInputSource.java:894) at java.lang.Thread.run(Thread.java:748) "Thread-134" #149 daemon prio=1 os_prio=0 tid=0x7f1e381e7800 nid=0x3a1 in Object.wait() [0x7f1ea6af9000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.io.PipedInputStream.read(PipedInputStream.java:326) - eliminated <0x0005d821ea90> (a com.jcraft.jsch.Channel$MyPipedInputStream) at java.io.PipedInputStream.read(PipedInputStream.java:377) - locked <0x0005d821ea90> (a com.jcraft.jsch.Channel$MyPipedInputStream) at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2909) at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2935) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1643) at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1553) at org.apache.commons.vfs2.provider.sftp.MySftpFileObject.doListChildrenResolved(MySftpFileObject.java:376) at org.apache.commons.vfs2.provider.AbstractFileObject.getChildren(AbstractFileObject.java:1081) - locked <0x0005cedf23f8> (a org.apache.commons.vfs2.provider.sftp.MySftpFileSystem) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.checkForNewChildren(MyFileMonitor.java:458) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.check(MyFileMonitor.java:543) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor$FileMonitorAgent.access$200(MyFileMonitor.java:376) at com.sqlstream.aspen.namespace.filevfs.MyFileMonitor.run(MyFileMonitor.java:321) at java.lang.Thread.run(Thread.java:748)
