Deadlock occurs in HTTP or WebDAV when all the connections are in use.
----------------------------------------------------------------------
Key: VFS-343
URL: https://issues.apache.org/jira/browse/VFS-343
Project: Commons VFS
Issue Type: Bug
Reporter: Ralph Goers
The stacktrace below shows several threads all waiting on a FileSystem lock.
The thread holding that lock is waiting for an HTTP connection from the
connection pool. However, each of the threads waiting on the FileSystem lock
has already acquired a connection from the pool. This issue can be resolved by
reducing the amount of locking that occurs on the FileSystem objects. In this
case the threads are attempting to get the lock to increment a counter.
Changing the counters to AtomicIntegers will resolve the problem.
Full thread dump Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode):
"Attach Listener" daemon prio=10 tid=0x0000000053834400 nid=0x23b1 waiting on
condition [0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"MultiThreadedHttpConnectionManager cleanup" daemon prio=10
tid=0x00000000538ff400 nid=0x21f0 in Object.wait()
[0x000000004077e000..0x000000004077ec10]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aab82694428> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x00002aab82694428> (a java.lang.ref.ReferenceQueue$Lock)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1082)
"Thread 2" prio=10 tid=0x00002aabc00a0800 nid=0x21ef waiting for monitor entry
[0x0000000040d17000..0x0000000040d17c90]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
at
org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"Thread 3" prio=10 tid=0x00002aabc009f800 nid=0x21ec in Object.wait()
[0x0000000042593000..0x0000000042593d10]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aab826c67a8> (a
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.doGetConnection(MultiThreadedHttpConnectionManager.java:509)
- locked <0x00002aab826c67a8> (a
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.getConnectionWithTimeout(MultiThreadedHttpConnectionManager.java:394)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:152)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.execute(WebdavFileObject.java:349)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperties(WebdavFileObject.java:442)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperties(WebdavFileObject.java:417)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperty(WebdavFileObject.java:410)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.getProperty(WebdavFileObject.java:402)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.isDirectory(WebdavFileObject.java:382)
at
org.apache.commons.vfs.provider.webdav.WebdavFileObject.doGetType(WebdavFileObject.java:112)
at
org.apache.commons.vfs.provider.AbstractFileObject.getType(AbstractFileObject.java:441)
- locked <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.AbstractFileObject.exists(AbstractFileObject.java:424)
at
org.apache.commons.configuration.VFSFileSystem.locateFromURL(VFSFileSystem.java:293)
at
org.apache.commons.configuration.ConfigurationUtils.locate(ConfigurationUtils.java:466)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:245)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"Thread 4" prio=10 tid=0x00002aabc00ce000 nid=0x21eb waiting for monitor entry
[0x0000000042492000..0x0000000042492d90]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
at
org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"Thread 5" prio=10 tid=0x00002aabc00d2000 nid=0x21ea waiting for monitor entry
[0x0000000042391000..0x0000000042391a10]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
at
org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"Thread 6" prio=10 tid=0x00002aabc00af400 nid=0x21e9 waiting for monitor entry
[0x0000000042290000..0x0000000042290a90]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:294)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:86)
at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:66)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:692)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:648)
at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:604)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:99)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"Thread 7" prio=10 tid=0x00002aabc001cc00 nid=0x21e8 waiting for monitor entry
[0x000000004218f000..0x000000004218fb10]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
at
org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at java.lang.Thread.run(Thread.java:619)
"org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread" daemon
prio=10 tid=0x00002aabc001ec00 nid=0x21e7 in Object.wait()
[0x000000004208e000..0x000000004208eb90]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaabe1f0198> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x00002aaabe1f0198> (a java.lang.ref.ReferenceQueue$Lock)
at
org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread.run(SoftRefFilesCache.java:79)
"Low Memory Detector" daemon prio=10 tid=0x000000005360bc00 nid=0x21e4 runnable
[0x0000000000000000..0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=10 tid=0x0000000053609000 nid=0x21e3 waiting on
condition [0x0000000000000000..0x0000000041d8a4c0]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=10 tid=0x0000000053605400 nid=0x21e2 waiting on
condition [0x0000000000000000..0x000000004165c5d0]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x0000000053603400 nid=0x21e1 runnable
[0x0000000000000000..0x0000000040ed66a0]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x00000000535e0c00 nid=0x21e0 in Object.wait()
[0x000000004155c000..0x000000004155ca90]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaabe1f0278> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x00002aaabe1f0278> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x00000000535df000 nid=0x21df in
Object.wait() [0x000000004145b000..0x000000004145bb10]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00002aaabe1f2370> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x00002aaabe1f2370> (a java.lang.ref.Reference$Lock)
"Thread 1" prio=10 tid=0x0000000053427400 nid=0x21d5 waiting for monitor entry
[0x00000000412af000..0x00000000412afed0]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.commons.vfs.provider.AbstractFileSystem.streamOpened(AbstractFileSystem.java:602)
- waiting to lock <0x00002aab827f92c8> (a
org.apache.commons.vfs.provider.webdav.WebdavFileSystem)
at
org.apache.commons.vfs.provider.DefaultFileContent.streamOpened(DefaultFileContent.java:93)
at
org.apache.commons.vfs.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:373)
at
org.apache.commons.configuration.VFSFileSystem.getInputStream(VFSFileSystem.java:111)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:311)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:251)
at
org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:229)
at
org.apache.commons.configuration.AbstractHierarchicalFileConfiguration.load(AbstractHierarchicalFileConfiguration.java:180)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClientProduct(ConfigurationSynchronizer.java:862)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.synchronizeClient(ConfigurationSynchronizer.java:829)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.run(ConfigurationSynchronizer.java:416)
at
com.diginsite.services.configsync.ConfigurationSynchronizer.main(ConfigurationSynchronizer.java:1790)
"VM Thread" prio=10 tid=0x00000000535d9800 nid=0x21de runnable
"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000053432000 nid=0x21d6
runnable
"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000053433800 nid=0x21d7
runnable
"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000053435400 nid=0x21d8
runnable
"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000053436c00 nid=0x21d9
runnable
"GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000053438400 nid=0x21da
runnable
"GC task thread#5 (ParallelGC)" prio=10 tid=0x000000005343a000 nid=0x21db
runnable
"GC task thread#6 (ParallelGC)" prio=10 tid=0x000000005343b800 nid=0x21dc
runnable
"GC task thread#7 (ParallelGC)" prio=10 tid=0x000000005343d000 nid=0x21dd
runnable
"VM Periodic Task Thread" prio=10 tid=0x000000005360e000 nid=0x21e5 waiting on
condition
JNI global references: 902
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
