https://bz.apache.org/bugzilla/show_bug.cgi?id=63251
--- Comment #3 from harri...@gmail.com ---
(In reply to harristr from comment #2)
> The work-around of pre-loading the default FileSystem does not appear to
> solve the problem for logging, or maybe it only reduced the frequency of the
> occurrence. I am still seeing the following stacktrace from a Spring Boot
> Microservice running Tomcat 9.0.22 with JDK 1.8.0_222 64-bit on RedHat 7.5:
After further investigation I think this work-around actually CAUSED the
"deadlock" I am experiencing. I never saw this using Tomcat 9.0.17, it only
manifested after an upgrade directly to 9.0.22.
In an environment such as mine (a SpringBoot Microservice), Tomcat is NOT
running in the main thread or starting first, Spring Boot is. This means the
probability of reaching the "deadlock" is higher since Spring Boot is the one
running first and Tomcat is coming up in a different thread but is now trying
to prime the FileSystem right away.
I included more of the thread dump below.
"background-preinit" #9 prio=5 os_prio=0 tid=0x0000000002d11800 nid=0x726
waiting for monitor entry [0x00007f26ddd01000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Runtime.loadLibrary0(Runtime.java:862)
- waiting to lock <0x00000000c010be00> (a java.lang.Runtime)
at java.lang.System.loadLibrary(System.java:1122)
at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:573)
at sun.nio.fs.UnixNativeDispatcher$1.run(UnixNativeDispatcher.java:571)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.nio.fs.UnixNativeDispatcher.<clinit>(UnixNativeDispatcher.java:571)
at sun.nio.fs.UnixFileSystem.<init>(UnixFileSystem.java:67)
at sun.nio.fs.LinuxFileSystem.<init>(LinuxFileSystem.java:39)
at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:46)
at
sun.nio.fs.LinuxFileSystemProvider.newFileSystem(LinuxFileSystemProvider.java:39)
at
sun.nio.fs.UnixFileSystemProvider.<init>(UnixFileSystemProvider.java:56)
at
sun.nio.fs.LinuxFileSystemProvider.<init>(LinuxFileSystemProvider.java:41)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at
sun.nio.fs.DefaultFileSystemProvider.createProvider(DefaultFileSystemProvider.java:48)
at
sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:63)
at
java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108)
at
java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89)
at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98)
at
java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96)
at java.security.AccessController.doPrivileged(Native Method)
at
java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:96)
at
java.nio.file.FileSystems$DefaultFileSystemHolder.<clinit>(FileSystems.java:90)
at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
at org.apache.juli.logging.LogFactory.<init>(LogFactory.java:85)
at org.apache.juli.logging.LogFactory.<clinit>(LogFactory.java:66)
at
org.apache.catalina.mbeans.MBeanFactory.<clinit>(MBeanFactory.java:59)
at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$MBeanFactoryInitializer.run(BackgroundPreinitializer.java:139)
at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.runSafely(BackgroundPreinitializer.java:102)
at
org.springframework.boot.autoconfigure.BackgroundPreinitializer$1.run(BackgroundPreinitializer.java:94)
at java.lang.Thread.run(Thread.java:748)
"Service Thread" #8 daemon prio=9 os_prio=0 tid=0x0000000000fa0000 nid=0x6d7
runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x0000000000f90000
nid=0x6d6 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x0000000000f98800
nid=0x6d4 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x0000000000ebf800 nid=0x6d0
waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x0000000000e95000 nid=0x6c9 in
Object.wait() [0x00007f26e4c2d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c0166a10> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
- locked <0x00000000c0166a10> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x0000000000e8a800
nid=0x6c8 in Object.wait() [0x00007f26e4c6e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c0299c90> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000000c0299c90> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"main" #1 prio=5 os_prio=0 tid=0x0000000000b52000 nid=0x687 in Object.wait()
[0x00007f26fee02000]
java.lang.Thread.State: RUNNABLE
at java.nio.file.FileSystems.getDefault(FileSystems.java:176)
at java.nio.file.Paths.get(Paths.java:138)
at sun.misc.Launcher$ExtClassLoader.findLibrary(Launcher.java:235)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
- locked <0x00000000c010be00> (a java.lang.Runtime)
at java.lang.System.loadLibrary(System.java:1122)
at sun.security.ec.SunEC$1.run(SunEC.java:60)
at sun.security.ec.SunEC$1.run(SunEC.java:58)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ec.SunEC.<clinit>(SunEC.java:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:221)
at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206)
at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187)
- locked <0x00000000c055a6a0> (a sun.security.jca.ProviderConfig)
at sun.security.jca.ProviderList.getProvider(ProviderList.java:233)
at sun.security.jca.ProviderList.getService(ProviderList.java:331)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:157)
at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
- locked <0x00000000eb60aa50> (a java.lang.Class for
javax.net.ssl.SSLContext)
at javax.net.ssl.SSLSocketFactory.getDefault(SSLSocketFactory.java:122)
- locked <0x00000000eb608ec8> (a java.lang.Class for
javax.net.ssl.SSLSocketFactory)
at
javax.net.ssl.HttpsURLConnection.getDefaultSSLSocketFactory(HttpsURLConnection.java:332)
at javax.net.ssl.HttpsURLConnection.<init>(HttpsURLConnection.java:289)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.<init>(HttpsURLConnectionImpl.java:99)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:62)
at sun.net.www.protocol.https.Handler.openConnection(Handler.java:57)
at java.net.URL.openConnection(URL.java:991)
at
org.springframework.http.client.SimpleClientHttpRequestFactory.openConnection(SimpleClientHttpRequestFactory.java:187)
at
org.springframework.http.client.SimpleClientHttpRequestFactory.createRequest(SimpleClientHttpRequestFactory.java:145)
at
org.springframework.http.client.support.HttpAccessor.createRequest(HttpAccessor.java:87)
at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:731)
at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:691)
at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:618)
at
com.sas.commons.consul.util.ExchangeUtil.internalExchange(ExchangeUtil.java:161)
at
com.sas.commons.consul.util.ExchangeUtil.callExchangeMethod(ExchangeUtil.java:118)
at
com.sas.commons.consul.util.ExchangeUtil.getForObjectWithRetry(ExchangeUtil.java:320)
at
com.sas.commons.consul.impl.ServiceRegistrationImpl.getServiceDetails(ServiceRegistrationImpl.java:160)
at
com.sas.commons.consul.service.ConsulServiceResolver.getNodes(ConsulServiceResolver.java:249)
at
com.sas.commons.consul.service.ConsulServiceResolver.resolveServiceWithTags(ConsulServiceResolver.java:205)
at
com.sas.commons.consul.service.ConsulServiceResolver.resolveService(ConsulServiceResolver.java:180)
at
com.sas.commons.consul.service.ConsulServiceResolver.resolveService(ConsulServiceResolver.java:165)
at
com.sas.commons.rest.boot.initialization.env.VaultBootstrapPropertyLoader.resolveVaultFromConsul(VaultBootstrapPropertyLoader.java:65)
at
com.sas.commons.rest.boot.initialization.env.VaultBootstrapPropertyLoader.load(VaultBootstrapPropertyLoader.java:44)
at
com.sas.commons.rest.boot.initialization.env.BootstrapEnvironmentPropertyInitializer.onApplicationEvent(BootstrapEnvironmentPropertyInitializer.java:63)
at
com.sas.commons.rest.boot.initialization.env.BootstrapEnvironmentPropertyInitializer.onApplicationEvent(BootstrapEnvironmentPropertyInitializer.java:30)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at
org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
at
org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
at
org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:341)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:305)
at
org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:140)
- locked <0x00000000c05f83a0> (a
java.util.concurrent.atomic.AtomicBoolean)
at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.bootstrapServiceContext(BootstrapApplicationListener.java:203)
at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:114)
at
org.springframework.cloud.bootstrap.BootstrapApplicationListener.onApplicationEvent(BootstrapApplicationListener.java:71)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at
org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at
org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
at
org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
at
org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:341)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:305)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1214)
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1203)
at com.sas.transfer.Application.main(Application.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at
org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
"VM Thread" os_prio=0 tid=0x0000000000e81000 nid=0x6c7 runnable
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
