[
https://issues.apache.org/jira/browse/HADOOP-16159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531434#comment-17531434
]
John Zhuge edited comment on HADOOP-16159 at 5/3/22 11:33 PM:
--------------------------------------------------------------
Here is the deadlock in SPARK-39094:
thread-1
{noformat}
t0: URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory(conf))
when conf is not initialized. i.e. `properties` is null
t1: conf.get(...) ->
conf.getProps() acquires Configuration object ->
conf.loadResources ->
any code that acquires some sort of JAVA lock, e.g.
`URL.streamHandlerLock`{noformat}
thread-2
{noformat}
t2: random Java code e.g. JceSecurity static initializer ->
new URL(): acquire the same URL.streamHandlerLock ->
URL.getURLStreamHandler ->
FsUrlStreamHandlerFactory.createURLStreamHandler ->
FileSystem.getFileSystemClass ->
Configuration.getClass -> ...
Configuration.getProps acquires Configuration object{noformat}
was (Author: jzhuge):
thread-1
{noformat}
t0: URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory(conf))
when conf is not initialized. i.e. `properties` is null
t1: conf.get(...) ->
conf.getProps() acquires Configuration object ->
conf.loadResources ->
any code that acquires some sort of JAVA lock, e.g.
`URL.streamHandlerLock`{noformat}
thread-2
{noformat}
t2: random Java code e.g. JceSecurity static initializer ->
new URL(): acquire the same URL.streamHandlerLock ->
URL.getURLStreamHandler ->
FsUrlStreamHandlerFactory.createURLStreamHandler ->
FileSystem.getFileSystemClass ->
Configuration.getClass -> ...
Configuration.getProps acquires Configuration object{noformat}
> Deadlock when using FsUrlStreamHandlerFactory
> ---------------------------------------------
>
> Key: HADOOP-16159
> URL: https://issues.apache.org/jira/browse/HADOOP-16159
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.8.5, 3.1.2
> Reporter: Ajith S
> Priority: Critical
>
> URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory())
> This induces the thread lock
> Thread 1 : Does load class which will do below
> - waiting to lock <0x00000005c0c1e5e0> (a
> org.apache.hadoop.conf.Configuration)
> at
> org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:684)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1088)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1145)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2363)
> at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2840)
> at
> org.apache.hadoop.fs.FsUrlStreamHandlerFactory.createURLStreamHandler(FsUrlStreamHandlerFactory.java:74)
> at java.net.URL.getURLStreamHandler(URL.java:1142)
> at java.net.URL.<init>(URL.java:420)
> at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:812)
> at sun.misc.URLClassPath$JarLoader$3.run(URLClassPath.java:1094)
> at sun.misc.URLClassPath$JarLoader$3.run(URLClassPath.java:1091)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1090)
> at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1050)
> at sun.misc.URLClassPath.getResource(URLClassPath.java:239)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:365)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> - locked <0x00000005b7991168> (a org.apache.spark.util.MutableURLClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> Thread 2 : Create new URL
> - waiting to lock <0x00000005b7991168> (a
> org.apache.spark.util.MutableURLClassLoader)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.apache.xerces.parsers.ObjectFactory.findProviderClass(Unknown Source)
> at org.apache.xerces.parsers.ObjectFactory.newInstance(Unknown Source)
> at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
> at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source)
> at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
> at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
> at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
> at
> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
> Source)
> at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2737)
> at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2696)
> at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2579)
> - locked <0x00000005c0c1e5e0> (a org.apache.hadoop.conf.Configuration)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:1091)
> at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1145)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2363)
> at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2840)
> at
> org.apache.hadoop.fs.FsUrlStreamHandlerFactory.createURLStreamHandler(FsUrlStreamHandlerFactory.java:74)
> at java.net.URL.getURLStreamHandler(URL.java:1142)
> at java.net.URL.<init>(URL.java:599)
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
