[ https://issues.apache.org/jira/browse/HADOOP-16159 ]
John Zhuge deleted comment on HADOOP-16159: ------------------------------------- was (Author: jzhuge): Can we fix it by replacing `conf` with `this.conf` in this constructor? {code:java} public FsUrlStreamHandlerFactory(Configuration conf) { this.conf = new Configuration(conf); // force init of FileSystem code to avoid HADOOP-9041 try { FileSystem.getFileSystemClass("file", conf); <<<<=== replace with this.conf } catch (IOException io) { throw new RuntimeException(io); } this.handler = new FsUrlStreamHandler(this.conf); } {code} In this way, no other thread can lock this private Configuration object. > Deadlock when using FsUrlStreamHandlerFactory > --------------------------------------------- > > Key: HADOOP-16159 > URL: https://issues.apache.org/jira/browse/HADOOP-16159 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 2.8.5, 3.1.2 > Reporter: Ajith S > Priority: Critical > > URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory()) > This induces the thread lock > Thread 1 : Does load class which will do below > - waiting to lock <0x00000005c0c1e5e0> (a > org.apache.hadoop.conf.Configuration) > at > org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:684) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1088) > at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1145) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2363) > at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2840) > at > org.apache.hadoop.fs.FsUrlStreamHandlerFactory.createURLStreamHandler(FsUrlStreamHandlerFactory.java:74) > at java.net.URL.getURLStreamHandler(URL.java:1142) > at java.net.URL.<init>(URL.java:420) > at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:812) > at sun.misc.URLClassPath$JarLoader$3.run(URLClassPath.java:1094) > at sun.misc.URLClassPath$JarLoader$3.run(URLClassPath.java:1091) > at java.security.AccessController.doPrivileged(Native Method) > at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1090) > at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:1050) > at sun.misc.URLClassPath.getResource(URLClassPath.java:239) > at java.net.URLClassLoader$1.run(URLClassLoader.java:365) > at java.net.URLClassLoader$1.run(URLClassLoader.java:362) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:361) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > - locked <0x00000005b7991168> (a org.apache.spark.util.MutableURLClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:348) > Thread 2 : Create new URL > - waiting to lock <0x00000005b7991168> (a > org.apache.spark.util.MutableURLClassLoader) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at org.apache.xerces.parsers.ObjectFactory.findProviderClass(Unknown Source) > at org.apache.xerces.parsers.ObjectFactory.newInstance(Unknown Source) > at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source) > at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown Source) > at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source) > at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source) > at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source) > at > org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown > Source) > at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2737) > at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2696) > at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2579) > - locked <0x00000005c0c1e5e0> (a org.apache.hadoop.conf.Configuration) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1091) > at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1145) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2363) > at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2840) > at > org.apache.hadoop.fs.FsUrlStreamHandlerFactory.createURLStreamHandler(FsUrlStreamHandlerFactory.java:74) > at java.net.URL.getURLStreamHandler(URL.java:1142) > at java.net.URL.<init>(URL.java:599) > -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org