On Tue, 31 Aug 2021 12:11:46 GMT, wxiang
<[email protected]> wrote:
> Using jarIndex for Hibench, there is an unexpected behavior with the
> exception "Exception in thread "main"
> org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme
> "hdfs"".
>
> After investigating it, it is related to the usage of ServiceLoader with
> JarIndex.
> The below stack shows the issue with JDK11:
>
> getResource:1016, URLClassPath$JarLoader (jdk.internal.loader)
> getResource:937, URLClassPath$JarLoader (jdk.internal.loader)
> findResource:912, URLClassPath$JarLoader (jdk.internal.loader)
> next:341, URLClassPath$1 (jdk.internal.loader)
> hasMoreElements:351, URLClassPath$1 (jdk.internal.loader)
> hasNext:355, BuiltinClassLoader$1 (jdk.internal.loader)
> hasMoreElements:363, BuiltinClassLoader$1 (jdk.internal.loader)
> next:3032, CompoundEnumeration (java.lang)
> hasMoreElements:3041, CompoundEnumeration (java.lang)
> nextProviderClass:1203, ServiceLoader$LazyClassPathLookupIterator (java.util)
> hasNextService:1221, ServiceLoader$LazyClassPathLookupIterator (java.util)
> hasNext:1265, ServiceLoader$LazyClassPathLookupIterator (java.util)
> hasNext:1300, ServiceLoader$2 (java.util)
> hasNext:1385, ServiceLoader$3 (java.util)
>
> The below API tries to get all the resources with the same name.
>
> public Enumeration<URL> findResources(final String name,
> final boolean check)
> ```
> After using JarIndex, URLClassPath.findResources only returns 1 URL.
> It is the same as the description in JDK-6957241.
>
> The issue still exists in JDK18.
>
> Root cause:
>
> public Enumeration<URL> findResources(final String name,
> final boolean check) {
> return new Enumeration<>() {
> private int index = 0;
> private URL url = null;
>
> private boolean next() {
> if (url != null) {
> return true;
> } else {
> Loader loader;
> while ((loader = getLoader(index++)) != null) {
> url = loader.findResource(name, check);
> if (url != null) {
> return true;
> }
> }
> return false;
> }
> }
> ...
> };
> }
>
> With the JarIndex, there is only one loader which is corresponding to the jar
> with the index due to the implementation in JarLoader.getResource(final
> String name, boolean check, Set<String> visited).
>
> Loaders corresponding to other jar packages will not appear in this while.
> So it only returns 1 instance.
>
> To solve the issue, I change the implementation "private boolean next()".
> If the loader has index, traverse the index and get all the resource from the
> loader.
I created a new JBS https://bugs.openjdk.java.net/browse/JDK-8273401, and PR:
https://github.com/openjdk/jdk/pull/5383.
The PR is to remove the JarIndex support in URLClassPath and move JarIndex into
the jdk.jartool module.
-------------
PR: https://git.openjdk.java.net/jdk/pull/5316
