[
https://issues.apache.org/jira/browse/HADOOP-7549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416467#comment-15416467
]
Liu Shaohui commented on HADOOP-7549:
-------------------------------------
[~jxchen] [~tucu00]
I encountered same problem when calling CompressionCodecFactory from C++ code
through JNI.
{code}
java.util.ServiceConfigurationError:
org.apache.hadoop.io.compress.CompressionCodec: Provider
org.apache.hadoop.io.compress.BZip2Codec not found
at java.util.ServiceLoader.fail(ServiceLoader.java:231)
at java.util.ServiceLoader.access$300(ServiceLoader.java:181)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:116)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
{code}
The reason is that calling vm->AttachCurrentThread() in jni_helper.c of libhdfs
sets the context class loader to the “bootstrap loader”, which is insufficient
for ServiceLoader to function properly.
See:
https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/invocation.html#AttachCurrentThread.
In current implements of libhdfs, the first calling getJNIEnv will create a
java vm which doesn't change the context class loader. But later threads
calling getJNIEnv will attach thread to the java vm which will change the
context class loader. So It works when call FileSystem.loadFileSystems() in
the same thread where JVM is initialized.
A suggest from John Lilley is to setting the system class loader to be the
thread's context class loader:
{code}
java.lang.Thread.currentThread().setContextClassLoader(java.lang.ClassLoader.getSystemClassLoader());
{code}
ps: You must translate it to the JNI callings.
See : [Mail link |
http://mail-archives.apache.org/mod_mbox/hadoop-user/201409.mbox/%3cfc3d75136d434956ae9cc92621357...@dm2pr0701mb729.namprd07.prod.outlook.com%3E]
> Use JDK ServiceLoader mechanism to find FileSystem implementations
> ------------------------------------------------------------------
>
> Key: HADOOP-7549
> URL: https://issues.apache.org/jira/browse/HADOOP-7549
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.23.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 2.0.0-alpha
>
> Attachments: HADOOP-7549v1.patch, HADOOP-7549v2.patch,
> HADOOP-7549v3.patch
>
>
> Currently configuring FileSystem implementations must be done by declaring
> the FileSystem class in the Hadoop configuration files (core-default.xml,
> ...).
> Using JDK ServiceLoader mechanism this configuration step can be avoided.
> Adding the JAR file with the additional FileSystem implementation would
> suffice.
> This is similar to what is being proposed for compression codecs
> (HADOOP-7350).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
