[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253167#comment-15253167
]
Sangjin Lee commented on HBASE-15686:
-------------------------------------
[~jerryhe], yes, I see that the 2.0 list is pretty small. That said, if my
experience is any guide, getting this "system/exempt" classes list right is
trickier than it appears. This is the current default list in hadoop:
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/resources/org.apache.hadoop.application-classloader.properties
Even getting the JDK packages right is not so easy as they are not explicitly
defined in terms of packages. But the penalty for not getting it right is
pretty severe, as classes just won't get loaded. So over time, I found myself
repeatedly modifying and improving the list. I suspect a similar thing may
happen on your end too.
This JIRA is another example: HADOOP-11831. {{org.w3c.dom}} is a JDK package,
and as such should be in the system classes. However, it turns out a
third-party library (xerces) has a class called
{{org.w3c.dom.ElementTraversal}} that the JDK does not have. The moment that
class is used under these classloaders, it won't work. So users' customizing
this list proves to be a handy way to address those interesting edge cases.
Most custom classloader solutions (OSGi, servlet engines, ...) have a way to
modify/customize the system classes/packages list for that reason. Just my 2
cents.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> ---------------------------------------------------------------------------------
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 1.0.1
> Reporter: Sangjin Lee
> Attachments: 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
