[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259104#comment-15259104
]
Enis Soztutar commented on HBASE-15686:
---
if that is the case, we should name it included instead. Ted, please also
rename corresponding method names, etc.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258984#comment-15258984
]
Sangjin Lee commented on HBASE-15686:
-
The meaning of "excluded.classes" is these classes will be excluded from
exemption so that this is bit of a double negative. These classes will actually
be loaded by the coprocessor classloader. Could that be a source of confusion?
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258789#comment-15258789
]
Enis Soztutar commented on HBASE-15686:
---
The patch for this adds a configuration option for excluded classes. If that is
the target goal, it should be backportable. There is no BC implications and it
is a low risk patch.
We should rename the config option though, instead of "coproc.exclusion", it
should be something like {{hbase.coprocessor.classloader.excluded.classes}}
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258350#comment-15258350
]
Sangjin Lee commented on HBASE-15686:
-
My apologies for mixing discussions on HBASE-13706 and this JIRA itself.
I'd like to re-word this JIRA to discuss supporting an override mechanism for
the exempt classes, and move the discussion for backporting HBASE-13706 to that
JIRA. I believe Nick's and Enis's comments apply to HBASE-13706. Implementing
the override mechanism can be done in a later version.
Does that sound reasonable?
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257622#comment-15257622
]
Nick Dimiduk commented on HBASE-15686:
--
I'd prefer to not go changing this for 1.1 if we can avoid it. What's the
pressing need that prevents picking this up in 1.2? It's out already.
If the need is dire for 1.1.x, please take the branch-1 patch on HBASE-13706
for a spin, verifies it's working for you. By my read, applying that patch will
allow for a less strict set of coprocs to load, not more strick, so should be
acceptable by BC rules. Please also do a triple-check that it's not breaking
Phoenix, especially consider the classes from the phoenix jar that come in the
{{org.apache.hadoop.hbase}} package.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254342#comment-15254342
]
Enis Soztutar commented on HBASE-15686:
---
bq. Would it be a possibility to backport HBASE-13706 to the 1.0.x releases?
I'm not completely familiar with the currently supported release lines, but it
would be nice if we can get this in a 1.0.x line. Thanks!
There are no more scheduled 1.0.x releases unfortunately. We can make a case
for HBASE-13706 to be included in 1.1.5, which would be the earliest release.
cc [~ndimiduk]. Not sure the backwards compatibility implications.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253391#comment-15253391
]
Anoop Sam John commented on HBASE-15686:
Ping [~enis]
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
> Attachments: 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253381#comment-15253381
]
Sangjin Lee commented on HBASE-15686:
-
Would it be a possibility to backport HBASE-13706 to the 1.0.x releases? I'm
not completely familiar with the currently supported release lines, but it
would be nice if we can get this in a 1.0.x line. Thanks!
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
> Attachments: 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253331#comment-15253331
]
Jerry He commented on HBASE-15686:
--
Hi, [~sjlee0]
Thanks for the explanation.
I am neutral on creating a configurable co-processor class exempt mechanism.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
> Attachments: 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253167#comment-15253167
]
Sangjin Lee commented on HBASE-15686:
-
[~jerryhe], yes, I see that the 2.0 list is pretty small. That said, if my
experience is any guide, getting this "system/exempt" classes list right is
trickier than it appears. This is the current default list in hadoop:
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/resources/org.apache.hadoop.application-classloader.properties
Even getting the JDK packages right is not so easy as they are not explicitly
defined in terms of packages. But the penalty for not getting it right is
pretty severe, as classes just won't get loaded. So over time, I found myself
repeatedly modifying and improving the list. I suspect a similar thing may
happen on your end too.
This JIRA is another example: HADOOP-11831. {{org.w3c.dom}} is a JDK package,
and as such should be in the system classes. However, it turns out a
third-party library (xerces) has a class called
{{org.w3c.dom.ElementTraversal}} that the JDK does not have. The moment that
class is used under these classloaders, it won't work. So users' customizing
this list proves to be a handy way to address those interesting edge cases.
Most custom classloader solutions (OSGi, servlet engines, ...) have a way to
modify/customize the system classes/packages list for that reason. Just my 2
cents.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
> Attachments: 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253150#comment-15253150
]
Jerry He commented on HBASE-15686:
--
Hi, [~sjlee0]
bq. allow configuring the exempt classes
This is interesting.
But HBase already tries to put 'minimum' in the exempt list (classes that need
to be loaded from system classpath), meaning anything less would cause problem
for HBase.
For example, there is class casting between the host and co-processor.
In the 2.0 version, the only thing meaningful in the exempt list is native
hbase.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253102#comment-15253102
]
Sangjin Lee commented on HBASE-15686:
-
Either that (if that can be handled cleanly) or even an overloaded method...
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253090#comment-15253090
]
Ted Yu commented on HBASE-15686:
{code}
public HTableDescriptor addCoprocessor(String className, Path jarFilePath,
int priority, final Map kvs)
{code}
So you are suggesting that exclusion list be passed as a key-value pair through
kvs ?
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253070#comment-15253070
]
Sangjin Lee commented on HBASE-15686:
-
Thanks for the pointer [~jerryhe]. It looks like the earliest release that has
the fix for HBASE-13706 is 1.2.0? Is there a possibility of getting this into
the 1.0.x line?
Also, although HBASE-13706 should address our specific issue, I think it would
still be a good feature enhancement to allow configuring the exempt classes.
Otherwise, any modification to the exempt list would require a HBase release.
The changes can be defined through configuration or even as an additional
argument to {{addCoprocessor()}} or something to that effect...
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252934#comment-15252934
]
Ted Yu commented on HBASE-15686:
Sangjin:
Looks like after upgrading hbase, you should be good.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252882#comment-15252882
]
Jerry He commented on HBASE-15686:
--
Hopefully HBASE-13706 would help you.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252853#comment-15252853
]
Sangjin Lee commented on HBASE-15686:
-
In case of hadoop's {{ApplicationClassLoader}}, the main way to define/override
it is via configuration (although there are other ways).
It allows "-" as a way to exclude certain packages or classes from the exempt
list. {{ApplicationClassLoader}}'s constructor takes the exempt classes
("system classes") as an argument. Thus, I suppose any way you can pass in the
modified exempt classes would work, including programmatically.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15252590#comment-15252590
]
Ted Yu commented on HBASE-15686:
bq. ability to exclude certain classes
How would such classes be defined ?
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
