[
https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13527317#comment-13527317
]
Adrian Muraru commented on HBASE-7205:
--------------------------------------
Right, classloading java is tough, I agree
Regarding your comments:
{quote}
The reason why the above assertion failed was that the classloader for
jarFileOnHDFS2 was removed from classLoadersCache in the middle of the test
because of attempt of loading cpNameInvalid class.
{quote}
True, *cpNameInvalid* fails to load (no such class in cp jar) however the same
jar (i.e. its associated classloader) manages to successfully loads *another
coprocessor: cpName2*
Take a closer look on how the *test* table is created in
{{TestClassLoading#testClassLoadingFromHDFS}}:
{code:java}
htd.addFamily(new HColumnDescriptor("test"));
// without configuration values
htd.setValue("COPROCESSOR$1", jarFileOnHDFS1.toString() + "|" + cpName1 +
"|" + Coprocessor.PRIORITY_USER);
// with configuration values
htd.setValue("COPROCESSOR$2", jarFileOnHDFS2.toString() + "|" + cpName2 +
"|" + Coprocessor.PRIORITY_USER + "|k1=v1,k2=v2,k3=v3");
// invalid class name (should fail to load this class)
htd.setValue("COPROCESSOR$3", jarFileOnHDFS2.toString() + "|" +
cpNameInvalid + "|" + Coprocessor.PRIORITY_USER);
{code}
See, the same jar file {{jarFileOnHDFS2}} is used to load two different
coprocessor classes (one is successfully loaded, the other not).
What should we do in this case?
My take is to keep the classloader in cache and allow other regions to re-use
it.
That's the reason I removed classloaderCache.remove() and I strongly go for it.
Now, the fundamental question :
*Should we silently ignore failures in CP loading (excepting the warn message
in log) ?*
I think we should be more restrictive, propagate the failures upstream to the
table handler and fail to bring the HRegion online in this case.
What do you think?
{quote}
I think the above assertion places extra limit on how CoprocessorHost.load()
handles ClassNotFoundException. It assumes that the classloader corresponding
to attempt of loading invalid classname (more strictly, classname and jar file
mismatch) would be retained in cache.
{quote}
No, that's not true: The assertion is checking that *all region
active-classloaders (i.e. those that managed to successfully load at least one
CP) are all cached*
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, 7205-v3.txt, 7205-v4.txt, 7205-v5.txt,
> 7205-v6.txt, 7205-v7.txt, 7205-v8.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the
> coprocessor classes and a new instance of this CL is created for each single
> HRegion opened. This leads to OOME-PermGen when the number of regions go
> above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however
> we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
