[
https://issues.apache.org/jira/browse/HIVE-11878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033412#comment-15033412
]
Jason Dere commented on HIVE-11878:
-----------------------------------
The following test failures look like real errors and need to be fixed:
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_classloader_dynamic_dependency_resolution
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters
TestHiveAuthorizerShowFilters is failing with the following error in
hive-unit/target/tmp/log/hive.log:
{noformat}
2015-12-01T01:05:03,174 ERROR [main]: ql.Driver
(SessionState.java:printError(1010)) - FAILED: MockitoException
Mockito cannot mock this class: class
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters$MockedHiveAuthorizerFactory$1AuthorizerWithFilterCmdImpl
Mockito can only mock visible & non-final classes.
If you're not sure why you're getting this error, please report to the mailing
list.
org.mockito.exceptions.base.MockitoException:
Mockito cannot mock this class: class
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters$MockedHiveAuthorizerFactory$1AuthorizerWithFilterCmdImpl
Mockito can only mock visible & non-final classes.
If you're not sure why you're getting this error, please report to the mailing
list.
at
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters$MockedHiveAuthorizerFactory.createHiveAuthorizer(TestHiveAuthorizerShowFilters.java:98)
at
org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:773)
at
org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1436)
at
org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1034)
at
org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:179)
at org.apache.hadoop.hive.ql.metadata.Table.<init>(Table.java:121)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.addDbAndTabToOutputs(SemanticAnalyzer.java:11016)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10871)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:9960)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10060)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:222)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:462)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:317)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1227)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1276)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1152)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1140)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.runCmd(TestHiveAuthorizerShowFilters.java:265)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.beforeTest(TestHiveAuthorizerShowFilters.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: org.mockito.cglib.core.CodeGenerationException:
java.lang.reflect.InvocationTargetException-->null
at
org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:238)
at org.mockito.cglib.proxy.Enhancer.createHelper(Enhancer.java:378)
at org.mockito.cglib.proxy.Enhancer.createClass(Enhancer.java:318)
at
org.mockito.internal.creation.jmock.ClassImposterizer.createProxyClass(ClassImposterizer.java:110)
at
org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:62)
at
org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:56)
at
org.mockito.internal.creation.CglibMockMaker.createMock(CglibMockMaker.java:23)
at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:26)
at org.mockito.internal.MockitoCore.mock(MockitoCore.java:51)
at org.mockito.Mockito.mock(Mockito.java:1243)
... 36 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.mockito.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:385)
at
org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:220)
... 45 more
Caused by: java.lang.IllegalAccessError: class
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters$MockedHiveAuthorizerFactory$1AuthorizerWithFilterCmdImpl$$EnhancerByMockitoWithCGLIB$$53d16709
cannot access its superclass
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters$MockedHiveAuthorizerFactory$1AuthorizerWithFilterCmdImpl
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
... 51 more
{noformat}
> ClassNotFoundException can possibly occur if multiple jars are registered
> one at a time in Hive
> ------------------------------------------------------------------------------------------------
>
> Key: HIVE-11878
> URL: https://issues.apache.org/jira/browse/HIVE-11878
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 1.2.1
> Reporter: Ratandeep Ratti
> Assignee: Ratandeep Ratti
> Labels: URLClassLoader
> Attachments: HIVE-11878 ClassLoader Issues when Registering
> Jars.pptx, HIVE-11878.2.patch, HIVE-11878.3.patch, HIVE-11878.patch,
> HIVE-11878_approach3.patch,
> HIVE-11878_approach3_per_session_clasloader.patch,
> HIVE-11878_approach3_with_review_comments.patch,
> HIVE-11878_approach3_with_review_comments1.patch, HIVE-11878_qtest.patch
>
>
> When we register a jar on the Hive console. Hive creates a fresh URL
> classloader which includes the path of the current jar to be registered and
> all the jar paths of the parent classloader. The parent classlaoder is the
> current ThreadContextClassLoader. Once the URLClassloader is created Hive
> sets that as the current ThreadContextClassloader.
> So if we register multiple jars in Hive, there will be multiple
> URLClassLoaders created, each classloader including the jars from its parent
> and the one extra jar to be registered. The last URLClassLoader created will
> end up as the current ThreadContextClassLoader. (See details:
> org.apache.hadoop.hive.ql.exec.Utilities#addToClassPath)
> Now here's an example in which the above strategy can lead to a CNF exception.
> We register 2 jars *j1* and *j2* in Hive console. *j1* contains the UDF class
> *c1* and internally relies on class *c2* in jar *j2*. We register *j1* first,
> the URLClassLoader *u1* is created and also set as the
> ThreadContextClassLoader. We register *j2* next, the new URLClassLoader
> created will be *u2* with *u1* as parent and *u2* becomes the new
> ThreadContextClassLoader. Note *u2* includes paths to both jars *j1* and *j2*
> whereas *u1* only has paths to *j1* (For details see:
> org.apache.hadoop.hive.ql.exec.Utilities#addToClassPath).
> Now when we register class *c1* under a temporary function in Hive, we load
> the class using {code} class.forName("c1", true,
> Thread.currentThread().getContextClassLoader()) {code} . The
> currentThreadContext class-loader is *u2*, and it has the path to the class
> *c1*, but note that Class-loaders work by delegating to parent class-loader
> first. In this case class *c1* will be found and *defined* by class-loader
> *u1*.
> Now *c1* from jar *j1* has *u1* as its class-loader. If a method (say
> initialize) is called in *c1*, which references the class *c2*, *c2* will not
> be found since the class-loader used to search for *c2* will be *u1* (Since
> the caller's class-loader is used to load a class)
> I've added a qtest to explain the problem. Please see the attached patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
