LuciferYang commented on a change in pull request #29783:
URL: https://github.com/apache/spark/pull/29783#discussion_r490089526
##########
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/HiveMetastoreLazyInitializationSuite.scala
##########
@@ -31,6 +31,7 @@ class HiveMetastoreLazyInitializationSuite extends
SparkFunSuite {
.config("spark.hadoop.hive.metastore.uris", "thrift://127.0.0.1:11111")
.getOrCreate()
val originalLevel = org.apache.log4j.Logger.getRootLogger().getLevel
+ val originalClassLoader = Thread.currentThread().getContextClassLoader
Review comment:
cc @srowen I think this change is very Interesting,
`HiveThriftServer2ListenerSuite ` will ABORTED without this change
```
HiveThriftServer2ListenerSuite:
*** RUN ABORTED ***
java.lang.LinkageError: loader constraint violation: loader (instance of
net/bytebuddy/dynamic/loading/MultipleParentClassLoader) previously initiated
loading for a different type with name
"org/apache/hive/service/ServiceStateChangeListener"
```
The Context Class Loader is `NonClosableMutableURLClassLoader` in
`HiveMetastoreLazyInitializationSuite` running process and propagate to
`HiveThriftServer2ListenerSuite` , then
https://github.com/apache/spark/blob/92b75dc260eb43d906a425f9f9d8d63b78c48cee/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala#L172-L175
will use `MultipleParentClassLoader(NonClosableMutableURLClassLoader and
AppClassLoader)` to mock `HiveThriftServer2` instance in Scala 2.13, but it's
`AppClassLoader` only in Scala 2.12.
Then I do some experiments:
- Ignore `HiveMetastoreLazyInitializationSuite` all tests passed
- Run `HiveThriftServer2ListenerSuite` only will passed
- Copy test case `HiveMetastoreLazyInitializationSuite` to
`HiveThriftServer2ListenerSuite` then run `HiveThriftServer2ListenerSuite`
only, it aborted in both Scala 2.12 and Scala 2.13.
I guess that `HiveMetastoreLazyInitializationSuite` and
`HiveThriftServer2ListenerSuite` run in same thread in Scala 2.13 but different
thread in Scala 2.12, but I don't know why.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
