marton-bod commented on a change in pull request #1781:
URL: https://github.com/apache/iceberg/pull/1781#discussion_r527791291
##########
File path: build.gradle
##########
@@ -607,6 +607,8 @@ project(':iceberg-hive-runtime') {
if (jdkVersion == '8') {
compile project(':iceberg-hive3')
}
+ // including thrift so that the Tez AM can communicate with the HMS when
using Hive catalog
+ compile("org.apache.thrift:libfb303")
Review comment:
Thanks @shardulm94 , very good points.
Tez is responsible for its own classpath, Hive does not provide dependencies
to the Tez AM.
Right now pre-iceberg, the Tez AM does not communicate with HMS, therefore
has not needed the thrift library on its classpath before, only with Iceberg
tables now. Hence, the inclusion in the runtime jar.
I'm not sure it will work and find the classes properly if we use shading -
this will need to be tested. In theory I think it could be okay not to shade in
this specific case, since we are putting the same thrift version on the
classpath as the one already used by Hive. Using Hive with a thrift version
other than 0.9.3 would likely break Hive so having different thrift versions on
the classpath is not a hugely realistic cause for breakage I believe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
