Github user chesterxgchen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42810639
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -337,5 +339,82 @@ object YarnSparkHadoopUtil {
conf.getInt("spark.executor.instances", targetNumExecutors)
}
}
+
+ /**
+ * Obtains token for the Hive metastore. Exceptions are caught and logged
+ */
+ def obtainTokenForHiveMetastore(
+ sparkConf: SparkConf,
+ conf: Configuration): Option[Token[DelegationTokenIdentifier]] = {
+ try {
+ obtainTokenForHiveMetastoreInner(sparkConf, conf,
+ UserGroupInformation.getCurrentUser().getUserName)
+ } catch {
+ case e: java.lang.NoSuchMethodException =>
+ logInfo("Hive Method not found " + e)
+ None
+ case e: java.lang.ClassNotFoundException =>
+ logInfo("Hive Class not found " + e)
+ None
+ case e: java.lang.NoClassDefFoundError =>
+ logDebug("Hive Class not found: " + e)
--- End diff --
This may not be class not found exception, but it could be an
initialization exception, where the HiveConf or Hadoop Conf is not available
during class contruction
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
