Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/9232#discussion_r42939231
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnSparkHadoopUtil.scala ---
@@ -142,6 +145,104 @@ class YarnSparkHadoopUtil extends SparkHadoopUtil {
val containerIdString =
System.getenv(ApplicationConstants.Environment.CONTAINER_ID.name())
ConverterUtils.toContainerId(containerIdString)
}
+
+ /**
+ * Obtains token for the Hive metastore, using the current user as the
principal.
+ * Some exceptions are caught and downgraded to a log message.
+ * @param conf hadoop configuration; the Hive configuration will be
based on this
+ * @return a token, or `None` if there's no need for a token (no
metastore URI or principal
+ * in the config), or if a binding exception was caught and
downgraded.
+ */
+ def obtainTokenForHiveMetastore(conf: Configuration):
Option[Token[DelegationTokenIdentifier]] = {
+ try {
+ obtainTokenForHiveMetastoreInner(conf,
UserGroupInformation.getCurrentUser().getUserName)
+ } catch {
+ case e: NoSuchMethodException =>
+ logInfo("Hive Method not found", e)
+ None
+ case e: ClassNotFoundException =>
--- End diff --
I kinda feel that everything but this exception should be propagated.
That's really the only valid error you might expect; someone running Spark
without Hive classes, and even that might change (I think there's a bug
tracking removal of the hive profile from the maven build).
Every other error means that either you have the wrong version of Hive in
the classpath (user error) or that your configuration says you need delegation
tokens, but you can't get them for some reason.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
