[
https://issues.apache.org/jira/browse/SPARK-35747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-35747:
------------------------------------
Assignee: (was: Apache Spark)
> Avoid printing full Exception stack trace, if HBase service is not running in
> a secure cluster
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-35747
> URL: https://issues.apache.org/jira/browse/SPARK-35747
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0, 3.1.0, 3.1.2
> Reporter: Vinod KC
> Priority: Minor
>
> In a secure Yarn cluster where HBase service is down, even if the spark
> application is not using HBase, during the application submit stage,
> HBaseDelegationTokenProvider prints full Exception Stack trace and it causes
> a noisy warning.
>
> {code:java}
> WARN security.HBaseDelegationTokenProvider: Failed to get token from service
> hbase
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn(HBaseDelegationT
> okenProvider.scala:93)
> more than 100 line exception stack trace{code}
> Also, Application submission taking more time as
> `HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn` retries
> the connection to HBase master multiple times before it gives up. This slows
> down the application submission steps. Hence, if HBase is not used in the
> user Application, it is better to suggest user to disable HBase Delegation
> Token generation.
> ie, spark.security.credentials.hbase.enabled=false
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
