Vinod KC created SPARK-35747:
--------------------------------
Summary: Avoid printing full Exception stack trace, if HBase
service is not running in a secure cluster
Key: SPARK-35747
URL: https://issues.apache.org/jira/browse/SPARK-35747
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.1.2, 3.1.0, 3.0.0
Reporter: Vinod KC
In a secure Yarn cluster where HBase service is down, even if the spark
application is not using HBase, during the application submit stage,
HBaseDelegationTokenProvider prints full Exception Stack trace and it causes a
noisy warning.
{code:java}
WARN security.HBaseDelegationTokenProvider: Failed to get token from service
hbase
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.security.HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn(HBaseDelegationT
okenProvider.scala:93)
{code}
Also, Application submission taking more time as
`HBaseDelegationTokenProvider.obtainDelegationTokensWithHBaseConn` retries the
connection to HBase master multiple times before it gives up. This slows down
the application submission steps. Hence, if HBase is not used in the user
Application, it is better to suggest user to disable HBase Delegation Token
generation.
ie, spark.security.credentials.hbase.enabled=false
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
