GitHub user WangTaoTheTonic opened a pull request:
https://github.com/apache/spark/pull/8048
[SPARK-8676][SQL]Lazy start event logger in sql application to avoid TGT
expiring in lâ¦
â¦ong connection
Now in Thrift Server/Spark SQL, it will login first in `Client.scala`, then
open the file stream in event logger. After that the login action will be
executed twice again, one of them is in `SparkSQLCLIService`, and another is in
`ThriftHttpCLIService/ThriftBinaryCLIService`.
If we open a long-connected file stream to HDFS between two login, the
UserGroupInformation that RPC took will be refreshed by second login. After the
TGT expires, it will cause "No invalid credentials" excepiton.
So in SQL application, we will start event logger only after the
HiveServer2 being launched.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/WangTaoTheTonic/spark SPARK-8676
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8048.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8048
----
commit 8ca90467cb31da40da27a30e4dae6c73ada862d9
Author: WangTaoTheTonic <[email protected]>
Date: 2015-08-08T06:54:25Z
Lazy start event logger in sql application to avoid TGT expiring in long
connection
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
