Github user Rucongzhang commented on the issue:
https://github.com/apache/flink/pull/3776
@tzulitai , you are right. There are two problems in yarn cluster mode:
1ãwhen we use the keytabï¼we do not set the HDFS delegation token to
yarn container context, but yarn need.
2ãwhen we user keytab, and also get HDFS delegation token. The UGI
contains both, but UGI use token first to communication with HDFS. The default
expire time of HDFS delegation token is 7 days. Flink does not refresh the
token.
So, I resolve this problem by following solution:
1ãwe user keytab and also get HDFS delegation token. The token is set to
yarn container context. And the UGI only use keytab.
Maybe the best solution I think the AM need refresh the token like spark.
Maybe we can create a FILP to do this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
