JackWangCS edited a comment on pull request #14841: URL: https://github.com/apache/flink/pull/14841#issuecomment-893501105
Hi @mbalassi and @gaborgsomogyi , thanks for your reply. You are right, this implementation now just introduces new APIs but does not address some main concerns. The initial idea for this PR is that we want to support Hive delegation token in a short-live sql-client, so it does not include features such as renewing tokens, propagating DTs to workers, etc and it's not an end-to-end solution, I will close this PR. If we want to refactor the whole DT handling in the Flink, it will be a huge job. For your questions, I only have some basic ideas currently: * I think it's better to re-obtain the tokens than renew the tokens. There are a few components now supporting the renew. Re-obtaining will be enough and simple. * For propagating DT to workers, it's better to implement a RPC to do this and don't depend on any external distributed storage such as HDFS. Also, it will be more complicated since Flink supports many deployment modes such as per-job, session and application modes. The `HadoopDelegationTokenManager` can be run at client as well as JobManager side and have different lifecyle in different deployment modes. @gaborgsomogyi , great to hear that you already have a FLIP, I am happy to contribute and collaborate with you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
