Hi all, I've a proposal to enhance the current mechanism to distribute delegation tokens and other secure tokens.
The summary is that the current mechanism is gated behind Kerberos even though the actual distribution does not require Kerberos except where the tokens themselves are Kerberos tokens. Cloud environments may not have a Kerberos setup and this creates an unnecessary setup step that users may have to perform. The current implementation of KafkaDelegationTokenProvider illustrates this. The implementation does not require Kerberos, yet it has to pass the Kerberos gates. The proposal then is to allow a second path that does not require the Kerberos gates unless the provider indicates that it be required. the design has minimal change to the existing code and is fully backward compatible. The proposal and corresponding JIRA are in [1], [2] I'd greatly appreciate it if committers can take some time to review and provide feedback Thanks Parth [1] https://docs.google.com/document/d/1PPqAoJAj48MdjMJNc7DlytXi745z-imFpVaFDnt18Xg/edit?tab=t.0#heading=h.21tncge82jbl [2] https://issues.apache.org/jira/browse/SPARK-57252
