You may have to implement a similar solution to pyarrow, where you pass the Kerberos cache for the user.
Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Udi Meiri <[email protected]> Sent: Thursday, February 4, 2021 5:53 pm To: user Subject: Re: Python SDK and Kerberos Thank you, this is useful. The HDFS client library that Beam uses supports Kerberos authentication [1], but it would require adding a flag to Beam to use it. I don't have experience with Kerberos, but it looks like running Kinit might be enough: "HTTPKerberosAuth normally uses the default principal (ie, the user for whom you last ran kinit or kswitch, or an SSO credential if applicable)". [2] I've opened a feature request with more details: https://issues.apache.org/jira/browse/BEAM-11750 Please let me know if you are interested in contributing and/or testing. [1] https://hdfscli.readthedocs.io/en/latest/api.html#module-hdfs.ext.kerberos (currently Beam uses the InsecureClient) [2] https://pypi.org/project/requests-kerberos/
