Thank you, this is useful. The HDFS client library that Beam uses supports Kerberos authentication [1], but it would require adding a flag to Beam to use it. I don't have experience with Kerberos, but it looks like running Kinit might be enough: "HTTPKerberosAuth normally uses the default principal (ie, the user for whom you last ran kinit or kswitch, or an SSO credential if applicable)". [2]
I've opened a feature request with more details: https://issues.apache.org/jira/browse/BEAM-11750 Please let me know if you are interested in contributing and/or testing. [1] https://hdfscli.readthedocs.io/en/latest/api.html#module-hdfs.ext.kerberos (currently Beam uses the InsecureClient) [2] https://pypi.org/project/requests-kerberos/
smime.p7s
Description: S/MIME Cryptographic Signature
