Hi
This is a write up of some notes I've written in Slack. Since a recent
Github Actions Runner image update, every CI run we do under the Hadoop
3 build profile dies on Hadoop impersonation tests with the following
error [1].
|Error:
TestImpersonationMetadata.setup:72->BaseTestImpersonation.startMiniDfsCluster:84->BaseTestImpersonation.startMiniDfsCluster:112
» IO Running in secure mode, but config doesn't have a keytab |
|Error:
TestImpersonationMetadata.setup:72->BaseTestImpersonation.startMiniDfsCluster:84->BaseTestImpersonation.startMiniDfsCluster:112
» IO Running in secure mode, but config doesn't have a keytab |
|...
|Note that these errors do not show up under the Hadoop 2 build profile
[2]. ||We actually had this exact problem in the CI a few months ago but
it was resolved on that occasion by a subsequent Github Runner image
update [3]. At that time we could also dodge the problem by downgrading
our Runner image from ubuntu-latest to ubuntu-20.04, but that little
trick does not work today. Upgrading Hadoop to the latest release,
3.3.6, also doesn't help here [4].
This issue hasn't ever been reproduced locally where the Hadoop Mini DFS
cluster remains in "simple" auth mode with the result that no keytab
file is sought and no test errors occur. One way to debug locally would
be to create a Github Runner image from source and run it in a VM,
container or chroot [5]. This looks unappetising to me so far, mainly
because of the needed tools that I don't have or want, but it might
prove to be the only way to stop shooting in the dark.
Regards
James
1.
https://github.com/apache/drill/actions/runs/5845539423/job/15849744146#step:4:15538
2. https://github.com/apache/drill/actions/runs/5834759769/job/15824986427
3. https://github.com/actions/runner-images/issues/7340
4. https://github.com/apache/drill/actions/runs/5834722006
5.
https://github.com/actions/runner-images/blob/main/docs/create-image-and-azure-resources.md
