LawrenceHunter commented on issue #2080: URL: https://github.com/apache/buildstream/issues/2080#issuecomment-3406771611
I continued investigating this and want to summarise my opinion: The primary issue is a result of the use of bubblewrap (bwrap) within Buildstream. bwrap is used on build steps to sandbox the build environment and to achieve the sandboxing it uses kernel namespaces. Using namespaces is pretty much a non-issue when working on a traditional Linux environment where a privileged container can be run. When running Buildstream on macOS it is not possible to do so natively due to buildbox-casd which cannot be built for macOS so some degree of virtualisation is required. A full Linux VM like QEMU or parallels essentially reverts back to a traditional Linux environment so is not an issue; however, the situation is more complex when trying to use containerisation tools such as Docker or Podman. Both of these containerisation tools use a virtual machine in the background to behave as the kernel on non-Linux platforms (e.g. Windows and Mac). This virtual machine is accessible and for Podman is a fedora VM with namespacing. However, when I tried to run Buildstream inside a Podman container no combination of `--userns`, `--user`, `--cap-add`, `--privileged`, and rootful or rootless machines would allow bwrap to work. The combinations produced two results 1) bwrap exits with an error that the kernel does not support namespaces, 2) bwrap exits that it's capabilities are incorrect and that setuid is not correct. The latter is linked to https://github.com/containers/bubblewrap/issues/380 but the suggestion to check `CapAmb` did not make a difference for me. I am unsure exactly why despite the Podman VM appearing to support namespacing and capabilities as if it was a normal Linux kernel it was not possible to correctly configure for bwrap to run. I suspect that it is linked to Apple's container framework which is a clear difference between Linux and macOS Podman/Docker; therefore, these is an issue here whether that is incorrect configuration, a bug, or a hard limit. I do not have any more time currently to investigate this problem but my next steps would be to investigate the fedora container used by Podman machine for it's configuration and if swapping to a similarly modified image from another distro or modification to the fedora image yielded success. Otherwise, I would begin to dive Podman code to understand what happens with namespacing on macOS as unfortunately the internet and docs are fairly sparse on the specifics of the Podman VM interface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
