Re: Tool to build Debian packages not requiring root in containers ?
Hi Charles, * Charles Plessy [2024-05-08 07:27]: I want to leverage our cluster to automate as much of the rebuilds as I can, but could not find the right tool. I tried to run sbuild in a Singularity image and this failed. However, I do not need the whole power of engines like sbuild, as none of the packages involved require root priviledges to build. Have you tried the unshare backend for sbuild? It uses Linux namespaces instead of full-blown root privileges, and works really great for my regular packaging work. I have not tried running it inside a virtualization container, though. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Re: Tool to build Debian packages not requiring root in containers ?
Le Wed, May 08, 2024 at 08:02:41AM -0700, Otto Kekäläinen a écrit : > > I read the docs on how Singularity is able to pull Docker images of Debian > Sid and build on top of them, and run and exec just like Docker/Podman. > Unfortunately it has its own Containerfile format ( > https://docs.sylabs.io/guides/3.5/user-guide/quick_start.html#singularity-definition-files) > and the commands have their own syntax. I guess Debcraft could be extended > to support it, but that would require at least one Singularity user as > frequent contributor to test and develop Singularity-compatibility. > > The entire code base is shell code. Perhaps you want to take a look if it > looks hackable for you? Hi Otto, I looked at the code, and while it would be easy to replace the podman commands to run containers, I wonder if there isn't a major roadblock: The main use of Singularity containers is to provide static images for software. The default is that the image is read-only and has write access to the host filesystems. Thus, running apt upgrade in a singularity container isn't something that is done usually. It might even be impossible, although I am not expert enough to make that statement firmly. Is there a chance debcraft can work from a static container provided by the user? I think that the key problem I have is that I want to use a build Debian packages that need no root access and that do not need to install dependencies that need root access, and I want to do that with user privileges only. Have a nice day, Charles -- Charles Plessy Nagahama, Yomitan, Okinawa, Japan Debian Med packaging team http://www.debian.org/devel/debian-med Tooting from home https://framapiaf.org/@charles_plessy - You do not have my permission to use this email to train an AI -
Re: Tool to build Debian packages not requiring root in containers ?
Hi! ti 7. toukok. 2024 klo 23.01 Charles Plessy kirjoitti: > Le Tue, May 07, 2024 at 08:17:31PM -0700, Otto Kekäläinen a écrit : > > > > Can you give me an example of a package you want to build and what is > > the starting point, and I can tell you what command to issue to > > https://salsa.debian.org/otto/debcraft to achieve it? > > > > It supports running Podman in user mode (=no root permissions needed), > > Hi Otto, > > it looks really great! > > Do you think you can make it work with Singularity/Apptainer instead of > Podman? Our cluster runs only singularity 3.5.2 > (https://docs.sylabs.io/guides/3.5/user-guide/). Debian has version > 4.1.2 in the singularity-container package. > > The conversion of a Docker container to the Singularity format is > simple, and Singularity already mounts most of the local storage to make > it visible and writable from within the container. > I read the docs on how Singularity is able to pull Docker images of Debian Sid and build on top of them, and run and exec just like Docker/Podman. Unfortunately it has its own Containerfile format ( https://docs.sylabs.io/guides/3.5/user-guide/quick_start.html#singularity-definition-files) and the commands have their own syntax. I guess Debcraft could be extended to support it, but that would require at least one Singularity user as frequent contributor to test and develop Singularity-compatibility. The entire code base is shell code. Perhaps you want to take a look if it looks hackable for you?
Re: Tool to build Debian packages not requiring root in containers ?
Le Tue, May 07, 2024 at 08:17:31PM -0700, Otto Kekäläinen a écrit : > > Can you give me an example of a package you want to build and what is > the starting point, and I can tell you what command to issue to > https://salsa.debian.org/otto/debcraft to achieve it? > > It supports running Podman in user mode (=no root permissions needed), Hi Otto, it looks really great! Do you think you can make it work with Singularity/Apptainer instead of Podman? Our cluster runs only singularity 3.5.2 (https://docs.sylabs.io/guides/3.5/user-guide/). Debian has version 4.1.2 in the singularity-container package. The conversion of a Docker container to the Singularity format is simple, and Singularity already mounts most of the local storage to make it visible and writable from within the container. The typical packages that I want to build are the r-bioc-* collection. Together, they represent a dependency graph deep of a dozen of layers, which makes transitions work-intensive. With tools like debcraft I would like to prepare a set of updated packages for which I know that the CI tests pass, and that can be uploaded all together at the same time when I we get green light from the Release team. (And to rebuild all of them if in the meantime the contents of Unstable have changed significantly). Have a nice day, Charles -- Charles Plessy Nagahama, Yomitan, Okinawa, Japan Debian Med packaging team http://www.debian.org/devel/debian-med Tooting from home https://framapiaf.org/@charles_plessy - You do not have my permission to use this email to train an AI -
Re: Tool to build Debian packages not requiring root in containers ?
Hi! On Tue, 7 May 2024 at 15:27, Charles Plessy wrote: .. > I want to leverage our cluster to automate as much of the rebuilds as I > can, but could not find the right tool. I tried to run sbuild in a > Singularity image and this failed. However, I do not need the whole > power of engines like sbuild, as none of the packages involved require > root priviledges to build. > > Do you have a suggestion for a tool can run in user mode in a container > image having access to local storage on the host, and that given a > Debian source control file will download the dependencies and build the > package ? Can you give me an example of a package you want to build and what is the starting point, and I can tell you what command to issue to https://salsa.debian.org/otto/debcraft to achieve it? It supports running Podman in user mode (=no root permissions needed), it loop-mounts a local directory (local storage), creates clean build containers on the fly similar to sbuild but is much easier and faster to use. Example of how to build one of your packages with just pointing it at the source git repo: $ debcraft build https://salsa.debian.org/med-team/altree.git Building container 'debcraft-debian-sid' in '/tmp/tmp.brCZRhn2lL/debcraft-container' for downloader use mkdir: created directory '/tmp/tmp.brCZRhn2lL/debcraft-container' STEP 1/10: FROM debian:sid ... $ ls -1 debcraft-build-altree-1715137513.a8c999a+master altree_1.3.2-2_amd64.build altree_1.3.2-2_amd64.buildinfo altree_1.3.2-2_amd64.changes altree_1.3.2-2_amd64.deb altree-dbgsym_1.3.2-2_amd64.deb altree-examples_1.3.2-2_all.deb control.log filelist.log lintian.log First build is a bit slow as it needs to download all the dependencies and create a container, but the second run of 'debcraft build' inside the source directory will be very fast as all container cache is reused.
Tool to build Debian packages not requiring root in containers ?
Hello everybody, I just re-suscribed :) At work I have access to a nice cluster with plenty of nodes rich of 128 cores and 512 Gb RAM. The nodes do not run Debian but Singularity is available for virtualisation (https://en.wikipedia.org/wiki/Singularity_(software)). And in Debian I am part of transitions involving more than 100 packages every 6 months (r-api-bioc-*)… I want to leverage our cluster to automate as much of the rebuilds as I can, but could not find the right tool. I tried to run sbuild in a Singularity image and this failed. However, I do not need the whole power of engines like sbuild, as none of the packages involved require root priviledges to build. Do you have a suggestion for a tool can run in user mode in a container image having access to local storage on the host, and that given a Debian source control file will download the dependencies and build the package ? Have a nice day, Charles -- Charles Plessy Nagahama, Yomitan, Okinawa, Japan Debian Med packaging team http://www.debian.org/devel/debian-med Tooting from work, https://fediscience.org/@charles_plessy Tooting from home, https://framapiaf.org/@charles_plessy
