laforge has submitted this change. (
https://gerrit.osmocom.org/c/docker-playground/+/33740 )
Change subject: debian-bookworm-build(-arm): new docker containers
......................................................................
debian-bookworm-build(-arm): new docker containers
Create debian bookworm (12) based containers for building all Osmocom
projects in master/gerrit/ttcn3 jenkins jobs.
This is a combination of debian-bullseye-build (ttcn3) and
debian-bullseye-jenkins(-arm) (master/gerrit), which were used for this
purpose but had several differences. Combining them solves the
following problems:
* The naming was confusing, as both the -jenkins and -build containers
were used in jenkins and were used to build the Osmocom stack.
* This lead to adding the dependencies to the wrong containers / not
adding them to both containers.
* Now we are sure that if a program builds from source in the master
and gerrit verifications, it will also build in the ttcn3 jobs.
Other notable changes from the debian 11 containers:
* Python2 is finally gone (OS#5950, for the few projects that still
need it like openbsc, I'll use debian 11 for the jenkins jobs)
* Removed osc and git-buildpackage, now that OBS package building is
done in different docker containers (see osmo-ci/scripts/obs)
* Combined multiple RUN commands into one as they the image build
down, used set -x to display what commands run exactly
* Use UID variable instead of hardcoding 1000
* Optimized order of commands, so e.g. the LLVM for Arm that we will
rarely change gets installed/cached before the big list of apt pkgs
* Replaced old git.osmocom.org urls
* Removed apt-get upgrade; the base image will get upgraded from time
to time, no need to upgrade within the image
* Add --depth=1 option to git clones and remove temporary git clones
* Removed generating locales code, it was only done by one of the two
images and I couldn't find in the git log why we would need this (if
we do need it we can add it back later)
The following patches add more debian bookworm containers and have
similar improvements.
Related: OS#6057
Change-Id: I49aaf62b5b97775f923453611df3b91354a640a0
---
A debian-bookworm-build-arm/Dockerfile
A debian-bookworm-build-arm/Makefile
A debian-bookworm-build/Dockerfile
A debian-bookworm-build/Makefile
4 files changed, 355 insertions(+), 0 deletions(-)
Approvals:
laforge: Looks good to me, approved
pespin: Looks good to me, but someone else must approve
Jenkins Builder: Verified
diff --git a/debian-bookworm-build-arm/Dockerfile
b/debian-bookworm-build-arm/Dockerfile
new file mode 100644
index 0000000..9a4b2b4
--- /dev/null
+++ b/debian-bookworm-build-arm/Dockerfile
@@ -0,0 +1,89 @@
+# Image used to run contrib/jenkins.sh scripts of a few Osmocom projects on
+# arm. Keep this image minimal so the rpis don't need forever to build it.
+# See master-builds.yml, gerrit-verifications.yml in osmo-ci.git.
+ARG REGISTRY=docker.io
+ARG UPSTREAM_DISTRO=debian:bookworm
+FROM ${REGISTRY}/${UPSTREAM_DISTRO}
+
+# Arguments used after FROM must be specified again
+ARG OSMOCOM_REPO_MIRROR="https://downloads.osmocom.org";
+ARG OSMOCOM_REPO_PATH="packages/osmocom:"
+ARG
OSMOCOM_REPO="${OSMOCOM_REPO_MIRROR}/${OSMOCOM_REPO_PATH}/nightly/Debian_12/"
+ARG UID
+
+# Copy from common dir
+COPY .common/Release.key /etc/apt/trusted.gpg.d/obs.osmocom.org.asc
+
+# Configure build user, disable installing man pages
+# * man pages: without them we avoid waiting for "generating manpages"
+RUN set -x && \
+ useradd --uid=${UID} build && \
+ mkdir /build && \
+ chown -R build:build /build /usr/local && \
+ \
+ echo "path-exclude=/usr/share/man/*" \
+ > /etc/dpkg/dpkg.cfg.d/exclude-man-pages && \
+ rm -rf /usr/share/man/
+
+# Install packages from Debian repositories (alphabetic order)
+RUN set -x && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ autoconf \
+ automake \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ g++ \
+ gcc \
+ git \
+ libboost-dev \
+ libboost-filesystem-dev \
+ libboost-program-options-dev \
+ libboost-thread-dev \
+ libfftw3-dev \
+ libgnutls28-dev \
+ libmnl-dev \
+ libortp-dev \
+ libpcsclite-dev \
+ libsctp-dev \
+ libtalloc-dev \
+ libtool \
+ liburing-dev \
+ libusb-1.0-0-dev \
+ make \
+ pkg-config \
+ python3-minimal \
+ python3-setuptools \
+ sdcc \
+ stow \
+ && \
+ apt-get clean
+
+# Install osmo-python-tests
+ADD
https://gerrit.osmocom.org/plugins/gitiles/python/osmo-python-tests/+/master?format=TEXT
/tmp/osmo-python-tests-commit
+RUN set -x && \
+ git clone --depth=1 https://gerrit.osmocom.org/python/osmo-python-tests
osmo-python-tests && \
+ cd osmo-python-tests && \
+ python3 setup.py clean build install && \
+ rm -rf osmo-python-tests
+
+# Install osmo-ci.git/scripts to /usr/local/bin
+ADD https://gerrit.osmocom.org/plugins/gitiles/osmo-ci/+/master?format=TEXT
/tmp/osmo-ci-commit
+RUN set -x && \
+ git clone --depth=1 https://gerrit.osmocom.org/osmo-ci osmo-ci && \
+ su build -c "cd osmo-ci/scripts && cp -v *.sh *.py /usr/local/bin" && \
+ rm -rf osmo-ci
+
+# Install packages from Osmocom OBS nightly repository
+# * osmo-trx: liblimesuite-dev, libuhd-dev
+ADD $OSMOCOM_REPO/Release /tmp/Release
+RUN set -x && \
+ echo "deb [signed-by=/etc/apt/trusted.gpg.d/obs.osmocom.org.asc]
$OSMOCOM_REPO ./" \
+ > /etc/apt/sources.list.d/osmocom-nightly.list && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ liblimesuite-dev \
+ libuhd-dev \
+ && \
+ apt-get clean
diff --git a/debian-bookworm-build-arm/Makefile
b/debian-bookworm-build-arm/Makefile
new file mode 100644
index 0000000..cdac90c
--- /dev/null
+++ b/debian-bookworm-build-arm/Makefile
@@ -0,0 +1,3 @@
+UPSTREAM_DISTRO=debian:bookworm
+DISTRO=debian-bookworm
+include ../make/Makefile
diff --git a/debian-bookworm-build/Dockerfile b/debian-bookworm-build/Dockerfile
new file mode 100644
index 0000000..f717241
--- /dev/null
+++ b/debian-bookworm-build/Dockerfile
@@ -0,0 +1,215 @@
+# Image for building all Osmocom projects in master/gerrit/ttcn3 jenkins jobs
+ARG REGISTRY=docker.io
+ARG UPSTREAM_DISTRO=debian:bookworm
+FROM ${REGISTRY}/${UPSTREAM_DISTRO}
+
+# Arguments used after FROM must be specified again
+ARG OSMOCOM_REPO_MIRROR="https://downloads.osmocom.org";
+ARG OSMOCOM_REPO_PATH="packages/osmocom:"
+ARG
OSMOCOM_REPO="$OSMOCOM_REPO_MIRROR/${OSMOCOM_REPO_PATH}/nightly/Debian_12/"
+ARG UID
+ARG LLVM_VERSION="14.0.0"
+
+# Copy from common dir
+COPY .common/respawn.sh /usr/local/bin/respawn.sh
+COPY .common/Release.key /etc/apt/trusted.gpg.d/obs.osmocom.org.asc
+
+# Configure build user, disable installing man pages
+# * /usr/local: osmo-python-tests's contrib/jenkins.sh writes there
+# * man pages: without them we avoid waiting for "generating manpages"
+RUN set -x && \
+ useradd --uid=${UID} build && \
+ mkdir /build && \
+ chown -R build:build /build /usr/local && \
+ \
+ echo "path-exclude=/usr/share/man/*" \
+ > /etc/dpkg/dpkg.cfg.d/exclude-man-pages && \
+ rm -rf /usr/share/man/
+
+# Install LLVM-embedded-toolchain-for-Arm
+RUN if [ "$(arch)" != "x86_64" ]; then \
+ echo "ERROR: use debian-bookworm-build-arm instead"; \
+ exit 1; \
+ fi && \
+ set -x && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ ca-certificates \
+ libtinfo5 \
+ wget \
+ && \
+ apt-get clean && \
+ wget
https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm/releases/download/release-${LLVM_VERSION}/LLVMEmbeddedToolchainForArm-${LLVM_VERSION}-linux.tar.gz
&& \
+ tar -xf LLVMEmbeddedToolchainForArm-${LLVM_VERSION}-linux.tar.gz && \
+ rm LLVMEmbeddedToolchainForArm-${LLVM_VERSION}-linux.tar.gz && \
+ mv LLVMEmbeddedToolchainForArm-${LLVM_VERSION} /opt/llvm-arm && \
+ /opt/llvm-arm/bin/clang --version && \
+ /opt/llvm-arm/bin/clang --print-targets
+
+# Install packages from Debian repositories (alphabetic order)
+RUN set -x && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ asciidoc \
+ asciidoc-dblatex \
+ autoconf \
+ autoconf-archive \
+ autogen \
+ automake \
+ bc \
+ bison \
+ build-essential \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ cmake \
+ coccinelle \
+ cppcheck \
+ dahdi-source \
+ dblatex \
+ dbus \
+ debhelper \
+ devscripts \
+ dh-autoreconf \
+ docbook5-xml \
+ doxygen \
+ equivs \
+ flex \
+ g++ \
+ gawk \
+ gcc \
+ gcc-arm-none-eabi \
+ ghostscript \
+ git \
+ gnupg \
+ graphviz \
+ htop \
+ inkscape \
+ iproute2 \
+ latexmk \
+ lcov \
+ libaio-dev \
+ libasound2-dev \
+ libboost-all-dev \
+ libc-ares-dev \
+ libcdk5-dev \
+ libcsv-dev \
+ libdbd-sqlite3 \
+ libdbi-dev \
+ libelf-dev \
+ libffi-dev \
+ libfftw3-dev \
+ libgmp-dev \
+ libgnutls28-dev \
+ libgps-dev \
+ libgsm1-dev \
+ libjansson-dev \
+ liblua5.3-dev \
+ libmnl-dev \
+ libncurses5-dev \
+ libnewlib-arm-none-eabi \
+ libnl-3-dev \
+ libnl-route-3-dev \
+ liboping-dev \
+ libortp-dev \
+ libpcap-dev \
+ libpcsclite-dev \
+ libreadline-dev \
+ libsctp-dev \
+ libsigsegv-dev \
+ libsnmp-dev \
+ libsofia-sip-ua-glib-dev \
+ libsqlite3-dev \
+ libssl-dev \
+ libtalloc-dev \
+ libtinfo5 \
+ libtool \
+ liburing-dev \
+ libusb-1.0-0-dev \
+ libusb-dev \
+ libxml2-utils \
+ libzmq3-dev \
+ locales \
+ lua-socket \
+ make \
+ mscgen \
+ ofono \
+ openssh-client \
+ patchelf \
+ picolibc-arm-none-eabi \
+ pkg-config \
+ python3 \
+ python3-gi \
+ python3-mako \
+ python3-nwdiag \
+ python3-pip \
+ python3-pyflakes \
+ python3-setuptools \
+ python3-usb \
+ python3-yaml \
+ rsync \
+ sdcc \
+ source-highlight \
+ sqlite3 \
+ stow \
+ sudo \
+ swig \
+ systemd \
+ tcpdump \
+ telnet \
+ tex-gyre \
+ texinfo \
+ unzip \
+ xsltproc \
+ && \
+ apt-get clean
+
+# Install pip dependencies (alphabetic order)
+# break-system-packages: inside docker it's fine to install pkgs system-wide
+ADD
https://gitea.osmocom.org/sim-card/pysim/raw/branch/master/requirements.txt
/tmp/pysim_requirements.txt
+RUN set -x && \
+ cat /tmp/pysim_requirements.txt && \
+ pip3 install --break-system-packages \
+ 'git+https://github.com/eriwen/lcov-to-cobertura-xml.git' \
+
'git+https://github.com/osmocom/sphinx-argparse@master#egg=sphinx-argparse' \
+ 'git+https://github.com/podshumok/python-smpplib.git' \
+ 'pydbus' \
+ 'pylint' \
+ 'pysispm' \
+ 'sphinx' \
+ 'sphinxcontrib-napoleon' \
+ -r /tmp/pysim_requirements.txt
+
+# Install osmo-python-tests
+ADD
https://gerrit.osmocom.org/plugins/gitiles/python/osmo-python-tests/+/master?format=TEXT
/tmp/osmo-python-tests-commit
+RUN set -x && \
+ git clone --depth=1 https://gerrit.osmocom.org/python/osmo-python-tests
osmo-python-tests && \
+ cd osmo-python-tests && \
+ python3 setup.py clean build install && \
+ rm -rf osmo-python-tests
+
+# Install osmo-ci.git/scripts to /usr/local/bin
+ADD https://gerrit.osmocom.org/plugins/gitiles/osmo-ci/+/master?format=TEXT
/tmp/osmo-ci-commit
+RUN set -x && \
+ git clone --depth=1 https://gerrit.osmocom.org/osmo-ci osmo-ci && \
+ su build -c "cd osmo-ci/scripts && cp -v *.sh *.py /usr/local/bin" && \
+ rm -rf osmo-ci
+
+# Install osmo-gsm-manuals to /opt/osmo-gsm-manuals
+ADD
https://gerrit.osmocom.org/plugins/gitiles/osmo-gsm-manuals/+/master?format=TEXT
/tmp/osmo-gsm-manuals-commit
+RUN git -C /opt clone --depth=1 https://gerrit.osmocom.org/osmo-gsm-manuals
+
+# Install packages from Osmocom OBS nightly repository
+# * osmo-remsim: libulfius
+# * osmo-trx: liblimesuite-dev, libuhd-dev
+ADD $OSMOCOM_REPO/Release /tmp/Release
+RUN set -x && \
+ echo "deb [signed-by=/etc/apt/trusted.gpg.d/obs.osmocom.org.asc]
$OSMOCOM_REPO ./" \
+ > /etc/apt/sources.list.d/osmocom-nightly.list && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ liblimesuite-dev \
+ libuhd-dev \
+ libulfius-dev \
+ && \
+ apt-get clean
diff --git a/debian-bookworm-build/Makefile b/debian-bookworm-build/Makefile
new file mode 100644
index 0000000..cdac90c
--- /dev/null
+++ b/debian-bookworm-build/Makefile
@@ -0,0 +1,3 @@
+UPSTREAM_DISTRO=debian:bookworm
+DISTRO=debian-bookworm
+include ../make/Makefile
--
To view, visit https://gerrit.osmocom.org/c/docker-playground/+/33740
To unsubscribe, or for help writing mail filters, visit
https://gerrit.osmocom.org/settings
Gerrit-Project: docker-playground
Gerrit-Branch: master
Gerrit-Change-Id: I49aaf62b5b97775f923453611df3b91354a640a0
Gerrit-Change-Number: 33740
Gerrit-PatchSet: 2
Gerrit-Owner: osmith <[email protected]>
Gerrit-Reviewer: Jenkins Builder
Gerrit-Reviewer: fixeria <[email protected]>
Gerrit-Reviewer: laforge <[email protected]>
Gerrit-Reviewer: pespin <[email protected]>
Gerrit-MessageType: merged
