Package: release.debian.org Severity: normal User: release.debian....@packages.debian.org Usertags: unblock
Hi! Please unblock packages crowdsec, crowdsec-custom-bouncer, and crowdsec-firewall-bouncer. I'm filing a single unblock request since all three packages are entangled, and Paul suggested this would be appropriate: - both bouncers get the exact same changes (only the context differs); - crowdsec gets an extra snippet to deal with the pending registration requested by either one or both bouncers. [ Reason ] RC bugs #1035499 and #1036985 on the bouncers: they might fail to install depending on the dpkg-level ordering as far as configuration goes, that is: if crowdsec (which is listed in Recommends) is unpacked but not configured, the `cscli` call fails and the postinst errors out. Back when the bouncers were prepared, both upstream and I verified that we could just install either bouncer package without a pre-existing crowdsec package installed, but as seen with Andreas' piuparts-based testing, we might get a different order over time, or across bouncers… In any case, at the moment, a freshly-deployed Bookworm VM can't get either bouncer installed, and I'd like to get that fixed in time for 12.0. The proposed changes keep the existing code paths, and add detection for the problematic case, queueing bouncer registration, letting crowdsec catch up when it's finally configured. To prevent bouncers from starting before crowdsec has dealt with the registration, I've added a condition to both their systemd units, and a `deb-systemd-invoke start` call once everything is ready. [ Impact ] Abysmal user experience without those fixes. [ Tests ] Both upstream and I have tested updated packages, stashed in a custom repository, installing 1 to 3 packages in various order, making sure the new code does the right thing. I've also verified this under piuparts, seeing that policy-rc.d is respected as it should (and the start request is ignored without triggering an error). [ Risks ] There might be tricky situations I haven't imagined or encountered, but since we're basically keeping existing code, and just adding detection and solution for a specific bad situation during the first installation, I'm not sure what could regress. [ Checklist ] [x] all changes are documented in the d/changelog [x] I reviewed all changes and I approve them [x] attach debdiff against the package in testing unblock crowdsec/1.4.6-4 unblock crowdsec-custom-bouncer/0.0.15-3 unblock crowdsec-firewall-bouncer/0.0.25-3 Cheers, -- Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/
diff -Nru crowdsec-1.4.6/debian/changelog crowdsec-1.4.6/debian/changelog --- crowdsec-1.4.6/debian/changelog 2023-03-19 00:25:07.000000000 +0100 +++ crowdsec-1.4.6/debian/changelog 2023-05-31 18:54:17.000000000 +0200 @@ -1,3 +1,16 @@ +crowdsec (1.4.6-4) unstable; urgency=medium + + * Implement support for pending registration: since bouncers list crowdsec + in Recommends, we cannot guarantee the order in which bouncers and + crowdsec are configured (See: #1035499, #1036985). Bouncers can now + queue triplets (systemd unit name, bouncer identifier and API key) in + /var/lib/crowdsec/pending-registration. crowdsec.postinst will register + those bouncers, and start their systemd units after removing that file + (satisfying their ConditionPathExists=! on it). + * Replace `exit 0` with `break` in the preceding code block. + + -- Cyril Brulebois <cy...@debamax.com> Wed, 31 May 2023 18:54:17 +0200 + crowdsec (1.4.6-3) unstable; urgency=medium * When performing an upgrade from pre-1.4.x versions, apply a workaround diff -Nru crowdsec-1.4.6/debian/crowdsec.postinst crowdsec-1.4.6/debian/crowdsec.postinst --- crowdsec-1.4.6/debian/crowdsec.postinst 2023-03-18 14:40:31.000000000 +0100 +++ crowdsec-1.4.6/debian/crowdsec.postinst 2023-05-31 17:01:15.000000000 +0200 @@ -280,15 +280,35 @@ for _ in $(seq 1 $MAX); do # Getting decisions means we can happily exit: if grep -qs 'added [0-9][0-9]* entries, deleted [0-9][0-9]* entries' $LOG; then - exit 0 + break fi # Getting 0 new entries means we can happily trigger a restart then exit: if grep -qs 'received 0 new entries (expected if you just installed crowdsec)' $LOG; then echo "W: Restarting manually to force a CAPI pull (upstream #2120)" >&2 deb-systemd-invoke restart 'crowdsec.service' >/dev/null || true - exit 0 + break fi # Don't poll too aggressively: sleep 1 done fi + +# Bouncer registration: they have crowdsec in Recommends only, so ordering isn't +# guaranteed (#1035499, #1036985). Process pending registration if any, then +# kick relevant systemd units once their ConditionPathExists is satisfied. +PENDING=/var/lib/crowdsec/pending-registration +if [ -f $PENDING ]; then + while read unit name key; do + units="${units:+$units }$unit" + bouncers="${bouncers:+$bouncers }$name" + # We don't need the API key to be echo'd back: + cscli --error -oraw bouncers add "$name" -k "$key" > /dev/null + done < $PENDING + rm -f $PENDING + echo "I: Registered bouncers: $bouncers" >&2 + + for unit in $units; do + deb-systemd-invoke start "$unit" + done + echo "I: Restarts units: $units" >&2 +fi
diff -Nru crowdsec-custom-bouncer-0.0.15/debian/changelog crowdsec-custom-bouncer-0.0.15/debian/changelog --- crowdsec-custom-bouncer-0.0.15/debian/changelog 2023-03-21 01:02:17.000000000 +0100 +++ crowdsec-custom-bouncer-0.0.15/debian/changelog 2023-05-31 19:01:19.000000000 +0200 @@ -1,3 +1,21 @@ +crowdsec-custom-bouncer (0.0.15-3) unstable; urgency=medium + + * Fix failure to install if crowdsec is unpacked but not configured + (Closes: #1035499): + - If both cscli and /etc/crowdsec/config.yaml exist, register as + previously, via `cscli bouncers add`. + - If cscli exists but /etc/crowdsec/config.yaml doesn't, create an + API key similarly to what cscli would do (32 hexadecimal digits), + and queue registration in /var/lib/crowdsec/pending-registration. + * Add ConditionPathExists=!/var/lib/crowdsec/pending-registration to the + systemd unit accordingly, so that it's skipped until crowdsec's + postinst deals with the pending-registration file, removes it, and + starts the units. + * Use mode 600 when creating crowdsec-custom-bouncer.yaml.local, since + it contains an API key. + + -- Cyril Brulebois <cy...@debamax.com> Wed, 31 May 2023 19:01:19 +0200 + crowdsec-custom-bouncer (0.0.15-2) unstable; urgency=medium * Add patch to honor log-related settings (upstream PR #54): diff -Nru crowdsec-custom-bouncer-0.0.15/debian/crowdsec-custom-bouncer.postinst crowdsec-custom-bouncer-0.0.15/debian/crowdsec-custom-bouncer.postinst --- crowdsec-custom-bouncer-0.0.15/debian/crowdsec-custom-bouncer.postinst 2023-03-21 00:58:41.000000000 +0100 +++ crowdsec-custom-bouncer-0.0.15/debian/crowdsec-custom-bouncer.postinst 2023-05-31 17:16:45.000000000 +0200 @@ -2,6 +2,7 @@ set -e CONFIG=/etc/crowdsec/bouncers/crowdsec-custom-bouncer.yaml +PENDING=/var/lib/crowdsec/pending-registration # Create a configuration override file only once, see README.Debian: if [ "$1" = configure ] && [ ! -f "$CONFIG.local" ]; then @@ -13,10 +14,21 @@ # machines; reuse the logic from crowdsec's postinst: unique=$(tr -dc 'a-zA-Z0-9' < /dev/urandom | fold -w 32 | head -n 1) id="CustomBouncer-$unique" - api_key=$(cscli --error -oraw bouncers add "$id") - if [ -z "$api_key" ]; then - echo "E: Local registration failed to yield an API key" >&2 - exit 1 + + # crowdsec might be unpacked but not configured (#1035499): + if [ -f /etc/crowdsec/config.yaml ]; then + api_key=$(cscli --error -oraw bouncers add "$id") + if [ -z "$api_key" ]; then + echo "E: Local registration failed to yield an API key" >&2 + exit 1 + fi + else + api_key=$(tr -dc 'a-f0-9' < /dev/urandom | fold -w 32 | head -n 1) + if [ ! -f $PENDING ]; then + touch $PENDING + chmod 600 $PENDING + fi + echo "crowdsec-custom-bouncer $id $api_key" >> $PENDING fi # Store it so that it can be unregistered when purging the package: @@ -28,6 +40,9 @@ DEFAULT_SCRIPT=/usr/bin/true echo "W: Using $DEFAULT_SCRIPT as default custom script" >&2 + touch "$CONFIG.local" + chmod 600 "$CONFIG.local" + if [ -n "$api_key" ]; then cat > "$CONFIG.local" <<EOF bin_path: $DEFAULT_SCRIPT diff -Nru crowdsec-custom-bouncer-0.0.15/debian/patches/0002-adjust-service.patch crowdsec-custom-bouncer-0.0.15/debian/patches/0002-adjust-service.patch --- crowdsec-custom-bouncer-0.0.15/debian/patches/0002-adjust-service.patch 2023-03-21 00:58:41.000000000 +0100 +++ crowdsec-custom-bouncer-0.0.15/debian/patches/0002-adjust-service.patch 2023-05-31 17:15:02.000000000 +0200 @@ -9,12 +9,13 @@ --- a/config/crowdsec-custom-bouncer.service +++ b/config/crowdsec-custom-bouncer.service -@@ -1,14 +1,11 @@ +@@ -1,14 +1,12 @@ [Unit] Description=The custom bouncer for CrowdSec -After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service - +After=network.target remote-fs.target nss-lookup.target crowdsec.service ++ConditionPathExists=!/var/lib/crowdsec/pending-registration [Service] Type=notify
diff -Nru crowdsec-firewall-bouncer-0.0.25/debian/changelog crowdsec-firewall-bouncer-0.0.25/debian/changelog --- crowdsec-firewall-bouncer-0.0.25/debian/changelog 2023-03-20 23:27:07.000000000 +0100 +++ crowdsec-firewall-bouncer-0.0.25/debian/changelog 2023-05-31 18:57:41.000000000 +0200 @@ -1,3 +1,21 @@ +crowdsec-firewall-bouncer (0.0.25-3) unstable; urgency=medium + + * Fix failure to install if crowdsec is unpacked but not configured + (Closes: #1036985): + - If both cscli and /etc/crowdsec/config.yaml exist, register as + previously, via `cscli bouncers add`. + - If cscli exists but /etc/crowdsec/config.yaml doesn't, create an + API key similarly to what cscli would do (32 hexadecimal digits), + and queue registration in /var/lib/crowdsec/pending-registration. + * Add ConditionPathExists=!/var/lib/crowdsec/pending-registration to the + systemd unit accordingly, so that it's skipped until crowdsec's + postinst deals with the pending-registration file, removes it, and + starts the units. + * Use mode 600 when creating crowdsec-firewall-bouncer.yaml.local, since + it contains an API key. + + -- Cyril Brulebois <cy...@debamax.com> Wed, 31 May 2023 18:57:41 +0200 + crowdsec-firewall-bouncer (0.0.25-2) unstable; urgency=medium * Stop shipping a logrotate configuration snippet, as the bouncer rotates diff -Nru crowdsec-firewall-bouncer-0.0.25/debian/crowdsec-firewall-bouncer.postinst crowdsec-firewall-bouncer-0.0.25/debian/crowdsec-firewall-bouncer.postinst --- crowdsec-firewall-bouncer-0.0.25/debian/crowdsec-firewall-bouncer.postinst 2022-12-15 03:37:41.000000000 +0100 +++ crowdsec-firewall-bouncer-0.0.25/debian/crowdsec-firewall-bouncer.postinst 2023-05-31 16:49:51.000000000 +0200 @@ -2,6 +2,7 @@ set -e CONFIG=/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml +PENDING=/var/lib/crowdsec/pending-registration # Create a configuration override file only once, see README.Debian: if [ "$1" = configure ] && [ ! -f "$CONFIG.local" ]; then @@ -13,10 +14,21 @@ # machines; reuse the logic from crowdsec's postinst: unique=$(tr -dc 'a-zA-Z0-9' < /dev/urandom | fold -w 32 | head -n 1) id="FirewallBouncer-$unique" - api_key=$(cscli --error -oraw bouncers add "$id") - if [ -z "$api_key" ]; then - echo "E: Local registration failed to yield an API key" >&2 - exit 1 + + # crowdsec might be unpacked but not configured (#1036985): + if [ -f /etc/crowdsec/config.yaml ]; then + api_key=$(cscli --error -oraw bouncers add "$id") + if [ -z "$api_key" ]; then + echo "E: Local registration failed to yield an API key" >&2 + exit 1 + fi + else + api_key=$(tr -dc 'a-f0-9' < /dev/urandom | fold -w 32 | head -n 1) + if [ ! -f $PENDING ]; then + touch $PENDING + chmod 600 $PENDING + fi + echo "crowdsec-firewall-bouncer $id $api_key" >> $PENDING fi # Store it so that it can be unregistered when purging the package: @@ -37,6 +49,9 @@ firewall=nftables fi + touch "$CONFIG.local" + chmod 600 "$CONFIG.local" + # Generate the override: if [ -n "$api_key" ]; then cat > "$CONFIG.local" <<EOF diff -Nru crowdsec-firewall-bouncer-0.0.25/debian/patches/0001-adjust-service.patch crowdsec-firewall-bouncer-0.0.25/debian/patches/0001-adjust-service.patch --- crowdsec-firewall-bouncer-0.0.25/debian/patches/0001-adjust-service.patch 2022-11-30 21:35:51.000000000 +0100 +++ crowdsec-firewall-bouncer-0.0.25/debian/patches/0001-adjust-service.patch 2023-05-31 16:49:51.000000000 +0200 @@ -14,12 +14,13 @@ --- a/config/crowdsec-firewall-bouncer.service +++ b/config/crowdsec-firewall-bouncer.service -@@ -1,12 +1,12 @@ +@@ -1,12 +1,13 @@ [Unit] Description=The firewall bouncer for CrowdSec -After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service +After=network.target remote-fs.target nss-lookup.target crowdsec.service Before=netfilter-persistent.service ++ConditionPathExists=!/var/lib/crowdsec/pending-registration [Service] Type=notify