Bug#1066731: libcloud: FTBFS: dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p "3.12 3.11" returned exit code 13

[Noah Meyerhans]

On Wed, Mar 13, 2024 at 03:57:06PM +0100, Lucas Nussbaum wrote:
> > === FAILURES 
> > ===
> > ___ TestUtils.test_init_once_and_debug_mode 
> > 
> > 
> > self =  > testMethod=test_init_once_and_debug_mode>
> > 
> > def test_init_once_and_debug_mode(self):
> > if have_paramiko:
> > paramiko_logger = logging.getLogger('paramiko')
> > paramiko_logger.setLevel(logging.INFO)
> > 
> > # Debug mode is disabled
> > _init_once()
> > 
> > >   self.assertIsNone(LoggingConnection.log)
> > E   AssertionError:  > 0x7f79cb14b090> is not None
> > 
> > libcloud/test/test_init.py:50: AssertionError

This seems to be an issue with the upstream test suite and is caused by
_init_once() having already been called earlier in the test execution process.
If we run just the failing test on its own, it is successful, but if we run
tests in libcloud/test/compute/test_ssh_client.py first, then it fails because
its setup method calls _init_once().  See below for logs indicating this
behavior.  I'm not sure of the best way to avoid this.  Can we somehow
undo the effect of _init_once()?  Or force these tests to run in a
separate process?  Someone who knows pytest better than me might have
ideas...

Note that this issue is still present in the libcloud 3.8.0 currently in
git.

Successful standalone test execution:

builder@b6dfde9f3b55:/src/libcloud/.pybuild/cpython3_3.11_libcloud/build$ 
python3 -m pytest libcloud/test/test_init.py -v
 test session starts 

platform linux -- Python 3.11.9, pytest-8.1.2, pluggy-1.5.0 -- /usr/bin/python3
cachedir: .pytest_cache
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False 
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10 
warmup=False warmup_iterations=10)
rootdir: /src/libcloud/.pybuild/cpython3_3.11_libcloud/build
configfile: pyproject.toml
plugins: xdist-3.5.0, benchmark-4.0.0, timeout-2.3.1, requests-mock-1.11.0
collected 5 items   


libcloud/test/test_init.py::TestUtils::test_factory PASSED  
  [ 20%]
libcloud/test/test_init.py::TestUtils::test_init_once_and_debug_mode PASSED 
  [ 40%]
libcloud/test/test_init.py::TestUtils::test_init_once_correct_chardet_version 
PASSED  [ 60%]
libcloud/test/test_init.py::TestUtils::test_init_once_detects_bad_yum_install_requests
 PASSED [ 80%]
libcloud/test/test_init.py::TestUtils::test_raises_error PASSED 
  [100%]

=== slowest 10 durations 

0.01s call libcloud/test/test_init.py::TestUtils::test_factory

(9 durations < 0.005s hidden.  Use -vv to show these durations.)
= 5 passed in 0.11s 
=

Failing execution following execution of a test from
libcloud/test/compute/test_ssh_client.py:

libcloud/test/compute/test_ssh_client.py::ParamikoSSHClientTests::test_key_material_valid_pem_keys_invalid_header_auto_converbuilder@b6dfde9f3b55:/src/libcloud/.pybuild/cpython3_3.11_libcloud/build$
 python3 -m pytest 
libcloud/test/compute/test_ssh_client.py::ParamikoSSHClientTests::test_password_protected_key_valid_password_provided
 libcloud/test/test_init.py -v
 test session starts 

platform linux -- Python 3.11.9, pytest-8.1.2, pluggy-1.5.0 -- /usr/bin/python3
cachedir: .pytest_cache
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False 
min_rounds=5 min_time=0.05 max_time=1.0 calibration_precision=10 
warmup=False warmup_iterations=10)
rootdir: /src/libcloud/.pybuild/cpython3_3.11_libcloud/build
configfile: pyproject.toml
plugins: xdist-3.5.0, benchmark-4.0.0, timeout-2.3.1, requests-mock-1.11.0
collected 6 items   


libcloud/test/compute/test_ssh_client.py::ParamikoSSHClientTests::test_password_protected_key_valid_password_provided
 PASSED [ 16%]
libcloud/test/test_init.py::TestUtils::test_factory PASSED  
  [ 33%]
libcloud/test/test_init.py::TestUtils::test_init_once_and_debug_mode FAILED 
  [ 50%]
libcloud/test/test_init.py::TestUtils::test_init_once_correct_chardet_version 
PASSED  [ 66%]

Bug#1070137: bullseye-pu: package cloud-init/22.4.2-1

[Noah Meyerhans]

Control: tag -1 - moreinfo

On Sun, May 12, 2024 at 09:14:14PM +0100, Jonathan Wiltshire wrote:
> > There are pros and cons to each option.  Given bullseye's age and
> > cloud-init's blast radius (a regression could potentially disrupt the
> > provisioning process of cloud VMs, which is particularly disruptive in
> > such environments) I lean toward option (2) above, as it minimizes the
> > changes.  The obvious drawback is that we now have two versions of
> > cloud-init in the bullseye repositories, which was not the case
> > previously.  The cloud team is committed to supporting this situation
> > for the duration of the bullseye LTS lifetime.
> 
> I think I lean towards option 2 as well. I assume the versioning is
> calendar-based not semantic, so it's hard to know how disruptive 20.x ->
> 22.x would be, and meaningful testing across all the platforms it could be
> deployed on is unrealistic.

Yes, cloud-init has a very large configuration interface, and is under
pretty constant development and refactoring.  We have in the past
discussed with the release team the possibility of updating it (and some
of the other cloud related packages) with some regularity, but have
never actually followed through with it.  While there are some good
reasons for doing a distro-wide update, this point in the bullseye
lifecycle probably isn't the best time to start.

> 
> Can you attach proposed debian/control and debian/changelog files please?
> 

See the complete debian/control attached.  The full diff between the
bookworm version and the proposed bullseye version is below.  There were
some minor changes to debian/rules to cope with the package name change.

diff --git a/debian/changelog b/debian/changelog
index 9bd33d11..bc3b921c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cloud-init-22.4.2 (22.4.2-2~bpo11+1) bullseye-security; urgency=medium
+
+  * Rename package to cloud-init-22.4.2 for bullseye-lts
+
+ -- Noah Meyerhans   Tue, 30 Apr 2024 13:14:06 -0700
+
 cloud-init (22.4.2-1) unstable; urgency=medium
 
   [ Bastian Blank ]
diff --git a/debian/control b/debian/control
index fff0d87f..0d26a1fa 100644
--- a/debian/control
+++ b/debian/control
@@ -1,4 +1,4 @@
-Source: cloud-init
+Source: cloud-init-22.4.2
 Maintainer: Debian Cloud Team 
 Uploaders:
  Bastian Blank ,
@@ -38,8 +38,11 @@ Vcs-Browser: https://salsa.debian.org/cloud-team/cloud-init
 Vcs-Git: https://salsa.debian.org/cloud-team/cloud-init.git
 Homepage: https://launchpad.net/cloud-init
 
-Package: cloud-init
+Package: cloud-init-22.4.2
 Architecture: all
+Conflicts: cloud-init
+Replaces: cloud-init (<< 22.4.2)
+Provides: cloud-init
 Depends:
  eject,
  fdisk | util-linux (<< 2.29.2-3~),
@@ -68,3 +71,7 @@ Description: initialization system for infrastructure cloud 
instances
  .
  Various methods are supported for passing data to the instance at launch
  time, including the standard interfaces of multiple platforms.
+ .
+ This versioned cloud-init-22.4.2 package exists to allow two
+ cloud-init package versions to co-exist in the Debian archive during
+ the Debian 11 LTS support period.
diff --git a/debian/rules b/debian/rules
index 3c302ff8..db302a4e 100755
--- a/debian/rules
+++ b/debian/rules
@@ -3,13 +3,24 @@
 PYTHON3S:=$(shell py3versions -vr)
 export PYBUILD_INSTALL_ARGS=--init-system=systemd,sysvinit_deb
 
+pkgver:=22.4.2
+
+execute_before_dh_prep:
+   for extension in install lintian-overrides manpages postinst postrm; do 
\
+ cp -v debian/cloud-init.$${extension} \
+   debian/cloud-init-$(pkgver).$${extension} ;\
+   done
+
+execute_after_dh_clean:
+   -rm -vf debian/cloud-init-$(pkgver).*
+
 %:
dh $@ --with python3 --buildsystem pybuild
 
 override_dh_auto_install:
dh_auto_install
-   mv 
$(CURDIR)/debian/cloud-init/etc/NetworkManager/dispatcher.d/hook-network-manager
 \
-   
$(CURDIR)/debian/cloud-init/etc/NetworkManager/dispatcher.d/cloud-init-hook-network-manager
+   mv 
$(CURDIR)/debian/cloud-init-$(pkgver)/etc/NetworkManager/dispatcher.d/hook-network-manager
 \
+   
$(CURDIR)/debian/cloud-init-$(pkgver)/etc/NetworkManager/dispatcher.d/cloud-init-hook-network-manager
 
 override_dh_auto_test:
 ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
Source: cloud-init-22.4.2
Maintainer: Debian Cloud Team 
Uploaders:
 Bastian Blank ,
 Thomas Goirand ,
 Noah Meyerhans ,
Section: admin
Priority: optional
Build-Depends:
 debhelper (>= 10),
 dh-python,
 iproute2,
 po-debconf,
 pylint,
 python3-all,
 python3-pep8,
 python3-setuptools,
Build-Depends-Indep:
 python3-configobj,
 python3-contextlib2,
 python3-httpretty (>= 0.9.5),
 python3-jinja2,
 python3-jsonpatch,
 python3-jsonschema,
 python3-mock,
 python3-netifaces,
 python3-oauthlib,
 python3-pyflakes,
 python3-pytest,
 python3-pytest-mock,
 python3-requests,
 python3-responses,
 python3-serial,
 python3-unitte

Bug#1070137: bullseye-pu: package cloud-init/22.4.2-1

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
Tags: bullseye
User: release.debian@packages.debian.org
Usertags: pu
X-Debbugs-Cc: cloud-i...@packages.debian.org, t...@security.debian.org
Control: affects -1 + src:cloud-init

Hi folks.  This isn't a straightforward stable proposed-updates
request, but I think starting with such a request is probably the
right approach...

The cloud team builds official Debian images for multiple cloud
environments, including OpenStack, Microsoft Azure, and Amazon EC2.
We support all supported stable releases, including those supported by
the LTS team.  We build images including the backports kernels in
addition to the standard kernels.

To make a long story short, we build our Azure images with the
cloud-init package from bullseye-backports.  Images targeting other
cloud environments use cloud-init from the main repo.

As bullseye-backports nears its EOL date, we're faced with the
possibility that our Azure images contain unsupportable packages, and
that eventually (when bullseye-backports is archived) we'll be unable
to build new images at all.  These are scenarios we'd very much like
to avoid.

We've got at least a few options:

1. With an upcoming bullseye point release (how many more are there?)
   we update cloud-init to the version that's in bookworm.  This is
   the 22.4.2 package, which is close to the 22.4.1 package we're
   currently shipping on Azure.  22.4.2 is well tested in bookworm
   across all major cloud services, though we have not performed any
   major testing in a bullseye environment yet.  For non Azure users,
   this would be an update from version 20.4.2, which is a pretty
   large change.

2. We introduce a new versioned cloud-init source and binary package
   in the bullseye security archive, e.g. something like
   cloud-init-22.4.1.  This would look similar to what the kernel team
   did with the linux-5.10 source package added to buster-security,
   and which I assume they plan on doing with linux-6.1 in
   bullseye-security.  The cloud team would transition to this new
   versioned package for the Azure images, but would continue using
   the existing bullseye package everywhere else.

3. We do nothing, and leave the bullseye Azure users without a
   supportable cloud-init package.

4. Something else?

There are pros and cons to each option.  Given bullseye's age and
cloud-init's blast radius (a regression could potentially disrupt the
provisioning process of cloud VMs, which is particularly disruptive in
such environments) I lean toward option (2) above, as it minimizes the
changes.  The obvious drawback is that we now have two versions of
cloud-init in the bullseye repositories, which was not the case
previously.  The cloud team is committed to supporting this situation
for the duration of the bullseye LTS lifetime.

I realize that the security and release teams won't specifically care
what choice we make once bullseye's final point release is issued, but
I suspect you'll both have useful insights into how best to approach
this situation, and we may need your signoff ahead of that event
depending on which path we choose.

Thanks.
noah

Bug#1064235:

[Noah Meyerhans]

On Thu, Apr 18, 2024 at 10:17:27AM +0200, Arthur LUTZ wrote:
>A small heads up on this issue, we have some minor impacts :
>* sudo commands show a warning
> 
>  sudo: unable to resolve host example: Name or service not known

Thanks for the report.  This is resolved with the 20240429 images.

noah

Bug#1069800: FTBFS: test failures in buildd environment

[Noah Meyerhans]

Source: cloud-init
Version: 21.1.4-1
Severity: serious
Tags: ftbfs upstream
Justification: fails to build from source

Cloud-init's test fails in a non-networked build environment, as visible in
recent buildd logs, e.g.
https://buildd.debian.org/status/fetch.php?pkg=cloud-init=all=24.1.4-1=1714007718=0

This appears to be an upstream issue, as it occurs in pristine upstream source 
tree.

-- System Information:
Debian Release: 12.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-20-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Bug#1063758: spamd: /etc/init.d/spamd still uses --name instead of --exec

[Noah Meyerhans]

On Wed, Apr 24, 2024 at 10:02:47AM +0200, Alessandro Vesely wrote:
> > > Stopping S.M.A.R.T. daemon smartd.
> > > Stopping SpamAssassin Mail Filter Daemon spamd.
> > > start-stop-daemon: warning: this system is not able to track process names
> > > longer than 15 characters. please use --exec instead of --name.
> > > 
> > > Next it hangs there for several seconds (~30).  The next string says:
> > > 
> > > Asking all remaining processes to terminate...done.
> > > 
> > > Is it not SpamAssassin causing that warning?
> > 
> > If it is, it doesn't do it on Debian.  Even with sysvinit, though, the
> > stop jobs during shutdown run in parallel, so that message may be coming
> > from somewhere else.
> > 
> > If you manually stop spamassassin with `/etc/init.d/spamd stop`, do you
> > get the same warning?
> 
> 
> Good question!  No, I don't.  It stops cleanly and quickly.

You may need to stop/start each service individually to find the
culprit.  We can reassign this bug accordingly when you do.

noah

Bug#1063758: spamd: /etc/init.d/spamd still uses --name instead of --exec

[Noah Meyerhans]

On Sun, Apr 14, 2024 at 02:09:05PM +0200, Alessandro Vesely wrote:
> Sorry for the delay.  Today a new kernel image was loaded, so I had to
> reboot. I attach a screenshot of the closing session.  Near the bottom, it
> says:
> 
> Stopping S.M.A.R.T. daemon smartd.
> Stopping SpamAssassin Mail Filter Daemon spamd.
> start-stop-daemon: warning: this system is not able to track process names
> longer than 15 characters. please use --exec instead of --name.
> 
> Next it hangs there for several seconds (~30).  The next string says:
> 
> Asking all remaining processes to terminate...done.
> 
> Is it not SpamAssassin causing that warning?

If it is, it doesn't do it on Debian.  Even with sysvinit, though, the
stop jobs during shutdown run in parallel, so that message may be coming
from somewhere else.

If you manually stop spamassassin with `/etc/init.d/spamd stop`, do you
get the same warning?

noah

Bug#1064235:

[Noah Meyerhans]

On Thu, Apr 18, 2024 at 10:17:27AM +0200, Arthur LUTZ wrote:
>Hi,
>A small heads up on this issue, we have some minor impacts :
>* sudo commands show a warning
> 
>  sudo: unable to resolve host example: Name or service not known
> 
>On a more problematic side (but not breaking) we have filebeat that
>generates a lot of logs with warnings  related to the change :
>* Failed looking up CNAME: lookup ip-10-31-51-7 on [1]10.31.48.2:53: no
>such host: failed looking up IP: lookup ip-10-31-51-7 on [2]10.31.48.2:53:
>no such host, using hostname = ip-10-31-51-7 as FQDN
>Our workaround so far is to install `libnss-resolve` in our packer build
>of the AMI that is based on the debian bookworm AMI.

Indeed, we needed to install libnss-myhostname in addition to removing
libnss-resolve.  This will be fixed in a future release, but in the
meantime you can work around it by installing libnss-myhostname
yourself.

noah

Bug#1068409: dovecot-core: avoid linking against libsystemd0

[Noah Meyerhans]

Control: tags -1 + upstream

On Tue, Apr 16, 2024 at 02:05:17PM -0700, Noah Meyerhans wrote:
> > in light of the recent xz security breach, I'd like to ask if it
> > would be possible to rework systemd readiness notification and socket
> > activation patches to not link against libsystemd as just achieved for
> > the openssh-server package in version 1:9.7p1-4 ?
> > This would avoid /usr/bin/dovecot being linked also to three compression
> > libraries (liblz4, liblzma, libzstd) and to libgpg-error.
> 
> Yes, I believe this is reasonable.  I believe the systemd upstream
> maintainers have just released an updated MIT-0 licensed example of the
> socket activation patches that avoids requiring libsystemd0.  I'll see
> about adapting this patch to dovecot.

The openssh package's change is at
https://salsa.debian.org/ssh-team/openssh/-/commit/cc5f37cb8405cba624a133f4b8f464fbe381c5c8

As Dovecot natively supports systemd, rather than adding support for it
via a downstream modification as openssh does, it'd be best for this
change to be handled upstream.

noah

Bug#1061445: linux-image-6.7-cloud-amd64: Built CONFIG_VIRTIO_BLK into kernel

[Noah Meyerhans]

On Wed, Apr 03, 2024 at 06:26:46PM +0200, Paul Menzel wrote:
> Sorry, I have the feeling we talk past each other. I do not want to create
> an initrd. I want to boot *without* an initrd, and the only missing piece is
> building VIRTIO_BLK into the Linux kernel.
> 
> Ubuntu also builds this into their “kvm” flavour [1].
> 
> If you think, that is unnecessary, could you please elaborate, how I would
> achieve the goal with virtiofs?

The cloud kernel generally targets VM guests on the Microsoft Azure and
Amazon EC2 cloud environments, neither of which benefit from VIRTIO_BLK
driver being statically linked as you describe.  I think that's the
primary reason for reluctance to make your requested change.

For background, the Azure and AWS clouds present well-defined device
models, making it straightforward for us to construct targeted kernel
configs for them.

noah

Bug#1068409: dovecot-core: avoid linking against libsystemd0

[Noah Meyerhans]

On Thu, Apr 04, 2024 at 08:34:28PM +0200, Jörg-Volker Peetz wrote:
> in light of the recent xz security breach, I'd like to ask if it
> would be possible to rework systemd readiness notification and socket
> activation patches to not link against libsystemd as just achieved for
> the openssh-server package in version 1:9.7p1-4 ?
> This would avoid /usr/bin/dovecot being linked also to three compression
> libraries (liblz4, liblzma, libzstd) and to libgpg-error.

Yes, I believe this is reasonable.  I believe the systemd upstream
maintainers have just released an updated MIT-0 licensed example of the
socket activation patches that avoids requiring libsystemd0.  I'll see
about adapting this patch to dovecot.

noah

Bug#1068478: dovecot-core: Wildcard !include statements fail if nothing matches

[Noah Meyerhans]

Control: tags -1 + confirmed upstream
Control: severity -1 minor

On Fri, Apr 05, 2024 at 10:04:29PM +, Einhard Leichtfuß wrote:
> when the Dovecot configuration contains an `!include` statement with a
> wildcard that does not match anything, dovecot prints an error and
> terminates.
> 
> Expected behaviour: Dovecot processes the configuration as if the
> `!include` statement was not present.
> 
> The upstream on-line documentation [0] says on `!include`:
> > It’s not an error if wildcards don’t result in any matching files.
> 
> [0] 
> 

Confirmed based on your repro steps.  The documentation does indeed
claim that this should not result in an error, but that's not consistent
with the code.  Some relevant snippets of code from
src/config/config-parser.c:

from config_parse_line():
if (strcmp(key, "!include") == 0)
return CONFIG_LINE_TYPE_INCLUDE;
if (strcmp(key, "!include_try") == 0)
return CONFIG_LINE_TYPE_INCLUDE_TRY;

This return value is later handled with a case statement in 
config_parser_apply_line():
case CONFIG_LINE_TYPE_INCLUDE:
case CONFIG_LINE_TYPE_INCLUDE_TRY:
(void)settings_include(ctx, fix_relative_path(value, 
ctx->cur_input),
   type == CONFIG_LINE_TYPE_INCLUDE_TRY);
break;

The result of the "type == CONFIG_LINE_TYPE_INCLUDE_TRY" statement is
passed as the bool ignore_errors parameter to bool ignore_errors(), so
if it evaluates to false as it does when type ==
CONFIG_LINE_TYPE_INCLUDE, then we return an error:

case GLOB_NOMATCH:
if (ignore_errors)
return 0;
ctx->error = "No matches";
return -1;

I will pass this along to upstream.  It's not clear from here whether
the issue is with the code or with the documentation.

noah

Bug#1064235: cloud.debian.org: systemd-resolved et al. status (bookworm)

[Noah Meyerhans]

On Wed, Apr 03, 2024 at 09:39:40PM -0700, Flavio Veloso Soares wrote:
> Hi Noah - I guess I'll be doing bullseye->bookworm installs in the meantime,
> until 12.6 so I can fill bug reports (if any).

It should be plenty to start with the bookworm images and simply remove
the libnss-resolve package.  It's quite a lot simpler.

noah

Bug#1064235: cloud.debian.org: systemd-resolved et al. status (bookworm)

[Noah Meyerhans]

Control: tags -1 + pending

On Wed, Apr 03, 2024 at 06:48:56PM -0700, Flavio Veloso Soares wrote:
> Is this the correct forum to report issues/ask question about the official
> Debian images on AWS?

The BTS isn't really the correct forum to ask questions about any Debian
topics, but it is the right place to report issues.

Note that the plan, as discussed in team meetings and described at
https://lists.debian.org/debian-cloud/2024/03/msg00018.html, is to stop
installing libnss-resolve as of the upcoming 12.6 release.  That will
effectively remove systemd-resolved from the name resolution process,
leaving it only to manage /etc/resolv.conf.

noah

Bug#1063758: spamd: /etc/init.d/spamd still uses --name instead of --exec

[Noah Meyerhans]

Control: tags -1 + moreinfo

On Mon, Feb 12, 2024 at 10:42:31AM +0100, Alessandro Vesely wrote:
> /etc/init.d/spamd still uses --name instead of --exec.
> This is noticeable on shutdown, what the system waits 
> for some time trying to kill spamd, and then complains 
> something about its inability to track process names 
> and suggests to use --exec injstead.

I do not experience this when running under sysvinit on Debian 12 with
the following package versions:

ii  init-system-helpers 1.65.2   all  helper tools for all init 
systems
ii  spamd   4.0.0-6  all  Server for SpamAssassin spam 
filtering daemon
ii  sysvinit-core   3.06-4   amd64System-V-like init

When stopping spamd, there is no indication of any delay or other issue
terminating spamd.

Manual termination:
root@debian-sysv:~# /etc/init.d/spamd status
spamd is running.
root@debian-sysv:~# /etc/init.d/spamd stop
Stopping SpamAssassin Mail Filter Daemon: spamd.
root@debian-sysv:~# /etc/init.d/spamd status
spamd is not running ... failed!

Console output on shutdown:
root@debian-sysv:~# /etc/init.d/spamd status
spamd is running.
root@debian-sysv:~# shutdown -h now
...
Stopping SpamAssassin Mail Filter Daemon: spamd.
...

What is the specific warning you're seeing?  Is spamd actually running
when the init script is executing?

noah

Bug#1062955: debian-cloud-images: unsatisfiable dependencies

[Noah Meyerhans]

On Sun, Feb 04, 2024 at 09:24:30AM +0200, Graham Inggs wrote:
> debian-cloud-images 0.0.7 introduced a dependency on systemd-timesyncd
> (not mentioned in the changelog), which makes the dependencies of
> debian-cloud-images-packages unsatisfiable [1] due to a conflict with
> chrony.
> 
> The dependencies of debian-cloud-images-packages=0.0.7 cannot be
> satisfied in unstable on amd64, arm64, and ppc64el because: conflict
> between chrony and systemd-timesyncd

Interesting.  I don't think this actually matters, as
debian-cloud-images doesn't actually contain any content and isn't
something you'd ever install.  It only exists to list the set of
packages on which the cloud images depend.  The dependency list is
autogenerated, which explains the lack of an exhaustive changelog.

We have images that use systemd-timesyncd and images that use chrony,
which is why they're both on the list of dependencies.  So the
dependency list is correct and accurate, and I'm inclined to tag this
bug "sid" and "wontfix" and leave it at that.  But what I'm not sure
about is whether we need this to propagate to testing.

noah

Bug#1060421: python3-botocore: botocore as a (useless) undeclared dependency on python3-six

[Noah Meyerhans]

On Fri, Jan 12, 2024 at 08:00:48AM +0100, Alexandre Detiste wrote:
> >> Or should the Debian package depend on python3-six?
> >
> > That s the fast solution
> 
> Yes please do that, I've checked awscli upstream and it's a bit of a mess
> with so many pull request; I'll check again later.

Here's the upstream change responsible for six being present in
botocore's compat.py: https://github.com/boto/botocore/pull/2814

So it seems that adding the dependency is the right thing for now, and I
will take care of that.

noah

Bug#1060760: python-s3transfer: please update to 0.10.0

[Noah Meyerhans]

Source: python-s3transfer
Severity: wishlist

Newer versions of python-boto3 will require python-s3transfer >= 0.10.0, so in
order to unblock those I'd like to request that python-s3transfer be updated
soon.

I'm happy to perform the upload myself with the maintainer's acknowledgement.

Thank you!
noah

-- System Information:
Debian Release: 12.4
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-17-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Bug#1060455: awscli: Please drop python-cryptography upper limit or increase it

[Noah Meyerhans]

On Thu, Jan 11, 2024 at 02:27:23PM -0500, Jeremy Bícha wrote:
> There is a diff at
> https://launchpad.net/ubuntu/+source/awscli/2.14.6-1ubuntu1 for this
> issue. You can ignore the build failure since that doesn't seem to
> currently affect Debian Unstable.

Thanks, I've applied that patch and will include it with the upcoming
2.15.9-1 package.

Bug#1059744: spamassassin-maintenance does not invoke amavisd-new update-hook

[Noah Meyerhans]

On Sun, Dec 31, 2023 at 11:39:42AM +0100, Damian wrote:
> The fix for #1028030 introduced a regression for amavisd-new. Its hook [1] is 
> never run as long as spamd is not installed.

Right, it seems like this:
if [ -d /etc/spamassassin/sa-update-hooks.d ]; then
run-parts --lsbsysinit /etc/spamassassin/sa-update-hooks.d
fi

should not be conditional on spamd running.

> I'd also like an opinion on a related issue: When spamd is running but
> amavisd-new is not, spamassassin-maintenance.service fails by virtue of [1]
> failing with a dedicated exit code. Would you like to consider this a
> spamassassin or an amavisd-new bug?
> 
> [1] /etc/spamassassin/sa-update-hooks.d/amavisd-new

I think the amavisd-new hook probably shouldn't exit nonzero if it's not
running.  But we can also ensure that we fall through to an explicit
"exit 0" in the maintenance script, which should also take care of this.

noah

Bug#1058652: RM: python-boto -- ROM; orphaned upstream, replaced by python-boto3

[Noah Meyerhans]

Package: ftp.debian.org
Severity: normal
User: ftp.debian@packages.debian.org
Usertags: remove
X-Debbugs-Cc: python-b...@packages.debian.org
Control: affects -1 + src:python-boto

Python-boto has long ago been superseded upstream by python-boto3, which has
been in Debian for several releases.  Python-boto is not maintained upstream
and is not being ported to modern python versions.

On behalf of the cloud team, please remove python-boto from unstable testing
(assuming the latter doesn't happen automatically following removal from
unstable).

Thanks
noah

Bug#1058237: python-boto: FTBFS: ModuleNotFoundError: No module named 'imp'

[Noah Meyerhans]

On Tue, Dec 12, 2023 at 08:14:41PM +0100, Andreas Tille wrote:
> maybe we need to remove python-boto and fix all its rdepends.
> 
> According to upstream[1]:
> 
>   This repo, boto was deprecated in 2020 and the last supported Python 3 
> version was 3.4. It's unlikely this package will ever work with Python 3.12.

Yes, python-boto should be removed from the archive.  Really it should
have been removed prior to bookworm, if not even earlier.

I'll take care of filing the appropriate bugs.

noah

Bug#1055454: spamassassin: Spamassassin in Bookworm dropped SysVinit script

[Noah Meyerhans]

On 11/6/2023 2:53 PM, Patrik Schindler wrote:
The removal of the sysvinit script was intentional. Per Debian policy 
section 9.3.1, "Packages including a service unit may optionally 
include an init script to support other init systems". Spamd provides 
a service unit. There is no requirement to support any other startup 
mechanisms or init systems, and I have no interest in supporting the 
old init script anymore.

Thank you for your clear words.


Hi Patrik. I apologize; I was getting ahead of myself when indicating 
that the spamassassin init script is gone by design. It's not, 
actually.  It's just moved to the spamd package, which was broken out 
from the base spamassassin package. /etc/init.d/spamd is what you're 
looking for.


noah

Bug#1056001: spamd: user_prefs include files are only read on the first scan

[Noah Meyerhans]

Control: tags -1 + pending


On 11/15/2023 2:31 PM, Martin Schwenke wrote:

This was reported upstream via:

   https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8130

and fixed in the trunk via commit:

   
https://svn.apache.org/viewvc/spamassassin/trunk/lib/Mail/SpamAssassin.pm?r1=1913789=1913803_format=h



Thank you.  Fixed with 
https://salsa.debian.org/debian/spamassassin/-/commit/5e3d193894de527a22dac427a308a4387dc4d890



Will upload this to unstable in the not-too-distant future.


noah

Bug#1055478: nmu: dovecot-antispam_2.0+20171229-1+b12

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu
X-Debbugs-Cc: dovecot-antis...@packages.debian.org
Control: affects -1 + src:dovecot-antispam

nmu dovecot-antispam_2.0+20171229-1+b12 . ANY . unstable . -m "Rebuild for 
dovecot 2.3.21"

dovecot-antispam needs to be rebuilt for the dovecot 2.3.21 ABI.

Bug#1055477: nmu: dovecot-fts-xapian_1.5.5-1+b3

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu
X-Debbugs-Cc: dovecot-fts-xap...@packages.debian.org
Control: affects -1 + src:dovecot-fts-xapian

nmu dovecot-fts-xapian_1.5.5-1+b3 . ANY . unstable . -m "Rebuild for dovecot 
2.3.21"

dovecot-fts-xapian needs to be rebuilt for the dovecot 2.3.21 ABI.

Bug#1055454: spamassassin: Spamassassin in Bookworm dropped SysVinit script

[Noah Meyerhans]

Control: tags -1 + wontfix
Control: severity -1 wishlist

On 11/6/2023 8:41 AM, Patrik Schindler wrote:

Surprisingly, after upgrading to Bookworm, the SysVinit script to start
spamassassin is no longer there. This renders spamassassin unusable for
SysVinit users without manual intervention.

Because SysVinit is still supported in Debian in general, please revive
SysVinit support.


The removal of the sysvinit script was intentional.  Per Debian policy 
section 9.3.1, "Packages including a service unit may optionally include 
an init script to support other init systems". Spamd provides a service 
unit. There is no requirement to support any other startup mechanisms or 
init systems, and I have no interest in supporting the old init script 
anymore.



Sorry.

noah

Bug#1020217: S3-backed snapshot implementation on AWS?

[Noah Meyerhans]

On Sun, Sep 24, 2023 at 04:09:31PM -0700, Noah Meyerhans wrote:
> > > Could we use the Debian AWS account to host that service?
> > 
> > I would assume that a service like snapshot would be within the scope
> > for our AWS usage.  Noah?
> 
> It makes sense and I will look into it.  Let's not start anything until
> we hear definitive confirmation.

OK, let's do it.

noah

Bug#1020217: S3-backed snapshot implementation on AWS?

[Noah Meyerhans]

On Fri, Sep 22, 2023 at 05:12:21PM +0200, Bastian Blank wrote:
> > Could we use the Debian AWS account to host that service?
> 
> I would assume that a service like snapshot would be within the scope
> for our AWS usage.  Noah?

It makes sense and I will look into it.  Let's not start anything until
we hear definitive confirmation.  Do we have a sense of how much
outgoing traffic the current snapshot service generates?

> >   It would
> > require one fairly powerful VM, and a large S3 bucket (approximately
> > 150-200 TB).
> 
> 200 TB should be no problem.

Agreed.

> However we need to talk about that "one […] VM", because this sounds
> like you intend to use AWS as VM hosting, which it is not.
> 
> Please think about this in form of services and there should be at least
> two:
> - the injestor, which can only exist once and writes, and
> - the web frontend, which should be able to exist several times and only
>   reads.
> 
> So you want to plan with running the multiple web frontends with load
> balancers and maybe even cloudfront.

I agree that it would be best to design something more cloud-oriented.
However, if there's an existing infrastructure that can be moved as a
"lift & shift" into AWS now, with architectural refactoring happening
later, that's an OK place to start.

noah

Bug#1040884: Acknowledgement (dovecot: stop shipping unsupported lucene fts backend)

[Noah Meyerhans]

This is done with https://salsa.debian.org/debian/dovecot/-/merge_requests/33

Bug#1040979: write_files: defer not working any more

[Noah Meyerhans]

Also, I'm curious to see what happens if you invoke the 
write_files_deferred module by hand with a command like


$ sudo cloud-init single --name cc_write_files_deferred \
  --frequency always --report

It should generate messages in /var/log/cloud-init.log.

Bug#1040979: write_files: defer not working any more

[Noah Meyerhans]

On 7/14/23 06:49, Sven Strickroth wrote:

I'm not able to reproduce the behavior you describe.  defer seems to
work as expected.  For example:

admin@ip-10-0-0-87:~$ ec2-metadata --ami-id
ami-id: ami-0544719b13af6edc3
admin@ip-10-0-0-87:~$ sudo cat /var/lib/cloud/instance/user-data.txt
#cloud-config
write_files:
- content: |
 testing. this file should be owned by the dynamically created user
   path: /test-file
   owner: admin:admin
   defer: true
admin@ip-10-0-0-87:~$ ls -l /test-file
-rw-r--r-- 1 admin admin 67 Jul 13 16:59 /test-file


That example does not work for me on Hetzner.cloud.


Strange. The functionality works as expected for me using the current 
bookworm AMIs on Amazon EC2, as shown in the output below.  I don't have 
an account with Hetzner, so I'm afraid you're going to have to collect 
some more information.


Can you show that the "defer" field in your cloud-config is making 
through to the input that's passed to cloud-init?


Can you show that write_files works without the defer flag?

The write_files modules are defined to run once per instance.  Is it 
possible that your launches are not showing up as new instances, 
resulting in cloud-init skipping their execution?


noah

admin@ip-10-0-0-199:~$ sudo cat /var/lib/cloud/instance/user-data.txt
#cloud-config
locale: de_DE.UTF-8
timezone: Europe/Berlin
package_update: true # Implied with `package_upgrade: true`
package_upgrade: true
packages: 
['joe','mariadb-server','aptitude','psmisc','bzip2','fail2ban','postfix','apticron','nagios-nrpe-server','monitoring-plugins','libmariadb3','borgbackup','logwatch']

write_files:
- content: |
[mysqldump]
max_allowed_packet = 60M
  path: /etc/mysql/mariadb.conf.d/51-dump.cnf
  defer: true
admin@ip-10-0-0-199:~$ cat /etc/mysql/mariadb.conf.d/51-dump.cnf
[mysqldump]
max_allowed_packet = 60M
admin@ip-10-0-0-199:~$ grep write-files-deferred /var/log/cloud-init.log
2023-07-14 21:28:41,034 - modules.py[DEBUG]: Running module 
write-files-deferred (from 
'/usr/lib/python3/dist-packages/cloudinit/config/cc_write_files_deferred.py'>) 
with frequency once-per-instance
2023-07-14 21:28:41,034 - handlers.py[DEBUG]: start: 
modules-final/config-write-files-deferred: running 
config-write-files-deferred with frequency once-per-instance
2023-07-14 21:28:41,036 - helpers.py[DEBUG]: Running 
config-write-files-deferred using lock ('/var/lib/cloud/instances/i-096c9d6b1c3c04be6/sem/config_write_files_deferred'>)
2023-07-14 21:28:41,036 - handlers.py[DEBUG]: finish: 
modules-final/config-write-files-deferred: SUCCESS: 
config-write-files-deferred ran successfully

Bug#1040979: write_files: defer not working any more

[Noah Meyerhans]

Control: tags -1 + moreinfo

On Thu, Jul 13, 2023 at 05:30:52PM +0200, Sven Strickroth wrote:
> starting with the shipped version in Debian 12 the write_files feature with 
> defer option does not work any more.

I'm not able to reproduce the behavior you describe.  defer seems to
work as expected.  For example:

admin@ip-10-0-0-87:~$ ec2-metadata --ami-id
ami-id: ami-0544719b13af6edc3
admin@ip-10-0-0-87:~$ sudo cat /var/lib/cloud/instance/user-data.txt
#cloud-config
write_files:
- content: |
testing. this file should be owned by the dynamically created user
  path: /test-file
  owner: admin:admin
  defer: true
admin@ip-10-0-0-87:~$ ls -l /test-file 
-rw-r--r-- 1 admin admin 67 Jul 13 16:59 /test-file

And in /var/log/cloud-init.log we see it being handled as expected:
2023-07-13 16:59:53,893 - modules.py[DEBUG]: Running module 
write-files-deferred () 
with frequency once-per-instance
2023-07-13 16:59:53,893 - handlers.py[DEBUG]: start: 
modules-final/config-write-files-deferred: running config-write-files-deferred 
with frequency once-per-instance
2023-07-13 16:59:53,893 - util.py[DEBUG]: Writing to 
/var/lib/cloud/instances/i-016aa1a2f2e89c783/sem/config_write_files_deferred - 
wb: [644] 24 bytes
2023-07-13 16:59:53,893 - helpers.py[DEBUG]: Running 
config-write-files-deferred using lock ()
2023-07-13 16:59:53,893 - util.py[DEBUG]: Writing to /test-file - wb: [644] 67 
bytes
2023-07-13 16:59:53,894 - util.py[DEBUG]: Changing the ownership of /test-file 
to 1000:1001
2023-07-13 16:59:53,894 - handlers.py[DEBUG]: finish: 
modules-final/config-write-files-deferred: SUCCESS: config-write-files-deferred 
ran successfully

I note that your cloud-config attempts to set the locale:

> #cloud-config
> locale: de_DE.UTF-8

I suspect that that's failing (see #955733) and interfering with
subsequent module execution.  If you remove the `locale` setting, does
your userdata work as expected?

noah

Bug#1040884: dovecot: stop shipping unsupported lucene fts backend

[Noah Meyerhans]

Source: dovecot
Version: 1:2.3.19.1+dfsg1-2.1
Severity: normal

The Dovecot Lucene FTS backend is not supported and should not be built or
shipped by Debian.  Supported alternatives (dovecot-solr and
dovecot-fts-xapian) are available in Debian and users should migrate to those
instead.


-- System Information:
Debian Release: 12.0
  APT prefers stable-security
  APT policy: (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-10-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Bug#1040665: nmu: dovecot-antispam_2.0+20171229-1+b11

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu
X-Debbugs-Cc: dovecot-antis...@packages.debian.org
Control: affects -1 + src:dovecot-antispam

nmu dovecot-antispam_2.0+20171229-1+b11 . ANY . unstable . -m "Rebuild for 
dovecot 1:2.3.20+dfsg1-1 ABI"

dovecot-antispam needs to be rebuilt against the dovecot 1.3.20 internal ABI in
order to allow the transition to testing.

Bug#1040648: nmu: dovecot-fts-xapian_1.5.5-1+b2

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu
X-Debbugs-Cc: dovecot-fts-xap...@packages.debian.org
Control: affects -1 + src:dovecot-fts-xapian

nmu dovecot-fts-xapian_1.5.5-1+b2 . ANY . unstable . -m "Rebuild for dovecot 
1:2.3.20+dfsg1-1 ABI"

dovecot-fts-xapian needs to be rebuilt against the dovecot 1.3.20 internal ABI
in order to allow the transition to testing.

Bug#955620: cloud-init - debian/rules clean fails from git repo

[Noah Meyerhans]

Control: tags -1 + patch

Proposed fix is at 
https://salsa.debian.org/cloud-team/cloud-init/-/merge_requests/9

Bug#955733: cloud-init does not set locale specified in user data

[Noah Meyerhans]

On Sat, Apr 04, 2020 at 11:49:42AM +0100, Nick Holloway wrote:
> Package: cloud-init
> Version: 18.3-6

Confirmed that the behavior as described is still an issue in cloud-init
22.4.2-1 in bookworm.

> The command "locale-gen" does not accept a locale as an argument, but
> only generates the locales listed in "/etc/locale.gen".  So the locale
> requested does not get generated, and causes the subsequent failure when
> setting locale.
> 
> 2020-04-04 10:00:43,196 - util.py[DEBUG]: Running command 
> ['update-locale', '--locale-file=/etc/default/locale', 'LANG=en_GB.UTF-8'] 
> with allowed return codes [0] (shell=False, capture=False)
> 

I think that each of the components is working correct if viewed in
isolation, but the end result is not what you expect.  If we document
that you must first populate /etc/locale.gen with the desired list of
available locales, that may be sufficient.  For example, the following
sets the desired locale:

admin@ip-10-0-3-7:~$ sudo cat /var/lib/cloud/instance/user-data.txt
#cloud-config
write_files:
- path: /etc/locale.gen
  content: ZW5fR0IuVVRGLTggVVRGLTgK
  encoding: base64
locale: en_GB.UTF-8
admin@ip-10-0-3-7:~$ echo "ZW5fR0IuVVRGLTggVVRGLTgK" | openssl base64 -d
en_GB.UTF-8 UTF-8

Do Ubuntu systems list all known locales in /etc/locale.gen?  That would
make the locale module work without additional configuration, but seems
unnecessarily expensive.

noah

Bug#1037453: libmail-dmarc-perl: FTBFS with test failures when there's no network

[Noah Meyerhans]

Control: tags -1 + pending


On 6/12/2023 8:58 PM, Steve Langasek wrote:

I've applied the attached (very dirty) patch to libmail-dmarc-perl in
Ubuntu, which is sufficient to let the package build in Launchpad.  Having
looked at some of the surrounding tests, I'm not sure this would let the
package build in a completely offline environment - I think Launchpad
provides name resolution of a limited subset of domains, so some tests which
pass in Launchpad may also fail when offline.


Thanks. It seems to build in a completely disconnected environment with 
your patch applied, so let's start with it.

Bug#1034639: unblock: spamassassin/4.0.0-5

[Noah Meyerhans]

Control: tags -1 - moreinfo

On Sun, Apr 30, 2023 at 08:08:55AM +0200, Paul Gevers wrote:
> > Please go ahead and remove the moreinfo tag once the upload happened.
> 
> Already several years we don't accept uploader built binaries in testing.
> Unfortunately, we can't binNMU arch:all binaries, so please upload a
> no-change *source only* version 4.0.0-6.

*sigh* Done, with apologies.

Bug#1034347: spamd timeout after reboot: dns: bad dns reply: bgread: recv() failed

[Noah Meyerhans]

On Wed, Apr 19, 2023 at 07:50:32AM +0800, Martin Michlmayr wrote:
> * Martin Michlmayr  [2023-04-13 18:23]:
> > A Google search has led to:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1985587
> > The proposed patch seems relevant to Debian's spamd service file too:
> > https://src.fedoraproject.org/rpms/spamassassin/pull-request/12#request_diff
> 
> I've now tested this change to the systemd file and it makes it work
> as expected.
> 
> Can you please add this fix as an update to bookworm.

Requested an unblock for bookworm at 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034639

Bug#1034639: unblock: spamassassin/4.0.0-5

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: unblock
X-Debbugs-Cc: spamassas...@packages.debian.org
Control: affects -1 + src:spamassassin

Please unblock package spamassassin

This is a targeted change that addresses #1034347, in which spamd can
sometimes be started with incorrect DNS server configuration.  This happens
because spamd startup happened too early in the boot process, and services
responsible for populating resolv.conf had not necessarily run yet.  The
solution was to order spamd.service After=network-online.target, as shown in
the attached debdiff.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034347

[ Reason ]
Spamd needs a valid DNS configuration for complete operation.

[ Impact ]
Without working DNS, spamd is unable to make use of many of its core mail
validation services, which diminishes its usefulness.

[ Tests ]
Manual validation by myself and the bug submitter.

[ Risks ]
It's a trivial change to the boot order and should not impact anything that
depends on spamd.

[ Checklist ]
  [x] all changes are documented in the d/changelog
  [x] I reviewed all changes and I approve them
  [x] attach debdiff against the package in testing

unblock spamassassin/4.0.0-5
diff -Nru spamassassin-4.0.0/debian/changelog 
spamassassin-4.0.0/debian/changelog
--- spamassassin-4.0.0/debian/changelog 2023-01-06 13:05:09.0 -0800
+++ spamassassin-4.0.0/debian/changelog 2023-04-16 13:28:05.0 -0700
@@ -1,3 +1,9 @@
+spamassassin (4.0.0-5) unstable; urgency=medium
+
+  * Update spamd.service to order after network-online.target (Closes: 
#1034347)
+
+ -- Noah Meyerhans   Sun, 16 Apr 2023 13:28:05 -0700
+
 spamassassin (4.0.0-4) unstable; urgency=medium
 
   * Revert "drop unused params from Makefile.PL"
diff -Nru spamassassin-4.0.0/debian/spamd.service 
spamassassin-4.0.0/debian/spamd.service
--- spamassassin-4.0.0/debian/spamd.service 2023-01-06 12:32:43.0 
-0800
+++ spamassassin-4.0.0/debian/spamd.service 2023-04-16 13:04:49.0 
-0700
@@ -1,6 +1,6 @@
 [Unit]
 Description=Perl-based spam filter using text analysis
-After=network.target
+After=syslog.target network-online.target
 
 [Service]
 Type=simple

Bug#1034427: spamd fails to filter

[Noah Meyerhans]

On Sat, Apr 15, 2023 at 03:56:02AM -0400, Ed Lawson wrote:
> 23-04-13T16:2056:26.075987-04:00 800-mini-1 spamd[938]: spamd: connection 
> from localhost [::1]:44842 to port 783, fd 5
> 2023-04-13T16:56:26.077471-04:00 800-mini-1 spamd[938]: spamd: setuid to 
> elawson succeeded
> 2023-04-13T16:56:26.089459-04:00 800-mini-1 spamd[938]: spamd: checking 
> message  for elawson:1000
> 2023-04-13T16:56:26.135987-04:00 800-mini-1 spamd[938]: dns: sendto() to 
> [127.0.0.1]:53 failed: Connection refused, failing over to [::1]:53
> 2023-04-13T16:56:26.137769-04:00 800-mini-1 spamd[938]: dns: sendto() to 
> [::1]:53 failed: Connection refused, failing over to [127.0.0.1]:53

These don't necessarily indicated an inability to filter messages, but
they are indicating an inability to talk to the DNS resolver on
localhost with either IPv4 or IPv6.  Is that the DNS server you expect
spamassassin to be using?  What's in `/etc/resolv.conf`?

The symptoms seem similar to those reported in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034347

noah

Bug#1034368: unblock: amazon-ec2-net-utils/2.3.0-3

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: unblock
X-Debbugs-Cc: amazon-ec2-net-ut...@packages.debian.org
Control: affects -1 + src:amazon-ec2-net-utils

Please unblock package amazon-ec2-net-utils

This is a targeted change to work around the debhelper dh_installsystemd issues
with management of unit files in /usr/lib/systemd/system (background in
#1031695). This closes RC bug #1034212 by moving these files to
/lib/systemd/system as shown below:

$ debdiff amazon-ec2-net-utils_2.3.0-2.dsc amazon-ec2-net-utils_2.3.0-3.dsc 
   
dpkg-source: warning: extracting unsigned source package 
(/home/noahm/src/debian/cloud/amazon-ec2-net-utils_2.3.0-2.dsc)
dpkg-source: warning: extracting unsigned source package 
(/home/noahm/src/debian/cloud/amazon-ec2-net-utils_2.3.0-3.dsc)
diff -Nru amazon-ec2-net-utils-2.3.0/debian/changelog 
amazon-ec2-net-utils-2.3.0/debian/changelog
--- amazon-ec2-net-utils-2.3.0/debian/changelog 2023-01-21 11:25:53.0 
-0800
+++ amazon-ec2-net-utils-2.3.0/debian/changelog 2023-04-13 10:22:32.0 
-0700
@@ -1,3 +1,9 @@
+amazon-ec2-net-utils (2.3.0-3) unstable; urgency=medium
+
+  * Install systemd services to /lib/systemd/system. (Closes: 1034212)
+
+ -- Noah Meyerhans   Thu, 13 Apr 2023 10:22:32 -0700
+
 amazon-ec2-net-utils (2.3.0-2) unstable; urgency=medium
 
   * Set Maintainer to the cloud team and add myself to Uploaders
diff -Nru amazon-ec2-net-utils-2.3.0/debian/patches/debian-changes 
amazon-ec2-net-utils-2.3.0/debian/patches/debian-changes
--- amazon-ec2-net-utils-2.3.0/debian/patches/debian-changes2023-01-21 
11:25:53.0 -0800
+++ amazon-ec2-net-utils-2.3.0/debian/patches/debian-changes2023-04-13 
10:22:32.0 -0700
@@ -1,3 +1,16 @@
+diff --git a/GNUmakefile b/GNUmakefile
+index d06847d..dd077a7 100644
+--- a/GNUmakefile
 b/GNUmakefile
+@@ -6,7 +6,7 @@ PREFIX?=/usr/local
+ BINDIR=${DESTDIR}${PREFIX}/bin
+ UDEVDIR=${DESTDIR}/usr/lib/udev/rules.d
+ SYSTEMDDIR=${DESTDIR}/usr/lib/systemd
+-SYSTEMD_SYSTEM_DIR=${SYSTEMDDIR}/system
++SYSTEMD_SYSTEM_DIR=${DESTDIR}/lib/systemd/system
+ SYSTEMD_NETWORK_DIR=${SYSTEMDDIR}/network
+ SHARE_DIR=${DESTDIR}/${PREFIX}/share/${pkgname}
+ 
 diff --git a/lib/lib.sh b/lib/lib.sh
 index d01dd23..02357d9 100644
 --- a/lib/lib.sh
$ debdiff amazon-ec2-net-utils_2.3.0-2_amd64.changes 
amazon-ec2-net-utils_2.3.0-3_amd64.changes
[The following lists of changes regard files as different if they have
different names, permissions or owners.]

Files in second .changes but not in first
-
-rw-r--r--  root/root   /lib/systemd/system/policy-routes@.service
-rw-r--r--  root/root   /lib/systemd/system/refresh-policy-routes@.service
-rw-r--r--  root/root   /lib/systemd/system/refresh-policy-routes@.timer

Files in first .changes but not in second
-
-rw-r--r--  root/root   /usr/lib/systemd/system/policy-routes@.service
-rw-r--r--  root/root   /usr/lib/systemd/system/refresh-policy-routes@.service
-rw-r--r--  root/root   /usr/lib/systemd/system/refresh-policy-routes@.timer

Control files: lines which differ (wdiff format)

Installed-Size: [-44-] {+47+}
Version: [-2.3.0-2-] {+2.3.0-3+}

[ Impact ]

Without this change, systemd services and timers installed by
amazon-ec2-net-utils may not be activated as expected when the package is
installed.

[ Tests ]

n/a

[ Risks ]

Worst case, if everything goes horribly wrong, is that certain non-default
network configuration in Amazon EC2 won't be applied. This includes
configuration of secondary network interfaces and secondary IPv4 addresses on
any interfaces.

[ Checklist ]
  [x] all changes are documented in the d/changelog
  [x] I reviewed all changes and I approve them
  [x] attach debdiff against the package in testing

[ Other info ]
(Anything else the release team should know.)

unblock amazon-ec2-net-utils/2.3.0-3

Bug#1033867: cloud.debian.org: Please add Amazon hibernation agent to EC2 AMIs

[Noah Meyerhans]

On 4/4/2023 9:26 AM, Noah Meyerhans wrote:

Hi Dirk.  Since we don't have this in the archive yet, I've refiled this
as an RFP.  Once it's packaged, we can add it to the AMIs.
Well, *now* I've refiled it as an RFP, now that I can correctly spell 
"wnpp" :)


noah

Bug#1033867: cloud.debian.org: Please add Amazon hibernation agent to EC2 AMIs

[Noah Meyerhans]

Control: reassign -1 wnpp.debian.org
Control: retitle -1 RFP: amazon-ec2-hibinit-agent -- Amazon EC2 instance 
hibernation support

> So the request is to also ship the agent preinstalled in the Debian AMIs. See
> https://packages.ubuntu.com/search?keywords=ec2-hibinit-agent=names
> for Ubuntu (source) packages or 
> https://github.com/aws/amazon-ec2-hibinit-agent
> for latest sources.
> 
> I've already verified that it works on Debian with a local rebuild of
> the latest Ubuntu source deb.

Hi Dirk.  Since we don't have this in the archive yet, I've refiled this
as an RFP.  Once it's packaged, we can add it to the AMIs.

I've also verified in the past that it works on newer instance types,
though I believe hibernation of the older generation Xen based instance
types requires kernel patches that we don't have.  However, I think it's
fine to ignore those old instance types and focus on making things work
with modern ones.

noah

Bug#1033795: spamassassin: Spamassassin does start

[Noah Meyerhans]

On Sat, Apr 01, 2023 at 09:53:06AM -0400, Ed Lawson wrote:
> Spamassassin does not start and is not functional. I have two computers
> running Debian Sid and spamassissin is not working on either.  When I
> run systemctl status spamassassin it reports "Units spamassassin.service
> could not be found."  If I run systemctl enable spamassassin.service it
> reports "Unit file spamassassin.service does not exist."

The service name is spamd.service, not spamassassin.service.

Bug#1029847: offlineimap3: netrc support broken

[Noah Meyerhans]

Package: offlineimap3
Version: 0.0~git20211018.e64c254+dfsg-2
Severity: important

I have an offlineimap configuration that retrieves authentication material
from .netrc.  This configuration stopped working with the most recent update
in testing.  See below for the offlineimap command output.  Note that
nothing has changed with the server and if I remove ~/.netrc and copy &
paste the values from it to the interactive offlineimap password prompt,
authentication is successful.  The problem appears specific to .netrc.

OfflineIMAP 8.0.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v3.05, Python v3.11.1, OpenSSL 3.0.7 1 Nov 2022
Account sync morgul:
 *** Processing account morgul
 Establishing connection to quickpost.morgul.net:993 (Remote)
 PLAIN authentication failed: b'[AUTHENTICATIONFAILED] Authentication failed.'
 LOGIN authentication failed: b'[AUTHENTICATIONFAILED] Authentication failed.'
 ERROR: All authentication types failed:
PLAIN: b'[AUTHENTICATIONFAILED] Authentication failed.'
LOGIN: b'[AUTHENTICATIONFAILED] Authentication failed.'
 *** Finished account 'morgul' in 0:09
ERROR: Exceptions occurred during the run!
ERROR: All authentication types failed:
PLAIN: b'[AUTHENTICATIONFAILED] Authentication failed.'
LOGIN: b'[AUTHENTICATIONFAILED] Authentication failed.'

Traceback:
  File "/usr/share/offlineimap3/offlineimap/accounts.py", line 298, in 
syncrunner
self.__sync()
  File "/usr/share/offlineimap3/offlineimap/accounts.py", line 374, in __sync
remoterepos.getfolders()
  File "/usr/share/offlineimap3/offlineimap/repository/IMAP.py", line 681, in 
getfolders
imapobj = self.imapserver.acquireconnection()
  ^^^
  File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 593, in 
acquireconnection
self.__authn_helper(imapobj)
  File "/usr/share/offlineimap3/offlineimap/imapserver.py", line 453, in 
__authn_helper
raise OfflineImapError("All authentication types "




-- System Information:
Debian Release: bookworm/sid
  APT prefers testing
  APT policy: (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-1-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages offlineimap3 depends on:
ii  ca-certificates   20211016
ii  python3   3.11.1-1
ii  python3-distro1.8.0-1
ii  python3-imaplib2  2.57-5.2

offlineimap3 recommends no packages.

Versions of packages offlineimap3 suggests:
pn  python3-gssapi  

-- no debconf information

Bug#1028031: Acknowledgement (Main spamassassin.service file missing)

[Noah Meyerhans]

On 1/5/2023 7:11 PM, Matt Corallo wrote:
Ah, sorry for the noise, the package was split. Seems funny to do in a 
backport.


backports pull changes from what's in bookworm (testing). Nothing was 
"done" in the backport, it was a straight pull.  Preserving bullseye's 
package structure would have been significant packaging work and 
wouldn't constitute a backport or be appropriate for bullseye-backports.


Should the package split get a changelog notice for the bullseye 
upgrade? I'd assume most users of spamassassin actually use the spamd 
feature, so it being silently removed may be surprising for an upgrade.


I guess apt didn't show /usr/share/doc/spamassassin/NEWS.Debian.gz when 
you upgraded, but it's all documented there.


noah

Bug#1027925: spamassassin FTBFS on IPV6-only buildds

[Noah Meyerhans]

On Wed, Jan 04, 2023 at 06:58:30PM +0200, Adrian Bunk wrote:
> https://buildd.debian.org/status/logs.php?pkg=spamassassin=amd64
> 
> ...
> Jan  4 03:57:23.254 [3488924] dbg: logger: adding facilities: all
> Jan  4 03:57:23.255 [3488924] dbg: logger: logging level is DBG
> Jan  4 03:57:23.257 [3488924] dbg: logger: successfully opened file 
> log/sa_check_spamd.g3uk5R/d.sa_check_spamd/spamd.err.0.timestamped
> Jan  4 03:57:23.257 [3488924] dbg: logger: successfully added file method
> Jan  4 03:57:23.257 [3488924] dbg: spamd: will perform setuids? 0
> Jan  4 03:57:23.257 [3488924] dbg: spamd: socket module of choice: 
> IO::Socket::IP 0.41, Socket 2.033, have PF_INET, have PF_INET6, using legacy 
> Socket6::getaddrinfo, AI_ADDRCONFIG is supported
> Jan  4 03:57:23.257 [3488924] dbg: spamd:  socket specification: "127.0.0.1", 
> IP address: 127.0.0.1, port: 61558
> server socket setup failed, retry 1: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 2: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 3: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 4: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 5: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 6: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 7: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 8: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> server socket setup failed, retry 9: spamd: invalid address for a listen 
> socket: "127.0.0.1": Address family for hostname not supported
> spamd: invalid address for a listen socket: "127.0.0.1": Address family for 
> hostname not supported
> ...
> Failed 30/213 test programs. 162/3101 subtests failed.
> make[2]: *** [Makefile:1473: test_dynamic] Error 255

I haven't been able to reproduce this on VMs with IPv6-only networking.
Is the buildd network environment documented anywhere?  There's clearly
something different about it than my test environment, but I haven't
been able to figure out what it is.

In the meantime, I'm looking into printing more information about the
Network configuration when running these failing tests.  See 
https://salsa.debian.org/debian/spamassassin/-/commit/531bf8ea45cde94a60852d62ac701f70c0db4b3d

On my IPv6-only build host, these changes print:
IP-DEBUG: have_inet4 returning false
IP-DEBUG: have_inet6 returning true
IP-DEBUG: Set spamdlocalhost to ::1
IP-DEBUG: Set spamdhost to ::1


noahm@localhost:~/spamassassin$ ip -4 addr ; ip -4 ro ; dpkg-buildpackage -uc 
-us > /tmp/build.log 2>&1
echo $?
tail -n20 /tmp/build.log
noahm@localhost:~/spamassassin$ echo $?
0
noahm@localhost:~/spamassassin$ tail -n20 /tmp/build.log
   dh_makeshlibs -a
   dh_shlibdeps -a
   dh_installdeb
   dh_gencontrol
dpkg-gencontrol: warning: package spamassassin: substitution variable 
${perl:Depends} unused, but is defined
dpkg-gencontrol: warning: package sa-compile: substitution variable 
${perl:Depends} unused, but is defined
dpkg-gencontrol: warning: Depends field of package spamd: substitution variable 
${shlibs:Depends} used, but is not defined
dpkg-gencontrol: warning: package spamd: substitution variable ${perl:Depends} 
unused, but is defined
   dh_md5sums
   dh_builddeb
dpkg-deb: building package 'spamassassin' in '../spamassassin_4.0.0-3_all.deb'.
dpkg-deb: building package 'spamc' in '../spamc_4.0.0-3_amd64.deb'.
dpkg-deb: building package 'spamc-dbgsym' in 
'../spamc-dbgsym_4.0.0-3_amd64.deb'.
dpkg-deb: building package 'sa-compile' in '../sa-compile_4.0.0-3_all.deb'.
dpkg-deb: building package 'spamd' in '../spamd_4.0.0-3_all.deb'.
 dpkg-genbuildinfo -O../spamassassin_4.0.0-3_amd64.buildinfo
 dpkg-genchanges -O../spamassassin_4.0.0-3_amd64.changes
dpkg-genchanges: info: not including original source code in upload
 dpkg-source --after-build .
dpkg-buildpackage: info: binary and diff upload (original source NOT included)

Bug#1023345: spamassassin: plugin eval failed with unknown encoding 'MIME-Header'

[Noah Meyerhans]

On Wed, Nov 02, 2022 at 05:31:33PM +0100, Francesco Potortì wrote:
> Starting few days ago, I get many of these in syslog:
> 
> plugin: eval failed: Unknown encoding 'MIME-Header' at 
> /usr/share/perl5/Mail/SpamAssassin/Message.pm line 1090,  line 8342.
> plugin: eval failed: Unknown encoding 'MIME-Header' at 
> /usr/share/perl5/Mail/SpamAssassin/Message.pm line 1090,  line 248.
> plugin: eval failed: Unknown encoding 'MIME-Header' at 
> /usr/share/perl5/Mail/SpamAssassin/Message.pm line 1090,  line 2683.

Hi Francesco.  Are these issues still happening for you?  I haven't
encountered this myself, and don't see anything that could cause it on
first glance.  If it's still happening with spamassassin 4.0.0-1, please
provide additional debug output as generated when running spamd with the
--debug flag.

noah

Bug#1008281: setting sysctl net.ipv4.ping_group_range

[Noah Meyerhans]

On Tue, Jan 03, 2023 at 03:26:37AM +0100, Marco d'Itri wrote:
> > Debian's default sysctl settings should reside in procps (as it owns
> > /sbin/sysctl and /etc/sysctl* settings) rather than some unrelated
> > package.
> Nowadays systemd is a source of common sysctl settings among different 
> distributions.

I've opened https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1027773 to
request this sysctl setting be applied globally there.  Will keep this
bug open against iputils-ping to track the removal of the
cap_net_raw/setuid settings when that's done.

noah

Bug#1008281: setting sysctl net.ipv4.ping_group_range

[Noah Meyerhans]

On Tue, Jan 03, 2023 at 01:48:29AM +0100, Adam Borowski wrote:
> Most settings are in /etc/sysctl.conf, especially network related ones.
> 
> That /usr/lib/sysctl.d/ path doesn't have its settings applied normally.

systemd-sysctl is run by default and processes /usr/lib/sysctl.d/.  This
is the case even on systems that don't have procps installed.

If the intent is to set sysctl values _everywhere_, then neither procps
nor systemd is the right place.

noah

Bug#1008281: setting sysctl net.ipv4.ping_group_range

[Noah Meyerhans]

On Tue, Jan 03, 2023 at 01:36:30AM +0100, Adam Borowski wrote:
> > > I'm entirely happy to reassign this request to systemd and have the
> > > setting applied more broadly.
> > Some options:
> > - conflict with systemd < version_with_the_new_default
> > - wait for a full release and then just drop it
> > - when sysctl in postinst reports the new default
> > - a mix of the last two options
> 
> Debian's default sysctl settings should reside in procps (as it owns
> /sbin/sysctl and /etc/sysctl* settings) rather than some unrelated
> package.

Is that documented anywhere?  It's certainly not the case today:

$ for i in /usr/lib/sysctl.d/*.conf; do
  dpkg -S $i
done
tracker-miner-fs: /usr/lib/sysctl.d/30-tracker.conf
bubblewrap: /usr/lib/sysctl.d/50-bubblewrap.conf
systemd-coredump: /usr/lib/sysctl.d/50-coredump.conf
systemd: /usr/lib/sysctl.d/50-pid-max.conf
procps: /usr/lib/sysctl.d/99-protect-links.conf

Bug#1008281: setting sysctl net.ipv4.ping_group_range

[Noah Meyerhans]

On Mon, Jan 02, 2023 at 10:09:44PM +0100, Marco d'Itri wrote:
> > With that in place, unprivileged users are able to excute ping for both
> > IPv4 and IPv6 targets without cap_net_raw (currently set as either a
> > file-based attribute on the ping binary or acquired via setuid).  But
> > since that applies system-wide, not just to the ping binary, there may
> > be objections.
> I do not think that the submitter made clear why this would be 
> preferable, so I had to research it myself. See:
> 
> https://fedoraproject.org/wiki/Changes/EnableSysctlPingGroupRange
> https://github.com/systemd/systemd/pull/13141
> 
> Since this is one of the systemd sysctl defaults (of which I think that 
> we should adopt more, especially the network-related ones!) I agree with 
> changing this.
> I would just do it in the systemd package package to allow all packages 
> to benefit from it without having to care if ping is installed.

I'm entirely happy to reassign this request to systemd and have the
setting applied more broadly.  The question that arises then is what to
do about the file-level capabilities on the ping binary.  Ideally we
drop them entirely (including the setuid fallback), but when?

I could leave things completely decoupled, and simply wait until systemd
makes the change and then upload iputils and assume that anybody
upgrading iputils is also upgrading systemd.  That seems to be what
Fedora did, according to the fedoraproject.org wiki cited above.
Alternatives would seem to involve some level of versioned dependency,
which doesn't feel right.

noah

Bug#895338: awscli: zsh completion doesn't work as expected

[Noah Meyerhans]

tags -1 + pending

> This package ships /usr/share/zsh/vendor-completions/_awscli which I
> expect it to work like other files in zsh/vendor-completions.
> 
> For example, I should have `aws` completion if I just called `compinit`
> in my zshrc.
> 
> But I still need to `source /usr/share/zsh/vendor-completions/_awscli`
> to get the completion working.

I think I have this fixed with
https://salsa.debian.org/noahm/awscli/-/commit/2636a965c5646b2f635da7fcdcba3f6050860897,
which should be included in the next upload.

noah

Bug#1026919: ITP: amazon-ec2-net-utils -- utilities for managing network interfaces in Amazon EC2

[Noah Meyerhans]

Package: wnpp
Severity: wishlist
Owner: Noah Meyerhans 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: amazon-ec2-net-utils
  Version : 2.3.0
  Upstream Contact: Noah Meyerhans 
* URL : https://github.com/amazonlinux/amazon-ec2-net-utils
* License : Apache 2.0
  Programming Lang: Shell
  Description : utilities for managing network interfaces in Amazon EC2

amazon-ec2-net-utils provides udev integration and helper utilities to manage
network configuration in the Amazon EC2 cloud environment.  It is responsible
for configuring systemd-networkd with appropriate configuration for DHCPv4 and
DHCPv6 as appropriate to the network environment in which an EC2 instance is
running.  It supports policy routing configuration to ensure compliance with
Virtual Private Cloud (VPC) network anti-spoofing restrictions.  IP aliasing
and prefix delegation are also supported.

I'm the upstream author and will maintain this package within the Debian cloud
team.

Bug#1026874: awscli installs a 5 million line manpage

[Noah Meyerhans]

Also it seems that generating this file triggers an OOM condition on
salsa. https://salsa.debian.org/noahm/awscli/-/jobs/3693800

Bug#1026874: awscli installs a 5 million line manpage

[Noah Meyerhans]

Package: awscli
Version: 2.9.9-1
Severity: normal

Just making sure this is recorded:

The aws(1) man page installed by awscli 2.9.9-1 is not useful.  Ideally it
should be split up in to per service pages, e.g. aws-vpc(1), aws-ec2(2), etc.
for all services (some services already have their own pages, but most don't.)
But until then, it should be replaced by something more along the lines of the
`aws help` output.

$ ls -lh /usr/share/man/man1/aws.1.gz
-rw-r--r-- 1 root root 17M Dec 22 14:15 /usr/share/man/man1/aws.1.gz

doom:~$ time man aws 2> /dev/null | wc -l
5150024
man aws 2> /dev/null  134.93s user 17.01s system 191% cpu 1:19.53 total

Bug#1025618: cloud-init and firewalld systemd unit files have ordering cycles

[Noah Meyerhans]

On 12/12/2022 6:44 AM, Sam Hartman wrote:

 >> From my quick read: Michael Biebl proposes dropping
 >> network-pre.target
 Ross> from cloud-init's After=, and replacing it with each of the
 Ross> config backends that cloud-init supports.  This sounds pretty
 Ross> reasonable, but also like something that upstream should
 Ross> address first.

Why wait for upstream?
It's a bug affecting Debian users, our systemd maintainer has a solution
that you (and I) think is reasonable.
The symptom is quite serious.
We often make changes before upstream in situations like that,
especially when the alternative is:

 Ross> Should we consider adding "Conflicts: firewalld" to cloud-init
 Ross> before the freeze?  That's not optimal of course, but it'd
 Ross> prevent a user from ending up in this situation for now.

I'd much rather see Debian local changes than conflicts.


We should simply move this discussion to an upstream pull request rather 
than wait passively for their response. I agree that diverging from 
upstream is preferable to unnecessary conflicts, but it shouldn't be 
done without first consulting with upstream on our proposed solution.


noah

Bug#1025828: cloud.debian.org: cloud-init may run time synchronization dependent jobs before the clock is synchronized

[Noah Meyerhans]

Package: cloud.debian.org
Severity: normal

Certain functionality built in to cloud-init depends on a reasonably
accurate clock, such as apt repo metadata signature verification.  In the
case where a system's hardware clock is far out of sync, chrony may not have
completed synchronization before apt tries to refresh apt caches in the
package_update_upgrade_install module or in a userdata shell script.  An
example of the resulting failure follows:

MS Name/IP address Stratum Poll Reach LastRx Last sample
===
^? 169.254.169.123   3   6 1 1   -2986h[ -2986h] +/-  586us
Sun Aug  7 13:25:16 UTC 2022
Get:2 http://cdn-aws.deb.debian.org/debian bullseye InRelease [116 kB]
Get:3 http://cdn-aws.deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://cdn-aws.deb.debian.org/debian bullseye-backports InRelease [49.0 
kB]
Reading package lists...
E: Release file for 
http://security.debian.org/debian-security/dists/bullseye-security/InRelease is 
not valid yet (invalid for another 124d 5h 58min 50s). Updates for this 
repository will not be applied.
E: Release file for 
http://cdn-aws.deb.debian.org/debian/dists/bullseye/InRelease is not valid yet 
(invalid for another 33d 20h 52min 34s). Updates for this repository will not 
be applied.
E: Release file for 
http://cdn-aws.deb.debian.org/debian/dists/bullseye-updates/InRelease is not 
valid yet (invalid for another 124d 6h 53min 11s). Updates for this repository 
will not be applied.
E: Release file for 
http://cdn-aws.deb.debian.org/debian/dists/bullseye-backports/InRelease is not 
valid yet (invalid for another 124d 6h 53min 11s). Updates for this repository 
will not be applied.
2022-08-07 13:25:17,663 - cc_scripts_user.py[WARNING]: Failed to run module 
scripts-user (scripts in /var/lib/cloud/instance/scripts)
2022-08-07 13:25:17,664 - util.py[WARNING]: Running module scripts-user 
() failed
Cloud-init v. 20.4.1 finished at Sun, 07 Aug 2022 13:25:17 +. Datasource 
DataSourceEc2Local.  Up 10.28 seconds

The userdata script was:

#!/bin/bash
chronyc sources
date
DEBIAN_FRONTEND=noninteractive apt-get update

A possible workaround to this is to run a command like `chronyc waitsync`
before apt.  It turns out to be not entirely trivial to insert such a
command in the right place with userdata, though.

We should make some reasonable effort to ensure the clock is in sync before
executing the cloud-final.service, which is where all the package
installation work happens.  It's not immediately clear exactly where we
should do this, whether it's in the chrony configs, cloud-init itself,
cloud-init's systemd configuration, or elsewhere.  And then, should the fix
be in one of the packages or the images?

Bug#1025351: ITP: aws-crt-python -- Python 3 bindings for the AWS Common Runtime

[Noah Meyerhans]

Package: wnpp
Severity: wishlist
Owner: Noah Meyerhans 
X-Debbugs-Cc: debian-de...@lists.debian.org, debian-cl...@lists.debian.org

* Package name: aws-crt-python
  Version : 0.15.3
  Upstream Author : Amazon Web Services
* URL : https://github.com/awslabs/aws-crt-python
* License : Apache 2.0
  Programming Lang: Python and C
  Description : Python 3 bindings for the AWS Common Runtime

aws-crt-python contains python interfaces for common low-level functionality
used when interfacing with Amazon Web Services service endpoints.  It
provides socket handling, the HTTP client implementation, authentication,
logging, and error handling.

The package is initially being prepared in order to support upgrading the
awscli package to version 2.x, which introduces it as a dependency.  It will
be maintained by the cloud team.

In the future, the C components in this package may be split into standalone
packages that can be used independently of the python bindings if needed.
For now the focus is on python.

noah

Bug#1024444: spamd: systemd hardening flags prevents bayes access to per-user dirs

[Noah Meyerhans]

Control: tags -1 + moreinfo

On Sat, Nov 19, 2022 at 10:23:12AM -0500, Michael Welsh Duggan wrote:
> The addition of the
> 
>   ProtectSystem=full
> 
> clause to the spamd service module prevents spamd from writing to user
> bayes files.  Here is a log from spamd:

Hi Michael.  Per the systemd documentation on the ProtectSystem setting:

Takes a boolean argument or the special values "full" or
"strict". If true, mounts the /usr/ and the boot loader
directories (/boot and /efi) read-only for processes invoked by
this unit. If set to "full", the /etc/ directory is mounted
read-only, too.

Access to /home is not restricted by this setting.  Is /home on your
system a symlink or otherwise not actually located at /home?

noah

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

On Fri, Nov 04, 2022 at 05:37:21PM +0100, Bastian Blank wrote:
> > > Are you sure this library can have a 1 as ABI?  Can you please reproduce
> > > the ABI stability promisses?
> > Allegedly upstream has recently committed to proper SONAME and ABI
> > management in support of efforts to get these packages accepted into
> > Fedora.  I'll see if I can find meaningful evidence of this...
> 
> The SOVERSION was set two years ago:
> 
> https://github.com/awslabs/aws-c-common/pull/702
> 
> So some more information would be nice.

According to the person who handled the Fedora review for aws-c-common
(https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2049379), davdunc
had indicated he'd take care for SONAME management until upstream gets
things worked out.  I haven't been able to get in touch with davdunc
yet to confirm this.

Given the number of packages involved, though, I expect it'll take a
while for everything to have a properly managed SONAME.  Given that, I
see a few alternatives:

1. We package aws-crt-python with all the aws-c-* packages included in a
   single source package.  Given that upstream maintains this structure
   in a single repo using git submodules and that their build system
   supports it directly, it appears to be the way they expect the package
   to be consumed.

2. We package the individual aws-c-* dependencies but only ship static
   libraries, and handle them as standard buid-deps for aws-crt-python.
 
3. We manage the SONAME versions ourselves until upstream does it for
   us.

4. We ignore awscli v2 and continue shipping v1.

I actually prefer #1 and suggest we do that.  My reasoning is:

1. It's well supported by upstream,

2. It prevents other packages from picking up the aws-c-* packages as
   dependencies before they expose a stable API/ABI,

3. It is simple to split out the submodules into standalone packages
   one-by-one as their interfaces stabilize.

4. It's quite simple to implement

What do folks think of this idea?

noah

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

On Fri, Nov 04, 2022 at 08:32:25AM +0100, Bastian Blank wrote:
> > My first pass only produces -dev packages with headers and static libraries.
> > To test them out, build the debian/sid branch from these repos, in this 
> > order:
> > - https://salsa.debian.org/rvandegrift/aws-c-common
> 
> Are you sure this library can have a 1 as ABI?  Can you please reproduce
> the ABI stability promisses?
> 
> I see in the public JSON headers the following interface changes:
> 
> | -typedef bool(aws_json_on_member_encountered_const_fn)(
> | +typedef int(aws_json_on_member_encountered_const_fn)(
> |  const struct aws_byte_cursor *key,
> |  const struct aws_json_value *value,
> | +bool *out_should_continue,
> |  void *user_data);
> 
> That's neither source nor binary compatible.
> 
> I don't think any of those libraries are reasonable ABI stable to be
> maintained as shaed libraries.

Allegedly upstream has recently committed to proper SONAME and ABI
management in support of efforts to get these packages accepted into
Fedora.  I'll see if I can find meaningful evidence of this...

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

On Thu, Nov 03, 2022 at 09:35:14PM -0700, Ross Vandegrift wrote:
> > Honestly, filing some of the ITPs would be quite helpful at this point.
> > We'll need to get the following projects packaged:
> > 
> > aws-c-auth
> > aws-c-cal
> > aws-c-compression
> > aws-c-event-stream
> > aws-c-http
> 
> I got this far, and then:
> 
> > SMTP send failure: {'sub...@bugs.debian.org': (451, b'sorry, only 5 reports 
> > per hour for
> > submission')}. You can retry, or save the report and exit. Do you want to 
> > retry [Y|n|q|?]?

That's hilarious.

I guess the rust team has been uploading their random crate dependencies
without an ITP bug at all, so we're probably not going to generate too
much unhappiness if we skip them.

One nice thing about having the wnpp bugs is that we can add them as
blockers to this bug, which is helpful for tracking progress:

$ bts block 966573 with 

I uploaded aws-c-common and aws-checksums today.  All the git repos
backing my uploads are owned by the cloud-team on salsa.  I'll try to
get another couple uploads done tomorrow.

noah

Bug#1023415: ITP: aws-checksums -- fast, cross-platform CRC32c/CRC32 implementations

[Noah Meyerhans]

Package: wnpp
Severity: wishlist
Owner: Noah Meyerhans 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: aws-checksums
  Version : 0.1.13
  Upstream Author : Amazon Web Services
* URL : https://github.com/awslabs/aws-checksums
* License : Apache 2.0
  Programming Lang: C
  Description : fast, cross-platform CRC32c/CRC32 implementations

 Cross-Platform HW accelerated CRC32c and CRC32 with fallback to
 efficient SW implementations. C interface with language bindings for
 each of our SDKs

This package will be maintained by the cloud team.  Packaging is initially
being driven by awscli v2 dependencies.

Bug#1023412: ITP: aws-c-common -- C99 package for AWS SDK for C

[Noah Meyerhans]

Package: wnpp
Severity: wishlist
Owner: Noah Meyerhans 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: aws-c-common
  Version : 0.8.4
  Upstream Author : Amazon Web Services
* URL : https://github.com/awslabs/aws-c-common
* License : Apache 2.0
  Programming Lang: C
  Description : C99 package for AWS SDK for C

 Core c99 package for AWS SDK for C. Includes cross-platform
 primitives, configuration, data structures, and error handling.

This package will be maintained by the cloud team.  It is initially being
packaged as a dependency for awscli v2.

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

On Wed, Nov 02, 2022 at 07:55:51PM -0700, Ross Vandegrift wrote:
> > > - the aws-cli2-temp repo is based on upstream, not our awscli repo.  I was
> > >   intentionally being sloppy to quickly get through a test.
> > 
> > Same.  I essentially Debianized the upstream v2 repo from scratch,
> > pulling in some of your packaging metadata as it made sense.  Given that
> > v2 is developed on a different branch and by now differs quite
> > significantly from v1, a case could be made for introducing a new
> > awscli2 package as a new source package and retiring the original awscli
> > package.  However, the debian package metadata isn't really all that
> > complex, so it may not actually be necessary.
> 
> Is there any reason to have both versions available to install at once?

No, there isn't.  And both should be installed as /usr/bin/aws anyway.

At this point my v2 package builds as awscli_2.8.7-1_all.deb and should
be a direct upgrade from the current 1.x packages.

> > Fortunately, aws-lc isn't an issue. But s2n-tls remains one.  Not sure
> > we're going to be able to do anything about that.  The difficult thing
> > is that it's typically expected to be used as a statically linked
> > library, which means updates end up being tedious.
> 
> Agreed, there's nothing we can do about it.  But I think it's good news to be
> able to use OpenSSL and just have one highly security sensitive package.

I just uploaded s2n, so it should be in the NEW queue momentarily.
Sources are at https://salsa.debian.org/cloud-team/s2n-tls.  So we'll
see how this goes.

> > I haven't pushed my changes anywhere, yet.  Once I do, the remaining
> > tasks will be to any lintian issues or other obvious problems and get
> > these packages into NEW.  I think they're in reasonably good shape, but
> > we don't have a lot of time before bookworm starts freezing, so I'd love
> > any help with these steps.
> 
> I might have some time to help.  Would it be useful to transfer my original
> repos to the cloud-team group?

I don't think so.  There were a couple repos where you had made a commit
on the upstream branch, which I blew away in order to stay in sync with
upstream commits.  So I'd need to force push.  Either way, I think the
repos I have locally can be pushed to empty salsa repos.

Honestly, filing some of the ITPs would be quite helpful at this point.
We'll need to get the following projects packaged:

aws-c-auth
aws-c-cal
aws-c-common
aws-c-compression
aws-c-event-stream
aws-c-http
aws-c-io
aws-c-mqtt
aws-c-s3
aws-c-sdkutils
aws-checksums
aws-crt-python

Bug#1023369: ITP: s2n-tls -- C99 implementation of the TLS/SSL protocols

[Noah Meyerhans]

Package: wnpp
Severity: wishlist
Owner: Noah Meyerhans 
X-Debbugs-Cc: debian-de...@lists.debian.org

* Package name: s2n-tls
  Version : 1.3.26
  Upstream Author : Amazon Web Services
* URL : https://github.com/aws/s2n-tls
* License : Apache 2.0
  Programming Lang: C
  Description : C99 implementation of the TLS/SSL protocols

 s2n-tls is a C99 implementation of the TLS/SSL protocols that is
 designed to be simple, small, fast, and with security as a
 priority. It is released and licensed under the Apache License 2.0.
 .
 The s2n-tls I/O APIs are designed to be intuitive to developers
 familiar with the widely-used POSIX I/O APIs, and s2n-tls supports
 blocking, non-blocking, and full-duplex I/O. Additionally there are
 no locks or mutexes within s2n-tls.
 .
 s2n-tls implements SSLv3, TLS1.0, TLS1.1, and TLS1.2. For encryption,
 s2n-tls supports 128-bit and 256-bit AES, in the CBC and GCM modes,
 ChaCha20, 3DES, and RC4. For forward secrecy, s2n-tls supports both
 DHE and ECDHE. s2n-tls also supports the Server Name Indicator (SNI),
 Application-Layer Protocol Negotiation (ALPN) and the Online
 Certificate Status Protocol (OCSP) TLS extensions. SSLv3, RC4, 3DES
 and DHE are each disabled by default for security reasons.
 .
 As it can be difficult to keep track of which encryption algorithms
 and protocols are best to use, s2n-tls features a simple API to use
 the latest "default" set of preferences. If you prefer to remain on a
 specific version for backwards compatibility, that is also supported.

This package will be maintained by the cloud team.  Initial packaging is
being driven by the awscli package, version 2 of which will depend on this
package.

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

On Mon, Oct 31, 2022 at 10:50:19PM -0700, Noah Meyerhans wrote:
> > aws-crt-python is the hard part.  It depends on a bunch of C libraries which
> > follow a more modern development style.  They:
> > - provide no abi/api stability guarantees.
> > - have versioned releases, but no one cares about the versions.  The 
> > intention
> >   is that everyone should be following the main branch.
> > - only nominally support dynamic linking (it can be enabled, but is not 
> > used or
> >   tested by upstream).

Actually, taking a look at Amazon Linux 2022's packaging of these
libraries, I see that they're distributed as .so files, and the
aws-crt-python packagin declares runtime dependencies on these
dynamically loaded libraries.  So I don't think we're in uncharted
waters if we decide to ship .so libraries.

noah

Bug#966573: progress packaging awscli v2

[Noah Meyerhans]

Hi Ross,

On Tue, Oct 05, 2021 at 11:10:43PM -0600, Ross Vandegrift wrote:
> > awscli v2 remains quite difficult to package, but it seems that upstream
> > is looking to address this.  See
> > https://github.com/aws/aws-cli/issues/6186 for details and tracking.
> 
> Using the source dist poc from https://github.com/aws/aws-cli/pull/6352 I've
> made enough progress to get a packaged aws-cli v2 working.  There's a lot more
> that needs to be done, but idea of the above linked PR could work for us.  I'm
> going to document my findings here.

So, only a year later, I've picked this up and made some additional
progress:

I have no name!@b02f1db79f9e:/src$ aws --version
aws-cli/2.8.7 Python/3.10.8 Linux/6.0.0-2-amd64 source/x86_64.debian prompt/off
I have no name!@b02f1db79f9e:/src$ dpkg -s awscli | grep Version
Version: 2.8.7-1

> aws-crt-python is the hard part.  It depends on a bunch of C libraries which
> follow a more modern development style.  They:
> - provide no abi/api stability guarantees.
> - have versioned releases, but no one cares about the versions.  The intention
>   is that everyone should be following the main branch.
> - only nominally support dynamic linking (it can be enabled, but is not used 
> or
>   tested by upstream).
> - seem to be mostly used via source inclusion.

Indeed, none of this has improved in the past year.  In fact, we've
actually gained a few more such dependencies that didn't exist last year
(aws-c-sdkutils, at least).

We might actually be able to make a case that all the aws-c-* libraries
end up being easier to maintain if we ship them as private internal
components of the aws-crt-python package rather than as first class
packages that could end up becoming dependencies of other downstream
packages.

> - some repos have tests disabled due to failing during builds.  So far, I 
> don't
>   know if these are real failures, or if upstream's build method.

I think I've got most of these fixed.

> - copyright attribution for aws-lc is very hard.  It's a fork of Google's
>   BoringSSL, which is a fork of pre-3.0 OpenSSL.
> 
> - That also means that aws-lc inherits the openssl gpl incompatibility.

Here's the good news: We don't actually need aws-lc at all.  awscli v2
and its various dependencies (including s2n-tls) can build against
OpenSSL 3.

> - the aws-cli2-temp repo is based on upstream, not our awscli repo.  I was
>   intentionally being sloppy to quickly get through a test.

Same.  I essentially Debianized the upstream v2 repo from scratch,
pulling in some of your packaging metadata as it made sense.  Given that
v2 is developed on a different branch and by now differs quite
significantly from v1, a case could be made for introducing a new
awscli2 package as a new source package and retiring the original awscli
package.  However, the debian package metadata isn't really all that
complex, so it may not actually be necessary.

> - aws-lc and s2n-tls may be hard to maintain.  Both are complicated, security
>   critical crypto libraries.

Fortunately, aws-lc isn't an issue. But s2n-tls remains one.  Not sure
we're going to be able to do anything about that.  The difficult thing
is that it's typically expected to be used as a statically linked
library, which means updates end up being tedious.

I haven't pushed my changes anywhere, yet.  Once I do, the remaining
tasks will be to any lintian issues or other obvious problems and get
these packages into NEW.  I think they're in reasonably good shape, but
we don't have a lot of time before bookworm starts freezing, so I'd love
any help with these steps.

We have to get at least the following packages into the archive:

aws-c-auth_0.6.18-1_amd64.changes
aws-c-cal_0.5.20-1_amd64.changes
aws-c-common_0.8.4-1_amd64.changes
aws-c-compression_0.2.15-1_amd64.changes
aws-c-event-stream_0.2.15-1_amd64.changes
aws-c-http_0.6.23-1_amd64.changes
aws-c-io_0.13.6-1_amd64.changes
aws-c-mqtt_0.7.12-1_amd64.changes
aws-c-s3_0.1.51-1_amd64.changes
aws-c-sdkutils_0.1.4-1_amd64.changes
aws-checksums_0.1.13-1_amd64.changes
aws-crt-python_0.15.1-1_amd64.changes
s2n-tls_1.3.26-1_amd64.changes

Bug#1022937: RFP: aws-sdk-go-v2 -- AWS SDK for the Go programming language, version 2

[Noah Meyerhans]

Package: wnpp
Severity: wishlist

* Package name: aws-sdk-go-v2
  Version : 2022-10-27
  Upstream Author : Amazon Web Services (https://aws.amazon.com/)
* URL : https://github.com/aws/aws-sdk-go-v2
* License : Apache 2.0
  Programming Lang: Go
  Description : AWS SDK for the Go programming language, version 2

aws-sdk-go-v2 is a replacement for the original AWS SDK for go, currently
packaged as golang-github-aws-aws-sdk-go.  Version 2 uses a more modular
interface consistent with current Go packaging idioms.

A number of applications packaged by Debian have migrated to the v2 SDK,
including amazon-ecr-credential-helper, which I'm looking to update.

Bug#1021978: spamassassin: non-standard version numbers break spamassassin's AIDE script

[Noah Meyerhans]

Control: reassign -1 aide
Control: retitle -1 aide's spamassassin script does not correctly handle 
spamassassin prerelease version strings

On Fri, Oct 21, 2022 at 01:05:21PM +0200, Thomas Dorner wrote:
> > > The problem is that the current version number 4.0.0~rc3-3.1 does
> > > not match the expected schema of N.N.N-N.  As only the first 3
> > > numbers are used anyway, I've created a patch (attached) making the
> > > regular expression less rigid by ignoring everything after those 3
> > > numbers. This works for me.  
> 
> Ooops, you're right, the file
> /usr/share/aide/config/aide/aide.conf.d/21_aide_spamassassin
> belongs to aide-common.  Could you forward this or should I create a
> new report?

Reassinged to aide.

noah

Bug#1021978: spamassassin: non-standard version numbers break spamassassin's AIDE script

[Noah Meyerhans]

On Tue, Oct 18, 2022 at 10:08:32AM +0200, Thomas Dorner wrote:
> The problem is that the current version number 4.0.0~rc3-3.1 does not
> match the expected schema of N.N.N-N.  As only the first 3 numbers are
> used anyway, I've created a patch (attached) making the regular
> expression less rigid by ignoring everything after those 3 numbers.
> This works for me.

Isn't this an AIDE issue that should be fixed in that package?  The
spamassassin version complies with Debian's package version numbering
expectations.

noah

Bug#1020859: spamd init script chooses wrong default

[Noah Meyerhans]

On Tue, Sep 27, 2022 at 07:28:22PM +0200, Matus UHLAR - fantomas wrote:
> the /etc/init.d/spamd init script loads /etc/default/spamassassin while
> package is bundled with /etc/default/spamd
> 
> this is apparently unintentional
> 
> 
> It may be welcome when upgrading from versions <4 if admin configured
> different defaults, unless upgrade script will be able to migrate PIDFILE
> and OPTIONS settings from /etc/default/spamassassin to /etc/default/spamd

Thanks for reporting this.  Yes, the intent is to preserve local admin
changes across the upgrade.  However, that process likely needs to be
handled differently, most likely in the spamd postinst scripts...

noah

Bug#1017584: bullseye launch failures in IPv6-only VPC subnets

[Noah Meyerhans]

Adding some tracing to the dhclient-script, I can see that
/etc/dhcp/dhclient-exit-hooks.d/rfc3442-classless-routes is trying to add
the routes with calls like:

ip -4 route add 169.254.169.254/32 via 169.254.0.1 dev ens5

However, because there's no route to 169.254.0.1, the call fails with
"Error: Nexthop has invalid gateway."

On systems where the routes are properly configured, there's a link scope
route to that address, e.g. on sid:

169.254.0.1 dev ens5 proto dhcp scope link src 169.254.2.187 metric 100

If we create this route on bullseye, then the rest of the IPv4 routes are
installed as expected.  Something like the following could work, but I'm
open to other options:

services/admin@i-07e63055bb304147c:~$ cat /etc/dhcp/dhclient-exit-hooks.d/gw
case "$reason" in
  BOUND|RENEW|REBIND|REBOOT)
if [ -z "$old_ip_address" ] ||
   [ "$old_ip_address" != "$new_ip_address" ]; then
case "$new_ip_address" in
  169.254.*)
ip -4 ro add 169.254.0.1 dev "$interface" scope link
  ;;
esac
fi
  ;;
esac

Bug#1017584: bullseye launch failures in IPv6-only VPC subnets

[Noah Meyerhans]

Package: cloud.debian.org
Severity: important
User: cloud.debian@packages.debian.org
Usertags: aws image

When AWS originally launched IPv6-only VPC subnets, the DHCPv4 server
handed out a link-local v4 address and a default route that was a
blackhole for most destinations.  It did route traffic to the instance
metadata endpoint (169.254.169.254), the timesync endpoint
(169.254.169.123) and a couple of others.  This provided compatibility
with existing software, but the blackhole default route lead to some
obvious problems for any software that would try to establish a
connection using IPv4 before IPv6.

AWS has recently started rolling out RFC 3442 routes to the specific
endpoints, and stopped distributing a default route.  This is the
correct thing to do, and works well on sid and bookworm, but it's
causing problems on bullseye.

The expected behavior when launching in an IPv6 subnet is that an
instance has one or more IPv6 addresses and a functioning IPv6 default
route.  It also should have an IPv4 address in the 169.254/16 range and
/32 routes to several endpoints, for example:

2: eth0:  mtu 9001 qdisc mq state UP
group default qlen 1000
inet 169.254.245.205/32 scope global dynamic eth0
   valid_lft 2789sec preferred_lft 2789sec

169.254.0.1 dev eth0 scope link
169.254.169.123 via 169.254.0.1 dev eth0 proto static
169.254.169.249 via 169.254.0.1 dev eth0 proto static
169.254.169.250/31 via 169.254.0.1 dev eth0 proto static
169.254.169.253 via 169.254.0.1 dev eth0 proto static
169.254.169.254 via 169.254.0.1 dev eth0 proto static

The actual behavior on the current bullseye AMIs is that IPv6 is
configured as expected, but the IPv4 routes are not configured:

admin@i-07e63055bb304147c:~$ ip -4 addr show dev ens5
2: ens5:  mtu 9001 qdisc mq state UP
group default qlen 1000
altname enp0s5
inet 169.254.109.198/32 scope global ens5
   valid_lft forever preferred_lft forever
admin@i-07e63055bb304147c:~$ ip -4 ro
admin@i-07e63055bb304147c:~$

Because cloud-init in bullseye does not know about the IPv6 instance
metadata endpoint and is unable to reach the IPv4 endpoint, it is unable
to complete the instance provisioning process.

Logs indicate the following:

Aug 18 00:07:10 debian systemd[1]: Started ifup for ens5.
Aug 18 00:07:10 debian dhclient[388]: Internet Systems Consortium DHCP Client 
4.4.1
Aug 18 00:07:10 debian sh[388]: Internet Systems Consortium DHCP Client 4.4.1
Aug 18 00:07:10 debian dhclient[388]: Copyright 2004-2018 Internet Systems 
Consortium.
Aug 18 00:07:10 debian sh[388]: Copyright 2004-2018 Internet Systems Consortium.
Aug 18 00:07:10 debian dhclient[388]: All rights reserved.
Aug 18 00:07:10 debian sh[388]: All rights reserved.
Aug 18 00:07:10 debian dhclient[388]: For info, please visit 
https://www.isc.org/software/dhcp/
Aug 18 00:07:10 debian sh[388]: For info, please visit 
https://www.isc.org/software/dhcp/
Aug 18 00:07:10 debian dhclient[388]:
Aug 18 00:07:10 debian dhclient[388]: Listening on LPF/ens5/06:6c:90:77:5b:a1
Aug 18 00:07:10 debian sh[388]: Listening on LPF/ens5/06:6c:90:77:5b:a1
Aug 18 00:07:10 debian sh[388]: Sending on   LPF/ens5/06:6c:90:77:5b:a1
Aug 18 00:07:10 debian sh[388]: Sending on   Socket/fallback
Aug 18 00:07:10 debian sh[388]: DHCPDISCOVER on ens5 to 255.255.255.255 port 67 
interval 4
Aug 18 00:07:10 debian dhclient[388]: Sending on   LPF/ens5/06:6c:90:77:5b:a1
Aug 18 00:07:10 debian dhclient[388]: Sending on   Socket/fallback
Aug 18 00:07:10 debian dhclient[388]: DHCPDISCOVER on ens5 to 255.255.255.255 
port 67 interval 4
Aug 18 00:07:10 debian dhclient[388]: DHCPOFFER of 169.254.109.198 from 
169.254.0.1
Aug 18 00:07:10 debian sh[388]: DHCPOFFER of 169.254.109.198 from 169.254.0.1
Aug 18 00:07:10 debian sh[388]: DHCPREQUEST for 169.254.109.198 on ens5 to 
255.255.255.255 port 67
Aug 18 00:07:10 debian sh[388]: DHCPACK of 169.254.109.198 from 169.254.0.1
Aug 18 00:07:10 debian dhclient[388]: DHCPREQUEST for 169.254.109.198 on ens5 
to 255.255.255.255 port 67
Aug 18 00:07:10 debian dhclient[388]: DHCPACK of 169.254.109.198 from 
169.254.0.1
Aug 18 00:07:10 debian sh[399]: RTNETLINK answers: File exists
Aug 18 00:07:10 debian root[415]: 
/etc/dhcp/dhclient-exit-hooks.d/rfc3442-classless-routes returned non-zero exit 
status 2
Aug 18 00:07:10 debian dhclient[388]: bound to 169.254.109.198 -- renewal in 
1498 seconds.

Bug#1016652: cloud.debian.org: AWS Marketplace usage instructions need to be updated

[Noah Meyerhans]

Package: cloud.debian.org
Severity: normal
User: cloud.debian@packages.debian.org
Usertags: aws infrastructure

Per recent email from AWS, we need to update the usage instructions associated
with our AWS Marketplace AMI listings.  The requirements are documented at
https://docs.aws.amazon.com/marketplace/latest/userguide/ami-product-usage-instructions.html

Bug#1016458: bullseye-pu: package dovecot/2.3.13+dfsg1-2+deb11u1

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
Tags: bullseye
User: release.debian@packages.debian.org
Usertags: pu

Dovecot 2.3.13+dfsg1-2+deb11u1 contains a backported fix for #1016351
(CVE-2022-30550).  The fix is cherry-picked from upstream and is identical
to the fix recently uploaded to unstable in dovecot_2.3.19.1+dfsg1-2.  The
stable security team and the package maintainers have determined that this
issue does not warrant a DSA and should be fixed in the next bullseye
point release.

Debdiff is attached.  Note that it contains one additional minor change to
switch the salsa gitlab ci configuration to use bullseye runners.

noah
diff -Nru dovecot-2.3.13+dfsg1/debian/changelog 
dovecot-2.3.13+dfsg1/debian/changelog
--- dovecot-2.3.13+dfsg1/debian/changelog   2021-07-20 08:05:19.0 
-0700
+++ dovecot-2.3.13+dfsg1/debian/changelog   2022-07-31 17:47:06.0 
-0700
@@ -1,3 +1,10 @@
+dovecot (1:2.3.13+dfsg1-2+deb11u1) bullseye; urgency=medium
+
+  * [4b5dac8] d/patches: cherry-pick fix for CVE-2022-30550 (Closes: #1016351)
+  * [597ba7f] salsa-ci: build with bullseye
+
+ -- Noah Meyerhans   Sun, 31 Jul 2022 17:47:06 -0700
+
 dovecot (1:2.3.13+dfsg1-2) unstable; urgency=high
 
   * Import upstream fixes for security issues (Closes: #990566):
diff -Nru 
dovecot-2.3.13+dfsg1/debian/patches/auth-Add-a-comment-about-updating-userdb_find.patch
 
dovecot-2.3.13+dfsg1/debian/patches/auth-Add-a-comment-about-updating-userdb_find.patch
--- 
dovecot-2.3.13+dfsg1/debian/patches/auth-Add-a-comment-about-updating-userdb_find.patch
 1969-12-31 16:00:00.0 -0800
+++ 
dovecot-2.3.13+dfsg1/debian/patches/auth-Add-a-comment-about-updating-userdb_find.patch
 2022-07-31 17:47:06.0 -0700
@@ -0,0 +1,22 @@
+From: Timo Sirainen 
+Date: Mon, 16 May 2022 14:58:45 +0200
+Subject: auth: Add a comment about updating userdb_find()
+
+---
+ src/auth/userdb.c | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+Index: dovecot/src/auth/userdb.c
+===
+--- dovecot.orig/src/auth/userdb.c
 dovecot/src/auth/userdb.c
+@@ -162,7 +162,8 @@ userdb_preinit(pool_t pool, const struct
+   userdb->id = ++auth_userdb_id;
+   userdb->iface = iface;
+   userdb->args = p_strdup(pool, set->args);
+-
++  /* NOTE: if anything else than driver & args are added here,
++ userdb_find() also needs to be updated. */
+   array_push_back(_modules, );
+   return userdb;
+ }
diff -Nru 
dovecot-2.3.13+dfsg1/debian/patches/auth-Fix-handling-passdbs-with-identical-driver-args-but-.patch
 
dovecot-2.3.13+dfsg1/debian/patches/auth-Fix-handling-passdbs-with-identical-driver-args-but-.patch
--- 
dovecot-2.3.13+dfsg1/debian/patches/auth-Fix-handling-passdbs-with-identical-driver-args-but-.patch
 1969-12-31 16:00:00.0 -0800
+++ 
dovecot-2.3.13+dfsg1/debian/patches/auth-Fix-handling-passdbs-with-identical-driver-args-but-.patch
 2022-07-31 17:47:06.0 -0700
@@ -0,0 +1,130 @@
+From: Timo Sirainen 
+Date: Mon, 9 May 2022 15:23:33 +0300
+Subject: auth: Fix handling passdbs with identical driver/args but different
+ mechanisms/username_filter
+
+The passdb was wrongly deduplicated in this situation, causing wrong
+mechanisms or username_filter setting to be used. This would be a rather
+unlikely configuration though.
+
+Fixed by moving mechanisms and username_filter from struct passdb_module
+to struct auth_passdb, which is where they should have been in the first
+place.
+---
+ src/auth/auth-request.c |  6 +++---
+ src/auth/auth.c | 18 ++
+ src/auth/auth.h |  5 +
+ src/auth/passdb.c   | 15 ++-
+ src/auth/passdb.h   |  4 
+ 5 files changed, 28 insertions(+), 20 deletions(-)
+
+Index: dovecot/src/auth/auth-request.c
+===
+--- dovecot.orig/src/auth/auth-request.c
 dovecot/src/auth/auth-request.c
+@@ -553,8 +553,8 @@ auth_request_want_skip_passdb(struct aut
+ struct auth_passdb *passdb)
+ {
+   /* if mechanism is not supported, skip */
+-  const char *const *mechs = passdb->passdb->mechanisms;
+-  const char *const *username_filter = passdb->passdb->username_filter;
++  const char *const *mechs = passdb->mechanisms;
++  const char *const *username_filter = passdb->username_filter;
+   const char *username;
+ 
+   username = request->fields.user;
+@@ -567,7 +567,7 @@ auth_request_want_skip_passdb(struct aut
+   return TRUE;
+   }
+ 
+-  if (passdb->passdb->username_filter != NULL &&
++  if (passdb->username_filter != NULL &&
+   !auth_request_username_accepted(username_filter, username)) {
+   auth_request_log_debug(request,
+  request-&g

Bug#1016351: dovecot: CVE-2022-30550

[Noah Meyerhans]

Control: tags -1 + pending

The fix targeting sid is pending review on salsa.

My inclination is that this won't need a DSA and can wait for a bullseye point 
release, but I'm open to other opinions.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Bug#1013439: nmu: dovecot-fts-xapian_1.5.5-1

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu

nmu dovecot-fts-xapian_1.5.5-1 . ANY . unstable . -m "rebuild for 
dovecot-abi-2.3.abiv19"

Bug#1013438: nmu: dovecot-antispam_2.0+20171229-1

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu

nmu dovecot-antispam_2.0+20171229-1 . ANY . unstable . -m "rebuild for 
dovecot-abi-2.3.abiv19"

Bug#1010137: RFP: libmail-dmarc-perl -- Perl implementation of DMARC

[Noah Meyerhans]

Package: wnpp
Severity: wishlist

* Package name: libmail-dmarc-perl
  Version : 1.20211209
  Upstream Author : Marc Bradshaw 
* URL : https://metacpan.org/pod/Mail::DMARC
* License : GPL-1+ or Artistic
  Programming Lang: Perl
  Description : Perl implementation of DMARC

Mail::DMARC is a set of Perl modules for implementing DMARC clients.

DMARC, which stands for “Domain-based Message Authentication, Reporting &
Conformance”, is an email authentication, policy, and reporting protocol. It
builds on the widely deployed SPF and DKIM protocols, adding linkage to the
author (“From:”) domain name, published policies for recipient handling of
authentication failures, and reporting from receivers to senders, to improve
and monitor protection of the domain from fraudulent email.

SpamAssassin 4.0 (currently in experimental) makes use of this module to
provide its DMARC support.  I don't mind (co-)maintaining this package, but
ideally it will be done with the help of the pkg-perl group.

Bug#1009979: spamassassin: sa-learn mbox parsing is broken

[Noah Meyerhans]

Package: spamassassin
Version: 4.0.0~0.0svn1899920-1
Severity: important
Tags: upstream experimental
Forwarded: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7977

When passing an mbox file to sa-learn, expected training does not occur:

noahm@74805e6e29ad:/tmp$ sa-learn --spam --mbox < spam-2022-04-20_1040 
Learned tokens from 0 message(s) (0 message(s) examined)
noahm@74805e6e29ad:/tmp$ echo $?
1

The previous version in experimental did not have this problem. 

noahm@c30d8e8ec714:/src/spamassassin$ sa-learn -D --spam --mbox < 
spam-2022-04-20_1040 2> debug-revert.log
Learned tokens from 2 message(s) (2 message(s) examined)
noahm@c30d8e8ec714:/src/spamassassin$ echo $?
0

Bug#1008582: cloud.debian.org: SSM public parameters for buster-backports AMIs aren't getting updated

[Noah Meyerhans]

Package: cloud.debian.org
Severity: normal
User: cloud.debian@packages.debian.org
Usertags: + aws infrastructure

The IDs for buster-backports AMIs for AWS are queryable via SSM public
parameters at /aws/service/debian/release/10-backports/  However, the release
pipeline is apparently not updating them, at least some of the time, and the
most recently published parameter is from last year:

$ aws ssm get-parameters-by-path \
   --path /aws/service/debian/release/10-backports \
   --recursive --output json \
   --query "max_by(Parameters[], )"
{
"Name": "/aws/service/debian/release/10-backports/20211229-871/arm64",
"Type": "String",
"Value": "ami-0d4fccbb20a209053",
"Version": 1,
"LastModifiedDate": 1640801031.624,
"ARN": 
"arn:aws:ssm:us-west-2::parameter/aws/service/debian/release/10-backports/20211229-871/arm64",
"DataType": "text"
}

Given that we have published buster-backports AMIs within the past two days,
this data is clearly stale.

Bullseye and Buster regular listings are refreshed as expected.  The problem is
likely related to the SSM publication code not handling the case when both
buster and buster-backports are being published in the same pipeline run.

Bug#1006346: cloud.debian.org: bullseye AMIs don't boot on Amazon EC2 Xen instances with Enhanced Networking

[Noah Meyerhans]

On Sat, Mar 19, 2022 at 10:41:39AM +0100, Salvatore Bonaccorso wrote:
> > >From the upstream discussion on the linux-pci mailing list [*]:
> > 
> > > Yes. My understanding is that the issue is because AWS is using older
> > > versions of Xen. They are in the process of updating their fleet to a
> > > newer version of Xen so the change introduced with Stefan's commit
> > > isn't an issue any longer.
> > > 
> > > I think the changes are scheduled to be completed in the next 10-12
> > > weeks. For now we are carrying a revert in the Fedora Kernel.
> > > 
> > > You can follow this Fedora CoreOS issue if you'd like to know more
> > > about when the change lands in their backend. We work closely with one
> > > of their partner engineers and he keeps us updated.
> > > https://github.com/coreos/fedora-coreos-tracker/issues/1066
> > 
> > Ideally we can revert the upstream commit from the stable kernels, since
> > otherwise Debian users on AWS Xen instance types may be stuck using
> > older, unsafe kernels.  Especially if we have time to include the change
> > in the upcoming bullseye and buster point releases.  If the kernel
> > updates for those stable updates have already been built, though, it
> > might be too late to matter.  By the time we publish our next kernel
> > builds, the AWS Xen update may be complete.
> 
> Wehere one can track the update status for their Xen version directly
> or is following the above the only reference?

It's just for reference; the deployment timeline isn't published.  As
far as I know, it's also subject to change in the event that unexpected
issues arise or it's preempted by some high severity issue.

> How frequent is this particular combination of hardware/software? We
> have the change already applied for a while in bullseye, buster would
> be impacted new since the last update done for security fixes

The impacted instance types aren't the most common, as they're not the
latest generation.  So I expect that the majority of the impact is felt
by people or organizations that haven't yet been able to make time to
switch to newer instance types.  The implication here, of course, is
that many of these deployments may be production environment where
stability is prioritized over migration to the new thing.

We get a little bit of data about what instance types are used with
Debian on AWS, but it's incomplete as it only reflects usage by AWS
customers who use access Debian via the AWS Marketplace.  Consider it
something like popcon data; it's essentially opt-in.  If the data we get
from the Marketplace covering the past 3 days worth of activity is
representative of the Debian usage in general, then it looks like
roughly 1% of Debian users on AWS are trying to use the impacted
instance types.

> Are there workarounds for the affected users of this combination? I
> see some options listed in 
> https://wiki.debian.org/Cloud/AmazonEC2Image/Bullseye 

People can use newer generation instance types, which are not impacted.
Depending on the use case, that could be a trivial change, but it could
also be disruptive.  Newer instance types aren't based on Xen at all and
expose a different hardware device model to the instance.  Debian
supports the newer instance types, but the end user workload may still
need additional nontrivial qualification.

> If we revert the commit it reverts a fix for a bug with Marvell NVME
> devices.
> 
> But we cannot just revert the commit for the cloud images.

Understood.

> If we know something about the release schedule from Amazon to update
> their Xen instances (which is the way to move forward, since upstream
> won't revert the commit) then we should leave the status as it is for
> bullseye (and now for buster). For bullseye there is there is
> CVE-2022-0847 fixes they would need to pick up.

Yes, the problem will go away when the Xen fleet is updated.  It sounds
like we're looking at roughly a 3 month timeline, after which point the
patch won't be a problem.  However, until then, people who need to use
Xen instances will be stuck either running an unsafe kernel or building
their own.

noah

Bug#1006346: cloud.debian.org: bullseye AMIs don't boot on Amazon EC2 Xen instances with Enhanced Networking

[Noah Meyerhans]

>From the upstream discussion on the linux-pci mailing list [*]:

> Yes. My understanding is that the issue is because AWS is using older
> versions of Xen. They are in the process of updating their fleet to a
> newer version of Xen so the change introduced with Stefan's commit
> isn't an issue any longer.
> 
> I think the changes are scheduled to be completed in the next 10-12
> weeks. For now we are carrying a revert in the Fedora Kernel.
> 
> You can follow this Fedora CoreOS issue if you'd like to know more
> about when the change lands in their backend. We work closely with one
> of their partner engineers and he keeps us updated.
> https://github.com/coreos/fedora-coreos-tracker/issues/1066

Ideally we can revert the upstream commit from the stable kernels, since
otherwise Debian users on AWS Xen instance types may be stuck using
older, unsafe kernels.  Especially if we have time to include the change
in the upcoming bullseye and buster point releases.  If the kernel
updates for those stable updates have already been built, though, it
might be too late to matter.  By the time we publish our next kernel
builds, the AWS Xen update may be complete.

noah

* 
https://lore.kernel.org/linux-pci/c4a65b9a-d1e2-bf0d-2519-aac718593...@redhat.com/

Bug#1007144: linux-image-cloud-amd64: Network doesn't come up on AWS Xen-based EC2 instances (ex c4.large)

[Noah Meyerhans]

Control: reassign -1 src:linux
Control: forcemerge 1006346 -1

On Sat, Mar 12, 2022 at 01:21:23AM +, Reilly Brogan wrote:
> I bisected this issue and it was introduced in kernel 5.10.88 as commit
> e5949933f313c9e2c30ba05b977a047148b5e38c "PCI/MSI: Mask MSI-X vectors
> only on success", thus present in linux-image-5.10.0-11-amd64 which uses
> the 5.10.92 kernel (and all newer versions of the package).

I've already reported this as
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006346

The change implicated has been backported to all the upstream stable
branches, so this basically impacts all the current Debian kernels (or
will, once they're updated).

Bug#1006867: nmu: dovecot-antispam_2.0+20171229-1+b9

[Noah Meyerhans]

Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: binnmu

dovecot-antispam needs to be rebuild against the latest dovecot ABI.

nmu dovecot-antispam_2.0+20171229-1+b9 . ANY . unstable . -m "rebuild for 
dovecot-abi-2.3.abiv18"

Bug#1006439: Spamassasin is ignoring own rules

[Noah Meyerhans]

Control: tags -1 + moreinfo

Hello Karsten,

On Fri, Feb 25, 2022 at 01:57:08PM +0100, karsten wrote:
> i think this is not a bug and spamassassin is simply not using the rules.
> First i have added my own rules to /etc/spamassassin/local.cf without success.
> Because this has no effect i put the files in an extra file 
> /etc/spamassassin/myrules.cf
> This is ignored too.

How are you testing your rule?  Does spamassassin --lint report anything
notable?  Also consider adding the '-D' flag to generate copious
additional debug output.

I have done the following to test it, and things seem to work as
expected:

1. Install the rules:

root@a103ba41188a:/# cat > /etc/spamassassin/test.cf
header CONTAINS_SUB Subject =~
/(Sex|Potenz|Dating|Finanz|FFP-2|M[aä]dchen|Bett|ficken)/i
score CONTAINS_SUB 5
describe CONTAINS_SUB Betreff enthält Spam-Wörter

2. Construct a message to trigger the rule:

root@a103ba41188a:/# sed 's,Subject:.*,Subject: Potenz,; 
s,GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL,,' 
/usr/share/doc/spamassassin/examples/sample-spam.txt > /tmp/test.txt

3. Process the message with spamassassin. Note the report results at the
bottom:

Date: Wed, 23 Jul 2003 23:30:00 +0200
From: Sender 
To: Recipient 
Precedence: junk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

This is the GTUBE, the
Generic
Test for
Unsolicited
Bulk
Email

If your spam filter supports it, the GTUBE provides a test by which you
can verify that the filter is installed correctly and is detecting incoming
spam. You can send yourself a test mail containing the following string of
characters (in upper case and with no white spaces and line breaks):

XJS*C4JDBQADN1.NSBN3*2IDNEN**C.34X

You should send this test mail from an account outside of your network.

Spam detection software, running on the system "a103ba41188a",
has NOT identified this incoming email as spam.  The original
message has been attached to this so you can view it or label
similar future email.  If you have any questions, see
@@CONTACT_ADDRESS@@ for details.

Content preview:  This is the GTUBE, the Generic Test for Unsolicited Bulk Email
   If your spam filter supports it, the GTUBE provides a test by which you can
   verify that the filter is installed correctly and is detecting incoming spam.
   You can send yourself a test mail containing t [...] 

Content analysis details:   (4.9 points, 5.0 required)

 pts rule name  description
 -- --
 5.0 CONTAINS_SUB   Betreff enthält Spam-Wörter
-0.0 NO_RELAYS  Informational: message was not relayed via SMTP
-0.0 NO_RECEIVEDInformational: message has no Received headers

Bug#1006346: cloud.debian.org: bullseye AMIs don't boot on Amazon EC2 Xen instances with Enhanced Networking

[Noah Meyerhans]

Control: reassign -1 src:linux
Control: tags -1 + upstream

> Amazon EC2 instance types with Enhanced Networking use the ixgbevf.ko
> driver.  The current AMIs successfully probe the ixgbevf driver and spawn
> dhclient as expected, but dhclient appears to never receive a lease.  Older
> AMIs do work on this class of instance.

Upstream commit 83dbf898a2d4 "PCI/MSI: Mask MSI-X vectors only on
success" seems to introduce a regression that breaks the "Enhanced
Networking" feature used on Amazon EC2 instances, which use PCI
passthrough access to Intel ethernet devices using the ixgbevf.ko
driver.  Systems using this hardware seem to probe their network
hardware as usual, and don't log any errors to the console, but are
never able to communicate over the NIC.

Device details:

00:03.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller 
Virtual Function (rev 01)
Physical Slot: 3
Flags: bus master, fast devsel, latency 64
Memory at f300 (64-bit, prefetchable) [size=16K]
Memory at f3004000 (64-bit, prefetchable) [size=16K]
Capabilities: 
Kernel driver in use: ixgbevf
Kernel modules: ixgbevf

The issue is present in Debian kernels in sid and experimental.

The patch has been backported to stable branches including those used in
our stable releaseѕ:

The 5.10.x (released with v5.10.88) is e5949933f313.  Since bullseye is
currently using v5.10.92, it is impacted.

The 4.19.x branch (released with v4.19.222) is 12ae8cd1c7e9.  Since
buster is still on v4.19.208, it is not yet impacted, but likely would
be with the next kernel update.

This issue has been reported elsewhere as well, for example Fedora
CoreOS at https://github.com/coreos/fedora-coreos-tracker/issues/1066

I have confirmed that reverting e5949933f313 from 5.10.x results in a
build that functions properly with this hardware on bullseye, but this
is probably not a reasonable thing to do generally.

noah

Bug#1006346: cloud.debian.org: bullseye AMIs don't boot on Amazon EC2 Xen instances with Enhanced Networking

[Noah Meyerhans]

Package: cloud.debian.org
Severity: important

(I suspect this is actually a kernel issue, but I'm starting with
cloud.debian.org as that's where I've observed the issue and I want to rule
out cloud configuration issues.)

Amazon EC2 instance types with Enhanced Networking use the ixgbevf.ko
driver.  The current AMIs successfully probe the ixgbevf driver and spawn
dhclient as expected, but dhclient appears to never receive a lease.  Older
AMIs do work on this class of instance.

Working AMI:

{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2021-12-21T01:39:18.000Z",
"ImageId": "ami-055b24b622e97e043",
"ImageLocation": "136693071363/debian-11-amd64-20211220-862",
"ImageType": "machine",
"Public": true,
"OwnerId": "136693071363",
"PlatformDetails": "Linux/UNIX",
"UsageOperation": "RunInstances",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": true,
"SnapshotId": "snap-0b1ec9931c8475322",
"VolumeSize": 8,
"VolumeType": "gp2",
"Encrypted": false
}
}
],
"Description": "Debian 11 (20211220-862)",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "debian-11-amd64-20211220-862",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}

Not working AMI:

{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2022-01-21T15:02:04.000Z",
"ImageId": "ami-0d0d8694ba492c02b",
"ImageLocation": "136693071363/debian-11-amd64-20220121-894",
"ImageType": "machine",
"Public": true,
"OwnerId": "136693071363",
"PlatformDetails": "Linux/UNIX",
"UsageOperation": "RunInstances",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/xvda",
"Ebs": {
"DeleteOnTermination": true,
"SnapshotId": "snap-0f15c399cd68cf47c",
"VolumeSize": 8,
"VolumeType": "gp2",
"Encrypted": false
}
}
],
"Description": "Debian 11 (20220121-894)",
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "debian-11-amd64-20220121-894",
"RootDeviceName": "/dev/xvda",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}

Relevant logs from the instance boot:

Feb 23 22:04:09 debian kernel: [1.763348] ixgbevf :00:03.0 ens3: 
renamed from eth0
...
Feb 23 22:04:09 debian kernel: [5.283750] ixgbevf :00:03.0: NIC Link is 
Up 10 Gbps
Feb 23 22:04:09 debian kernel: [5.287656] IPv6: ADDRCONF(NETDEV_CHANGE): 
ens3: link becomes ready
...
Feb 23 22:04:09 debian systemd-udevd[239]: Using default interface naming 
scheme 'v247'.
Feb 23 22:04:09 debian systemd-udevd[234]: ethtool: autonegotiation is unset or 
enabled, the speed and duplex are not writable.
Feb 23 22:04:09 debian systemd-udevd[239]: ethtool: autonegotiation is unset or 
enabled, the speed and duplex are not writable.
...
Feb 23 22:04:09 debian cloud-ifupdown-helper: Generated configuration for ens3
...
Feb 23 22:04:09 debian systemd[1]: Found device 82599 Ethernet Controller 
Virtual Function.
...
Feb 23 22:04:09 debian dhclient[345]: Internet Systems Consortium DHCP Client 
4.4.1
Feb 23 22:04:09 debian dhclient[345]: Copyright 2004-2018 Internet Systems 
Consortium.
Feb 23 22:04:09 debian dhclient[345]: All rights reserved.
Feb 23 22:04:09 debian dhclient[345]: For info, please visit 
https://www.isc.org/software/dhcp/
Feb 23 22:04:09 debian dhclient[345]: 
Feb 23 22:04:09 debian dhclient[345]: Listening on LPF/ens3/02:e0:5c:07:ed:e7
Feb 23 22:04:09 debian dhclient[345]: Sending on   LPF/ens3/02:e0:5c:07:ed:e7
Feb 23 22:04:09 debian dhclient[345]: Sending on   Socket/fallback
Feb 23 22:04:09 debian dhclient[345]: DHCPDISCOVER on ens3 to 255.255.255.255 
port 67 interval 5
Feb 23 22:04:09 debian dhclient[345]: DHCPDISCOVER on ens3 to 255.255.255.255 
port 67 interval 13
Feb 23 22:04:09 debian dhclient[345]: DHCPDISCOVER on ens3 to 255.255.255.255 
port 67 interval 8
Feb 23 22:04:09 debian dhclient[345]: DHCPDISCOVER on ens3 to 255.255.255.255 
port 67 interval 13
Feb 23 22:04:09 debian dhclient[345]: DHCPDISCOVER on ens3 to 255.255.255.255 
port 67 interval 10
Feb 23 22:04:09 

Bug#1006264: RFH: dhcpcd5 -- DHCPv4, IPv6RA and DHCPv6 client with IPv4LL support

[Noah Meyerhans]

On Tue, Feb 22, 2022 at 08:27:10PM +0100, Marco d'Itri wrote:
> On Feb 22, Noah Meyerhans  wrote:
> 
> > For servers, the ideal situation is somewhat less clear, but there was
> > at least some interest in using systemd-networkd (with or without
> > netplan).
> Why even consider netplan, I wonder?

It's not something I'm interested in, but there were some arguments made
in favor of it in the earlier thread.
https://lists.debian.org/debian-devel/2021/09/msg00410.html

Bug#1006264: RFH: dhcpcd5 -- DHCPv4, IPv6RA and DHCPv6 client with IPv4LL support

[Noah Meyerhans]

On Tue, Feb 22, 2022 at 11:22:08AM +0200, Martin-Éric Racine wrote:
> Given how upstream ISC will stop development of its DHCP suite by the end of 
> 2022 [1], Debian will need to select a new stock DHCP client to ship with 
> Priority:Important.
> 
> dhcpcd5 seems like the most potential replacement. It covers most IPv4 and 
> IPv6 usage cases, and upstream regularly updates the code. However, the 
> Debian package hasn't been updated in ages.

We talked a little bit about the future of DHCP clients in #995189,
though we didn't come up with a definitive plan.  Regardless of what
happens with dhcpcd5, we do need to move forward with something.  Doing
nothing (which effectively leaves us with an unsupported dhclient by
default) is not a good option.

As far as I know, there are no drop-in replacements for dhclient.  Thus,
the change is going to be pretty significant for dhclient users.  On the
plus side, we get to use this as an opportunity to figure out what we
really want to support long-term.

For desktop systems running NetworkManager, it sounds like we don't need
a dedicated DHCP client at all; nm has DHCP client support built in and
uses it by default.

For servers, the ideal situation is somewhat less clear, but there was
at least some interest in using systemd-networkd (with or without
netplan).

So we may not need to specifically promote a DHCP client like dhcpcd5 to
priority:important.  Systemd-networkd could be the default, and
NetworkManager could be used when it's present.

noah

Bug#1005964: spamassassin: sa-update fails due to Perl experimental feature in regex

[Noah Meyerhans]

Control: tags -1 + pending

On Fri, Feb 18, 2022 at 10:16:23AM +0100, Vincent Lefevre wrote:
> I got the following mail due to sa-update error via /etc/cron.daily:
> 
> 
> /etc/cron.daily/spamassassin:
> config: invalid regexp for __URI_TRY_3LD 
> 'm,^https?://(?:try(?!r\.codeschool)|start|get(?!\.adobe)|save|check(?!out)|act|compare|join|learn(?!ing)|request|visit(?!or|\.vermont)|my(?!sub|turbotax|news\.apple|a\.godaddy|account|support|build|blob)\w)[^.]*\.[^/]+\.(?  Variable length lookbehind is experimental in regex; marked by <-- HERE in 
> m/(?i)^https?://(?:try(?!r\.codeschool)|start|get(?!\.adobe)|save|check(?!out)|act|compare|join|learn(?!ing)|request|visit(?!or|\.vermont)|my(?!sub|turbotax|news\.apple|a\.godaddy|account|support|build|blob)\w)[^.]*\.[^/]+\.(?  <-- HERE /
> 
> channel 'updates.spamassassin.org': lint check of update failed, channel 
> failed
> sa-update failed for unknown reasons
> 
> 
> Is the version of Perl too old in Debian 11 (bullseye)?

This was a bug in the spamassassin rules distributed via the sa-update
channels.  Upstream has resolved the associated bug, but it'd be nice to
get confirmation from people who have been impacted.

noah

Bug#1005851: systemd: networkd does not reliably configure hot-plugged interfaces

[Noah Meyerhans]

Control: forarded -1 https://github.com/systemd/systemd/issues/22538

On Wed, Feb 16, 2022 at 08:11:15AM +0100, Michael Biebl wrote:
> Am 16.02.22 um 02:14 schrieb Noah Meyerhans:
> > However, starting with the systemd 250 upstream releases, configuration of
> > these interfaces fails intermittently, with networkd not properly 
> > associating
> > the interface with the .network file:
> 
> Is that a regression compared to v249?
> Would be great if you can file this upstream
> https://github.com/systemd/systemd/

Done, thanks.  It definitely seems to be a regression.  I can run the
attach/test/detach operation in a loop 100x (and counting) with no
failures using 249, but it fails ~30% of the time in 250.

noah

Bug#1005851: systemd: networkd does not reliably configure hot-plugged interfaces

[Noah Meyerhans]

Package: systemd
Version: 250.3-2
Severity: important

Dear Maintainer,

I'm using systemd-networkd on an Amazon EC2 virtual machine with the following
.network file:

admin@ip-10-0-0-80:~$ cat /etc/systemd/network/20-eni.network
[Match]
Driver=ena

[Link]
MTUBytes=9001

[Network]
DHCP=yes
IPv6DuplicateAddressDetection=0
LLMNR=no

[DHCPv4]
UseHostname=no

This hardware supports hotplugging, and it's a fairly common operation to
perform with it.  When an interface matching this configuration attached, the
expected behavior is that networkd configures it per the .network file, as
shown:

admin@ip-10-0-0-80:~$ networkctl status ens6
● 3: ens6
 Link File: /usr/lib/systemd/network/99-default.link
  Network File: /etc/systemd/network/20-eni.network
  Type: ether
 State: routable (configured)
  Online state: online
 Alternative Names: enp0s6
  Path: pci-:00:06.0
Driver: ena
Vendor: Amazon.com, Inc.
 Model: Elastic Network Adapter (ENA)
HW Address: 02:3f:ef:3b:20:71
   MTU: 9001 (min: 128, max: 9216)
 QDisc: mq
  IPv6 Address Generation Mode: eui64
  Queue Length (Tx/Rx): 2/2
   Address: 10.0.0.8 (DHCP4 via 10.0.0.1)
2600:1f14:eeb:2203:5dbb:f7c8:9e47:822e
2600:1f14:eeb:2203:8ff0:610d:6df4:6b3e
fe80::3f:efff:fe3b:2071
   Gateway: 10.0.0.1
fe80::8f:53ff:fe6f:64ef
   DNS: 10.0.0.2
 Activation Policy: up
   Required For Online: yes
   DHCP4 Client ID: IAID:0x6618dd42/DUID
 DHCP6 Client IAID: 0x6618dd42
 DHCP6 Client DUID: DUID-EN/Vendor:ab11b98c8625365b6386

However, starting with the systemd 250 upstream releases, configuration of
these interfaces fails intermittently, with networkd not properly associating
the interface with the .network file:

admin@ip-10-0-0-165:~$ networkctl status ens6
● 6: ens6   
 Link File: /usr/lib/systemd/network/99-default.link
  Network File: n/a
  Type: ether
 State: off (unmanaged)
  Online state: unknown
 Alternative Names: enp0s6
  Path: pci-:00:06.0
Driver: ena
Vendor: Amazon.com, Inc.
 Model: Elastic Network Adapter (ENA)
  Hardware Address: 02:3f:ef:3b:20:71
   MTU: 1500 (min: 128, max: 9216)
 QDisc: noop
  IPv6 Address Generation Mode: eui64
  Queue Length (Tx/Rx): 2/2
 Activation Policy: up
   Required For Online: yes

The networkd logs don't indicate any errors, but they also don't show
the link bring brought up or otherwise configured:

Feb 15 23:11:56 ip-10-0-0-165 kernel: pci :00:06.0: [1d0f:ec20] type 00 
class 0x02
Feb 15 23:11:56 ip-10-0-0-165 kernel: pci :00:06.0: reg 0x10: [mem 
0x-0x3fff]
Feb 15 23:11:56 ip-10-0-0-165 kernel: pci :00:06.0: BAR 0: assigned [mem 
0x8000-0x80003fff]
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0: enabling device ( 
-> 0002)
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0: ENA device version: 0.10
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0: ENA controller version: 
0.0.1 implementation version 1
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0: LLQ is not supported 
Fallback to host mode policy.
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0: Elastic Network Adapter 
(ENA) found at mem 8000, mac addr 02:3f:ef:3b:20:71
Feb 15 23:11:56 ip-10-0-0-165 kernel: ena :00:06.0 ens6: renamed from eth0
Feb 15 23:11:56 ip-10-0-0-165 systemd-udevd[517]: Using default interface 
naming scheme 'v250'.
Feb 15 23:11:56 ip-10-0-0-165 systemd-networkd[371]: eth0: Interface name 
change detected, renamed to ens6.

udev seems to see the correct information for this interface:

admin@ip-10-0-0-165:~$ udevadm info /sys/class/net/ens6 
P: /devices/pci:00/:00:06.0/net/ens6
L: 0
E: DEVPATH=/devices/pci:00/:00:06.0/net/ens6
E: INTERFACE=ens6
E: IFINDEX=5
E: SUBSYSTEM=net
E: USEC_INITIALIZED=172589369
E: ID_NET_NAMING_SCHEME=v250
E: ID_NET_NAME_MAC=enx023fef3b2071
E: ID_NET_NAME_PATH=enp0s6
E: ID_NET_NAME_SLOT=ens6
E: ID_BUS=pci
E: ID_VENDOR_ID=0x1d0f
E: ID_MODEL_ID=0xec20
E: ID_PCI_CLASS_FROM_DATABASE=Network controller
E: ID_PCI_SUBCLASS_FROM_DATABASE=Ethernet controller
E: ID_VENDOR_FROM_DATABASE=Amazon.com, Inc.
E: ID_MODEL_FROM_DATABASE=Elastic Network Adapter (ENA)
E: 

Bug#970827: ping: socket: Operation not permitted while apt dist-upgrade is in progress

[Noah Meyerhans]

Control: reassign -1 src:dpkg
Control: severity -1 wishlist

> root@debian:~# ls -l `which ping`
> -rwxr-xr-x 1 root root 77432 Aug 23 19:08 /usr/bin/ping
> root@debian:~# getcap `which ping`
> /usr/bin/ping cap_net_raw=ep
> root@debian:~#
> 
> 
> This looks like a limitation that would only be possible to solve by
> dpkg and extending tar / cpio probably.
> 
> >From what I found it is possible to do this with tar and
> --xattrs-include='security.capability'  when packing and unpacking.
> 
> There is some hacky non-standard patches for cpio,
> https://github.com/initlove/cpio/commit/531cabc88e9ecdc3231fad6e4856869baa9a91ef
> , but afaik not upstreamed.
> And even more hacky support in kernel for initramfs uses:
> https://lists.gnu.org/archive/html/bug-cpio/2019-05/msg1.html
> 
> I didn't see any real updates on this topic, last one is from middle of 2019.

I'm reassigning this to dpkg as a wishlist item.  If the problem is
going to be fixed, it's going to happen at a layer more fundamental to
package management.

Context for the dpkg maintainers:

Ping requires elevated privileges in order to open its ICMP network
sockets.  The postinst script attempts to set a file-based cap_net_raw
capability on the binary after installation, and falls back to setuid in
case that fails (usually due to missing filesystem support for file
capabilities).  This workflow is racy, however, as there's a period of
time when the file exists on disk but has not had any privilege
acquisition mechanism applied to it.  During this period of time,
unprivileged users cannot run this program, when otherwise they could.
Elimination of this race situation would likely require the ability for
dpkg to initially create files with additional file-based capabilities.

noah

Bug#1004843: debina bullseye: ping (from iputils-ping) throwing inappropriate error message if IPv6 is disabled

[Noah Meyerhans]

Control: tags -1 + bullseye

On Wed, Feb 02, 2022 at 08:36:22AM +0100, Binarus wrote:
> IMPORTANT NOTE:
> According to other reports of the same problem, ping behaves correctly when 
> IPv6 is *not* disabled at the kernel command line, but *is* instead disabled 
> via sysctl (e.g. sysctl -w net.ipv6.conf.all.disable_ipv6=1). The problem 
> occurs only when IPv6 is disabled at the kernel command line. Please note 
> also that disabling it via sysctl *in addition* to disabling it at the kernel 
> command line does not solve the problem. Actually, it does not change 
> anything, because the respective sysctl option does not have any effect if 
> IPv6 is already disabled at the kernel command line.
> 
> ping's misbehavior in this situation is not acceptable. Many people have IPv6 
> disabled at the kernel command line, and ping's misbehavior breaks a lot of 
> Nagios scripts for them, which is why I have assigned the "critical" severity 
> level.
> 
> Please fix this as soon as possible. This bug is well-known and has been 
> fixed upstream 4 (yes: four!) months ago. Please refer to the following 
> discussion:
> 
> https://github.com/iputils/iputils/issues/293

This bug by itself is entirely cosmetic and doesn't impact ping's
functionality, so I don't think it'd warrant a fix to stable by itself.
However, #920434, which covers measurement inaccuracies in ping, is
significantly more important and does warrant a stable update.  So I
think we can bundle the fix for this one with its fix and get both of
them resolved in a stable update.

Will prepare an upload and follow up with the release team.

noah