[Sts-sponsors] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak
Hi Mauricio,
It seems riscv64 passes on Groovy due to tests being skipped on the
riscv64 architecture.
>From Groovy's build log:
https://paste.ubuntu.com/p/NCJPDVSbSW/
If you look at the man page for dh_auto_test it mentions:
If the DEB_BUILD_OPTIONS environment variable contains nocheck, no tests
will be performed.
nocheck was added to riscv64 by default for all packages in Groovy as a
part of this change to dpkg in bug 1891686.
The test cases basic-realistic.sh and tls-basic-realistic.sh fail on
Focal because they attempt to send 100,000 packets between the server
and the client, and we get to various stages, like 00029000 msgs sent,
and now 00047000 msgs sent with some changes William made to the
builders, before it times out and assumes the channel is dead, and the
test fails.
https://paste.ubuntu.com/p/hwYXSbKPPV/
We aren't going to hit the 100,000 packets on riscv anytime soon. I
think I will open a new bug to adjust the packet counts from 100,000
down to 10,000 for basic-realistic.sh and tls-basic-realistic.sh, which
resembles what has been done for receiver-abort.sh and tls-receiver-
abort.sh.
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1908473
Title:
rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
leads to file descriptor leak
Status in librelp package in Ubuntu:
Fix Released
Status in rsyslog package in Ubuntu:
Fix Released
Status in librelp source package in Focal:
Fix Committed
Status in rsyslog source package in Focal:
Won't Fix
Status in librelp source package in Groovy:
Fix Committed
Status in rsyslog source package in Groovy:
Fix Released
Status in librelp source package in Hirsute:
Fix Released
Status in rsyslog source package in Hirsute:
Fix Released
Bug description:
[Impact]
In recent versions of rsyslog and librelp, the imrelp module leaks
file descriptors due to a bug where it does not correctly close
sockets, and instead, leaves them in the CLOSE_WAIT state.
This causes rsyslogd on busy servers to eventually hit the limit of
maximum open files allowed, which locks rsyslogd up until it is
restarted.
A workaround is to restart rsyslogd every month or so to manually
close all of the open sockets.
Only users of the imrelp module are affected, and not rsyslog users in
general.
[Testcase]
Install the rsyslog-relp module like so:
$ sudo apt install rsyslog rsyslog-relp
Next, generate a working directory, and make a config file that loads
the relp module.
$ sudo mkdir /workdir
$ cat << EOF >> ./spool.conf
\$LocalHostName spool
\$AbortOnUncleanConfig on
\$PreserveFQDN on
global(
workDirectory="/workdir"
maxMessageSize="256k"
)
main_queue(queue.type="Direct")
module(load="imrelp")
input(
type="imrelp"
name="imrelp"
port="601"
ruleset="spool"
MaxDataSize="256k"
)
ruleset(name="spool" queue.type="direct") {
}
# Just so rsyslog doesn't whine that we do not have outputs
ruleset(name="noop" queue.type="direct") {
action(
type="omfile"
name="omfile"
file="/workdir/spool.log"
)
}
EOF
Verify that the config is valid, then start a rsyslog server.
$ sudo rsyslogd -f ./spool.conf -N9
$ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid
Fetch the rsyslogd PID and check for open files.
$ RLOGPID=$(cat /workdir/rsyslogd.pid)
$ sudo ls -l /proc/$RLOGPID/fd
total 0
lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'
We have 3 sockets open by default. Next, use netcat to open 100
connections:
$ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done
Now check for open file descriptors, and there will be an extra 100 sockets
in the list:
$ sudo ls -l /proc/$RLOGPID/fd
https://paste.ubuntu.com/p/f6NQVNbZcR/
We can check the state of these sockets with:
$ ss -t
https://paste.ubuntu.com/p/7Ts2FbxJrg/
The listening sockets will be in CLOSE-WAIT, and the netcat sockets
will be in FIN-WAIT-2.
$ ss -t | grep CLOSE-WAIT | wc -l
100
If you install the test package available in the following ppa:
https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test
When you open connections with netcat, these will be closed properly,
and the file descriptor leak will be fixed.
[Where problems could occur]
If a regression were to occur, it would be limited to users of the
imrelp module, which is a part of the rsyslogd-relp package, and
depends on librelp.
rsyslog-relp is not part of a default installation of rsyslog, and is
opt in by changing a configuration file to
[Sts-sponsors] [Bug 1817651] Re: Primary slave on the bond not getting set.
** Tags removed: bonding verification-needed -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1817651 Title: Primary slave on the bond not getting set. Status in netplan.io package in Ubuntu: Fix Released Status in netplan.io source package in Bionic: Fix Committed Status in netplan.io source package in Focal: Fix Released Bug description: [Impact] primary slave fails to get set in netplan bonding configuration [Test Case] 0. created vm with 3 nics ( ens33, ens38, ens39 ) 1. setup netplan as below - https://pastebin.ubuntu.com/p/JGqhYXYY6r/ - ens38, ens39 is virtual nic, and dummy2 is not. 2. netplan apply 3. shows error [Where problems could occur] As this patch is related to bond, bond may have issue if there is problem. [Others] original description The primary slave fails to get set in netplan bonding configuration: network: version: 2 ethernets: e1p1: addresses: - x.x.x.x/x gateway4: x.x.x.x match: macaddress: xyz mtu: 9000 nameservers: addresses: - x.x.x.x set-name: e1p1 p1p1: match: macaddress: xx mtu: 1500 set-name: p1p1 p1p2: match: macaddress: xx mtu: 1500 set-name: p1p2 bonds: bond0: mtu: 9000 interfaces: [p1p1, p1p2] parameters: mode: active-backup mii-monitor-interval: 100 primary: p1p2 ~$ sudo netplan --debug apply sudo netplan --debug apply ** (generate:7353): DEBUG: 13:22:31.480: Processing input file /etc/netplan/50-cloud-init.yaml.. ** (generate:7353): DEBUG: 13:22:31.480: starting new processing pass ** (generate:7353): DEBUG: 13:22:31.480: Processing input file /etc/netplan/60-puppet-netplan.yaml.. ** (generate:7353): DEBUG: 13:22:31.480: starting new processing pass ** (generate:7353): DEBUG: 13:22:31.480: recording missing yaml_node_t bond0 ** (generate:7353): DEBUG: 13:22:31.480: recording missing yaml_node_t bond0 ** (generate:7353): DEBUG: 13:22:31.480: recording missing yaml_node_t bond0 ** (generate:7353): DEBUG: 13:22:31.480: recording missing yaml_node_t bond0 ** (generate:7353): DEBUG: 13:22:31.480: recording missing yaml_node_t bond0 ** (generate:7353): DEBUG: 13:22:31.480: starting new processing pass Error in network definition /etc/netplan/60-puppet-netplan.yaml line 68 column 17: bond0: bond already has a primary slave: p1p2 What's wrong here?? #apt-cache policy netplan.io netplan.io: Installed: 0.40.1~18.04.4 Candidate: 0.40.1~18.04.4 Version table: *** 0.40.1~18.04.4 500 500 http://mirrors.rc.nectar.org.au/ubuntu bionic-security/main amd64 Packages 500 http://mirrors.rc.nectar.org.au/ubuntu bionic-updates/main amd64 Packages 100 /var/lib/dpkg/status 0.36.1 500 500 http://mirrors.rc.nectar.org.au/ubuntu bionic/main amd64 Packages #cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS" regards, Shahaan To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/1817651/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1898129] Re: Cannot configure 'cryptsetup luksFormat' at install time
** Tags removed: sts-sponsor-mfo
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1898129
Title:
Cannot configure 'cryptsetup luksFormat' at install time
Status in partman-crypto package in Ubuntu:
Invalid
Status in ubiquity package in Ubuntu:
Fix Released
Status in partman-crypto source package in Focal:
Fix Released
Status in ubiquity source package in Focal:
Fix Released
Status in partman-crypto source package in Groovy:
Invalid
Status in ubiquity source package in Groovy:
Won't Fix
Status in partman-crypto source package in Hirsute:
Invalid
Status in ubiquity source package in Hirsute:
Fix Released
Status in partman-crypto package in Debian:
Unknown
Bug description:
[Impact]
* Users cannot specify options for 'cryptsetup luksFormat'
that is used by the installer.
* Some deployments need the installed disks in LUKS1 format
for backward compatibility with older releases that don't
support LUKS2, for backup/audit/management purposes.
* However, on Focal and later, cryptsetup defaults to LUKS2,
which broke that functionality.
* Currently it's not possible to request the LUKS format in
the installer, so this patch allows for that w/ a preseed
option ('partman-crypto/luksformat_options') for the user.
[Test Case]
* Default behavior: LUKS2
- Install Ubuntu (Focal/later); check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...
* Opt-in behavior: LUKS1 (for example; can use other options)
- Install Ubuntu (Focal/later) with preseed file/option:
ubiquity partman-crypto/luksformat_options string \
--type luks1
- Check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4
Version: 1
...
- Check install logs for confirmation:
$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat:
'--type luks1'
[Where problems could occur]
* The changes are contained within the partman-crypto functionality,
so only install with encrypted disks should be affected by issues.
* Any additional options specified to 'cryptsetup luksFormat' are
opt-in _and_ specified by the user via the preseed option, thus
errors are probably tied to particular options (mis) used.
* If the preseed option is not specified, original behavior remains.
[Other Info]
* This patch is applied in Hirsute.
* This patch is not needed in Groovy (rationale in comment #15.)
* This patch is targeted at Focal (cryptsetup defaulted to LUKS2.)
* This patch is not needed in Bionic/earlier (^defaults to LUKS1.)
[Original Description]
Most users should be fine with the options to
'cryptsetup luksFormat' used by the installer.
However, some users may have reasons to use
other options, and that is not possible now.
Let's provide a new preseed option for that:
'partman-crypto/luksformat_options'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/partman-crypto/+bug/1898129/+subscriptions
--
Mailing list: https://launchpad.net/~sts-sponsors
Post to : sts-sponsors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~sts-sponsors
More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak
Hi Matthew,
I've kicked one or two rebuilds today, and still see the (same) one or
two tests failing.
I'm out this week so can't look much more into it (althogh I'm
admittedly curious about it :)
If you can take a look at these FTBFS test case failures, that would be
awesome; all archs must pass for the SRU to be released. Maybe you can
get access to a riscv64 instance, or the issue perhaps reproduces in
QEMU?
Otherwise I can try and take a look on Monday.
cheers,
Mauricio
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1908473
Title:
rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
leads to file descriptor leak
Status in librelp package in Ubuntu:
Fix Released
Status in rsyslog package in Ubuntu:
Fix Released
Status in librelp source package in Focal:
Fix Committed
Status in rsyslog source package in Focal:
Won't Fix
Status in librelp source package in Groovy:
Fix Committed
Status in rsyslog source package in Groovy:
Fix Released
Status in librelp source package in Hirsute:
Fix Released
Status in rsyslog source package in Hirsute:
Fix Released
Bug description:
[Impact]
In recent versions of rsyslog and librelp, the imrelp module leaks
file descriptors due to a bug where it does not correctly close
sockets, and instead, leaves them in the CLOSE_WAIT state.
This causes rsyslogd on busy servers to eventually hit the limit of
maximum open files allowed, which locks rsyslogd up until it is
restarted.
A workaround is to restart rsyslogd every month or so to manually
close all of the open sockets.
Only users of the imrelp module are affected, and not rsyslog users in
general.
[Testcase]
Install the rsyslog-relp module like so:
$ sudo apt install rsyslog rsyslog-relp
Next, generate a working directory, and make a config file that loads
the relp module.
$ sudo mkdir /workdir
$ cat << EOF >> ./spool.conf
\$LocalHostName spool
\$AbortOnUncleanConfig on
\$PreserveFQDN on
global(
workDirectory="/workdir"
maxMessageSize="256k"
)
main_queue(queue.type="Direct")
module(load="imrelp")
input(
type="imrelp"
name="imrelp"
port="601"
ruleset="spool"
MaxDataSize="256k"
)
ruleset(name="spool" queue.type="direct") {
}
# Just so rsyslog doesn't whine that we do not have outputs
ruleset(name="noop" queue.type="direct") {
action(
type="omfile"
name="omfile"
file="/workdir/spool.log"
)
}
EOF
Verify that the config is valid, then start a rsyslog server.
$ sudo rsyslogd -f ./spool.conf -N9
$ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid
Fetch the rsyslogd PID and check for open files.
$ RLOGPID=$(cat /workdir/rsyslogd.pid)
$ sudo ls -l /proc/$RLOGPID/fd
total 0
lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'
We have 3 sockets open by default. Next, use netcat to open 100
connections:
$ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done
Now check for open file descriptors, and there will be an extra 100 sockets
in the list:
$ sudo ls -l /proc/$RLOGPID/fd
https://paste.ubuntu.com/p/f6NQVNbZcR/
We can check the state of these sockets with:
$ ss -t
https://paste.ubuntu.com/p/7Ts2FbxJrg/
The listening sockets will be in CLOSE-WAIT, and the netcat sockets
will be in FIN-WAIT-2.
$ ss -t | grep CLOSE-WAIT | wc -l
100
If you install the test package available in the following ppa:
https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test
When you open connections with netcat, these will be closed properly,
and the file descriptor leak will be fixed.
[Where problems could occur]
If a regression were to occur, it would be limited to users of the
imrelp module, which is a part of the rsyslogd-relp package, and
depends on librelp.
rsyslog-relp is not part of a default installation of rsyslog, and is
opt in by changing a configuration file to enable imrelp.
The changes to rsyslog implement a testcase which exercises the problematic
code to ensure things are working as expected; this
can be enabled manually on build, and has been verified to pass (#7).
[Other]
Upstream bug list:
https://github.com/rsyslog/rsyslog/issues/4350
https://github.com/rsyslog/rsyslog/issues/4005
https://github.com/rsyslog/librelp/issues/188
https://github.com/rsyslog/librelp/pull/193
The following commits fix the problem:
rsyslogd
commit baee0bd5420649329793746f0daf87c4f59fe6a6
Author: Andre lorbach
Date: Thu Apr 9 13:00:35 2020 +0200
Subject: testbench:
[Sts-sponsors] [Bug 1898129] Re: Cannot configure 'cryptsetup luksFormat' at install time
The functionality is present in the desktop ISO daily build of
2021-01-21 [1] , per the ISO manifest file [2]:
ubiquity 20.04.15.3
It has been verified to work correctly on testing of that ISO, with the
following in a preseed file:
ubiquity partman-crypto/luksformat_options string \
--type luks1
cheers,
Mauricio
[1] https://cdimage.ubuntu.com/focal/daily-live/20210121/
[2]
https://cdimage.ubuntu.com/focal/daily-live/20210121/focal-desktop-amd64.manifest
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1898129
Title:
Cannot configure 'cryptsetup luksFormat' at install time
Status in partman-crypto package in Ubuntu:
Invalid
Status in ubiquity package in Ubuntu:
Fix Released
Status in partman-crypto source package in Focal:
Fix Released
Status in ubiquity source package in Focal:
Fix Released
Status in partman-crypto source package in Groovy:
Invalid
Status in ubiquity source package in Groovy:
Won't Fix
Status in partman-crypto source package in Hirsute:
Invalid
Status in ubiquity source package in Hirsute:
Fix Released
Status in partman-crypto package in Debian:
Unknown
Bug description:
[Impact]
* Users cannot specify options for 'cryptsetup luksFormat'
that is used by the installer.
* Some deployments need the installed disks in LUKS1 format
for backward compatibility with older releases that don't
support LUKS2, for backup/audit/management purposes.
* However, on Focal and later, cryptsetup defaults to LUKS2,
which broke that functionality.
* Currently it's not possible to request the LUKS format in
the installer, so this patch allows for that w/ a preseed
option ('partman-crypto/luksformat_options') for the user.
[Test Case]
* Default behavior: LUKS2
- Install Ubuntu (Focal/later); check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...
* Opt-in behavior: LUKS1 (for example; can use other options)
- Install Ubuntu (Focal/later) with preseed file/option:
ubiquity partman-crypto/luksformat_options string \
--type luks1
- Check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4
Version: 1
...
- Check install logs for confirmation:
$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat:
'--type luks1'
[Where problems could occur]
* The changes are contained within the partman-crypto functionality,
so only install with encrypted disks should be affected by issues.
* Any additional options specified to 'cryptsetup luksFormat' are
opt-in _and_ specified by the user via the preseed option, thus
errors are probably tied to particular options (mis) used.
* If the preseed option is not specified, original behavior remains.
[Other Info]
* This patch is applied in Hirsute.
* This patch is not needed in Groovy (rationale in comment #15.)
* This patch is targeted at Focal (cryptsetup defaulted to LUKS2.)
* This patch is not needed in Bionic/earlier (^defaults to LUKS1.)
[Original Description]
Most users should be fine with the options to
'cryptsetup luksFormat' used by the installer.
However, some users may have reasons to use
other options, and that is not possible now.
Let's provide a new preseed option for that:
'partman-crypto/luksformat_options'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/partman-crypto/+bug/1898129/+subscriptions
--
Mailing list: https://launchpad.net/~sts-sponsors
Post to : sts-sponsors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~sts-sponsors
More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1906720] Re: Fix the disable_ssl_certificate_validation option
original problem: maas cli fails to work with apis over https with self-signed certificates due to the lack of disable_ssl_certificate_validation option with python 3.5. [0] attachment MAAS version (2.8.2) Python version (3.5 or less) Based on Robie's comment. there are 2 options 1. we continue to sru this patch 2. we ask for a monkey patch to MAAS One recent previously monkey patched by MAAS https://bugs.launchpad.net/maas/+bug/1741913 ** Attachment added: "Screenshot from 2020-12-15 12-48-12.png" https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1906720/+attachment/5455213/+files/Screenshot%20from%202020-12-15%2012-48-12.png -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1906720 Title: Fix the disable_ssl_certificate_validation option Status in maas package in Ubuntu: New Status in python-httplib2 package in Ubuntu: Fix Released Status in maas source package in Bionic: New Status in python-httplib2 source package in Bionic: Incomplete Status in maas source package in Focal: New Status in python-httplib2 source package in Focal: Fix Released Status in maas source package in Groovy: New Status in python-httplib2 source package in Groovy: Fix Released Status in maas source package in Hirsute: New Status in python-httplib2 source package in Hirsute: Fix Released Bug description: [Environment] Bionic python3-httplib2 | 0.9.2+dfsg-1ubuntu0.2 [Description] maas cli fails to work with apis over https with self-signed certificates due to the lack of disable_ssl_certificate_validation option with python 3.5. [Distribution/Release, Package versions, Platform] cat /etc/lsb-release; dpkg -l | grep maas DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" ii maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all "Metal as a Service" is a physical cloud and IPAM ii maas-cli 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS client and command-line interface ii maas-common 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server common files ii maas-dhcp 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS DHCP server ii maas-proxy 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS Caching Proxy ii maas-rack-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Rack Controller for MAAS ii maas-region-api 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region controller API service for MAAS ii maas-region-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region Controller for MAAS ii python3-django-maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server Django web framework (Python 3) ii python3-maas-client 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS python API client (Python 3) ii python3-maas-provisioningserver 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server provisioning libraries (Python 3) [Steps to Reproduce] - prepare a maas server(installed by packages for me and the customer). it doesn't have to be HA to reproduce - prepare a set of certificate, key and ca-bundle - place a new conf[2] in /etc/nginx/sites-enabled and `sudo systemctl restart nginx` - add the ca certificates to the host sudo mkdir /usr/share/ca-certificates/extra sudo cp -v ca-bundle.crt /usr/share/ca-certificates/extra/ dpkg-reconfigure ca-certificates - login with a new profile over https url - when not added the ca-bundle to the trusted ca cert store, it fails to login and '--insecure' flag also doesn't work[3] [Known Workarounds] None [Test] # Note even though this change only affects Python3 # I tested it with Python2 with no issues and was able to connect. Also please make note of the 2 packages. One is for Python2 the other Python3 Python2 ===> python-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb Python3 ===> python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb helpful urls: https://maas.io/docs/deb/2.8/cli/installation https://maas.io/docs/deb/2.8/cli/configuration-journey https://maas.io/docs/deb/2.8/ui/configuration-journey # create bionic VM/lxc container lxc launch ubuntu:bionic lp1906720 # get source code from repo pull-lp-source python-httplib2 bionic # install maas-cli apt-get install maas-cli # install maas server apt-get install maas # init maas sudo maas init # answer questions # generate self signed cert and key openssl req -newkey rsa:4096 -x509 -sha256 -days 60 -nodes -out localhost.crt -keyout localhost.key # add certs sudo cp -v test.crt /usr/share/ca-certificates/extra/ # add new cert to list sudo dpkg-reconfigure ca-certificates # select yes with spacebar # save # create api key files touch api_key touch api-key-file # remove any packages with this # or this python3-httplib2 apt-cache search python-httplib2 apt-get remove python-httplib2 apt-get remove
[Sts-sponsors] [Bug 1906720] Re: Fix the disable_ssl_certificate_validation option
** Description changed:
[Environment]
Bionic
python3-httplib2 | 0.9.2+dfsg-1ubuntu0.2
[Description]
maas cli fails to work with apis over https with self-signed certificates due
to the lack
of disable_ssl_certificate_validation option with python 3.5.
[Distribution/Release, Package versions, Platform]
cat /etc/lsb-release; dpkg -l | grep maas
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
ii maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all "Metal as a Service" is a
physical cloud and IPAM
ii maas-cli 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS client and
command-line interface
ii maas-common 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server common
files
ii maas-dhcp 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS DHCP server
ii maas-proxy 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS Caching Proxy
ii maas-rack-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Rack
Controller for MAAS
ii maas-region-api 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region
controller API service for MAAS
ii maas-region-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region
Controller for MAAS
ii python3-django-maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS
server Django web framework (Python 3)
ii python3-maas-client 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS
python API client (Python 3)
ii python3-maas-provisioningserver 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1
all MAAS server provisioning libraries (Python 3)
[Steps to Reproduce]
- prepare a maas server(installed by packages for me and the customer). it
doesn't have to be HA to reproduce
- prepare a set of certificate, key and ca-bundle
- place a new conf[2] in /etc/nginx/sites-enabled and `sudo systemctl restart
nginx`
- add the ca certificates to the host
sudo mkdir /usr/share/ca-certificates/extra
sudo cp -v ca-bundle.crt /usr/share/ca-certificates/extra/
dpkg-reconfigure ca-certificates
- login with a new profile over https url
- when not added the ca-bundle to the trusted ca cert store, it fails to
login and '--insecure' flag also doesn't work[3]
[Known Workarounds]
None
[Test]
# Note even though this change only affects Python3
# I tested it with Python2 with no issues and was able to connect.
Also please make note of the 2 packages. One is for Python2 the other Python3
Python2 ===> python-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb
Python3 ===> python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb
helpful urls:
https://maas.io/docs/deb/2.8/cli/installation
https://maas.io/docs/deb/2.8/cli/configuration-journey
https://maas.io/docs/deb/2.8/ui/configuration-journey
# create bionic VM/lxc container
lxc launch ubuntu:bionic lp1906720
# get source code from repo
pull-lp-source python-httplib2 bionic
# install maas-cli
apt-get install maas-cli
# install maas server
apt-get install maas
# init maas
sudo maas init
# answer questions
# generate self signed cert and key
openssl req -newkey rsa:4096 -x509 -sha256 -days 60 -nodes -out localhost.crt
-keyout localhost.key
# add certs
sudo cp -v test.crt /usr/share/ca-certificates/extra/
# add new cert to list
sudo dpkg-reconfigure ca-certificates
# select yes with spacebar
# save
# create api key files
touch api_key
touch api-key-file
# remove any packages with this
# or this python3-httplib2
apt-cache search python-httplib2
apt-get remove python-httplib2
apt-get remove python3-httplib2
# create 2 admin users
sudo maas createadmin testadmin
sudo maas createadmin secureadmin
# generate maas api keys
sudo maas apikey --username=testadmin > api_key
sudo maas apikey --username=secureadmin > api-key-file
# make sure you can login to maas-cli without TLS
# by running this script
# this is for the non-tls user
# this goes into a script called maas-login.sh
touch maas-login.sh
sudo chmod +rwx maas-login.sh
#!/bin/sh
PROFILE=testadmin
API_KEY_FILE=/home/ubuntu/api_key
API_SERVER=127.0.0.1:5240
MAAS_URL=http://$API_SERVER/MAAS
maas login $PROFILE $MAAS_URL - < $API_KEY_FILE
sudo chmod +rwx https-maas.sh
# another script called https-maas.sh
# for the tls user
#!/bin/sh
PROFILE=secureadmin
API_KEY_FILE=/home/ubuntu/api-key-file
API_SERVER=127.0.0.1
MAAS_URL=https://$API_SERVER/MAAS
maas login --insecure $PROFILE $MAAS_URL - < $API_KEY_FILE
# try to login
./maas-login.sh
cd /etc/nginx/sites-enabled
sudo touch maas-https-default
#example nginx config for maas https
server {
listen 443 ssl http2;
server_name _;
ssl_certificate /home/ubuntu/localhost.crt;
ssl_certificate_key /home/ubuntu/localhost.key;
location / {
proxy_pass http://localhost:5240;
include /etc/nginx/proxy_params;
}
[Sts-sponsors] [Bug 1898129] Re: Cannot configure 'cryptsetup luksFormat' at install time
This bug was fixed in the package ubiquity - 20.04.15.3
---
ubiquity (20.04.15.3) focal; urgency=medium
* Automatic update of included source packages:
partman-crypto 101ubuntu4.1, shim-signed 1.40.4. (LP: #1898129)
* Fix FTBFS with new pyflakes (fixed in Groovy/20.10.4, no LP bug):
autopilot/ubiquity_autopilot_tests/configs/english_label_conf.py:48:
.format(...) has unused arguments at position(s): 0
-- Mauricio Faria de Oliveira Wed, 13 Jan 2021
11:42:57 -0300
** Changed in: ubiquity (Ubuntu Focal)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1898129
Title:
Cannot configure 'cryptsetup luksFormat' at install time
Status in partman-crypto package in Ubuntu:
Invalid
Status in ubiquity package in Ubuntu:
Fix Released
Status in partman-crypto source package in Focal:
Fix Released
Status in ubiquity source package in Focal:
Fix Released
Status in partman-crypto source package in Groovy:
Invalid
Status in ubiquity source package in Groovy:
Won't Fix
Status in partman-crypto source package in Hirsute:
Invalid
Status in ubiquity source package in Hirsute:
Fix Released
Status in partman-crypto package in Debian:
Unknown
Bug description:
[Impact]
* Users cannot specify options for 'cryptsetup luksFormat'
that is used by the installer.
* Some deployments need the installed disks in LUKS1 format
for backward compatibility with older releases that don't
support LUKS2, for backup/audit/management purposes.
* However, on Focal and later, cryptsetup defaults to LUKS2,
which broke that functionality.
* Currently it's not possible to request the LUKS format in
the installer, so this patch allows for that w/ a preseed
option ('partman-crypto/luksformat_options') for the user.
[Test Case]
* Default behavior: LUKS2
- Install Ubuntu (Focal/later); check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...
* Opt-in behavior: LUKS1 (for example; can use other options)
- Install Ubuntu (Focal/later) with preseed file/option:
ubiquity partman-crypto/luksformat_options string \
--type luks1
- Check LUKS header version:
$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4
Version: 1
...
- Check install logs for confirmation:
$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat:
'--type luks1'
[Where problems could occur]
* The changes are contained within the partman-crypto functionality,
so only install with encrypted disks should be affected by issues.
* Any additional options specified to 'cryptsetup luksFormat' are
opt-in _and_ specified by the user via the preseed option, thus
errors are probably tied to particular options (mis) used.
* If the preseed option is not specified, original behavior remains.
[Other Info]
* This patch is applied in Hirsute.
* This patch is not needed in Groovy (rationale in comment #15.)
* This patch is targeted at Focal (cryptsetup defaulted to LUKS2.)
* This patch is not needed in Bionic/earlier (^defaults to LUKS1.)
[Original Description]
Most users should be fine with the options to
'cryptsetup luksFormat' used by the installer.
However, some users may have reasons to use
other options, and that is not possible now.
Let's provide a new preseed option for that:
'partman-crypto/luksformat_options'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/partman-crypto/+bug/1898129/+subscriptions
--
Mailing list: https://launchpad.net/~sts-sponsors
Post to : sts-sponsors@lists.launchpad.net
Unsubscribe : https://launchpad.net/~sts-sponsors
More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1906720] Re: Fix the disable_ssl_certificate_validation option
** Also affects: maas (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1906720 Title: Fix the disable_ssl_certificate_validation option Status in maas package in Ubuntu: New Status in python-httplib2 package in Ubuntu: Fix Released Status in maas source package in Bionic: New Status in python-httplib2 source package in Bionic: Incomplete Status in maas source package in Focal: New Status in python-httplib2 source package in Focal: Fix Released Status in maas source package in Groovy: New Status in python-httplib2 source package in Groovy: Fix Released Status in maas source package in Hirsute: New Status in python-httplib2 source package in Hirsute: Fix Released Bug description: [Environment] Bionic python3-httplib2 | 0.9.2+dfsg-1ubuntu0.2 [Description] maas cli fails to work with apis over https with self-signed certificates due to the lack of disable_ssl_certificate_validation option with python 3.5. [Distribution/Release, Package versions, Platform] cat /etc/lsb-release; dpkg -l | grep maas DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" ii maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all "Metal as a Service" is a physical cloud and IPAM ii maas-cli 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS client and command-line interface ii maas-common 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server common files ii maas-dhcp 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS DHCP server ii maas-proxy 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS Caching Proxy ii maas-rack-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Rack Controller for MAAS ii maas-region-api 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region controller API service for MAAS ii maas-region-controller 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all Region Controller for MAAS ii python3-django-maas 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server Django web framework (Python 3) ii python3-maas-client 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS python API client (Python 3) ii python3-maas-provisioningserver 2.8.2-8577-g.a3e674063-0ubuntu1~18.04.1 all MAAS server provisioning libraries (Python 3) [Steps to Reproduce] - prepare a maas server(installed by packages for me and the customer). it doesn't have to be HA to reproduce - prepare a set of certificate, key and ca-bundle - place a new conf[2] in /etc/nginx/sites-enabled and `sudo systemctl restart nginx` - add the ca certificates to the host sudo mkdir /usr/share/ca-certificates/extra sudo cp -v ca-bundle.crt /usr/share/ca-certificates/extra/ dpkg-reconfigure ca-certificates - login with a new profile over https url - when not added the ca-bundle to the trusted ca cert store, it fails to login and '--insecure' flag also doesn't work[3] [Known Workarounds] None [Test] # Note even though this change only affects Python3 # I tested it with Python2 with no issues and was able to connect. Also please make note of the 2 packages. One is for Python2 the other Python3 Python2 ===> python-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb Python3 ===> python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb helpful urls: https://maas.io/docs/deb/2.8/cli/installation https://maas.io/docs/deb/2.8/cli/configuration-journey https://maas.io/docs/deb/2.8/ui/configuration-journey # create bionic VM/lxc container lxc launch ubuntu:bionic lp1906720 # get source code from repo pull-lp-source python-httplib2 bionic # install maas-cli apt-get install maas-cli # install maas server apt-get install maas # init maas sudo maas init # answer questions # generate self signed cert and key openssl req -newkey rsa:4096 -x509 -sha256 -days 60 -nodes -out localhost.crt -keyout localhost.key # add certs sudo cp -v test.crt /usr/share/ca-certificates/extra/ # add new cert to list sudo dpkg-reconfigure ca-certificates # select yes with spacebar # save # create api key files touch api_key touch api-key-file # remove any packages with this # or this python3-httplib2 apt-cache search python-httplib2 apt-get remove python-httplib2 apt-get remove python3-httplib2 # create 2 admin users sudo maas createadmin testadmin sudo maas createadmin secureadmin # generate maas api keys sudo maas apikey --username=testadmin > api_key sudo maas apikey --username=secureadmin > api-key-file # make sure you can login to maas-cli without TLS # by running this script # this is for the non-tls user # this goes into a script called maas-login.sh touch maas-login.sh sudo chmod +rwx maas-login.sh #!/bin/sh PROFILE=testadmin API_KEY_FILE=/home/ubuntu/api_key API_SERVER=127.0.0.1:5240
[Sts-sponsors] [Bug 1910432] Re: dirmngr doesn't work with kernel parameter ipv6.disable=1
** Changed in: gnupg2 (Ubuntu Groovy) Status: Incomplete => In Progress -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910432 Title: dirmngr doesn't work with kernel parameter ipv6.disable=1 Status in gnupg2 package in Ubuntu: Fix Released Status in gnupg2 source package in Bionic: In Progress Status in gnupg2 source package in Focal: Fix Committed Status in gnupg2 source package in Groovy: In Progress Status in gnupg2 source package in Hirsute: Fix Released Bug description: [Impact] apt-key fails to fetch keys with "Address family not supported by protocol" [Description] We've had users report issues about apt-key being unable to fetch keys when IPv6 is disabled. As the mentioned kernel command line parameter disables IPV6 socket support, servers that allow/respond with IPv6 will cause connect_server() to fail with EAFNOSUPPORT. As this error is not being handled in some version of dirmngr, it'll simply fail the connection and could cause other processes to fail as well. In the test scenario below, it's easy to demonstrate this behaviour through apt-key. This has been reported upstream, and has been fixed with the following commit: - dirmngr: Handle EAFNOSUPPORT at connect_server. (109d16e8f644) The fix has been present upstream starting with GnuPG 2.22, so it's not currently available in any Ubuntu releases. [Test Case] 1. Spin up Focal VM 2. Disable IPv6: $ sudo vi /etc/default/grub (...) GRUB_CMDLINE_LINUX="ipv6.disable=1" $ sudo update-grub 3. Reboot the VM 4. Try to fetch a key: sudo apt-key adv --fetch-keys https://www.postgreSQL.org/media/keys/ACCC4CF8.asc You'll get the following error: gpg: WARNING: unable to fetch URI https://www.postgresql.org/media/keys/ACCC4CF8.asc: Address family not supported by protocol [Regression Potential] The patch introduces additional error handling when connecting to servers, to properly mark remote hosts as having valid IPv4 and/or IPv6 connectivity. We should look out for potential regressions when connecting to servers with exclusive IPv4 or IPv6 connectivity, to make sure the server is not getting marked as 'dead' due to missing one of the versions. This commit has also been tested in the corresponding Ubuntu series, and has been deemed safe for backporting to stable branches of upstream GnuPG. The overall regression potential for this change should be fairly low, and breakage should be easily spotted. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnupg2/+bug/1910432/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
[Sts-sponsors] [Bug 1910432] Re: dirmngr doesn't work with kernel parameter ipv6.disable=1
> I suggest you use 2.2.20-1ubuntu1.20.10.1 based on https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation#Update_the_packaging. Alternatively this problem will be resolved as soon as 2.2.20-1ubuntu2 in hirsute-proposed migrates. It occurred to me that 2.2.20-1ubuntu1.20.10.1 won't help either. We need the Hirsute release pocket to move beyond 2.2.20-1ubuntu1. -- You received this bug notification because you are a member of STS Sponsors, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1910432 Title: dirmngr doesn't work with kernel parameter ipv6.disable=1 Status in gnupg2 package in Ubuntu: Fix Released Status in gnupg2 source package in Bionic: In Progress Status in gnupg2 source package in Focal: Fix Committed Status in gnupg2 source package in Groovy: Incomplete Status in gnupg2 source package in Hirsute: Fix Released Bug description: [Impact] apt-key fails to fetch keys with "Address family not supported by protocol" [Description] We've had users report issues about apt-key being unable to fetch keys when IPv6 is disabled. As the mentioned kernel command line parameter disables IPV6 socket support, servers that allow/respond with IPv6 will cause connect_server() to fail with EAFNOSUPPORT. As this error is not being handled in some version of dirmngr, it'll simply fail the connection and could cause other processes to fail as well. In the test scenario below, it's easy to demonstrate this behaviour through apt-key. This has been reported upstream, and has been fixed with the following commit: - dirmngr: Handle EAFNOSUPPORT at connect_server. (109d16e8f644) The fix has been present upstream starting with GnuPG 2.22, so it's not currently available in any Ubuntu releases. [Test Case] 1. Spin up Focal VM 2. Disable IPv6: $ sudo vi /etc/default/grub (...) GRUB_CMDLINE_LINUX="ipv6.disable=1" $ sudo update-grub 3. Reboot the VM 4. Try to fetch a key: sudo apt-key adv --fetch-keys https://www.postgreSQL.org/media/keys/ACCC4CF8.asc You'll get the following error: gpg: WARNING: unable to fetch URI https://www.postgresql.org/media/keys/ACCC4CF8.asc: Address family not supported by protocol [Regression Potential] The patch introduces additional error handling when connecting to servers, to properly mark remote hosts as having valid IPv4 and/or IPv6 connectivity. We should look out for potential regressions when connecting to servers with exclusive IPv4 or IPv6 connectivity, to make sure the server is not getting marked as 'dead' due to missing one of the versions. This commit has also been tested in the corresponding Ubuntu series, and has been deemed safe for backporting to stable branches of upstream GnuPG. The overall regression potential for this change should be fairly low, and breakage should be easily spotted. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gnupg2/+bug/1910432/+subscriptions -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
