** Description changed:
[Impact]
As @cell explains, the problem here is that the errors are going to
stdout, not to stderr. This means apt's Acquire::http::ProxyAutoDetect
is not able to figure out what the proxy URL is.
[Test Plan]
The idea is to create a VM with several interfaces, to cache them plus
to check configuration and do the discovery. Here you can see the test
for Focal, but it has been the same behaviour for Hirsute, Bionic and
Impish.
Note: In comment #22 there's a very straightforward test (Impish)
without the need for extra ifaces.
Steps to reproduce (thanks to @cpaelzer and @sergiodj):
- # 1
- # Creating VM and the IPv4 addresses
+ #0
+ # On the test Host install apt-cacher-ng
+ # you need to do so before creating the guest VM to propagate
+ # and configure correctly when spawned
+ $ sudo apt install apt-cacher-ng
+
+
+ # 1
+ # On the Host, create the IPv4 addresses and the VM. Stop the VM to
+ # attach the ifaces and start again the VM to log in.
$ for i in 1 2 3 4; do lxc network create testnet${i}; done
$ lxc network list
$ lxc launch ubuntu-daily:focal test-bug1505670-focal --vm
$ lxc stop test-bug1505670-focal
$ for i in 1 2 3 4; do lxc network attach testnet${i}
test-bug1505670-focal; done
$ lxc start test-bug1505670-focal
$ lxc shell test-bug1505670-focal
- $ sudo apt update
-
- #2
+
+ # Now at the VM, install prereq packages
+ $ sudo apt update
+ $ sudo apt install avahi-utils squid-deb-proxy-client
+
+ #3
# check if all are interfaces are configured
$ ip a
# (you might need to edit netplan or E/N/I to e.g. dhcp on all of
them) to have the following content (check the names of your interfaces
from the 'ip a' command above and change them if needed):
$ vim /etc/netplan/50-cloud-init.yaml
network:
version: 2
renderer: networked
ethernet:
enp5s0:
dhcp4: true
enp6s0:
dhcp4: true
enp7s0:
dhcp4: true
enp8s0:
dhcp4: true
enp09s0:
dhcp4: true
#apply the changes to the ifaces:
$ sudo netplan apply
- # 3
- #Caching
- $ sudo apt install -y apt-cacher-ng
# 4
# avahi probes all ifaces
$ avahi-browse -kprtf _apt_proxy._tcp
+;enp8s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp7s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp6s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp6s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp5s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;enp5s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
+;lo;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local
=;enp8s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;fd42:701b:b04:c7ab:216:3eff:feda:f1e7;3142;
=;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;10.8.60.155;3142;
=;enp7s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;fd42:3406:c5ba:899a:216:3eff:fecd:de74;3142;
=;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;10.232.244.137;3142;
=;enp6s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;fd42:bddd:a962:e084:216:3eff:fe11:a50c;3142;
=;enp6s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;10.115.171.242;3142;
=;enp5s0;IPv6;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;fd42:ce31:bcba:21af:216:3eff:fe79:c522;3142;
=;enp5s0;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;10.47.127.209;3142;
=;lo;IPv4;apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;_apt_proxy._tcp;local;test-bug1505670-focal.local;127.0.0.1;3142;
+;enp8s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
+;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
+;enp6s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
+;enp7s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
+;enp6s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
+;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local
=;enp8s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;fd42:701b:b04:c7ab::1;3142;
=;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;10.8.60.1;3142;
=;enp6s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;fd42:bddd:a962:e084::1;3142;
=;enp7s0;IPv6;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;fd42:3406:c5ba:899a::1;3142;
=;enp6s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;10.115.171.1;3142;
=;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032Camelot-TITAN;_apt_proxy._tcp;local;Camelot-TITAN.local;10.232.244.1;3142;
# 5
# After being announced stop the cacher on the host
$ sudo systemctl stop apt-cacher-ng.service
#6
# Check it is now dead from the guests POV with curl <addr>:3142, i.e.:
$ curl 10.8.60.155:3142
curl: (7) Failed to connect to 10.8.60.155 port 3142: Connection refused
$ curl 10.232.244.137:3142
curl: (7) Failed to connect to 10.232.244.137 port 3142: Connection refused
$ curl 10.115.171.242:3142
curl: (7) Failed to connect to 10.115.171.242 port 3142: Connection refused
$ curl 10.47.127.209:3142
curl: (7) Failed to connect to 10.47.127.209 port 3142: Connection refused
#7
# Ensure that it is still announced in avahi in the guest
$ avahi-browse -kprtf _apt_proxy._tcp | grep '^=' | cut -d";" -f 4,8-9 |
grep -v '::'
...
root@test-bug1505670-focal:~# avahi-browse -kprtf _apt_proxy._tcp | grep '^='
| cut -d";" -f 4,8-9 | grep -v '::'
[...]
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;fd42:701b:b04:c7ab:216:3eff:feda:f1e7;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;fd42:3406:c5ba:899a:216:3eff:fecd:de74;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;fd42:bddd:a962:e084:216:3eff:fe11:a50c;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;fd42:ce31:bcba:21af:216:3eff:fe79:c522;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;10.8.60.155;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;10.232.244.137;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;10.115.171.242;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;10.47.127.209;3142
apt-cacher-ng\032proxy\032on\032test-bug1505670-focal;127.0.0.1;3142
### Bad response:
#8
# Now run apt-avahi-discover which shows the python errors
# bleeding into the output even with stderr redirected
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover 2>/dev/null
root@test-bug1505670-focal:~#
/usr/share/squid-deb-proxy-client/apt-avahi-discover 2>/dev/null
error: uncaptured python exception, closing channel <AptAvahiClient>
('fd42:701b:b04:c7ab:216:3eff:feda:f1e7', 3142, 0, 0): 9223372036854775807
(<class 'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('fd42:3406:c5ba:899a:216:3eff:fecd:de74', 3142, 0, 0): 9223372036854775807
(<class 'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('fd42:bddd:a962:e084:216:3eff:fe11:a50c', 3142, 0, 0): 9223372036854775807
(<class 'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('fd42:ce31:bcba:21af:216:3eff:fe79:c522', 3142, 0, 0): 9223372036854775807
(<class 'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.8.60.155', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.232.244.137', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.115.171.242', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.47.127.209', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('127.0.0.1', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.8/asyncore.py|read|83]
[/usr/lib/python3.8/asyncore.py|handle_read_event|417]
[/usr/lib/python3.8/asyncore.py|handle_connect_event|425])
http://10.115.171.1:3142/
### Good response, after applying the fix from proposed package
# now no output bleeding through
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover 2>/dev/null
# actually since there is no proxy left there is juts nothing now in this
setup
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover
# But as soon as the cacher is back up it works:
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover
http://10.115.171.1:3142/
[Where problems could occur]
As the change itself redirects the errors to the proper logs using
the proper output flow (stderr), the risk of losing any kind of
information is avoided as well as the risk to produce any kind of
interference with the rest of the system. It uses standard python calls
to do it, so it doesn't have to rely on anything in particular.
[Other Info]
The change is cherry-picked from upstream for fixing this bug
precisely: https://github.com/mvo5/squid-deb-
proxy/commit/604ba3f98beff25a8fd51783d3ffc4db5e987dab (Thanks to @mvo)
[Original Report]
-------------------------------------------------------------------
Steps to reproduce (for later SRU work)
#0
# On the test Host install apt-cacher-ng
# you need to do so before creating the guest to propagate
# and configure correctly when spawned
$ sudo apt install apt-cacher-ng
#1
# create a VM guest or container with at least three IPv4 adresses
# In the example below I used 4 virtio net in KVM all with DHCP configured
#2
# install prereq packages
$ sudo apt install avahi-utils squid-deb-proxy-client
#3
# check if all are interfaces are configured and avahi probes them all
# (you might need to edit netplan or E/N/I to e.g. dhcp on all of them)
$ avahi-browse -kprtf _apt_proxy._tcp
+;enp9s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp9s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local
+;enp9s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp8s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local
+;enp8s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp7s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local
+;enp7s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp1s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;enp1s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local
+;enp1s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
+;lo;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local
=;enp8s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local;Keschdeichel.local;192.168.122.1;3142;
=;enp9s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local;Keschdeichel.local;192.168.122.1;3142;
=;enp7s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local;Keschdeichel.local;192.168.122.1;3142;
=;enp1s0;IPv4;apt-cacher-ng\032proxy\032on\032Keschdeichel;_apt_proxy._tcp;local;Keschdeichel.local;192.168.122.1;3142;
=;enp9s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;fe80::5054:ff:fe3a:53e7;8000;
=;enp8s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;fe80::5054:ff:fe73:b27d;8000;
=;enp7s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;fe80::5054:ff:fe3a:53e7;8000;
=;enp1s0;IPv6;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;fe80::5054:ff:fe4e:2923;8000;
=;enp9s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;192.168.122.125;8000;
=;enp8s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;192.168.122.202;8000;
=;enp7s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;192.168.122.186;8000;
=;enp1s0;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;192.168.122.42;8000;
=;lo;IPv4;Squid\032deb\032proxy\032on\032i-deb-proxy;_apt_proxy._tcp;local;i-deb-proxy.local;127.0.0.1;8000;
#5
# After being announced stop the cacher on the host
$ sudo systemctl stop apt-cacher-ng.service
#6
Check it is now dead from the guests POV
$ curl 192.168.122.1:3142
curl: (7) Failed to connect to 192.168.122.1 port 3142: Connection refused
#7
# Ensure that it is still announced in avahi in the guest
$ avahi-browse -kprtf _apt_proxy._tcp | grep '^=' | cut -d";" -f 4,8-9 | grep
-v '::'
...
apt-cacher-ng\032proxy\032on\032Keschdeichel;192.168.122.1;3142
#8
# Now run apt-avahi-discover which shows the python errors
# bleeding into the output even with stderr redirected
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover 2>/dev/null
error: uncaptured python exception, closing channel <AptAvahiClient>
('192.168.122.1', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.9/asyncore.py|read|83]
[/usr/lib/python3.9/asyncore.py|handle_read_event|417]
[/usr/lib/python3.9/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('192.168.122.1', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.9/asyncore.py|read|83]
[/usr/lib/python3.9/asyncore.py|handle_read_event|417]
[/usr/lib/python3.9/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('192.168.122.1', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.9/asyncore.py|read|83]
[/usr/lib/python3.9/asyncore.py|handle_read_event|417]
[/usr/lib/python3.9/asyncore.py|handle_connect_event|425])
error: uncaptured python exception, closing channel <AptAvahiClient>
('192.168.122.1', 3142): 9223372036854775807 (<class
'ConnectionRefusedError'>:[Errno 111] Connection refused
[/usr/lib/python3.9/asyncore.py|read|83]
[/usr/lib/python3.9/asyncore.py|handle_read_event|417]
[/usr/lib/python3.9/asyncore.py|handle_connect_event|425])
http://192.168.122.42:8000/
P.S. I'm (paelzer) not entirely sure we need "multiple" IPs an announced
but down entry might be enough, but we want to stick close to what was
reported.
---
I get the following error when running the discovery script on the
command line.
$ /usr/share/squid-deb-proxy-client/apt-avahi-discover
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.1.2.3', 3142): 2147483647 (<class 'socket.error'>:[Errno 111] Connection
refused [/usr/lib/python2.7/asyncore.py|read|83]
[/usr/lib/python2.7/asyncore.py|handle_read_event|446]
[/usr/lib/python2.7/asyncore.py|handle_connect_event|454])
error: uncaptured python exception, closing channel <AptAvahiClient>
('10.0.3.1', 3142): 2147483647 (<class 'socket.error'>:[Errno 111] Connection
refused [/usr/lib/python2.7/asyncore.py|read|83]
[/usr/lib/python2.7/asyncore.py|handle_read_event|446]
[/usr/lib/python2.7/asyncore.py|handle_connect_event|454])
error: uncaptured python exception, closing channel <AptAvahiClient>
('172.24.74.129', 3142): 2147483647 (<class 'socket.error'>:[Errno 111]
Connection refused [/usr/lib/python2.7/asyncore.py|read|83]
[/usr/lib/python2.7/asyncore.py|handle_read_event|446]
[/usr/lib/python2.7/asyncore.py|handle_connect_event|454])
http://172.24.74.145:3142/
The last line still returns the proper proxy URI so as far as I can tell
things are still working. The IP 10.1.2.3 is for an n2n VPN. This is
on trusty with version 0.8.6ubuntu1.
To trigger the bug the environment setup needs to be in a specific way.
It seems for the problem to occur it need more than one host/IP discovered
via avahi. This can be probed via $ avahi-browse -kprtf _apt_proxy._tcp
and e.g. the common LXD setup of IPv4 + ipv6 is NOT enough to trigger it.
TODO: a sample output of the above command in an affected environment
could be helpful.
TODO: if possible outlining how the environment can be configured to
have this multi host/IP reply in avahi would be helpful as well.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1505670
Title:
python exceptions bleeding into stdout
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/squid-deb-proxy/+bug/1505670/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
