** Description changed:
+ [ Impact ]
+
+ If for any reason the user's system has any non-utf8 character as part
+ of the `apt list --installed` command, the Pro Client will fail to parse
+ the return output and break.
+
+ Independent of why this happens, the Pro Client should not be relying on
+ CLI output when there are Python bindings available to perform the same
+ task - and python3-apt provides ways to list installed packages.
+
+ The fix to this bug is to rely on the programmatic integration with APT
+ instead of calling the CLI.
+
+
+ [ Test Plan ]
+
+ Listing installed packages is part of the regular operation of the Pro
+ Client in many scenarios covered by the integration / acceptance tests.
+
+ For this particular issue:
+ - Launch a container for the release to be tested
+ - make `apt list --installed` return a non-utf8 character (can be done by
mocking the `apt` binary in the system)
+ - see it fail
+ - install the package from -proposed
+ - see it does not fail anymore
+
+
+ [ Where problems could occur ]
+
+ Switching from the CLI call to python3-apt brings the risk of wrong
+ implementation of the integration, which would lead to a different
+ package list to be returned by python3-apt. The regression potential is
+ mitigated by extensive testing - unit, integration and acceptance tests
+ rely on this functionality.
+
+
+ [ Original description ]
+
["2024-04-10T08:28:17.952", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:28:17.953", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:28:18.575", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
- last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
+ last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 1475, in
action_status\n status, ret = actions.status(\n File
\"/usr/lib/python3/dist-package
s/uaclient/actions.py\", line 173, in status\n status =
ua_status.status(cfg=cfg, show_all=show_all)\n File
\"/usr/lib/python3/dist-packages/uaclient/
status.py\", line 418, in status\n response = _unattached_status(cfg)\n
File \"/usr/lib/python3/dist-packages/uaclient/status.py\", line 283, in _unat
tached_status\n resources = get_available_resources(cfg)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 680, in
get_available_re
sources\n resources = client.available_resources()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 114, in
available_resources\n
- activity_info = self._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"deskt
+ activity_info = self._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"deskt
op\": system.is_desktop(),\n File
\"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in
is_desktop\n for package in apt.get_installed_pa
ckages():\n File \"/usr/lib/python3/dist-packages/uaclient/apt.py\", line
727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"
--installed\"])\n File
\"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-pa
ckages/uaclient/system.py\", line 649, in _subp\n out_result =
out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't
decode b
yte 0xe1 in position 109: invalid continuation byte"}]
["2024-04-10T08:30:25.404", "DEBUG", "ubuntupro.config", "parse_config", 625,
"Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:30:25.408", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:30:25.409", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:30:25.421", "DEBUG", "ubuntupro.http", "configure_web_proxy",
146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.http", "configure_web_proxy",
154, "Setting global proxy dict", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed
with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin
lock starting for pro attach", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmparyyoirn", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmpn1yxl91o", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpwqyq85v1", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/machine-id", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:30:25.425", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file:
/run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:30:25.971", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
- last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
+ last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n
return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n
File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in
action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File
\"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n
File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in
add_contract_machine\n activity_info = se
lf._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line
397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in
get_installed_packages\n out, _ = system.subp([\"apt\", \"list\",
\"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out
else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
["2024-04-10T08:38:29.763", "DEBUG", "ubuntupro.config", "parse_config", 625,
"Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:38:29.767", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:38:29.768", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy",
146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy",
154, "Setting global proxy dict", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed
with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin
lock starting for pro attach", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmplx6ehdag", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmp6ml_ltfj", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556,
"Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpvxf_b1v0", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.files.files", "read", 60,
"Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/machine-id", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /etc/os-release", {}]
["2024-04-10T08:38:29.784", "DEBUG", "ubuntupro.system", "load_file", 517,
"Reading file: /proc/version_signature", {}]
["2024-04-10T08:38:30.366", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file:
/run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:38:30.367", "DEBUG", "ubuntupro.system",
"ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:38:30.367", "ERROR", "ubuntupro.cli", "wrapper", 1768,
"Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent
call
- last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
+ last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\",
line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n
return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File
\"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n
return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n
File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in
action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File
\"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n
File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in
add_contract_machine\n activity_info = se
lf._get_activity_info()\n File
\"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in
_get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line
397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in
get_installed_packages\n out, _ = system.subp([\"apt\", \"list\",
\"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n
out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out
else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubuntu-advantage-tools 31.2~20.04
ProcVersionSignature: Ubuntu 5.15.0-101.111~20.04.1-generic 5.15.143
Uname: Linux 5.15.0-101-generic x86_64
NonfreeKernelModules: falcon_lsm_serviceable falcon_nf_netcontain falcon_kal
falcon_lsm_pinned_16407 falcon_lsm_pinned_16206
ApportVersion: 2.20.11-0ubuntu27.27
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed Apr 10 08:28:11 2024
PackageArchitecture: all
ProcEnviron:
- TERM=xterm
- PATH=(custom, no user)
- LANG=es_ES
- SHELL=/bin/bash
+ TERM=xterm
+ PATH=(custom, no user)
+ LANG=es_ES
+ SHELL=/bin/bash
SourcePackage: ubuntu-advantage-tools
UpgradeStatus: No upgrade log present (probably fresh install)
apparmor_logs.txt:
-
+
cloud-id.txt-error: Invalid command specified 'cloud-id'.
livepatch-status.txt-error: Invalid command specified
'/snap/bin/canonical-livepatch status'.
ua-status.json-error: Failed running command 'pro status --format json'
[exit(1)]. Message:
uaclient.conf:
- contract_url: https://contracts.canonical.com
- log_level: debug
+ contract_url: https://contracts.canonical.com
+ log_level: debug
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2060769
Title:
pro-client execs `apt list --installed` instead of using python3-apt -
leading to utf-8 errors
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2060769/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
