Public bug reported:
Creating a ping health monitor always results in ERROR state. This
happens because the haproxy service running in the amphora image does
not have enough permissions to run the file /var/lib/octavia/ping-
wrapper.sh.
In Octavia Logs:
2025-01-17 17:47:06.714 40655 DEBUG
octavia.amphorae.drivers.health.heartbeat_udp [-] member
8c4c9a83-96ff-4533-867b-6b451acdc255 status has changed from OFFLINE to ERROR,
updating db. _update_status
/usr/lib/python3/dist-packages/octavia/amphorae/drivers/health/heartbeat_udp.py:229
2025-01-17 17:47:06.719 40655 DEBUG
octavia.amphorae.drivers.health.heartbeat_udp [-] pool
00d6affb-3e1e-46df-9162-7b47c21da98f status has changed from ONLINE to ERROR,
updating db. _update_status
/usr/lib/python3/dist-packages/octavia/amphorae/drivers/health/heartbeat_udp.py:229
2025-01-17 17:47:06.724 40655 DEBUG
octavia.amphorae.drivers.health.heartbeat_udp [-] loadbalancer
8f720506-28fc-4589-a33b-de1cba13c93f status has changed from ONLINE to ERROR,
updating db. _update_status
/usr/lib/python3/dist-packages/octavia/amphorae/drivers/health/heartbeat_udp.py:229
In the amphora:
Jan 21 16:53:14 amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145 ip[19071]: [ALERT]
(19071) : Failed to exec process for external health check: Permission
denied. Aborting.
Jan 21 16:53:19 amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145 ip[19073]: [ALERT]
(19073) : Failed to exec process for external health check: Permission
denied. Aborting.
Looking at the permissions for /var/lib/octavia:
root@amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145:/var/lib# ls -lha octavia
total 36K
drwxr-x--- 4 octavia octavia 4.0K Jan 21 15:49 .
drwxr-xr-x 43 root root 4.0K Jan 20 19:42 ..
drwxr-xr-x 3 root root 4.0K Jan 20 19:42 certs
drwxr-xr-x 2 root root 4.0K Jan 21 16:52
d0a9cf91-1c0c-499d-a86c-3553099fa43d
srw-rw-rw- 1 root root 0 Jan 20 19:43
d0a9cf91-1c0c-499d-a86c-3553099fa43d.sock
-rw-r--r-- 1 root root 25 Jan 20 19:11 haproxy-default-user-group.conf
-rwxr-xr-x 1 root root 207 Jan 20 20:05 ping-wrapper.sh
-rw-r--r-- 1 root root 23 Jan 20 19:42 plugged_interfaces
-rw-r--r-- 1 root root 23 Jan 20 19:42 plugged_interfaces.sorted
-rw-r----- 1 root root 85 Jan 21 16:54 stats_counters.json
root@amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145:/var/lib#
The haproxy runs in a separate namespace as the nobody:nogroup config:
root@amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145:/var/lib# grep user
octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d/haproxy.cfg
user nobody
stats socket /var/lib/octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d.sock
mode 0666 level user
root@amphora-ae26c49f-53c8-4f74-ac29-74046f6a5145:/var/lib# grep group
octavia/haproxy-default-user-group.conf
group nogroup
Service config:
[Service]
# Force context as we start haproxy under "ip netns exec"
SELinuxContext=system_u:system_r:haproxy_t:s0
Environment="CONFIG=/var/lib/octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d/haproxy.cfg"
"USERCONFIG=/var/lib/octavia/haproxy-default-user-group.conf"
"PIDFILE=/var/lib/octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d/d0a9cf91-1c0c-499d-a86c-3553099fa43d.pid"
ExecStartPre=/usr/sbin/haproxy -f $CONFIG -f $USERCONFIG -c -q -L
PhDoY7WOeBqhwysZ0Dzu6CxNZgU
ExecReload=/usr/sbin/haproxy -c -f $CONFIG -f $USERCONFIG -L
PhDoY7WOeBqhwysZ0Dzu6CxNZgU
process:
nobody 1728 0.0 1.3 113372 13716 ? S Jan20 0:10
/usr/sbin/haproxy -sf 1651 -Ws -f
/var/lib/octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d/haproxy.cfg -f
/var/lib/octavia/haproxy-default-user-group.conf -p
/var/lib/octavia/d0a9cf91-1c0c-499d-a86c-3553099fa43d/d0a9cf91-1c0c-499d-a86c-3553099fa43d.pid
-L PhDoY7WOeBqhwysZ0Dzu6CxNZgU
Given that the haproxy user configuration is set in the octavia source
files, some possible solutions to run the ping-wrapper.sh are:
1) Change the /var/lib/octavia folder permissions to 755 instead of 750.
This will allow the haproxy service to run the ping-wrapper.sh file
which already has a 755 permission. This however exposes all the folder
contents. This can be done either during the octavia-diskimage-retrofit
build process or in the octavia package sources files.
2) Change the /var/lib/octavia ownership to nobody:nogroup. This allows
access to the folder but also gives permission to create files in the
folder. This can be done either during the octavia-diskimage-retrofit
build process or in the octavia package sources files.
3) Move ping-wrapper.sh out of the /var/lib/octavia folder, to a public
place. This will also require changing the octavia source haproxy
template files which have the location of the file hardcoded to
/var/lib/octavia.
** Affects: snap-octavia-diskimage-retrofit
Importance: Undecided
Status: New
** Affects: octavia (Ubuntu)
Importance: Undecided
Status: New
** Also affects: octavia (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2095432
Title:
Ping health-monitor does not work
To manage notifications about this bug go to:
https://bugs.launchpad.net/snap-octavia-diskimage-retrofit/+bug/2095432/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
