We restaged our environment
https://github.com/inverse-inc/packetfence/issues/7403 describes some
similar symptoms, so I have added some additional debug below from
cat /etc/network/interfaces
/usr/local/pf/sbin/pfperl-api get /api/v1/config/interfaces | jq
ip -br a
docker container ls
Following some of the triage steps in 7403, I additionally enabled debug on
pfperl-api, restarted the service and hit the Wizard Step 1 again, and
while I was capturing output for this email (below), the interfaces all of
a sudden appeared. The log output is also below from the process
restart, and I note there are some WARN level messages related to an ip
command exiting with a non-zero value for interfaces that do not appear in
my interface list.
packetfence.log:Jan 17 00:33:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(15) WARN: [mac:[undef]] Problem trying to run command: LANG=C
sudo ip -4 -o addr show veth17036df called from (eval). Child exited with
non-zero value 1 (pf::util::pf_run)
It seemed odd, so I reverted the DEBUG log to INFO, and rebooted the system
to see if I could reproduce the behavior. Sure enough, sitting on the
wizard step 1, as I had believed to have done previously, yielded no
interfaces in the Configurator and no WARN messages in the log.
It seemed that the service restart somehow resulted in the GUI being
updated. So we restart the service, and sure enough the interfaces
populated in the Configurator. No sign of the WARN messages either in the
log which seemed suppressed when set to INFO.
I re-ran the dump of the interfaces via API, and it gave the following
strange result
pf5:~# /usr/local/pf/sbin/pfperl-api get /api/v1/config/interfaces | jq
Device "veth7629818" does not exist.
parse error: Invalid numeric literal at line 1, column 19
Device "veth7629818" does not exist.
Device "veth6d6535d" does not exist.
Device "veth6d6535d" does not exist.
Unable to flush stdout: Broken pipe
cheers,
Ian
packetfence.log:Jan 17 00:31:15 pf5 pfperl-api-docker-wrapper[69046]:
Running with args --sig-proxy=true --rm --name=pfperl-api
--add-host=containers-gateway.internal:host-gateway -h pf5 -v
/var/lib/mysql:/var/lib/mysql -v /etc/sudoers:/etc/sudoers -v
/etc/sudoers.d/:/etc/sudoers.d/ -v
/usr/local/fingerbank/conf:/usr/local/fingerbank/conf -v
/usr/local/fingerbank/db:/usr/local/fingerbank/db -v
/usr/local/pf/var/run:/usr/local/pf/var/run -ePF_UID=996 -e PF_GID=996
-eFINGERBANK_UID=997 -e FINGERBANK_GID=997 -eIS_A_CLASSIC_PF_CONTAINER=yes
-v /etc/localtime:/etc/localtime:ro -v
/usr/local/pf/conf:/usr/local/pf/conf -v
/usr/local/pf/raddb/certs:/usr/local/pf/raddb/certs --privileged -v
/run/systemd/system:/run/systemd/system -v
/var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v
/usr/local/fingerbank/conf:/usr/local/fingerbank/conf -v
/etc/sysconfig/:/etc/sysconfig -v /etc/network:/etc/network -v
/etc/resolv.conf:/etc/resolv.conf --network=host -e HOST_OS
-v/usr/local/pf/var/conf/:/usr/local/pf/var/conf/
-v/usr/local/pf/html/captive-portal/profile-templates:/usr/local/pf/html/captive-portal/profile-templates
packetfence.log:Jan 17 00:31:15 pf5 pfperl-api-docker-wrapper[69053]:
Error: No such container: pfperl-api
packetfence.log:Jan 17 00:31:17 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] invalid IP: from cluster::__ANON__
(pf::util::valid_ip)
packetfence.log:Jan 17 00:31:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache get for namespace='configfiles',
key='/usr/local/pf/conf/roles.conf', cache='Redis:l1_cache', time='0ms':
MISS (not in cache) (CHI::Driver::_log_get_result)
packetfence.log:Jan 17 00:31:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache get for namespace='Default',
key='HASH(0x55f9c40983b0)', cache='RawMemory', time='0ms': MISS (not in
cache) (CHI::Driver::_log_get_result)
packetfence.log:Jan 17 00:31:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache set for namespace='Default',
key='{"encoding":null,"reconnect":"60","server":"containers-gateway.internal:6379"}',
size=1, expires='never', cache='RawMemory', time='0ms'
(CHI::Driver::_log_set_result)
packetfence.log:Jan 17 00:31:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache get for namespace='configfiles',
key='/usr/local/pf/conf/roles.conf', cache='Redis', time='1ms': HIT
(CHI::Driver::_log_get_result)
packetfence.log:Jan 17 00:31:19 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache set for namespace='configfiles',
key='/usr/local/pf/conf/roles.conf', size=1, expires='never',
cache='Redis:l1_cache', time='0ms' (CHI::Driver::_log_set_result)
packetfence.log:Jan 17 00:31:20 pf5 pfperl-api-docker-wrapper[69063]:
pfperl-api(8) DEBUG: [mac:[undef]] cache get for namespace='configfiles',
key='/usr/local/pf/conf/switches.conf', cache='Redis:l1_cache', time='0ms':
MISS (not in cache) (CHI::Driver::_log_get_result)
