You can certainly see messages in the logs if you setup the ha logs (see: https://kea.readthedocs.io/en/kea-2.2.0/arm/logging.html and in particluar this chart https://kea.readthedocs.io/en/kea-2.2.0/arm/logging.html#id3 which shows all of the available loggers - you will want to for sure setup "kea-dhcp4.ha-hooks"). You can also access the status via the API (see https://kea.readthedocs.io/en/kea-2.2.0/arm/ctrl-channel.html#management-api) with ha-heartbeat (https://kea.readthedocs.io/en/kea-2.2.0/api.html#ha-heartbeat).
On Mon, Mar 6, 2023 at 7:51 AM Kraishak Mahtha <kraishak....@gmail.com> wrote: > > Yes Thanks Darren, found the issue after changing the file name it is working > fine and do you have any idea like how can we get the failover status > In ISC DHCP failover we can see them in the lease file (communication > interrupted, recover, partner-down....etc) with the failover name being > specified but I don't see any such info in the kea-leas file > > Thanks > Kraishak > > On Mon, Mar 6, 2023 at 4:23 PM Darren Ankney <darren.ank...@gmail.com> wrote: >> >> Looks like your socket locations don't match between the kea config >> and the ctrl agent configuration. Those are filenames, so they are >> going to need to match exactly. Kea is in the driver's seat there as >> it actually creates the file. the ctrl agent "talks" to kea through >> such file. >> >> example, your primary kea config says: /tmp/kea4-ctrl-socket1 and >> your ctrl agent config on the primary says: /tmp/control_socket >> >> On Mon, Mar 6, 2023 at 4:38 AM Kraishak Mahtha <kraishak....@gmail.com> >> wrote: >> > >> > Hi All, >> > >> > I am trying to configure the kea-HA setup in my local environment but I am >> > facing an issue the heartbeat is getting failed and the logs are as >> > follows: >> > Logs on Failover >> > ============= >> > 2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] >> > HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to >> > http://192.168.0.168:8000/ >> > 2023-03-06 09:04:43.761 DEBUG [kea-dhcp4.http/28550.140575742654656] >> > HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent >> > to http://192.168.0.168:8000/: >> > POST / HTTP/1.1 >> > Host: 192.168.0.168 >> > Content-Length: 53 >> > Content-Type: application/json >> > >> > { "command": "ha-heartbeat", "service": [ "dhcp4" ] } >> > 2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] >> > HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from >> > http://192.168.0.168:8000/ >> > 2023-03-06 09:04:43.762 DEBUG [kea-dhcp4.http/28550.140575742654656] >> > HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about >> > well-formed response received from http://192.168.0.168:8000/: >> > HTTP/1.1 200 OK >> > Content-Length: 133 >> > Content-Type: application/json >> > Date: Mon, 06 Mar 2023 09:04:43 GMT >> > >> > [ { "result": 1, "text": "unable to forward command to the dhcp4 service: >> > Connection refused. The server is likely to be offline" } ] >> > 2023-03-06 09:04:43.762 WARN [kea-dhcp4.ha-hooks/28550.140575742654656] >> > HA_HEARTBEAT_FAILED heartbeat to kea-dhcp-168 (http://192.168.0.168:8000/) >> > failed: unable to forward command to the dhcp4 service: Connection >> > refused. The server is likely to be offline, error code 1 >> > 2023-03-06 09:04:43.762 WARN [kea-dhcp4.ha-hooks/28550.140575742654656] >> > HA_COMMUNICATION_INTERRUPTED communication with kea-dhcp-168 is interrupted >> > >> > >> > Logs on Primary >> > ============== >> > 2023-03-06 09:01:50.143 DEBUG [kea-dhcp4.dhcpsrv/13620.140081380149440] >> > DHCPSRV_TIMERMGR_START_TIMER starting timer: reclaim-expired-leases >> > 2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] >> > HTTP_CLIENT_REQUEST_SEND sending HTTP request POST / HTTP/1.1 to >> > http://192.168.0.169:8000/ >> > 2023-03-06 09:01:52.145 DEBUG [kea-dhcp4.http/13620.140081380149440] >> > HTTP_CLIENT_REQUEST_SEND_DETAILS detailed information about request sent >> > to http://192.168.0.169:8000/: >> > POST / HTTP/1.1 >> > Host: 192.168.0.169 >> > Content-Length: 53 >> > Content-Type: application/json >> > >> > { "command": "ha-heartbeat", "service": [ "dhcp4" ] } >> > 2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] >> > HTTP_SERVER_RESPONSE_RECEIVED received HTTP response from >> > http://192.168.0.169:8000/ >> > 2023-03-06 09:01:52.146 DEBUG [kea-dhcp4.http/13620.140081380149440] >> > HTTP_SERVER_RESPONSE_RECEIVED_DETAILS detailed information about >> > well-formed response received from http://192.168.0.169:8000/: >> > HTTP/1.1 200 OK >> > Content-Length: 133 >> > Content-Type: application/json >> > Date: Mon, 06 Mar 2023 09:01:52 GMT >> > >> > [ { "result": 1, "text": "unable to forward command to the dhcp4 service: >> > Connection refused. The server is likely to be offline" } ] >> > 2023-03-06 09:01:52.146 WARN [kea-dhcp4.ha-hooks/13620.140081380149440] >> > HA_HEARTBEAT_FAILED heartbeat to appliance-169 >> > (http://192.168.0.169:8000/) failed: unable to forward command to the >> > dhcp4 service: Connection refused. The server is likely to be offline, >> > error code 1 >> > 2023-03-06 09:01:53.558 INFO >> > [kea-ctrl-agent.commands/13581.140159667833024] COMMAND_RECEIVED Received >> > command 'ha-heartbeat' >> > 2023-03-06 09:01:53.558 INFO >> > [kea-ctrl-agent.ctrl-agent/13581.140159667833024] >> > CTRL_AGENT_COMMAND_RECEIVED command ha-heartbeat received from remote >> > address 192.168.0.169 >> > >> > >> > ========= >> > Primary config >> > ============== >> > ##DHCPv4 192.168.0.168 >> > { >> > "Dhcp4": { >> > "valid-lifetime": 300, >> > "renew-timer": 150, >> > "rebind-timer": 200, >> > "server-tag": "192.168.0.168", >> > "control-socket": { >> > "socket-name": "/tmp/kea4-ctrl-socket1", >> > "socket-type": "unix" >> > }, >> > >> > "lease-database": { >> > "type": "memfile", >> > "persist": true, >> > "name": "/var/lib/kea/dhcp4.leases" >> > }, >> > >> > "interfaces-config": { >> > "interfaces": ["eth0"] >> > }, >> > >> > "hooks-libraries": [ >> > { >> > "library": >> > "/usr/local/lib/kea/hooks/libdhcp_lease_cmds.so", >> > "parameters": { } >> > }, >> > { >> > "library": "/usr/local/lib/kea/hooks/libdhcp_bootp.so", >> > "parameters": { } >> > }, >> > { >> > "library": >> > "/usr/local/lib/kea/hooks/libdhcp_ha.so", >> > "parameters": { >> > "high-availability": [{ >> > "this-server-name": >> > "kea-dhcp-168", >> > "mode": "hot-standby", >> > "heartbeat-delay": 10000, >> > "max-response-delay": >> > 60000, >> > "max-ack-delay": 5000, >> > "max-unacked-clients": 5, >> > "peers": [{ >> > "name": >> > "kea-dhcp-168", >> > "url": >> > "http://192.168.0.168:8000/";, >> > "role": "primary", >> > "auto-failover": >> > true >> > }, { >> > "name": >> > "appliance-169", >> > "url": >> > "http://192.168.0.169:8000/";, >> > "role": "standby", >> > "auto-failover": >> > true >> > }] >> > }] >> > } >> > }], >> > "subnet4": [{ >> > "subnet": "4.0.0.0/24", >> > "option-data": [ >> > { >> > "data": "2.2.2.2, 8.8.4.4", >> > "name": "domain-name-servers" >> > }, >> > { >> > "data": "4.0.0.1", >> > "name": "routers" >> > } >> > ], >> > "pools": [ >> > { >> > "pool": "4.0.0.25 - 4.0.0.250" >> > } >> > ] >> > >> > }], >> > "loggers": [ >> > { >> > "name": "kea-dhcp4", >> > "debuglevel": 99, >> > "output_options": [ >> > { >> > "output": "/usr/local/var/log/kea-dhcp4.log" >> > } >> > ], >> > "severity": "DEBUG" >> > } >> > ] >> > } >> > } >> > kea-ctrl-agent.conf on Primary >> > ========================= >> > >> > { >> > "Control-agent": { >> > "http-host": "192.168.0.168", >> > "http-port": 8000, >> > "control-sockets": { >> > "dhcp4": { >> > "socket-type": "unix", >> > "socket-name": "/tmp/control_socket" >> > } >> > }, >> > "cert-required": false >> > } >> > } >> > Netstat output >> > ============= >> > [root@kea-dhcp-168 ~]# netstat -tulpna | grep 8000 >> > tcp 0 0 192.168.0.168:8000 0.0.0.0:* LISTEN >> > 13581/kea-ctrl-agen >> > tcp 0 0 192.168.0.168:57308 192.168.0.169:8000 >> > ESTABLISHED 13620/kea-dhcp4 >> > tcp 0 0 192.168.0.168:8000 192.168.0.169:58432 >> > ESTABLISHED 13581/kea-ctrl-agen >> > [root@kea-dhcp-168 ~]# >> > >> > [root@kea-dhcp-168 ~]# netstat -tulpna | grep 67 >> > udp 0 0 192.168.0.168:67 0.0.0.0:* >> > 13620/kea-dhcp4 >> > [root@kea-dhcp-168 ~]# >> > >> > >> > Failover DHCPv4 config >> > =================== >> > #DHCPv4 192.168.0.169 >> > { >> > "Dhcp4": { >> > "valid-lifetime": 300, >> > "renew-timer": 150, >> > "rebind-timer": 200, >> > "server-tag": "192.168.0.169", >> > "control-socket": { >> > "socket-name": "/opt/testUAT/keaFolder/kea4-ctrl-socket", >> > "socket-type": "unix" >> > }, >> > >> > "lease-database": { >> > "type": "memfile", >> > "persist": true, >> > "name": "/opt/testUAT/var/lib/kea/dhcp4.leases" >> > }, >> > >> > "interfaces-config": { >> > "interfaces": ["eth0"] >> > }, >> > >> > "hooks-libraries": [ >> > { >> > "library": >> > "/opt/testUAT/lib/kea/hooks/libdhcp_lease_cmds.so", >> > "parameters": { } >> > }, >> > { >> > "library": "/opt/testUAT/lib/kea/hooks/libdhcp_bootp.so", >> > "parameters": { } >> > }, >> > { >> > "library": >> > "/opt/testUAT/lib/kea/hooks/libdhcp_ha.so", >> > "parameters": { >> > "high-availability": [{ >> > "this-server-name": >> > "appliance-169", >> > "mode": "hot-standby", >> > "heartbeat-delay": 10000, >> > "max-response-delay": >> > 60000, >> > "max-ack-delay": 5000, >> > "max-unacked-clients": 5, >> > "peers": [{ >> > "name": >> > "kea-dhcp-168", >> > "url": >> > "http://192.168.0.168:8000/";, >> > "role": "primary", >> > "auto-failover": >> > true >> > }, { >> > "name": >> > "appliance-169", >> > "url": >> > "http://192.168.0.169:8000/";, >> > "role": "standby", >> > "auto-failover": >> > true >> > }] >> > }] >> > } >> > }], >> > "subnet4": [{ >> > "subnet": "4.0.0.0/24", >> > "option-data": [ >> > { >> > "data": "2.2.2.2, 8.8.4.4", >> > "name": "domain-name-servers" >> > }, >> > { >> > "data": "4.0.0.1", >> > "name": "routers" >> > } >> > ], >> > "pools": [ >> > { >> > "pool": "4.0.0.25 - 4.0.0.250" >> > } >> > ] >> > >> > }], >> > "loggers": [ >> > { >> > "name": "kea-dhcp4", >> > "debuglevel": 99, >> > "output_options": [ >> > { >> > "output": "/opt/testUAT/var/log/kea-dhcp4.log" >> > } >> > ], >> > "severity": "DEBUG" >> > } >> > ] >> > } >> > } >> > >> > kea-ctrl-agent.conf on Failover >> > ======================= >> > { >> > "Control-agent": { >> > "http-host": "192.168.0.169", >> > "http-port": 8000, >> > "control-sockets": { >> > "dhcp4": { >> > "socket-type": "unix", >> > "socket-name": "/tmp/control_socket" >> > } >> > }, >> > "cert-required": false >> > } >> > } >> > Netstat output >> > ============= >> > >> > [root@appliance-169 ~]# netstat -tulpna | grep 8000 >> > tcp 0 0 192.168.0.169:8000 0.0.0.0:* LISTEN >> > 28520/kea-ctrl-agen >> > tcp 0 0 192.168.0.169:8000 192.168.0.168:57308 >> > ESTABLISHED 28520/kea-ctrl-agen >> > tcp 0 0 192.168.0.169:58432 192.168.0.168:8000 >> > ESTABLISHED 28550/kea-dhcp4 >> > [root@appliance-169 ~]# >> > >> > [root@kea-dhcp-168 ~]# netstat -tulpna | grep 67 >> > udp 0 0 192.168.0.169:67 0.0.0.0:* >> > 28550/kea-dhcp4 >> > [root@kea-dhcp-168 ~]# >> > >> > I am not sure why it is unable to forward commands to dhcp4 service, Can >> > someone guide or help me if you face the same issue or are aware of the >> > issue? >> > >> > Thanks in advance >> > Kraishak >> > >> > -- >> > ISC funds the development of this software with paid support >> > subscriptions. Contact us at https://www.isc.org/contact/ for more >> > information. >> > >> > To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. >> > >> > Kea-users mailing list >> > Kea-users@lists.isc.org >> > https://lists.isc.org/mailman/listinfo/kea-users >> -- >> ISC funds the development of this software with paid support subscriptions. >> Contact us at https://www.isc.org/contact/ for more information. >> >> To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. >> >> Kea-users mailing list >> Kea-users@lists.isc.org >> https://lists.isc.org/mailman/listinfo/kea-users -- ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. Kea-users mailing list Kea-users@lists.isc.org https://lists.isc.org/mailman/listinfo/kea-users
