Hi Darren, Thank you for your message.
In fact, we (simply ^^) would like to implement a redundant DHCP cluster with the following characteristics : * A frontend DHCP cluster with 2 KEA instances * A backend GALERA cluster with 4 MariaDB instances * Leases sync / update disabled between the 2 KEA instances for relying only on the database data sync. I can confirm that the GALERA cluster works well for other applications without persistent database connections. In this case, we use HAPROXY with a VIP and the requests to MariaDB instances are load balanced between the available database nodes. By configuring our KEA instances in such a way, we can confirm that the DHCP service works, but KEA logs show a lot of error messages (which seems to be due to the fact that the database connections don't persist with a load balancer). According your comments, we'll try other setups such as : * 2 KEA + 2 MariaDB with HA hook * 4 KEA + 4 MariaDB without HA hook In the meantime, we would be delighted if anyone can share a successful implementation of such a cluster. Kind regards. Le jeu. 9 janv. 2025 à 17:08, Darren Ankney <darren.ank...@gmail.com> a écrit : > Hi Sébastien, > > I do not have any familiarity with HAPROXY. However, it seems there > might be a problem between HAPROXY <-> Mariadb from your description > of the behavior. This may not have anything to do with Kea. Are you > able to test with some other software package to see if the same > problem exists? As far as what can be done in Kea to mitigate this > issue, you could setup retries which you've already done. > > As far as the issue > (https://gitlab.isc.org/isc-projects/kea/-/issues/1746) you mentioned > is concerned. This seems to be describing adding support to Kea to > connect to multiple database servers. It does not seem to pertain to > HAPROXY. If Kea supported this, then you wouldn't need HAPROXY as Kea > would be able to be configured with all of your database IP addresses > directly and would somehow choose one to connect to. I think the > comment you mention specifically is a workaround where you run one Kea > server per database node. If the database node fails, that Kea server > would also fail. That is a workable strategy if you were interested > in it. Of course, the HA hook would not be usable if you have more > than two servers. The result of not using the HA hook would be that > all servers would try to answer each client. You would also want to > run the random lease allocator to avoid collisions. > > Thank you, > Darren Ankney > > > On Thu, Jan 9, 2025 at 4:51 AM Sébastien Nauer <sna...@hotcity.lu> wrote: > > > > Hello Daren, > > > > Thank you for your comments. > > > > We use MariaDB v11.4.3 which seems to be fully compatible with KEA. > > > > In MariaDB logs, we only see such entries : > > > 2025-01-09 9:06:50 8232949 [Warning] Aborted connection 8232949 to > db: 'kea' user: 'kea_user' host: '1.2.3.4' real ip: '5.6.7.8' (Got an error > reading communication packets) > > > > As we don't see such errors by connecting directly a KEA node to a > MariaDB node, it seems that the issue only occurs when using HAPROXY in > front of the GALERA Cluster. > > > > Also, according the following link, it seems that we should have to > configure 4 KEA instances (as we use 4 MariaDB instances which each KEA > instance being connected to one MariaDB node) ??? > https://gitlab.isc.org/isc-projects/kea/-/issues/1746 > > > > For troubleshooting this issue, we currently configured HAPROXY with > only 1 MariaDB node and source IP stickiness, the aims being : > > * to persist the KEA database connection with the same MariaDB node > > * to be able to connect KEA to another MariaDB node if the current node > fails > > > > We can confirm that the error occurs in such a simple situation (with > only one database node) and we don't understand why KEA generates > DATABASE_MYSQL_FATAL_ERROR messages. > > > > Could you confirm if we can use HAPROXY in such a manner ? > > Are there specific settings to activate in KEA and/or HAPROXY ? > > Or finaly, is it better not to use HAPROXY in such a scenario ? > > > > Any advice for configuring this kind of KEA cluster will be appreciated. > > Kind regards. > > > > Le mer. 8 janv. 2025 à 22:59, Darren Ankney <darren.ank...@gmail.com> a > écrit : > >> > >> Hi Sébastien, > >> > >> The frequent disconnections could be anything. Is there anything in > the MySQL logs? Also, is this MySQL or MariaDB? MySQL 8 has some > performance issues that affect Kea pretty badly (See here: > https://kea.readthedocs.io/en/kea-2.6.1/arm/admin.html#mysql-5-7-vs-mysql-8-vs-mariadb-10-and-11 > ) > >> > >> If you haven't already found them, there are a couple of articles about > MySQL clusters and Kea: > >> https://kb.isc.org/docs/getting-started-with-galera-or-percona-for-kea > >> https://kb.isc.org/docs/experimental-mysql-clusters-for-kea (this one > might not be relevant) > >> > >> Thank you, > >> Darren Ankney > >> > >> On Wed, Jan 8, 2025 at 4:51 AM Sébastien Nauer via Kea-users < > kea-users@lists.isc.org> wrote: > >>> > >>> Hello, > >>> > >>> We are experiencing issues with configuring a KEA cluster in > high-availability mode with a GALERA Cluster database backend. > >>> > >>> As we'd like to use the database cluster for managing leases > synchronization : > >>> > >>> we configured KEA High Availability mode like this for disabling lease > sync & update : > >>> > >>> "high-availability": [ > >>> { > >>> "this-server-name": "kea01", > >>> "mode": "load-balancing", > >>> "heartbeat-delay": 10000, > >>> "max-response-delay": 10000, > >>> "max-ack-delay": 5000, > >>> "max-unacked-clients": 5, > >>> "send-lease-updates": false, > >>> "sync-leases": false, > >>> "sync-page-limit": 10000, > >>> "sync-timeout": 30000, > >>> "multi-threading": { > >>> "enable-multi-threading": true, > >>> "http-dedicated-listener": true, > >>> "http-listener-threads": 0, > >>> "http-client-threads": 0 > >>> }, > >>> "peers": [ > >>> { > >>> "name": "kea0&", > >>> "url": "http://1.2.3.4:8080/";, > >>> "role": "primary", > >>> "auto-failover": true > >>> }, > >>> { > >>> "name": "kea04", > >>> "url": "http://5.6.7.8:8080/";, > >>> "role": "secondary", > >>> "auto-failover": true > >>> } > >>> ] > >>> } > >>> ] > >>> > >>> we configured the lease database like this for connecting to the > database cluster : > >>> > >>> "lease-database": { > >>> "type": "mysql", > >>> "name": "kea", > >>> "user": "KEA_USER", > >>> "password": "KEA_PASSWORD", > >>> "host": "1.1.1.1", > >>> "max-reconnect-tries" : 10, > >>> "reconnect-wait-time" : 30000 > >>> }, > >>> > >>> we tried to connect to the database backend : > >>> > >>> through a VIP with load balancing > >>> through HAPROXY with / without load balancing > >>> > >>> When we start KEA service and the servers start, KEA logs show a lot > of DATABASE_MYSQL_FATAL_ERROR messages like this : > >>> > >>> 2025-01-07 16:18:42.483 ERROR [kea-dhcp4.database/4349.0x34cf8b308500] > DATABASE_MYSQL_FATAL_ERROR Unrecoverable MySQL error occurred: unable to > execute for <SELECT h.host_id, h.dhcp_identifier, h.dhcp_identifier_type, > h.dhcp4_subnet_id, h.dhcp6_subnet_id, h.ipv4_address, h.hostname, > h.dhcp4_client_classes, h.dhcp6_client_classes, h.user_context, > h.dhcp4_next_server, h.dhcp4_server_hostname, h.dhcp4_boot_file_name, > h.auth_key, o.option_id, o.code, o.value, o.formatted_value, o.space, > o.persistent, o.cancelled, o.user_context FROM hosts AS h LEFT JOIN > dhcp4_options AS o ON h.host_id = o.host_id WHERE h.dhcp4_subnet_id = ? AND > h.dhcp_identifier_type = ? AND h.dhcp_identifier = ? ORDER BY h.host_id, > o.option_id>, reason: Server has gone away (error code: 2006). > >>> 2025-01-07 16:18:43.533 INFO [kea-dhcp4.dhcp4/4349.0x34cf8aa12000] > DHCP4_DB_RECONNECT_LOST_CONNECTION database connection lost. > >>> 2025-01-07 16:18:43.533 INFO [kea-dhcp4.hosts/4349.0x34cf8aa12000] > DHCPSRV_MYSQL_HOST_DB opening MySQL hosts database: host=10.194.50.12 > max-reconnect-tries=10 name=prod_kea password=***** > reconnect-wait-time=30000 type=mysql universe=4 user=keaCluster > >>> 2025-01-07 16:18:43.586 INFO [kea-dhcp4.dhcp4/4349.0x34cf8aa12000] > DHCP4_DB_RECONNECT_SUCCEEDED database connection recovered. > >>> > >>> As we noticed that the database connection is frequently lost for no > apparent reason we suspect that there is an issue with using persistent > database connections through a VIP with load balancing (maybe related to > multithreading also) > >>> > >>> Has anyone successfully configured such a setup ? How ? > >>> What do you suggest for avoiding this error message and configuring a > high availability DHCP cluster with a high availability database cluster ? > >>> > >>> Any feedback from the community will be appreciated ! > >>> Kind regards > >>> > >>> > >>> > >>> -- > >>> > >>> > >>> > >>> Sébastien NAUER| Technical Manager > >>> 11, Avenue Guillaume | L-1651 Luxembourg > >>> Phone: (+352) 2663 2660 | Fax: (+352) 2663 2665 > >>> Facebook | Twitter > >>> -- > >>> ISC funds the development of this software with paid support > subscriptions. Contact us at https://www.isc.org/contact/ for more > information. > >>> > >>> To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. > >>> > >>> Kea-users mailing list > >>> Kea-users@lists.isc.org > >>> https://lists.isc.org/mailman/listinfo/kea-users > > > > > > > > -- > > > > > > > > Sébastien NAUER| Technical Manager > > 11, Avenue Guillaume | L-1651 Luxembourg > > Phone: (+352) 2663 2660 | Fax: (+352) 2663 2665 > > Facebook | Twitter > -- [image: Inline-Bild 1] *Sébastien NAUER <sna...@hotcity.lu>*| Technical Manager 11, Avenue Guillaume | L-1651 Luxembourg <http://goo.gl/maps/GqQcf> Phone: (+352) 2663 2660 | Fax: (+352) 2663 2665 Facebook <https://www.facebook.com/hotcity.lu> | Twitter <https://twitter.com/hotcity_wifi>
-- ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. Kea-users mailing list Kea-users@lists.isc.org https://lists.isc.org/mailman/listinfo/kea-users
