Send dhcp-users mailing list submissions to dhcp-users@lists.isc.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.isc.org/mailman/listinfo/dhcp-users or, via email, send a message with subject or body 'help' to dhcp-users-requ...@lists.isc.org You can reach the person managing the list at dhcp-users-ow...@lists.isc.org When replying, please edit your Subject line so it is more specific than "Re: Contents of dhcp-users digest..." Today's Topics: 1. ISC DHCP performance, an example (Peter Rathlev) 2. DHCP Server Fails When Forking (Jay Foster) 3. Re: DHCP Server Fails When Forking (Christopher Barry) 4. Re: DHCP Server Fails When Forking (Jay Foster) 5. Re: DHCP Server Fails When Forking (Christopher Barry) ---------------------------------------------------------------------- Message: 1 Date: Mon, 06 Jan 2020 18:20:05 +0100 From: Peter Rathlev <pe...@rathlev.dk> To: dhcp-users <dhcp-us...@isc.org> Subject: ISC DHCP performance, an example Message-ID: <56ca38f6ea49ee77e5fedd7fabae753e748af8e1.ca...@rathlev.dk> Content-Type: text/plain; charset="UTF-8" I've been reading up on some (rather old) mails on this list and I can see that there have been a few discussions about performance. I thought I would post what we have at my current job, just as an example for future reference. We have two sets of DHCP servers, one set serving guests and the other set serving everything else. Each set have a standard failover config. Extra info follow below the stats. Stats for the set servicing everything but guests: ISC DHCP version 4.4.1 Number of range statements: 3048 Total number of addresses in range statements: 1641487 Leases in lease file (touched leases): 959816 (58%) Primary server: Server type: Physical Processors: 2 x Intel Xeon E5-2640 @2.40 GHz (20 cores) RAM: 16 GB Load average (24h): 0.6 Lease file storage: RAM-disk with periodic backup to HDD DISCOVERS per day: 127732 (1621 unique clients) Restart time: 21.2 seconds Secondary server: Server type: Virtual Processors (vCPUs): 4 x Intel Xeon E7-4880 v2 @2.5GHz RAM: 4 GB Load average (24h): 0.3 Lease file storage: SSD based virtual disk DISCOVERS per day: 119340 (1632 unique clients) Restart time: 16.0 seconds Stats for the set servicing guests: ISC DHCP version 4.3.6b1 Number of range statements: 516 Total number of addresses in range statements: 236146 Leases in lease file (touched leases): 194592 (82%) Primary server: Server type: Virtual Processors (vCPUs): 4 x Intel Xeon Gold 6154 @3.00 GHz RAM: 4 GB Load average (24h): 0.5 Restart time: 6.5 seconds DISCOVERS per day: 98265 (4564 unique clients) Lease file storage: SSD based virtual disk Secondary server: Server type: Virtual Processors (vCPUs): 4 x Intel Xeon E7-4880 v2 @2.50 GHz RAM: 4 GB Load average (24h): 0.1 Restart time: 5.7 seconds DISCOVERS per day: 119340 (4627 unique clients) Lease file storage: SSD based virtual disk Restart times are measured with "time systemctl restart dhcpd". I have only measured this once on every server for these numbers, but it's my impression that it's nominal. I have included the number of DHCP DISCOVER messages logged and number of unique clients as a measure of their approximate load. Data was for what I hope is a "typical" day, Tuesday December 10th 2019. There are always at least two DHCP relays and often four, multiplying the number of DISCOVER messages logged compared to actual client DISCOVERs. Default lease times are typically 8 days with a few exceptions. We have a bunch of misbehaving clients that result in a lot more DISCOVERs that unique clients per day even though the lease time is much longer. (We have top-talkers warnings and send requests for inspection to local people when one client sticks out.) All the servers also act as a caching resolvers (BIND + anycast) for all our clients and record (via a rotating "tshark") at least the last 24 hours of all DNS and DHCP traffic for troubleshooting purposes. This adds a bit to the load average but not much. We normally restart the secondary server immediately after the primary and have never seen any problems with this. After a normal restart they enter functional state with no pause. -- Peter ------------------------------ Message: 2 Date: Mon, 6 Jan 2020 10:42:05 -0800 From: Jay Foster <j...@systech.com> To: dhcp-users@lists.isc.org Subject: DHCP Server Fails When Forking Message-ID: <a4839503-a7e0-feac-0a5c-957a7eeff...@systech.com> Content-Type: text/plain; charset=utf-8; format=flowed I am using the ISC DHCP server (dhcpd) version 4.3.6 from a rocko Yocto build.? When dhcpd is started without the '-f' or '-d' options (forks to the background) it does not work.? DHCP DISCOVER requests are sent, but the dhcpd application does not respond.? I can attach strace to the dhcpd process and see that dhcpd just sleeps in an futex() call. If I start dhcpd with either the '-f' or '-d' options so it does not fork, then it works properly. I have run dhcpd (both forking and non forking) using strace to see if I could spot any differences.? The only thing I notice is that when forking, the parent process terminates (expected) along with 3 other child processes/threads.? It looks like these other threads are supposed to handle the ISC tasks/messages, but are not cloned by the fork and are not present after forking. For reference, with an older version of ISC DHCP server (4.1.1-P1) on an older product, this problem does not occur. Any ideas where to look? ------------------------------ Message: 3 Date: Mon, 6 Jan 2020 15:58:32 -0500 From: Christopher Barry <christopher.r.ba...@gmail.com> To: Users of ISC DHCP <dhcp-users@lists.isc.org> Subject: Re: DHCP Server Fails When Forking Message-ID: <caoajpmrdq2ew+owdgucwbur99ehwtlwp9vdaipsa+vzv3je...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Make sure user/group perms are valid. On Mon, Jan 6, 2020, 1:42 PM Jay Foster <j...@systech.com> wrote: > I am using the ISC DHCP server (dhcpd) version 4.3.6 from a rocko Yocto > build. When dhcpd is started without the '-f' or '-d' options (forks to > the background) it does not work. DHCP DISCOVER requests are sent, but > the dhcpd application does not respond. I can attach strace to the > dhcpd process and see that dhcpd just sleeps in an futex() call. > > If I start dhcpd with either the '-f' or '-d' options so it does not > fork, then it works properly. > > I have run dhcpd (both forking and non forking) using strace to see if I > could spot any differences. The only thing I notice is that when > forking, the parent process terminates (expected) along with 3 other > child processes/threads. It looks like these other threads are supposed > to handle the ISC tasks/messages, but are not cloned by the fork and are > not present after forking. > > For reference, with an older version of ISC DHCP server (4.1.1-P1) on an > older product, this problem does not occur. > > Any ideas where to look? > > > > > > _______________________________________________ > dhcp-users mailing list > dhcp-users@lists.isc.org > https://lists.isc.org/mailman/listinfo/dhcp-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.isc.org/pipermail/dhcp-users/attachments/20200106/b7811fda/attachment-0001.htm> ------------------------------ Message: 4 Date: Mon, 6 Jan 2020 13:14:53 -0800 From: Jay Foster <j...@systech.com> To: Users of ISC DHCP <dhcp-users@lists.isc.org>, Christopher Barry <christopher.r.ba...@gmail.com> Subject: Re: DHCP Server Fails When Forking Message-ID: <e5d0a2ba-d8f3-b6ca-bf69-85fdcb8ed...@systech.com> Content-Type: text/plain; charset="utf-8"; Format="flowed" Can you be more specific?? /usr/sbin/dhcpd is root:root. /usr/sbin/dhcpd is being run as root. On 1/6/2020 12:58 PM, Christopher Barry wrote: > Make sure user/group perms are valid. > > > On Mon, Jan 6, 2020, 1:42 PM Jay Foster <j...@systech.com > <mailto:j...@systech.com>> wrote: > > I am using the ISC DHCP server (dhcpd) version 4.3.6 from a rocko > Yocto > build.? When dhcpd is started without the '-f' or '-d' options > (forks to > the background) it does not work.? DHCP DISCOVER requests are > sent, but > the dhcpd application does not respond.? I can attach strace to the > dhcpd process and see that dhcpd just sleeps in an futex() call. > > If I start dhcpd with either the '-f' or '-d' options so it does not > fork, then it works properly. > > I have run dhcpd (both forking and non forking) using strace to > see if I > could spot any differences.? The only thing I notice is that when > forking, the parent process terminates (expected) along with 3 other > child processes/threads.? It looks like these other threads are > supposed > to handle the ISC tasks/messages, but are not cloned by the fork > and are > not present after forking. > > For reference, with an older version of ISC DHCP server (4.1.1-P1) > on an > older product, this problem does not occur. > > Any ideas where to look? > > > > > > _______________________________________________ > dhcp-users mailing list > dhcp-users@lists.isc.org <mailto:dhcp-users@lists.isc.org> > https://lists.isc.org/mailman/listinfo/dhcp-users > > > _______________________________________________ > dhcp-users mailing list > dhcp-users@lists.isc.org > https://lists.isc.org/mailman/listinfo/dhcp-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.isc.org/pipermail/dhcp-users/attachments/20200106/d6fac1ee/attachment-0001.htm> ------------------------------ Message: 5 Date: Mon, 6 Jan 2020 16:49:04 -0500 From: Christopher Barry <christopher.r.ba...@gmail.com> Cc: Users of ISC DHCP <dhcp-users@lists.isc.org> Subject: Re: DHCP Server Fails When Forking Message-ID: <CAOajpmq43V72OCZ8bvNmE9jiAHQo6qofFNb61E9+=14icpk...@mail.gmail.com> Content-Type: text/plain; charset="utf-8" Generally, when backgrounded, dhcpd runs with reduced privs, e.g. as dhcpd:dhcpd (your configured user/group may be different). Can you su as that user and run dhcpd in the foreground? I'm suspecting that user may not have write perms to the leases file. On Mon, Jan 6, 2020, 4:15 PM Jay Foster <j...@systech.com> wrote: > Can you be more specific? /usr/sbin/dhcpd is root:root. /usr/sbin/dhcpd > is being run as root. > > On 1/6/2020 12:58 PM, Christopher Barry wrote: > > Make sure user/group perms are valid. > > > On Mon, Jan 6, 2020, 1:42 PM Jay Foster <j...@systech.com> wrote: > >> I am using the ISC DHCP server (dhcpd) version 4.3.6 from a rocko Yocto >> build. When dhcpd is started without the '-f' or '-d' options (forks to >> the background) it does not work. DHCP DISCOVER requests are sent, but >> the dhcpd application does not respond. I can attach strace to the >> dhcpd process and see that dhcpd just sleeps in an futex() call. >> >> If I start dhcpd with either the '-f' or '-d' options so it does not >> fork, then it works properly. >> >> I have run dhcpd (both forking and non forking) using strace to see if I >> could spot any differences. The only thing I notice is that when >> forking, the parent process terminates (expected) along with 3 other >> child processes/threads. It looks like these other threads are supposed >> to handle the ISC tasks/messages, but are not cloned by the fork and are >> not present after forking. >> >> For reference, with an older version of ISC DHCP server (4.1.1-P1) on an >> older product, this problem does not occur. >> >> Any ideas where to look? >> >> >> >> >> >> _______________________________________________ >> dhcp-users mailing list >> dhcp-users@lists.isc.org >> https://lists.isc.org/mailman/listinfo/dhcp-users >> > > _______________________________________________ > dhcp-users mailing > listdhcp-us...@lists.isc.orghttps://lists.isc.org/mailman/listinfo/dhcp-users > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.isc.org/pipermail/dhcp-users/attachments/20200106/a69b3f45/attachment.htm> ------------------------------ Subject: Digest Footer _______________________________________________ dhcp-users mailing list dhcp-users@lists.isc.org https://lists.isc.org/mailman/listinfo/dhcp-users ------------------------------ End of dhcp-users Digest, Vol 135, Issue 1 ******************************************