Thanks for your support. Unless I miss it in the logs it might also be a good idea to have error message if both servers have the same server-id
Thanks Nitzan On Tue, May 7, 2019 at 11:45 AM Marcin Siodelski <[email protected]> wrote: > Hello Nitzan, > > Thank you for sharing this information. We'll try to reproduce this > behavior on our end. If the problem occurs, we'll investigate and fix it > because the duplicate server id is surely not a good reason for leaking > memory.... ;-) > > Marcin Siodelski > > DHCP Software Engineer, > ISC > > On 07/05/2019 10:26, Nitzan Tzelniker wrote: > > Hi, > > > > Just for anybody else that has this issue > > The issue was that I had the same server-id on both servers as we clone > > the VM of the first server after it create the server-id > > I dont know why it cause memory leak. > > > > Thenks > > > > Nitzan > > > > > > On Mon, May 6, 2019 at 6:56 PM Marcin Siodelski <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi Nitzan, > > > > We're trying to reproduce this issue using our test tools. If the > leak > > is related to the traffic volume processed by the server, it would be > > useful for us to know the average number of packets/second that your > > primary server is receiving. > > > > You say that the memory consumption grows to 28% within an hour but I > > don't know how many packets the server has received and processed to > > reach that level. If you can provide the average DHCP traffic rate we > > can craft the test that best mirrors your situation and confirm > whether > > or not we see the same thing. > > > > Thanks in advance, > > > > Marcin Siodelski > > DHCP Software Engineer, > > ISC > > > > On 05/05/2019 16:29, Nitzan Tzelniker wrote: > > > Hi, > > > > > > It seems that the issue is related to the HA > > > After I commented the HA part the memory consumption is stayed low > > > I am adding the relevant configuration if you have any idea > > > > > > Thanks > > > > > > Nitzan > > > > > > Standby server ha configuration part: > > > > > > "hooks-libraries": [ > > > { > > > "library": > "/usr/local/lib/hooks/libdhcp_lease_cmds.so", > > > "parameters": { } > > > }, > > > { > > > "library": "/usr/local/lib/hooks/libdhcp_ha.so", > > > "parameters": { > > > "high-availability": [ { > > > "this-server-name": "KEA_DHCPv6_2", > > > "mode": "hot-standby", > > > "heartbeat-delay": 10000, > > > "max-response-delay": 20000, > > > "max-ack-delay": 10000, > > > "max-unacked-clients": 0, > > > "peers": [ > > > { > > > "name": "KEA_DHCPv6_1", > > > "role": "primary", > > > "auto-failover": true > > > }, > > > { > > > "name": "KEA_DHCPv6_2", > > > "role": "standby", > > > "auto-failover": true > > > } > > > ] > > > } ] > > > } > > > } > > > ] > > > } > > > > > > kea-ctrl-agent.conf from both servers (Only the ip address is > > different ) > > > > > > { > > > > > > "Control-agent": { > > > "http-host": "1.1.1.X", > > > "http-port": 8080, > > > > > > "control-sockets": { > > > "dhcp4": { > > > "socket-type": "unix", > > > "socket-name": "/tmp/kea-dhcp4-ctrl.sock" > > > }, > > > "dhcp6": { > > > "socket-type": "unix", > > > "socket-name": "/tmp/kea-dhcp6-ctrl.sock" > > > } > > > }, > > > > > > "hooks-libraries": [ > > > ] > > > }, > > > > > > "Logging": > > > { > > > "loggers": [ > > > { > > > "name": "kea-ctrl-agent", > > > "output_options": [ > > > { > > > "output": "/usr/local/var/log/kea-ctrl-agent.log" > > > } > > > ], > > > "severity": "INFO", > > > > > > "debuglevel": 0 > > > } > > > ] > > > } > > > } > > > ---------- Forwarded message --------- > > > From: *Nitzan Tzelniker* <[email protected] > > <mailto:[email protected]> > > > <mailto:[email protected] > > <mailto:[email protected]>>> > > > Date: Fri, May 3, 2019 at 10:13 PM > > > Subject: Memory leak on dhcp6 > > > To: <[email protected] <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> > > > > > > > > > Hi, > > > > > > I am running two dhcpv6 server with high-availability > > > We have about 2500 regular leases and 2500 PD leases > > > The setup is working but it look like kea-dhcp6 leak memory > > > When it start (with all of the leases in the file ) it take less > > then 1% > > > of the memory but after two hours it take 28% and if I will give > it 8 > > > hours it will crash > > > > > > I don't see issue on the standby server only on the primary server > > > > > > Anybody saw this behavior > > > Any idea how to debug it > > > > > > kea-dhcp6 -V > > > 1.5.0 > > > tarball > > > linked with: > > > log4cplus 1.1.3 > > > OpenSSL 1.0.2k-fips 26 Jan 2017 > > > database: > > > Memfile backend 2.1 > > > > > > Running on CentOS 7.6.1810 VM kernel 3.10.0-957.1.3.el7.x86 with 2 > * > > > vCPU and 2GB RAM > > > > > > bellow is the dhcp6 config file omitting 102 subsets > > > > > > { > > > "Dhcp6": { > > > "interfaces-config": { > > > "interfaces": [ "ens192/1111:aef:aa77:83::100" ] > > > }, > > > "mac-sources": [ "docsis-cmts", "client-link-addr-option" ], > > > > > > "control-socket": { > > > "socket-type": "unix", > > > "socket-name": "/tmp/kea-dhcp6-ctrl.sock" > > > }, > > > > > > "lease-database": { > > > "type": "memfile", > > > "lfc-interval": 3600 > > > }, > > > "expired-leases-processing": { > > > "reclaim-timer-wait-time": 10, > > > "flush-reclaimed-timer-wait-time": 25, > > > "hold-reclaimed-time": 3600, > > > "max-reclaim-leases": 100, > > > "max-reclaim-time": 250, > > > "unwarned-reclaim-cycles": 5 > > > }, > > > > > > "renew-timer": 1000, > > > "rebind-timer": 2000, > > > "preferred-lifetime": 3000, > > > "valid-lifetime": 4000, > > > "option-data": [ > > > { > > > "name": "dns-servers", > > > "data": "1111:aef:aa77:1::22, 1111:aef:aa77:2::22" > > > } > > > ], > > > "subnet6": [ > > > { > > > "subnet": "1111:aec:0:10::/64", > > > "pools": [ { "pool": > > > "1111:aec:0:10::10-1111:aec:0:10:ffff::" } ], > > > "pd-pools": [ > > > { > > > "prefix": "1111:aec:10::", > > > "prefix-len": 44, > > > "delegated-len": 56 > > > } > > > ] > > > }, > > > //.... > > > // 101 more subnets > > > //.... > > > "hooks-libraries": [ > > > { > > > "library": "/usr/local/lib/hooks/libdhcp_ha.so", > > > "parameters": { > > > "high-availability": [ { > > > "this-server-name": "KEA_DHCPv6_1", > > > "mode": "hot-standby", > > > "heartbeat-delay": 10000, > > > "max-response-delay": 20000, > > > "max-ack-delay": 10000, > > > "max-unacked-clients": 0, > > > "peers": [ > > > { > > > "name": "KEA_DHCPv6_1", > > > "role": "primary", > > > "auto-failover": true > > > }, > > > { > > > "name": "KEA_DHCPv6_2", > > > "role": "standby", > > > "auto-failover": true > > > } > > > ] > > > } ] > > > } > > > } > > > ] > > > }, > > > > > > "Logging": > > > { > > > "loggers": [ > > > { > > > "name": "kea-dhcp6", > > > "output_options": [ > > > { > > > "output": "/usr/local/var/log/kea-dhcp6.log" > > > } > > > ], > > > "severity": "INFO", > > > > > > "debuglevel": 0 > > > } > > > ] > > > } > > > } > > > Thanks > > > > > > Nitzan > > > > > > _______________________________________________ > > > Kea-users mailing list > > > [email protected] <mailto:[email protected]> > > > https://lists.isc.org/mailman/listinfo/kea-users > > > > > > >
_______________________________________________ Kea-users mailing list [email protected] https://lists.isc.org/mailman/listinfo/kea-users
