Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Anton, > (gdb) print/x $rax > $7 = 0x0 > (gdb) print stmt->mysql > $8 = (MYSQL *) 0x0 > (gdb) print *stmt->mysql > Cannot access memory at address 0x0 >> (gdb) list libmysql.c:4134 >> 4134 if (stmt->mysql->options.report_data_truncation) That null pointer seems to be the problem here, as the mysql client dereferences it unconditionally. >> I looked through upstream git history and commits [1] and [2] might be >> related: they disable automatic reconnects. >> No such commit seem to have reached the stretch version of kea-dhcp: >> ./isc-kea-1.1.0/src/lib/dhcpsrv/mysql_connection.cc:138: my_bool >> auto_reconnect = MLM_TRUE; > Hmmm ... but if you disable autoreconnect, doesn't this means each time > you restart your database server for any reason your dhcp server would > become in a not working state and it would require restart also? I fear this needs to be answered by the package maintainers or upstream developers. Kind regards, Bernhard
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Burnhard,
On 2018-12-15 9:02 a.m., Bernhard Übelacker wrote:
Hello Anton,
On Fri, 14 Dec 2018 16:34:23 -0500 Anton Avramov wrote:
Hello Bernhard,
Well no. I've actually installed. apt install
libmariadbclient18=10.1.26-0+deb9u1 libmariadbclient18-dbgsym
(gdb) display/i $pc
2: x/i $pc
=> 0x7479eccc : movzbl 0x451(%rax),%eax
At this instruction $rax seems to contain the address stored in stmt->mysql.
This address seems to be invalid in your process.
And therefore accessing the options member crashes.
Could you please add the output of following commands, when the crash happened:
print/x $rax
print stmt->mysql
print stmt
set print pretty on
print *stmt
print *stmt->mysql
set print pretty off
up
print
conn_.statements_[isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID]
up
x/6xb identifier_begin
Here is the output of the requested commands:
(gdb) print/x $rax
$7 = 0x0
(gdb) print stmt->mysql
$8 = (MYSQL *) 0x0
(gdb) print stmt
$9 = (MYSQL_STMT *) 0x558f6be8
(gdb) set print pretty on
(gdb) print *stmt
$10 = {
mem_root = {
free = 0x558f6f28,
used = 0x558f9cb8,
pre_alloc = 0x558f6f28,
min_malloc = 32,
block_size = 2009,
block_num = 6,
first_block_usage = 0,
error_handler = 0x0
},
list = {
prev = 0x0,
next = 0x558f2c70,
data = 0x558f6be8
},
mysql = 0x0,
params = 0x558f9cd0,
bind = 0x558f9e20,
fields = 0x558f7770,
result = {
data = 0x0,
embedded_info = 0x0,
alloc = {
free = 0x558f8c88,
used = 0x0,
pre_alloc = 0x558f8c88,
min_malloc = 24,
block_size = 4057,
block_num = 4,
first_block_usage = 0,
error_handler = 0x0
},
rows = 0,
fields = 0,
---Type to continue, or q to quit---
extension = 0x0
},
data_cursor = 0x0,
read_row_func = 0x7479d630 ,
affected_rows = 18446744073709551615,
insert_id = 0,
stmt_id = 4269,
flags = 0,
prefetch_rows = 1,
server_status = 2,
last_errno = 2013,
param_count = 3,
field_count = 18,
state = MYSQL_STMT_PREPARE_DONE,
last_error = "Lost connection to MySQL server during query", '\000'
,
sqlstate = "HY000",
send_types_to_server = 1 '\001',
bind_param_done = 1 '\001',
bind_result_done = 1 '\001',
unbuffered_fetch_cancelled = 0 '\000',
update_max_length = 0 '\000',
extension = 0x558e7948
}
(gdb) print *stmt->mysql
Cannot access memory at address 0x0
(gdb) set print pretty off
(gdb) up
#1 0x77a9ed19 in
isc::dhcp::MySqlHostDataSourceImpl::getHostCollection (this=0x558d6840,
stindex=isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID,
bind=0x7fffd340, exchange=...,
result=std::vector of length 0, capacity 0, single=true)
at ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc:2262
2262 ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc: Няма
такъв файл или директория.
(gdb) print
conn_.statements_[isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID]
$11 = (st_mysql_stmt *) 0x558f6be8
(gdb) up
#2 0x77a9f540 in isc::dhcp::MySqlHostDataSourceImpl::getHost
(this=0x558d6840,
subnet_id=@0x7fffd8ac: 1, identifier_type=@0x55883830:
isc::dhcp::Host::IDENT_HWADDR,
identifier_begin=0x55834300 "\b", identifier_len=6,
stindex=isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID,
exchange=...)
at ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc:2345
2345 in ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc
(gdb) x/6xb identifier_begin
0x55834300: 0x08 0x00 0x27 0x04 0xcc 0x0e
The last line should output 6 bytes showing the MAC address of the
requesting client. Maybe you could check if that crash is
triggered always by the same client or kind of client.
Each time the identifier is different, so I would say it is not caused
by a particular client.
The clients are identical dhclient that is the default with debian
I looked through upstream git history and commits [1] and [2] might be
related: they disable automatic reconnects.
No such commit seem to have reached the stretch version of kea-dhcp:
./isc-kea-1.1.0/src/lib/dhcpsrv/mysql_connection.cc:138:my_bool
auto_reconnect = MLM_TRUE;
Hmmm ... but if you disable autoreconnect, doesn't this means each time
you restart your database server for any reason your dhcp server would
become in a not working state and it would require restart also?
Kind regards,
Bernhard
Thank you very much for all your effort.
Best regards.
(gdb) list libmysql.c:4134
4129 field->type, param_count);
4130 DBUG_RETURN(1);
4131}
4132 }
4133 stmt->bind_result_done= BIND_RESULT_DONE;
4134 if (stmt->mysql->options.report_data_truncation)
4135stmt->bind_result_done|= REPORT_DATA_TRUNCATION;
4136
4137 DBUG_RETURN(0);
4138}
[1]
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Anton, On Fri, 14 Dec 2018 16:34:23 -0500 Anton Avramov wrote: > Hello Bernhard, > > Well no. I've actually installed. apt install > libmariadbclient18=10.1.26-0+deb9u1 libmariadbclient18-dbgsym > > (gdb) display/i $pc > 2: x/i $pc > => 0x7479eccc : movzbl 0x451(%rax),%eax At this instruction $rax seems to contain the address stored in stmt->mysql. This address seems to be invalid in your process. And therefore accessing the options member crashes. Could you please add the output of following commands, when the crash happened: print/x $rax print stmt->mysql print stmt set print pretty on print *stmt print *stmt->mysql set print pretty off up print conn_.statements_[isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID] up x/6xb identifier_begin The last line should output 6 bytes showing the MAC address of the requesting client. Maybe you could check if that crash is triggered always by the same client or kind of client. I looked through upstream git history and commits [1] and [2] might be related: they disable automatic reconnects. No such commit seem to have reached the stretch version of kea-dhcp: ./isc-kea-1.1.0/src/lib/dhcpsrv/mysql_connection.cc:138:my_bool auto_reconnect = MLM_TRUE; Kind regards, Bernhard (gdb) list libmysql.c:4134 4129 field->type, param_count); 4130 DBUG_RETURN(1); 4131} 4132 } 4133 stmt->bind_result_done= BIND_RESULT_DONE; 4134 if (stmt->mysql->options.report_data_truncation) 4135stmt->bind_result_done|= REPORT_DATA_TRUNCATION; 4136 4137 DBUG_RETURN(0); 4138} [1] https://gitlab.isc.org/isc-projects/kea/commit/9881ef6d772f27de82c048e198ba0ff9e71b9351 [2] https://gitlab.isc.org/isc-projects/kea/commit/6b278a3f54ecf6bd6e2d381047a9eced4bf165f5 # From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916288#5 Thread 1 "kea-dhcp4" received signal SIGSEGV, Segmentation fault. 0x747f525c in mysql_stmt_bind_result () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 (gdb) (gdb) bt #0 0x747f525c in mysql_stmt_bind_result () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #1 0x77a9ed19 in ?? () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #2 0x77a9f540 in ?? () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #3 0x77aa0551 in isc::dhcp::MySqlHostDataSource::get4(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const ()from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #4 0x77a5acba in isc::dhcp::HostMgr::get4(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const ()from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #5 0x779eeb8c in boost::shared_ptr boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::call(isc::dhcp::HostMgr* const&, void const*, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*&, unsigned long&) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #6 0x779ed9b1 in boost::shared_ptr boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::operator()(isc::dhcp::HostMgr* const&, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #7 0x779ec881 in boost::shared_ptr boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> >::operator(), boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::rrlist4 >(boost::_bi::type >, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>&, boost::_bi::rrlist4&, long) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #8 0x779eadbe in boost::shared_ptr boost::_bi::bind_t, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >::operator()(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*&&, unsigned long&&) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #9 0x779e83e8 in boost::detail::function::function_obj_invoker4, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >, boost::shared_ptr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Bernhard, On 2018-12-14 12:38 p.m., Bernhard Übelacker wrote: Hello Anton, sorry, did completely miss the option that upstream could also provide dbgsym packages ... Just to be sure, the last backtrace was retrieved with upstream binaries of version 10.2.19+maria~stretch? And debug symbols are contained in libmariadb3-dbgsym? Well no. I've actually installed. apt install libmariadbclient18=10.1.26-0+deb9u1 libmariadbclient18-dbgsym I've tried libmariadb3 libmariadb3-dbgsym, but then the servers doesn't lease an IP at all. In the log I just see: 2018-12-14 21:29:28.325 INFO [kea-dhcp4.leases/2286] DHCP4_LEASE_ADVERT [hwtype=1 08:00:27:04:cc:0e], cid=[no info], tid=0xc3158b1a: lease 172.16.66.12 will be advertised 2018-12-14 21:29:28.327 ERROR [kea-dhcp4.alloc-engine/2286] ALLOC_ENGINE_V4_ALLOC_ERROR [hwtype=1 08:00:27:04:cc:0e], cid=[no info], tid=0xc3158b1a: error during attempt to allocate an IPv4 address: unable to bind parameters for valid_lifetime, expire, subnet_id, fqdn_fwd, fqdn_rev, hostname, state) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)>, reason: (error code 0) Because your last backtrace shows mysql_stmt_bind_result from "./libmysql/libmysql.c:4134". But in [1] I just found a "./libmysqld/libmysql.c" (the directory is not equal). And in that line 4134 is the end of the expected function. But attaching a debugger to a live process and setting a breakpoint to mysql_stmt_bind_result shows ./libmariadb/libmariadb/mariadb_stmt.c:1210 in my test environment. And in that file is another implementation for mysql_stmt_bind_result ... Therefore you probably can supply another backtrace with the output of following additional commands? display/i $pc info share (gdb) display/i $pc 2: x/i $pc => 0x7479eccc : movzbl 0x451(%rax),%eax (gdb) info share From To Syms Read Shared Object Library 0x77dd9aa0 0x77df5070 Yes /lib64/ld-linux-x86-64.so.2 0x779c2330 0x77b0aad0 Yes /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 0x7747fd00 0x774a3802 Yes /usr/lib/x86_64-linux-gnu/libkea-eval.so.4 0x77204d80 0x7722901c Yes /usr/lib/x86_64-linux-gnu/libkea-dhcp_ddns.so.1 0x76f9aa20 0x76fa8ab5 Yes /usr/lib/x86_64-linux-gnu/libkea-stats.so.1 0x76d49190 0x76d6316a Yes /usr/lib/x86_64-linux-gnu/libkea-cfgclient.so.2 0x76a184f0 0x76ab5b1b Yes /usr/lib/x86_64-linux-gnu/libkea-dhcp++.so.4 0x766b5fb0 0x766cd099 Yes /usr/lib/x86_64-linux-gnu/libkea-asiolink.so.3 0x76457c00 0x7646c56d Yes /usr/lib/x86_64-linux-gnu/libkea-cc.so.1 0x76229420 0x7622ce37 Yes /usr/lib/x86_64-linux-gnu/libkea-cryptolink.so.1 0x75ffb2d0 0x7600f8b6 Yes /usr/lib/x86_64-linux-gnu/libkea-hooks.so.2 0x75d95ce0 0x75daeca9 Yes /usr/lib/x86_64-linux-gnu/libkea-log.so.2 0x75b0ac90 0x75b3be06 Yes /usr/lib/x86_64-linux-gnu/libkea-util.so.2 0x758b2370 0x758b29f3 Yes /usr/lib/x86_64-linux-gnu/libkea-exceptions.so.0 0x756ae060 0x756aed06 Yes (*) /usr/lib/x86_64-linux-gnu/libboost_system.so.1.62.0 0x753b67e0 0x7545ed49 Yes (*) /usr/lib/x86_64-linux-gnu/libstdc++.so.6 0x75116a90 0x75126895 Yes (*) /lib/x86_64-linux-gnu/libgcc_s.so.1 0x74d94940 0x74ebe3d3 Yes /lib/x86_64-linux-gnu/libc.so.6 0x74799e00 0x7483ffec Yes /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 0x7454b8e0 0x74565ab4 Yes (*) /usr/lib/x86_64-linux-gnu/libpq.so.5 0x7432aab0 0x74337811 Yes /lib/x86_64-linux-gnu/libpthread.so.0 0x74017d70 0x740c3103 Yes /usr/lib/x86_64-linux-gnu/libkea-dns++.so.1 0x73d184c0 0x73d1b4f7 Yes /usr/lib/x86_64-linux-gnu/libkea-threads.so.1 0x73ac57f0 0x73af8e91 Yes (*) /usr/lib/x86_64-linux-gnu/liblog4cplus-1.1.so.9 0x73646000 0x737dca39 Yes (*) /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 0x733b8d80 0x733b994e Yes /lib/x86_64-linux-gnu/libdl.so.2 0x730b9680 0x731258da Yes /lib/x86_64-linux-gnu/libm.so.6 0x72eae0e0 0x72eb0ecf Yes /lib/x86_64-linux-gnu/librt.so.1 0x72c941c0 0x72ca4afe Yes (*) /lib/x86_64-linux-gnu/libz.so.1 0x72a21980 0x72a6b3e6 Yes (*) /usr/lib/x86_64-linux-gnu/libssl.so.1.1 0x727c4f80 0x727f46b9 Yes (*) /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 0x725763a0 0x725a5aa2 Yes (*) /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 0x722b3740 0x7230e1e5 Yes (*) /usr/lib/x86_64-linux-gnu/libkrb5.so.3 0x7205fa70 0x7207c790 Yes (*) /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 ---Type to continue, or q to quit--- 0x71e583c0 0x71e58f83 Yes (*)
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Anton,
sorry, did completely miss the option that upstream could
also provide dbgsym packages ...
Just to be sure, the last backtrace was retrieved with upstream
binaries of version 10.2.19+maria~stretch?
And debug symbols are contained in libmariadb3-dbgsym?
Because your last backtrace shows mysql_stmt_bind_result
from "./libmysql/libmysql.c:4134".
But in [1] I just found a "./libmysqld/libmysql.c"
(the directory is not equal).
And in that line 4134 is the end of the expected function.
But attaching a debugger to a live process and setting a breakpoint
to mysql_stmt_bind_result shows ./libmariadb/libmariadb/mariadb_stmt.c:1210
in my test environment.
And in that file is another implementation for mysql_stmt_bind_result ...
Therefore you probably can supply another backtrace with the output of
following additional commands?
display/i $pc
info share
And maybe what this command outputs:
ls -lisah /usr/lib/x86_64-linux-gnu/{*maria*,*mysql*}
Kind regards,
Bernhard
[1] Get:1 http://mirror.23media.de/mariadb/repo/10.2/debian stretch/main
mariadb-10.2 10.2.19+maria~stretch (dsc) [3275 B]
Get:2 http://mirror.23media.de/mariadb/repo/10.2/debian stretch/main
mariadb-10.2 10.2.19+maria~stretch (tar) [45.6 MB]
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hi Bernhard, Thank you for replying. On 2018-12-13 11:15 a.m., Bernhard Übelacker wrote: Hello Anton Avramov, not involved in packaging of isc-kea I just read your report and may have some points. Why do you use the upstream binary of libmariadbclient.so.18. Is the stretch packaged version 10.1.37-0+deb9u1 too old? [1] Yes. We are using mariadb 2.10 for our implementation across hundreds of installations. We don't have problems with the other services connecting to mariadb using the installed libmariadbclient And it looks like you installed the debug symbol package kea-dhcp4-server-dbgsym? Maybe you could also add kea-common-dbgsym? And if it fails with libmariadbclient18 from stretch also I would have suggested to install libmariadbclient18-dbgsym, but unfortunately it looks like there are no debug symbols from security packages broadly available [2]. OK. I've installed libmariadbclient18-dbgsym from the upstream packages with the corresponding libmariadbclient18 package. Here is the result: --- Thread 1 "kea-dhcp4" received signal SIGSEGV, Segmentation fault. mysql_stmt_bind_result (stmt=0x558f6be8, my_bind=) at ./libmysql/libmysql.c:4134 4134 ./libmysql/libmysql.c: Няма такъв файл или директория. (gdb) (gdb) (gdb) bt #0 mysql_stmt_bind_result (stmt=0x558f6be8, my_bind=out>) at ./libmysql/libmysql.c:4134 #1 0x77a9ed19 in isc::dhcp::MySqlHostDataSourceImpl::getHostCollection (this=0x558d6840, stindex=isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID, bind=0x7fffd340, exchange=..., result=std::vector of length 0, capacity 0, single=true) at ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc:2262 #2 0x77a9f540 in isc::dhcp::MySqlHostDataSourceImpl::getHost (this=0x558d6840, subnet_id=@0x7fffd8ac: 1, identifier_type=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, identifier_begin=0x55884550 "\b", identifier_len=6, stindex=isc::dhcp::MySqlHostDataSourceImpl::GET_HOST_SUBID4_DHCPID, exchange=...) at ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc:2345 #3 0x77aa0551 in isc::dhcp::MySqlHostDataSource::get4 (this=0x55849370, subnet_id=@0x7fffd8ac: 1, identifier_type=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, identifier_begin=0x55884550 "\b", identifier_len=6) at ../../../../src/lib/dhcpsrv/mysql_host_data_source.cc:2521 #4 0x77a5acba in isc::dhcp::HostMgr::get4 (this=0x558492b0, subnet_id=@0x7fffd8ac: 1, identifier_type=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, identifier_begin=0x55884550 "\b", identifier_len=6) at ../../../../src/lib/dhcpsrv/host_mgr.cc:134 #5 0x779eeb8c in boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::callconst, unsigned int const, isc::dhcp::Host::IdentifierType const, unsigned char const*, unsigned long> (this=0x7fffd988, u=@0x7fffd998: 0x558492b0, b1=@0x7fffd8ac: 1, b2=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, b3=@0x7fffd630: 0x55884550 "\b", b4=@0x7fffd680: 6) at /usr/include/boost/bind/mem_fn_template.hpp:561 #6 0x779ed9b1 in boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::operator() (this=0x7fffd988, u=@0x7fffd998: 0x558492b0, a1=@0x7fffd8ac: 1, a2=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, a3=0x55884550 "\b", a4=6) at /usr/include/boost/bind/mem_fn_template.hpp:571 #7 0x779ec881 in boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> >::operator(), boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::rrlist4const&, unsigned char const*, unsigned long> > (this=0x7fffd998, f=..., a=...) at /usr/include/boost/bind/bind.hpp:521 #8 0x779eadbe in boost::_bi::bind_t, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >::operator()const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*&&, unsigned long&&) (this=0x7fffd988, a1=@0x7fffd8ac: 1, a2=@0x55914250: isc::dhcp::Host::IDENT_HWADDR, a3=in /usr/lib/debug/.build-id/f9/bc9131c938ea0212083231a71f5a9cbed76ac0.debug, CU 0xbaca, DIE 0x68cf1>, a4=/usr/lib/debug/.build-id/f9/bc9131c938ea0212083231a71f5a9cbed76ac0.debug, CU 0xbaca, DIE 0x68cfd>) at /usr/include/boost/bind/bind.hpp:1342 #9 0x779e83e8 in boost::detail::function::function_obj_invoker4const>, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&,
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Hello Anton Avramov, not involved in packaging of isc-kea I just read your report and may have some points. Why do you use the upstream binary of libmariadbclient.so.18. Is the stretch packaged version 10.1.37-0+deb9u1 too old? [1] And it looks like you installed the debug symbol package kea-dhcp4-server-dbgsym? Maybe you could also add kea-common-dbgsym? And if it fails with libmariadbclient18 from stretch also I would have suggested to install libmariadbclient18-dbgsym, but unfortunately it looks like there are no debug symbols from security packages broadly available [2]. (Maybe a set of local rebuilt packages for 10.1.37-0+deb9u1 including the dbgsym package?) Kind regards, Bernhard [1] https://packages.debian.org/stretch/libmariadbclient18 [2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=894081
Bug#916288: kea-dhcp4-server: Segfault after running for several minutes
Package: kea-dhcp4-server Version: 1.1.0-1 Severity: important Dear Maintainer, I'm trying kea with mariadb as a backend. Relevant package installed from mariadb website (libmariadbclient18=10.2.19+maria~stretch). I have inserted several host reservations. Initially the server gives out leases but in some point (I suspect on renew) it crashes. Here is a debug trace of what I've collected: -- [New Thread 0x7fffef838700 (LWP 25166)] Thread 1 "kea-dhcp4" received signal SIGSEGV, Segmentation fault. 0x747f525c in mysql_stmt_bind_result () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 (gdb) (gdb) bt #0 0x747f525c in mysql_stmt_bind_result () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #1 0x77a9ed19 in ?? () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #2 0x77a9f540 in ?? () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #3 0x77aa0551 in isc::dhcp::MySqlHostDataSource::get4(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #4 0x77a5acba in isc::dhcp::HostMgr::get4(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #5 0x779eeb8c in boost::shared_ptr boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::callconst, unsigned int const, isc::dhcp::Host::IdentifierType const, unsigned char const*, unsigned long>(isc::dhcp::HostMgr* const&, void const*, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*&, unsigned long&) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #6 0x779ed9b1 in boost::shared_ptr boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::operator()(isc::dhcp::HostMgr* const&, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #7 0x779ec881 in boost::shared_ptr boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> >::operator(), boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::rrlist4const&, unsigned char const*, unsigned long> >(boost::_bi::type >, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>&, boost::_bi::rrlist4const&, unsigned char const*, unsigned long>&, long) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #8 0x779eadbe in boost::shared_ptr boost::_bi::bind_t, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >::operator()const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*&&, unsigned long&&) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #9 0x779e83e8 in boost::detail::function::function_obj_invoker4const>, boost::_mfi::cmf4, isc::dhcp::HostMgr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>, boost::_bi::list5, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >, boost::shared_ptr, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::invoke(boost::detail::function::function_buffer&, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #10 0x779e0a28 in boost::function4, unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long>::operator()(unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long) const () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #11 0x779de266 in void isc::dhcp::AllocEngine::findReservationInternal(isc::dhcp::AllocEngine::ClientContext4&, boost::function (unsigned int const&, isc::dhcp::Host::IdentifierType const&, unsigned char const*, unsigned long)> const&) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #12 0x779d2ef2 in isc::dhcp::AllocEngine::findReservation(isc::dhcp::AllocEngine::ClientContext4&) () from /usr/lib/x86_64-linux-gnu/libkea-dhcpsrv.so.6 #13 0x555d6c6c in isc::dhcp::Dhcpv4Exchange::Dhcpv4Exchange (this=0x7fffdc90, alloc_engine=..., query=..., subnet=...) at
