Re: [pmacct-discussion] crashes since update to 1.7.6
Hello Paolo, yeah, I see the issue. char* host is not checked and might be void, so strstr tries to read from 0x0. I use version 1.7.6 by package system, so switching to a git and master branch is not an option. But I can apply a patch. As quickfix I simpy supplied a sql_host setting in conf, which prevents the crashes. Thanks Goeran Am 15.04.2021 um 02:21 schrieb Paolo Lucente: Hi Goran, Thanks for the extra info, it helped. It should be the case that master code has already a fix for the issue. See: https://github.com/pmacct/pmacct/commit/1922062277d87da024938cc3057df9a4c3725e76 I may suggest to run master code from GitHub or to apply this patch to 1.7.6 to test it working for you. Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] crashes since update to 1.7.6
Hi Goran, Thanks for the extra info, it helped. It should be the case that master code has already a fix for the issue. See: https://github.com/pmacct/pmacct/commit/1922062277d87da024938cc3057df9a4c3725e76 I may suggest to run master code from GitHub or to apply this patch to 1.7.6 to test it working for you. Paolo On 14/04/2021 19:08, Göran Bruns wrote: Hello Paola, I installed valgrind and recompiled glibc and pmacct with debugging symbols. This is the output. rapna ~ # valgrind /usr/sbin/uacctd -d -f /etc/pmacctd/uacctd.conf ==3260== Memcheck, a memory error detector ==3260== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==3260== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info ==3260== Command: /usr/sbin/uacctd -d -f /etc/pmacctd/uacctd.conf ==3260== DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'default'/'core'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in'/'mysql'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'out'/'mysql'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in1'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'out1'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in2'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] daemonize:true DEBUG: [/etc/pmacctd/uacctd.conf] syslog:daemon DEBUG: [/etc/pmacctd/uacctd.conf] plugin_pipe_zmq:true DEBUG: [/etc/pmacctd/uacctd.conf] plugin_pipe_zmq_profile:medium DEBUG: [/etc/pmacctd/uacctd.conf] uacctd_group:5 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in]:dst_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[out]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in]:( dst net 192.168.2.0/24 or dst net 192.168.222.0/24 or dst net 172.16.3.0/24 ) and not dst host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[out]:( src net 192.168.2.0/24 or src net 192.168.222.0/24 or src net 172.16.3.0/24 ) and not src host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] sql_db:net_stats DEBUG: [/etc/pmacctd/uacctd.conf] sql_table[in]:traffic_wan_in DEBUG: [/etc/pmacctd/uacctd.conf] sql_table[out]:traffic_wan_out DEBUG: [/etc/pmacctd/uacctd.conf] sql_optimize_clauses:true DEBUG: [/etc/pmacctd/uacctd.conf] sql_passwd:max3ds DEBUG: [/etc/pmacctd/uacctd.conf] sql_user:root DEBUG: [/etc/pmacctd/uacctd.conf] sql_refresh_time:3600 DEBUG: [/etc/pmacctd/uacctd.conf] sql_history:1h DEBUG: [/etc/pmacctd/uacctd.conf] sql_history_roundoff:h DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in1]:dst_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[out1]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in1]:( dst net 192.168.2.0/24 or dst net 192.168.222.0/24 or dst net 172.16.3.0/24 ) and not dst host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[out1]:( src net 192.168.2.0/24 or src net 192.168.222.0/24 or src net 172.16.3.0/24 ) and not src host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[in1]:/var/run/pmacct_wan_in.pipe DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[out1]:/var/run/pmacct_wan_out.pipe DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in2]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in2]:dst host 192.168.2.27 DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[in2]:/var/run/pmacct_arvid_in.pipe DEBUG: [/etc/pmacctd/uacctd.conf] debug:true ==3260== ==3260== HEAP SUMMARY: ==3260== in use at exit: 135,884 bytes in 105 blocks ==3260== total heap usage: 164 allocs, 59 frees, 229,203 bytes allocated ==3260== ==3260== LEAK SUMMARY: ==3260== definitely lost: 288 bytes in 1 blocks ==3260== indirectly lost: 3,452 bytes in 35 blocks ==3260== possibly lost: 0 bytes in 0 blocks ==3260== still reachable: 132,144 bytes in 69 blocks ==3260== suppressed: 0 bytes in 0 blocks ==3260== Rerun with --leak-check=full to see details of leaked memory ==3260== ==3260== For lists of detected and suppressed errors, rerun with: -s ==3260== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ==3267== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==3267== at 0x54CACFC: sendto (in /lib64/libpthread-2.32.so) ==3267== by 0x621A7F2: nfnl_send (in /usr/lib64/libnfnetlink.so.0.2.0) ==3267== by 0x621C462: nfnl_query (in /usr/lib64/libnfnetlink.so.0.2.0) ==3267== by 0x50CB341: ??? (in /usr/lib64/libnetfilter_log.so.1.1.0) ==3267== by 0x113E50: main (in /usr/sbin/uacctd) ==3267== Address 0x1ffeff4239 is on thread 1's stack ==3267== ==3289== ==3289== HEAP SUMMARY: ==3289== in use at exit: 807,638 bytes in 405 blocks ==3289== total heap usage: 4,470 allocs, 4,065 frees, 12,736,275 bytes allocated ==3289== ==3295== ==3295== HEAP SUMMARY: ==3295== in use at exit: 998,559 bytes in 493 blocks ==3295== total heap usage: 3,478 allocs, 2,985 frees, 9,554,499 bytes allocated ==3295== ==3283== ==3283== HEAP SUMMARY: ==3283== in use at exit: 712,487 bytes in 342 blocks ==3283== total heap usage: 4,465 allocs, 4,123 frees, 12,893,134
Re: [pmacct-discussion] crashes since update to 1.7.6
Hello Paola, I installed valgrind and recompiled glibc and pmacct with debugging symbols. This is the output. rapna ~ # valgrind /usr/sbin/uacctd -d -f /etc/pmacctd/uacctd.conf ==3260== Memcheck, a memory error detector ==3260== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==3260== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info ==3260== Command: /usr/sbin/uacctd -d -f /etc/pmacctd/uacctd.conf ==3260== DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'default'/'core'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in'/'mysql'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'out'/'mysql'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in1'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'out1'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] plugin name/type: 'in2'/'memory'. DEBUG: [/etc/pmacctd/uacctd.conf] daemonize:true DEBUG: [/etc/pmacctd/uacctd.conf] syslog:daemon DEBUG: [/etc/pmacctd/uacctd.conf] plugin_pipe_zmq:true DEBUG: [/etc/pmacctd/uacctd.conf] plugin_pipe_zmq_profile:medium DEBUG: [/etc/pmacctd/uacctd.conf] uacctd_group:5 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in]:dst_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[out]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in]:( dst net 192.168.2.0/24 or dst net 192.168.222.0/24 or dst net 172.16.3.0/24 ) and not dst host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[out]:( src net 192.168.2.0/24 or src net 192.168.222.0/24 or src net 172.16.3.0/24 ) and not src host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] sql_db:net_stats DEBUG: [/etc/pmacctd/uacctd.conf] sql_table[in]:traffic_wan_in DEBUG: [/etc/pmacctd/uacctd.conf] sql_table[out]:traffic_wan_out DEBUG: [/etc/pmacctd/uacctd.conf] sql_optimize_clauses:true DEBUG: [/etc/pmacctd/uacctd.conf] sql_passwd:max3ds DEBUG: [/etc/pmacctd/uacctd.conf] sql_user:root DEBUG: [/etc/pmacctd/uacctd.conf] sql_refresh_time:3600 DEBUG: [/etc/pmacctd/uacctd.conf] sql_history:1h DEBUG: [/etc/pmacctd/uacctd.conf] sql_history_roundoff:h DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in1]:dst_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[out1]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in1]:( dst net 192.168.2.0/24 or dst net 192.168.222.0/24 or dst net 172.16.3.0/24 ) and not dst host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[out1]:( src net 192.168.2.0/24 or src net 192.168.222.0/24 or src net 172.16.3.0/24 ) and not src host 192.168.2.201 DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[in1]:/var/run/pmacct_wan_in.pipe DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[out1]:/var/run/pmacct_wan_out.pipe DEBUG: [/etc/pmacctd/uacctd.conf] aggregate[in2]:src_host DEBUG: [/etc/pmacctd/uacctd.conf] aggregate_filter[in2]:dst host 192.168.2.27 DEBUG: [/etc/pmacctd/uacctd.conf] imt_path[in2]:/var/run/pmacct_arvid_in.pipe DEBUG: [/etc/pmacctd/uacctd.conf] debug:true ==3260== ==3260== HEAP SUMMARY: ==3260== in use at exit: 135,884 bytes in 105 blocks ==3260== total heap usage: 164 allocs, 59 frees, 229,203 bytes allocated ==3260== ==3260== LEAK SUMMARY: ==3260== definitely lost: 288 bytes in 1 blocks ==3260== indirectly lost: 3,452 bytes in 35 blocks ==3260== possibly lost: 0 bytes in 0 blocks ==3260== still reachable: 132,144 bytes in 69 blocks ==3260== suppressed: 0 bytes in 0 blocks ==3260== Rerun with --leak-check=full to see details of leaked memory ==3260== ==3260== For lists of detected and suppressed errors, rerun with: -s ==3260== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ==3267== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==3267== at 0x54CACFC: sendto (in /lib64/libpthread-2.32.so) ==3267== by 0x621A7F2: nfnl_send (in /usr/lib64/libnfnetlink.so.0.2.0) ==3267== by 0x621C462: nfnl_query (in /usr/lib64/libnfnetlink.so.0.2.0) ==3267== by 0x50CB341: ??? (in /usr/lib64/libnetfilter_log.so.1.1.0) ==3267== by 0x113E50: main (in /usr/sbin/uacctd) ==3267== Address 0x1ffeff4239 is on thread 1's stack ==3267== ==3289== ==3289== HEAP SUMMARY: ==3289== in use at exit: 807,638 bytes in 405 blocks ==3289== total heap usage: 4,470 allocs, 4,065 frees, 12,736,275 bytes allocated ==3289== ==3295== ==3295== HEAP SUMMARY: ==3295== in use at exit: 998,559 bytes in 493 blocks ==3295== total heap usage: 3,478 allocs, 2,985 frees, 9,554,499 bytes allocated ==3295== ==3283== ==3283== HEAP SUMMARY: ==3283== in use at exit: 712,487 bytes in 342 blocks ==3283== total heap usage: 4,465 allocs, 4,123 frees, 12,893,134 bytes allocated ==3283== ==3289== LEAK SUMMARY: ==3289== definitely lost: 936 bytes in 2 blocks ==3289== indirectly lost: 32,276 bytes in 71 blocks ==3289== possibly lost: 640 bytes in 2 blocks ==3289== still reachable: 773,786 bytes in 330 blocks ==3289== of which reachable via heuristic: ==3289==
Re: [pmacct-discussion] crashes since update to 1.7.6
Hello Paolo, I got neither gdb nor valgrind ready. But I can provide core and binary. https://brunsware.de/pmacct/uacctd https://brunsware.de/pmacct/core Crash message was: 2021-04-14 14:59:41.467 debug uacctd[4758]: ( in/mysql ) *** Purging queries queue *** 2021-04-14 14:59:41.467 debug uacctd[4762]: ( out/mysql ) *** Purging queries queue *** 2021-04-14 14:59:41.471 info kernel: uacctd[4758]: segfault at 0 ip 7fba3490eacd sp 7fff30d985b8 error 4 2021-04-14 14:59:41.471 info kernel: uacctd[4762]: segfault at 0 ip 7fba3490eacd sp 7fff30d985b8 error 4 2021-04-14 14:59:41.471 warning kernel: in libc-2.32.so[7fba3488e000+143000] 2021-04-14 14:59:41.471 info kernel: Code: ff 0f 00 00 66 0f 60 c9 48 3d bf 0f 00 00 66 0f 60 d2 66 0f 61 c9 66 0f 61 d2 66 0f 70 c9 00 66 0f 70 d2 00 0f 87 03 03 00 00 0f 6f 1f 66 0f ef ed f3 0f 6f 67 01 66 0f 6f f3 66 0f 74 d9 66 2021-04-14 14:59:41.471 warning kernel: in libc-2.32.so[7fba3488e000+143000] 2021-04-14 14:59:41.471 info kernel: Code: ff 0f 00 00 66 0f 60 c9 48 3d bf 0f 00 00 66 0f 60 d2 66 0f 61 c9 66 0f 61 d2 66 0f 70 c9 00 66 0f 70 d2 00 0f 87 03 03 00 00 0f 6f 1f 66 0f ef ed f3 0f 6f 67 01 66 0f 6f f3 66 0f 74 d9 66 2021-04-14 14:59:41.471 info uacctd[4754]: INFO ( default/core ): OK, Exiting ... 2021-04-14 14:59:41.471 warning uacctd[4754]: WARN ( default/core ): connection lost to 'out1-memory'; closing connection. Is this enough ? Otherwise I will install gdb. regards Goeran Am 14.04.2021 um 14:39 schrieb Paolo Lucente: Hi Goran, Can you please gather more information about the crash following these instructions: https://github.com/pmacct/pmacct/blob/master/QUICKSTART#L2876-#L2896 Output from either a gdb back trace or valgrind would be of help. Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] crashes since update to 1.7.6
Hi Goran, Can you please gather more information about the crash following these instructions: https://github.com/pmacct/pmacct/blob/master/QUICKSTART#L2876-#L2896 Output from either a gdb back trace or valgrind would be of help. Paolo On 13/04/2021 19:40, Göran Bruns wrote: Hi there, since the update to version 1.7.6, I noticed crashes of uacctd in log files. It looks like this: 2021-04-13 19:07:45.781 info kernel: uacctd[4955]: segfault at 0 ip 7fbe399c7acd sp 7ffed50ed338 error 4 in libc-2.32.so[7fbe39947000+143000] 2021-04-13 19:07:45.781 info kernel: Code: ff 0f 00 00 66 0f 60 c9 48 3d bf 0f 00 00 66 0f 60 d2 66 0f 61 c9 66 0f 61 d2 66 0f 70 c9 00 66 0f 70 d2 00 0f 87 03 03 00 00 0f 6f 1f 66 0f ef ed f3 0f 6f 67 01 66 0f 6f f3 66 0f 74 d9 66 It seems related to writing to mysql database. It occurs in intervals and there are no new traffic data in db. I read the quickstart guide. The system is not memory constrained, uacct runs as root and the configuration did not change. So before I try to recompile with debug ... are there any other things I can do to solve the problem ? regards Goeran ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
