Hi Rainer, sorry, I needed to prioritise some other tasks before continuing with this. Now it is time to carry on with it .
I have recreated the rsyslog7.2.5 package for Ubuntu with debug support and launched it using valgrind. I have run the command: valgrind --leak-check=full --show-reachable=yes -v /usr/sbin/rsyslogd -n I copied you the last part of the output below. Don't know if you would prefer to have the full output but then I need some approval to send the attachment. Please let me know if I can help you on debugging it deeper. Thanks a log, Xavi ==16621== 8,097 bytes in 1 blocks are still reachable in loss record 1,050 of 1,053 ==16621== at 0x4C28DBC: malloc (vg_replace_malloc.c:270) ==16621== by 0x6ED6025: dbgmalloc (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x6EC25BF: ??? (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x504E9C9: start_thread (pthread_create.c:300) ==16621== by 0x640021C: clone (clone.S:112) ==16621== ==16621== 8,192 bytes in 2 blocks are still reachable in loss record 1,051 of 1,053 ==16621== at 0x4C28DBC: malloc (vg_replace_malloc.c:270) ==16621== by 0x4350A5: dbgmalloc (in /usr/sbin/rsyslogd) ==16621== by 0x43CC6D: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x415BB0: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x415FC0: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x44F255: actionCallDoAction (in /usr/sbin/rsyslogd) ==16621== by 0x44F80A: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x44FCAA: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x445C3D: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x450463: actionWriteToAction (in /usr/sbin/rsyslogd) ==16621== by 0x450652: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x448918: ??? (in /usr/sbin/rsyslogd) ==16621== ==16621== 12,096 bytes in 18 blocks are still reachable in loss record 1,052 of 1,053 ==16621== at 0x4C27B49: calloc (vg_replace_malloc.c:593) ==16621== by 0x445ACE: qqueueConstruct (in /usr/sbin/rsyslogd) ==16621== by 0x44D5DB: actionConstructFinalize (in /usr/sbin/rsyslogd) ==16621== by 0x44E37F: addAction (in /usr/sbin/rsyslogd) ==16621== by 0x425C46: cflineDoAction (in /usr/sbin/rsyslogd) ==16621== by 0x41EE51: cnfstmtNewLegaAct (in /usr/sbin/rsyslogd) ==16621== by 0x41A567: yyparse (in /usr/sbin/rsyslogd) ==16621== by 0x4295B7: load (in /usr/sbin/rsyslogd) ==16621== by 0x410292: realMain (in /usr/sbin/rsyslogd) ==16621== by 0x6335C4C: (below main) (libc-start.c:226) ==16621== ==16621== 80,000 bytes in 1 blocks are still reachable in loss record 1,053 of 1,053 ==16621== at 0x4C28DBC: malloc (vg_replace_malloc.c:270) ==16621== by 0x4350A5: dbgmalloc (in /usr/sbin/rsyslogd) ==16621== by 0x4443C6: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x445F17: qqueueStart (in /usr/sbin/rsyslogd) ==16621== by 0x40E529: createMainQueue (in /usr/sbin/rsyslogd) ==16621== by 0x4279C4: activate (in /usr/sbin/rsyslogd) ==16621== by 0x41032B: realMain (in /usr/sbin/rsyslogd) ==16621== by 0x6335C4C: (below main) (libc-start.c:226) ==16621== ==16621== LEAK SUMMARY: ==16621== definitely lost: 0 bytes in 0 blocks ==16621== indirectly lost: 0 bytes in 0 blocks ==16621== possibly lost: 864 bytes in 3 blocks ==16621== still reachable: 200,339 bytes in 1,566 blocks ==16621== suppressed: 0 bytes in 0 blocks ==16621== ==16621== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 13 from 7) ==16621== ==16621== 1 errors in context 1 of 3: ==16621== Thread 2: ==16621== Jump to the invalid address stated on the next line ==16621== at 0x0: ??? ==16621== by 0x6ECA6E1: MsgSetInputName (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x6EC1EE4: ??? (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) ==16621== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) ==16621== by 0x504E9C9: start_thread (pthread_create.c:300) ==16621== by 0x640021C: clone (clone.S:112) ==16621== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==16621== --16621-- --16621-- used_suppression: 11 U1004-ARM-_dl_relocate_object --16621-- used_suppression: 2 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a ==16621== ==16621== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 13 from 7) ==16621== could not unlink /tmp/vgdb-pipe-from-vgdb-to-16621-by-root-on-??? ==16621== could not unlink /tmp/vgdb-pipe-to-vgdb-from-16621-by-root-on-??? ==16621== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-16621-by-root-on-??? Killed On 21 February 2013 13:42, Rainer Gerhards <[email protected]> wrote: > On Thu, 2013-02-21 at 13:28 +0100, Xavier Fustero wrote: > > Hiya, > > > > sorry to send many emails. But I realized there were more options in > > valgrind so I executed it again using: > > > ah, you already got the interesting part. > > > --leak-check=full --show-reachable=yes -v > > > > I have attached the output. > > > > Thanks a log, > > Xavi > > > > > > > > > > On 21 February 2013 13:18, Xavier Fustero <[email protected]> wrote: > > > > > Sorry to hassle today with my bug... I added another dump adding the > > > --leak-check=full option. > > > > > > root@rsyslog:/var/tmp# /opt/valgrind/bin/valgrind --leak-check=full > > > /usr/sbin/rsyslogd -n > > > ==31957== Memcheck, a memory error detector > > > ==31957== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et > al. > > > ==31957== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright > info > > > ==31957== Command: /usr/sbin/rsyslogd -n > > > ==31957== > > > rsyslogd: Could no open output pipe '/dev/xconsole': No such file or > > > directory [try http://www.rsyslog.com/e/2039 ] > > > ==31957== Thread 2: > > > ==31957== Jump to the invalid address stated on the next line > > > ==31957== at 0x0: ??? > > > ==31957== by 0x6ECA6E1: MsgSetInputName (in > > > /usr/lib/rsyslog/imuxsock.so) > > would it be possible to compile with debug support and maybe -O0. This > somehow looks a bit like things got mixed up, but I am missing the line > number (which debug build will bring us, at least if the symbols are > installed on that system). You may also use --enable-valgrind with > rsyslog configure, which just prevents unloading of modules, so it has > no effect on the actual code. > > Rainer > > > ==31957== by 0x6EC1EE4: ??? (in /usr/lib/rsyslog/imuxsock.so) > > > ==31957== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) > > > ==31957== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) > > > ==31957== by 0x504E9C9: start_thread (pthread_create.c:300) > > > ==31957== by 0x640021C: clone (clone.S:112) > > > ==31957== Address 0x0 is not stack'd, malloc'd or (recently) free'd > > > ==31957== > > > ==31957== > > > ==31957== Process terminating with default action of signal 11 > (SIGSEGV) > > > ==31957== Bad permissions for mapped region at address 0x0 > > > ==31957== at 0x0: ??? > > > ==31957== by 0x6ECA6E1: MsgSetInputName (in > > > /usr/lib/rsyslog/imuxsock.so) > > > ==31957== by 0x6EC1EE4: ??? (in /usr/lib/rsyslog/imuxsock.so) > > > ==31957== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) > > > ==31957== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) > > > ==31957== by 0x504E9C9: start_thread (pthread_create.c:300) > > > ==31957== by 0x640021C: clone (clone.S:112) > > > ==31957== > > > ==31957== HEAP SUMMARY: > > > ==31957== in use at exit: 201,203 bytes in 1,569 blocks > > > ==31957== total heap usage: 2,228 allocs, 659 frees, 289,130 bytes > > > allocated > > > ==31957== > > > ==31957== Thread 1: > > > ==31957== 288 bytes in 1 blocks are possibly lost in loss record 1,001 > of > > > 1,053 > > > ==31957== at 0x4C27B49: calloc (vg_replace_malloc.c:593) > > > ==31957== by 0x4012495: _dl_allocate_tls (dl-tls.c:300) > > > ==31957== by 0x504F728: pthread_create@@GLIBC_2.2.5 > > > (allocatestack.c:561) > > > ==31957== by 0x441642: wtpAdviseMaxWorkers (in /usr/sbin/rsyslogd) > > > ==31957== by 0x446E62: qqueueEnqObj (in /usr/sbin/rsyslogd) > > > ==31957== by 0x40EF7B: submitMsg (in /usr/sbin/rsyslogd) > > > ==31957== by 0x40F15C: logmsgInternal (in /usr/sbin/rsyslogd) > > > ==31957== by 0x410398: realMain (in /usr/sbin/rsyslogd) > > > ==31957== by 0x6335C4C: (below main) (libc-start.c:226) > > > ==31957== > > > ==31957== 576 bytes in 2 blocks are possibly lost in loss record 1,019 > of > > > 1,053 > > > ==31957== at 0x4C27B49: calloc (vg_replace_malloc.c:593) > > > ==31957== by 0x4012495: _dl_allocate_tls (dl-tls.c:300) > > > ==31957== by 0x504F728: pthread_create@@GLIBC_2.2.5 > > > (allocatestack.c:561) > > > ==31957== by 0x450ACD: thrdCreate (in /usr/sbin/rsyslogd) > > > ==31957== by 0x427A98: activate (in /usr/sbin/rsyslogd) > > > ==31957== by 0x41032B: realMain (in /usr/sbin/rsyslogd) > > > ==31957== by 0x6335C4C: (below main) (libc-start.c:226) > > > ==31957== > > > ==31957== LEAK SUMMARY: > > > ==31957== definitely lost: 0 bytes in 0 blocks > > > ==31957== indirectly lost: 0 bytes in 0 blocks > > > ==31957== possibly lost: 864 bytes in 3 blocks > > > ==31957== still reachable: 200,339 bytes in 1,566 blocks > > > ==31957== suppressed: 0 bytes in 0 blocks > > > ==31957== Reachable blocks (those to which a pointer was found) are not > > > shown. > > > ==31957== To see them, rerun with: --leak-check=full > --show-reachable=yes > > > ==31957== > > > ==31957== For counts of detected and suppressed errors, rerun with: -v > > > ==31957== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 13 from > 7) > > > ==31957== could not unlink > /tmp/vgdb-pipe-from-vgdb-to-31957-by-root-on-??? > > > ==31957== could not unlink > /tmp/vgdb-pipe-to-vgdb-from-31957-by-root-on-??? > > > ==31957== could not unlink > > > /tmp/vgdb-pipe-shared-mem-vgdb-31957-by-root-on-??? > > > Killed > > > > > > > > > Thanks, > > > Xavi > > > > > > > > > > > > > > > On 21 February 2013 12:41, Xavier Fustero <[email protected]> wrote: > > > > > >> Hi Rainer, > > >> > > >> I launched the rsyslog using valgrind: /opt/valgrind/bin/valgrind > > >> /usr/sbin/rsyslogd -n > > >> and then send a message using logger. This is what it dumps: > > >> > > >> ==31904== Memcheck, a memory error detector > > >> ==31904== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et > al. > > >> ==31904== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright > > >> info > > >> ==31904== Command: /usr/sbin/rsyslogd -n > > >> ==31904== > > >> rsyslogd: Could no open output pipe '/dev/xconsole': No such file or > > >> directory [try http://www.rsyslog.com/e/2039 ] > > >> ==31904== Thread 2: > > >> ==31904== Jump to the invalid address stated on the next line > > >> ==31904== at 0x0: ??? > > >> ==31904== by 0x6ECA6E1: MsgSetInputName (in > > >> /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x6EC1EE4: ??? (in /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) > > >> ==31904== by 0x504E9C9: start_thread (pthread_create.c:300) > > >> ==31904== by 0x640021C: clone (clone.S:112) > > >> ==31904== Address 0x0 is not stack'd, malloc'd or (recently) free'd > > >> ==31904== > > >> ==31904== > > >> ==31904== Process terminating with default action of signal 11 > (SIGSEGV) > > >> ==31904== Bad permissions for mapped region at address 0x0 > > >> ==31904== at 0x0: ??? > > >> ==31904== by 0x6ECA6E1: MsgSetInputName (in > > >> /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x6EC1EE4: ??? (in /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x6EC2BD2: ??? (in /usr/lib/rsyslog/imuxsock.so) > > >> ==31904== by 0x450F2E: ??? (in /usr/sbin/rsyslogd) > > >> ==31904== by 0x504E9C9: start_thread (pthread_create.c:300) > > >> ==31904== by 0x640021C: clone (clone.S:112) > > >> ==31904== > > >> ==31904== HEAP SUMMARY: > > >> ==31904== in use at exit: 201,203 bytes in 1,569 blocks > > >> ==31904== total heap usage: 2,228 allocs, 659 frees, 289,130 bytes > > >> allocated > > >> ==31904== > > >> ==31904== LEAK SUMMARY: > > >> ==31904== definitely lost: 0 bytes in 0 blocks > > >> ==31904== indirectly lost: 0 bytes in 0 blocks > > >> ==31904== possibly lost: 864 bytes in 3 blocks > > >> ==31904== still reachable: 200,339 bytes in 1,566 blocks > > >> ==31904== suppressed: 0 bytes in 0 blocks > > >> ==31904== Rerun with --leak-check=full to see details of leaked memory > > >> ==31904== > > >> ==31904== For counts of detected and suppressed errors, rerun with: -v > > >> ==31904== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 13 > from 7) > > >> ==31904== could not unlink > > >> /tmp/vgdb-pipe-from-vgdb-to-31904-by-root-on-??? > > >> ==31904== could not unlink > > >> /tmp/vgdb-pipe-to-vgdb-from-31904-by-root-on-??? > > >> ==31904== could not unlink > > >> /tmp/vgdb-pipe-shared-mem-vgdb-31904-by-root-on-??? > > >> Killed > > >> > > >> > > >> Thanks, > > >> Xavi > > >> > > >> > > >> > > >> > > >> On 21 February 2013 11:18, Rainer Gerhards <[email protected] > >wrote: > > >> > > >>> On Thu, 2013-02-21 at 11:14 +0100, Xavier Fustero wrote: > > >>> > Hi, > > >>> > > > >>> > I would like to ask if there is any Ubuntu expert here who could > help > > >>> me on > > >>> > creating the rsyslog package to have rsyslog 7.2.5 running in my > Ubuntu > > >>> > 10.04. > > >>> > > > >>> > I have created several packages: > > >>> > > > >>> > json-c_0.9-1_amd64.deb > > >>> > libee_0.4.1-1_amd64.deb > > >>> > libestr_0.1.4-1_amd64.deb > > >>> > liblognorm_0.3.5-1_amd64.deb > > >>> > librelp_1.0.1-1_amd64.deb > > >>> > rsyslog_7.2.5-1_amd64.deb > > >>> > > > >>> > Then I have 2 scenarios where one is working and the other no. > > >>> > > > >>> > > > >>> > - *Scenario1:* > > >>> > > > >>> > > > >>> > I have installed all above packages except > rsyslog_7.2.5-1_amd64.deb. > > >>> Then > > >>> > I have installed any extra required package, downloaded the > rsyslog and > > >>> > compiled it in this box: > > >>> > > > >>> > apt-get install uuid-dev > > >>> > > > >>> > cd rsyslog-7.2.5 > > >>> > > > >>> > ./configure --libdir=/usr/lib --includedir=/usr/include > > >>> > --enable-gssapi-krb5 --enable-kmsg --enable-debug --enable-memcheck > > >>> > --enable-diagtools --enable-usertools --enable-mysql > > >>> --enable-elasticsearch > > >>> > --enable-gnutls --enable-mmnormalize --enable-mmjsonparse > --enable-relp > > >>> > --enable-imfile --enable-imptcp --enable-impstats > --enable-pmlastmsg > > >>> > > > >>> > This seems to be working for me. If I launch the rsyslog by hand > > >>> > /usr/local/sbin/rsyslogd the PrivDrop also seems to be working > fine. It > > >>> > takes a while for the root process to disappear but it does. > > >>> > > > >>> > > > >>> > > > >>> > - *Scenario 2* > > >>> > > > >>> > > > >>> > This is the desired scenario as I am building server through > templates > > >>> and > > >>> > I would like just to install packages at boot time. > > >>> > I have installed all above packages. I don't know why but my > rsyslog > > >>> > package produces a segmentation fault every time I send a message. > I > > >>> have > > >>> > created it according the above configuration parameters. > > >>> > > > >>> > But when I launch it in debug mode I get the last line when I get a > > >>> message: > > >>> > > > >>> > ... > > >>> > 6250.465837291:7fbc779ce700: regular consumer finished, iret=4, > szlog > > >>> > 0 sz phys 0 > > >>> > 6250.465841734:7fbc779ce700: main Q:Reg/w0: worker IDLE, waiting > for > > >>> work. > > >>> > 6253.511342997:7fbc799d2700: Message from UNIX socket: #3 > > >>> > Segmentation fault > > >>> > > > >>> can you run this under valgrind? It hopefully points to the actual > > >>> proiblem location. Make sure symbols are not stripped during > > >>> compile/link. > > >>> > > >>> Rainer > > >>> > I might miss something during the package creation because I am not > > >>> able to > > >>> > make a solid package for the rsyslog. > > >>> > Does anyone have any hint on how to fix this? > > >>> > > > >>> > Thanks a log, > > >>> > Xavi > > >>> > _______________________________________________ > > >>> > rsyslog mailing list > > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> > http://www.rsyslog.com/professional-services/ > > >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > >>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT > POST if > > >>> you DON'T LIKE THAT. > > >>> > > >>> _______________________________________________ > > >>> rsyslog mailing list > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> http://www.rsyslog.com/professional-services/ > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > >>> DON'T LIKE THAT. > > >>> > > >> > > >> > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
