I've been running the latest 8.20.0 RPMs from the "testing" repo for the last couple of days and rsyslog crashed several times with the following dmesg error: in:imfile[23319] trap divide error ip:7f179bec15ca sp:7f1794dad140 error:0 in rsyslogd[7f179be8c000+93000]
Any ideas on how to troubleshoot this? Thanks, Alec On Fri, Jul 29, 2016 at 6:37 AM, Rainer Gerhards <[email protected]> wrote: > David, > > if I have the debian control file like this: > > ========================= > Source: libfastjson > Priority: extra > Maintainer: Florian Riedl <[email protected]> > Build-Depends: cdbs, debhelper (>= 7), autotools-dev, pkg-config, > dh-apparmor, dh-autoreconf > Standards-Version: 3.9.2 > Section: libs > Homepage: https://github.com/rsyslog/libfastjson > #Vcs-Git: git://git.debian.org/collab-maint/libestr.git > #Vcs-Browser: http://git.debian.org/?p=collab-maint/libestr.git;a=summary > > Package: libfastjson-dev > Section: libdevel > Architecture: any > Depends: libfastjson4 (= ${binary:Version}), ${misc:Depends} > Replaces: libfastjson-dev > Breaks: libfastjson-dev > Description: Development files for the 'libfastjson' package > The libfastjson-dev package contains the header files and libraries > needed to develop applications using libksi. > . > This package contains the development files. > > Package: libfastjson4 > Section: libs > Architecture: any > Depends: ${shlibs:Depends}, ${misc:Depends} > Description: LibFastJSON > The 'libfastjson' library contains the > a modified json-c library with better performance > . > This package contains the shared library. > ================= > > libfastjson builds ok on trusty, but I reveceive this error on the > launchpad build: > > libtool: link: gcc -std=gnu99 -shared -fPIC -DPIC > .libs/liblognorm_la-liblognorm.o .libs/liblognorm_la-pdag.o > .libs/liblognorm_la-annot.o .libs/liblognorm_la-samp.o > .libs/liblognorm_la-lognorm.o .libs/liblognorm_la-parser.o > .libs/liblognorm_la-enc_syslog.o .libs/liblognorm_la-enc_csv.o > .libs/liblognorm_la-enc_xml.o .libs/liblognorm_la-v1_liblognorm.o > .libs/liblognorm_la-v1_parser.o .libs/liblognorm_la-v1_ptree.o > .libs/liblognorm_la-v1_samp.o -lfastjson -lestr -O2 -Wl,-soname > -Wl,liblognorm.so.5 -o .libs/liblognorm.so.5.0.0 > /usr/bin/ld.bfd.real: > > /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/libfastjson.a(libfastjson_la-json_object.o): > relocation R_X86_64_32 against `.rodata.str1.1' can not be used when > making a shared object; recompile with -fPIC > /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/libfastjson.a: error > adding symbols: Bad value > > > The same problem happens on xenial, but not yakkety. > > > If I now just change "libfastjson4" to "libfastjson" (number 4 removed > from "libfastjson4", two occurences) in control, both the libfastjson > and liblognorm builds succeed. No other change is done to any file or > the build process itself. > > This is where I am totally stuck. I just thought I give it a re-try > again as I was able to brush up some configure macros. > > If anyone has an explanation why this happens, I would be grateful to > learn the reason. > > Rainer > > 2016-07-27 19:03 GMT+02:00 Rainer Gerhards <[email protected]>: > > As I said, I tried for almost two weeks with different settings, for sure > > more than 100 tries. This is just the latest state. In the end I focused > on > > trusty, to save some time. If we can get that pic error go away, we can > look > > at the other ones. > > > > Rainer > > > > Sent from phone, thus brief. > > > > > > Am 27.07.2016 18:44 schrieb "David Lang" <[email protected]>: > >> > >> On Wed, 27 Jul 2016, Rainer Gerhards wrote: > >> > >>> 2016-07-27 16:41 GMT+02:00 David Lang <[email protected]>: > >>>> > >>>> On Wed, 27 Jul 2016, Rainer Gerhards wrote: > >>>> > >>>>> That's unfortunately not the case for the Ubuntu repo. I always > >>>>> consistently recompiled everything. Actually the problem occurs > during > >>>>> that > >>>>> process, when e.g. liblognorm is compiled. > >>>> > >>>> > >>>> > >>>> I am only building for trusty, but I am using the rsyslog-pkg-ubuntu > >>>> scripts > >>>> to do the compiles, everything up to the point of uploading the > results > >>>> to > >>>> the PPA > >>>> > >>>> are you getting the failure on trusty, or on other versions? > >>> > >>> > >>> I get them on trusty and also on other versions. See here: > >>> > >>> https://launchpad.net/~adiscon/+archive/ubuntu/v8-devel/+packages > >>> > >>> I have the impression that the exact versions which fail change, but > >>> trusty seems always affected. > >> > >> > >> what I am seeing at that link is an error that it wasn't able to install > >> libfastjson-dev (exact message: Depends: libfastjson-dev but it is not > >> installable) > >> > >> hmm, digging further, it looks like the trusty amd64 build errors out > with > >> the -fPIC message, but all the other errors I see from the 22nd are > >> uninstallable libfastjson-dev > >> > >> and indeed, there is not a libfastjson package for xenial or wily > >> > >> have you tried rebuilding libfastjson just in case something went wrong > >> there? > >> > >> David Lang > >> > >>> Rainer > >>>> > >>>> > >>>> David Lang > >>>> > >>>>> Rainer > >>>>> > >>>>> Sent from phone, thus brief. > >>>>> > >>>>> Am 27.07.2016 12:06 schrieb "David Lang" <[email protected]>: > >>>>> > >>>>>> On Wed, 27 Jul 2016, Rainer Gerhards wrote: > >>>>>> > >>>>>> > >>>>>>> Thanks for the elaborate output. I think we may be getting closer. > >>>>>>> This triggered my attention: > >>>>>>> > >>>>>>> --> Processing Dependency: libjson-c.so.2()(64bit) for package: > >>>>>>> liblognorm1-1.1.3-1.el6.x86_64 > >>>>>>> > >>>>>>> I think we need to rebuild liblognorm with the new libfastjson > >>>>>>> package. Otherwise we can see subtle problems with the json > handlers, > >>>>>>> as there core structures are different (much more in 0.99.3 than > >>>>>>> 0.99.2 of libfastjson). This doesn't explain yet the line number > >>>>>>> problems we saw, but in anycase that's something to look at. I will > >>>>>>> let you know when we have updates in the repo. > >>>>>>> > >>>>>> > >>>>>> Yes, as I have been building with the git master for libfastjson and > >>>>>> liblognorm for several months, I found that if I didn't recompile > >>>>>> everything at once to be on the new version of libraries that both > >>>>>> used, > >>>>>> bad things happened. > >>>>>> > >>>>>> I've been suspecting that this sort of skew has been a large part of > >>>>>> the > >>>>>> repo problems of the last few weeks as new versions of libraries > have > >>>>>> been > >>>>>> released at different times than rsyslog packages. I've been > >>>>>> suspecting > >>>>>> that if everything was recompiled, things would work (so 8.19.0 > >>>>>> packages > >>>>>> break with the new libraries, but I expected 8.20 packages compiled > >>>>>> against > >>>>>> the new libraries to then work) > >>>>>> > >>>>>> David Lang > >>>>>> > >>>>>> In parallel, I also install a CentOS6 (don't have RHEL) machine and > >>>>>>> > >>>>>>> > >>>>>>> see what exactly it installes when we pull from the testing repo. > >>>>>>> > >>>>>>> Thanks again, > >>>>>>> Rainer > >>>>>>> > >>>>>>> 2016-07-27 1:27 GMT+02:00 Alec Swan <[email protected]>: > >>>>>>> > >>>>>>>> I configured /etc/yum.repos.d/rsyslog.repo to point to the > "testing" > >>>>>>>> rsyslog repo and removed all other yum repos. After that I > >>>>>>>> downloaded > >>>>>>>> all > >>>>>>>> RPMS from the "testing" repo using yumdownloader and installed > them > >>>>>>>> with > >>>>>>>> "rpm -ivh". The end result is the same. The following shows the > >>>>>>>> output > >>>>>>>> of > >>>>>>>> the test. > >>>>>>>> > >>>>>>>> # more /etc/yum.repos.d/rsyslog.repo > >>>>>>>> [rsyslog_v8] > >>>>>>>> name=Adiscon CentOS-$releasever - local packages for $basearch > >>>>>>>> baseurl= > http://rpms.adiscon.com/testing/epel-$releasever/$basearch > >>>>>>>> enabled=1 > >>>>>>>> gpgcheck=0 > >>>>>>>> gpgkey=http://rpms.adiscon.com/RPM-GPG-KEY-Adiscon > >>>>>>>> protect=1 > >>>>>>>> > >>>>>>>> # yumdownloader --resolve rsyslog-mmnormalize-8.20.0 > >>>>>>>> rsyslog-elasticsearch-8.20.0 rsyslog-mmutf8fix-8.20.0 > >>>>>>>> Loaded plugins: fastestmirror > >>>>>>>> Loading mirror speeds from cached hostfile > >>>>>>>> ... > >>>>>>>> rsyslog_v8 > >>>>>>>> > >>>>>>>> | 2.5 kB 00:00 > >>>>>>>> rsyslog_v8/primary_db > >>>>>>>> > >>>>>>>> | 264 kB 00:00 > >>>>>>>> --> Running transaction check > >>>>>>>> ---> Package rsyslog-elasticsearch.x86_64 0:8.20.0-1.el6 will be > >>>>>>>> installed > >>>>>>>> --> Processing Dependency: rsyslog = 8.20.0-1.el6 for package: > >>>>>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64 > >>>>>>>> ---> Package rsyslog-mmnormalize.x86_64 0:8.20.0-1.el6 will be > >>>>>>>> installed > >>>>>>>> --> Processing Dependency: liblognorm.so.2()(64bit) for package: > >>>>>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64 > >>>>>>>> ---> Package rsyslog-mmutf8fix.x86_64 0:8.20.0-1.el6 will be > >>>>>>>> installed > >>>>>>>> --> Running transaction check > >>>>>>>> ---> Package liblognorm1.x86_64 0:1.1.3-1.el6 will be installed > >>>>>>>> --> Processing Dependency: libjson-c.so.2()(64bit) for package: > >>>>>>>> liblognorm1-1.1.3-1.el6.x86_64 > >>>>>>>> ---> Package rsyslog.x86_64 0:8.20.0-1.el6 will be installed > >>>>>>>> --> Processing Dependency: libfastjson.so.4()(64bit) for package: > >>>>>>>> rsyslog-8.20.0-1.el6.x86_64 > >>>>>>>> --> Running transaction check > >>>>>>>> ---> Package json-c.x86_64 0:0.11-11.el6 will be installed > >>>>>>>> ---> Package libfastjson4.x86_64 0:0.99.3-2.el6 will be installed > >>>>>>>> --> Finished Dependency Resolution > >>>>>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 13 kB 00:00 > >>>>>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 24 kB 00:00 > >>>>>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 12 kB 00:00 > >>>>>>>> rsyslog-8.20.0-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 690 kB 00:00 > >>>>>>>> json-c-0.11-11.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 51 kB 00:00 > >>>>>>>> libfastjson4-0.99.3-2.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 46 kB 00:00 > >>>>>>>> liblognorm1-1.1.3-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> | 46 kB 00:00 > >>>>>>>> > >>>>>>>> # ls > >>>>>>>> json-c-0.11-11.el6.x86_64.rpm > >>>>>>>> liblognorm1-1.1.3-1.el6.x86_64.rpm > >>>>>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64.rpm > >>>>>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64.rpm > >>>>>>>> libfastjson4-0.99.3-2.el6.x86_64.rpm > >>>>>>>> rsyslog-8.20.0-1.el6.x86_64.rpm > >>>>>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64.rpm > >>>>>>>> > >>>>>>>> # rpm -ivh *.rpm > >>>>>>>> warning: libfastjson4-0.99.3-2.el6.x86_64.rpm: Header V3 RSA/SHA1 > >>>>>>>> Signature, key ID e00b8985: NOKEY > >>>>>>>> Preparing... > >>>>>>>> ########################################### > >>>>>>>> [100%] > >>>>>>>> 1:libfastjson4 > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 14%] > >>>>>>>> 2:rsyslog > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 29%] > >>>>>>>> 3:json-c > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 43%] > >>>>>>>> 4:liblognorm1 > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 57%] > >>>>>>>> 5:rsyslog-mmnormalize > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 71%] > >>>>>>>> 6:rsyslog-elasticsearch > >>>>>>>> ########################################### > >>>>>>>> [ > >>>>>>>> 86%] > >>>>>>>> 7:rsyslog-mmutf8fix > >>>>>>>> ########################################### > >>>>>>>> [100%] > >>>>>>>> > >>>>>>>> # service rsyslog start > >>>>>>>> Starting system logger: [ OK > ] > >>>>>>>> > >>>>>>>> # ps -ax | grep rsys > >>>>>>>> Warning: bad syntax, perhaps a bogus '-'? See > >>>>>>>> /usr/share/doc/procps-3.2.8/FAQ > >>>>>>>> 1020 pts/0 S+ 0:00 grep rsys > >>>>>>>> > >>>>>>>> # dmesg > >>>>>>>> ... > >>>>>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp > >>>>>>>> 00007f799e66f7e8 > >>>>>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] > >>>>>>>> > >>>>>>>> # service rsyslog start > >>>>>>>> Starting system logger: [ OK > ] > >>>>>>>> > >>>>>>>> # dmesg > >>>>>>>> ... > >>>>>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp > >>>>>>>> 00007f799e66f7e8 > >>>>>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] > >>>>>>>> rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp > >>>>>>>> 00007f9a3c1de878 > >>>>>>>> error 6 in libfastjson.so.4.0.0[7f9a44246000+9000] > >>>>>>>> > >>>>>>>> # service rsyslog start > >>>>>>>> Starting system logger: [ OK > ] > >>>>>>>> > >>>>>>>> # dmesg > >>>>>>>> ... > >>>>>>>> rs:main Q:Reg[1009]: segfault at 0 ip 00007f79a5e4d2b6 sp > >>>>>>>> 00007f799e66f7e8 > >>>>>>>> error 4 in libc-2.12.so[7f79a5d25000+18a000] > >>>>>>>> rs:main Q:Reg[1063]: segfault at 1b ip 00007f9a44249778 sp > >>>>>>>> 00007f9a3c1de878 > >>>>>>>> error 6 in libfastjson.so.4.0.0[7f9a44246000+9000] > >>>>>>>> rs:main Q:Reg[1093]: segfault at 0 ip 00007f32e00c22b6 sp > >>>>>>>> 00007f32d88e47e8 > >>>>>>>> error 4 in libc-2.12.so[7f32dff9a000+18a000] > >>>>>>>> > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Alec > >>>>>>>> > >>>>>>>> On Tue, Jul 26, 2016 at 10:18 AM, Rainer Gerhards < > >>>>>>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Am 26.07.2016 17:58 schrieb "Alec Swan" <[email protected]>: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> I downloaded the packages from adiscon "testing" repo, then > >>>>>>>>>> uploaded > >>>>>>>>>> them > >>>>>>>>>> to our local Nexus repo and then installed them with yum. So, > the > >>>>>>>>>> source > >>>>>>>>>> > >>>>>>>>> is > >>>>>>>>> > >>>>>>>>>> our local Nexus repo. > >>>>>>>>>> > >>>>>>>>>> Unfortunately, the "testing" repo does not seem to have libee.so > >>>>>>>>>> > >>>>>>>>> dependency > >>>>>>>>> > >>>>>>>>>> required by libfastjson, so I can't just use "testing" repo to > >>>>>>>>>> install > >>>>>>>>>> > >>>>>>>>> all > >>>>>>>>> > >>>>>>>>>> rsyslog dependencies. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> Something is fishy here. I just checked the spec file, and > >>>>>>>>> libfastjson > >>>>>>>>> does > >>>>>>>>> not have a dependency on libee. Is there a problem with the > staging > >>>>>>>>> process? Or can you try from the original repo, not the mirror? > >>>>>>>>> > >>>>>>>>> Rainer > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> Alec > >>>>>>>>>> > >>>>>>>>>> On Tue, Jul 26, 2016 at 9:36 AM, Rainer Gerhards < > >>>>>>>>>> > >>>>>>>>> [email protected] > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> 2016-07-26 17:35 GMT+02:00 Alec Swan <[email protected]>: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Rainer, per Florian's request in "experimental rpms for > >>>>>>>>>>>> > >>>>>>>>>>> rsyslog-8.20.0" I > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> downloaded RPMs from the "testing" repo baseurl= > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> http://rpms.adiscon.com/testing/epel-$releasever/$basearch > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Can you check where the system says it actually installed them > >>>>>>>>>>> from. > >>>>>>>>>>> The line numbers do not match the RPMs Florian created, thus I > >>>>>>>>>>> ask. > >>>>>>>>>>> > >>>>>>>>>>> Rainer > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> > >>>>>>>>>>>> Alec > >>>>>>>>>>>> > >>>>>>>>>>>> On Tue, Jul 26, 2016 at 7:30 AM, Rainer Gerhards < > >>>>>>>>>>>> > >>>>>>>>>>> [email protected]> > >>>>>>>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> I today tried quite a while to reproduce the issue with your > >>>>>>>>>>>> config, > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> unfortunately without any success. For now, I conclude that > >>>>>>>>>>>>> this > >>>>>>>>>>>>> is > >>>>>>>>>>>>> not a general problem, but one induced by your environment > and > >>>>>>>>>>>>> data. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I have then looked again at the traces and the few line > numbers > >>>>>>>>>>>>> it > >>>>>>>>>>>>> contains. Now comes the interesting fact. I notice that there > >>>>>>>>>>>>> is a > >>>>>>>>>>>>> mismatch between the line numbers shown and the code that > >>>>>>>>>>>>> actually > >>>>>>>>>>>>> > >>>>>>>>>>>> was > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> used to build the packages (judging from git and the source > RPMs). > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> So > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> I wonder what might have gone wrong here. Can you please let me > >>>>>>>>>> know > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> which exact repro you installed from and potentially which > >>>>>>>>>>>>> other > >>>>>>>>>>>>> > >>>>>>>>>>>> repos > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> you have configured on that machine. It might also be useful to > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> query > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> the system from where exactly the packages have ben installed. I > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> found > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> [1], which hopefully helps to get that information. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Once I have this information, I can check if there are any > >>>>>>>>>>>>> version > >>>>>>>>>>>>> quirks (note that commit [2] fixed a situation that could > cause > >>>>>>>>>>>>> exactly what you see). If there is no issue with the package > >>>>>>>>>>>>> > >>>>>>>>>>>> sources, > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> I unfortunately need your help in further troubleshooting this > in > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> your > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> environment. But let's first see what the repo check will bring > >>>>>>>>>> up. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Rainer > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > http://serverfault.com/questions/62026/how-to-know-from-which-yum-repository-a-package-has-been-installed > >>>>>>>>> > >>>>>>>>>> [2] > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > https://github.com/rsyslog/libfastjson/commit/0499ec8df09c913156ead426c171031b566dbe7d > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>>>> 2016-07-23 19:28 GMT+02:00 Alec Swan <[email protected]>: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Yes, that should be OK. Is there a link to the build > server/CI > >>>>>>>>>>>>>> env > >>>>>>>>>>>>>> > >>>>>>>>>>>>> I > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> can > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> use? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Alec > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Sat, Jul 23, 2016 at 7:08 AM, Rainer Gerhards < > >>>>>>>>>>>>>> > >>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks. Would it be OK for your if a somewhat mangled > version > >>>>>>>>>>>>>> of > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> these > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> configs show up inside the testbench? I think about adding a > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> specific > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> test for it, that would be great for the future as well. I'd > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> probably > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> try to strip it a bit down (e.g. no ES but writing to file, > which > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> would enable runs on all CI environments and when no ES is > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> available > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> -- I assume this would also trigger the bug). > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Please let me know, > >>>>>>>>>>>>>>> Rainer > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> 2016-07-22 17:46 GMT+02:00 Alec Swan <[email protected]>: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Rainer, I attached /etc/rsyslog.conf and > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> /etc/rsyslog.d/rsyslog-custom.conf > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Alec > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Fri, Jul 22, 2016 at 12:42 AM, Rainer Gerhards < > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> unfortunately, we do not seem to have debug symbols, so I > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> don't > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> see > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> much more. Can you share your full config (including all > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> includes, if > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> any)? Maybe I can get it to abort in my lab as well. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Rainer > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 2016-07-21 22:02 GMT+02:00 Alec Swan <[email protected] > >: > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Rainer, here is the output from running rsyslog under > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> valgrind: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>>>>>>>>> [root valgrind-3.11.0]# valgrind rsyslogd > >>>>>>>>>>>>>>>>>> ==2371== Memcheck, a memory error detector > >>>>>>>>>>>>>>>>>> ==2371== Copyright (C) 2002-2015, and GNU GPL'd, by > Julian > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Seward > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> et > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> al. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> ==2371== Using Valgrind-3.11.0 and LibVEX; rerun with -h > for > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> copyright > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> info > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== Command: rsyslogd > >>>>>>>>>>>>>>>>>> ==2371== > >>>>>>>>>>>>>>>>>> ==2371== Conditional jump or move depends on > uninitialised > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> value(s) > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> ==2371== at 0x16A414: cnfexprDestruct (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== by 0x16B36D: cnfstmtOptimize (in > >>>>>>>>>>>>>>>>>> /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2371== by 0x146317: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2371== by 0x12E5BE: llExecFunc (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2371== by 0x146176: rulesetOptimizeAll (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /sbin/rsyslogd) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2371== by 0x123EF6: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== by 0x158A68: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2371== by 0x159008: main (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2371== > >>>>>>>>>>>>>>>>>> ==2375== Warning: invalid file descriptor 99990 in > syscall > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> close() > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> ==2371== > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== HEAP SUMMARY: > >>>>>>>>>>>>>>>>>> ==2371== in use at exit: 209,122 bytes in 2,121 > blocks > >>>>>>>>>>>>>>>>>> ==2371== total heap usage: 11,004 allocs, 8,883 frees, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 4,319,525 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> bytes > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> allocated > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== > >>>>>>>>>>>>>>>>>> ==2371== LEAK SUMMARY: > >>>>>>>>>>>>>>>>>> ==2371== definitely lost: 26 bytes in 1 blocks > >>>>>>>>>>>>>>>>>> ==2371== indirectly lost: 0 bytes in 0 blocks > >>>>>>>>>>>>>>>>>> ==2371== possibly lost: 88 bytes in 2 blocks > >>>>>>>>>>>>>>>>>> ==2371== still reachable: 209,008 bytes in 2,118 > blocks > >>>>>>>>>>>>>>>>>> ==2371== suppressed: 0 bytes in 0 blocks > >>>>>>>>>>>>>>>>>> ==2371== Rerun with --leak-check=full to see details of > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> leaked > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> memory > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ==2371== > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== For counts of detected and suppressed errors, > >>>>>>>>>>>>>>>>>> rerun > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> with: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> -v > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ==2371== Use --track-origins=yes to see where uninitialised > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> values > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> come > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2371== ERROR SUMMARY: 1 errors from 1 contexts > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (suppressed: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 114 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> from 7) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> [root valgrind-3.11.0]# ==2375== Thread 6 rs:main Q:Reg: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== Invalid read of size 8 > >>>>>>>>>>>>>>>>>> ==2375== at 0x56607BE: fjson_object_iter_next > >>>>>>>>>>>>>>>>>> (json_object_iterator.c:119) > >>>>>>>>>>>>>>>>>> ==2375== by 0x5660863: fjson_object_iter_begin > >>>>>>>>>>>>>>>>>> (json_object_iterator.c:77) > >>>>>>>>>>>>>>>>>> ==2375== by 0x126F09: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== Address 0xf57d338 is 0 bytes after a block of > >>>>>>>>>>>>>>>>>> size > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 72 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> alloc'd > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> ==2375== at 0x4A05FEF: calloc (vg_replace_malloc.c:711) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x7659871: json_object_new > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (json_object.c:178) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x765A61A: json_object_new_object > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (json_object.c:354) > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ==2375== by 0x744A355: ln_normalize (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /usr/lib64/liblognorm.so.2.0.0) > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x7242EEB: ??? (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> ==2375== Invalid read of size 4 > >>>>>>>>>>>>>>>>>> ==2375== at 0x5660778: fjson_object_get > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (json_object.c:176) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x126F57: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x13FB5D: wtiWorker (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== Address 0x60000001e is not stack'd, malloc'd > or > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (recently) > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> free'd > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> ==2375== Process terminating with default action of > signal > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 11 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> (SIGSEGV) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> ==2375== Access not within mapped region at address > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 0x60000001E > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== at 0x5660778: fjson_object_get > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (json_object.c:176) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x126F57: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x1270A0: msgAddJSON (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x7242F2D: ??? (in > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /lib64/rsyslog/mmnormalize.so) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== by 0x14EA64: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F561: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x14F923: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x146AB6: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1473DA: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x156FE3: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x1457D8: ??? (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== by 0x13FB5D: wtiWorker (in /sbin/rsyslogd) > >>>>>>>>>>>>>>>>>> ==2375== If you believe this happened as a result of a > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> stack > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== overflow in your program's main thread (unlikely > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> but > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== possible), you can try to increase the size of the > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== main thread stack using the --main-stacksize= > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> flag. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== The main thread stack size used in this run was > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 10485760. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== HEAP SUMMARY: > >>>>>>>>>>>>>>>>>> ==2375== in use at exit: 2,611,014 bytes in 11,732 > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> blocks > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== total heap usage: 27,013 allocs, 15,281 frees, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 8,524,299 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> bytes > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> allocated > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> ==2375== LEAK SUMMARY: > >>>>>>>>>>>>>>>>>> ==2375== definitely lost: 4,117 bytes in 11 blocks > >>>>>>>>>>>>>>>>>> ==2375== indirectly lost: 24,922 bytes in 20 blocks > >>>>>>>>>>>>>>>>>> ==2375== possibly lost: 4,464 bytes in 12 blocks > >>>>>>>>>>>>>>>>>> ==2375== still reachable: 2,577,511 bytes in 11,689 > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> blocks > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> ==2375== suppressed: 0 bytes in 0 blocks > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== Rerun with --leak-check=full to see details of > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> leaked > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> memory > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ==2375== > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== For counts of detected and suppressed errors, > >>>>>>>>>>>>>>>>>> rerun > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> with: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> -v > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> ==2375== Use --track-origins=yes to see where uninitialised > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> values > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> come > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> ==2375== ERROR SUMMARY: 5 errors from 3 contexts > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> (suppressed: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 120 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> from 7) > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> [root valgrind-3.11.0]# rsyslogd -v > >>>>>>>>>>>>>>>>>> rsyslogd 8.20.0, compiled with: > >>>>>>>>>>>>>>>>>> PLATFORM: x86_64-redhat-linux-gnu > >>>>>>>>>>>>>>>>>> PLATFORM (lsb_release -d): > >>>>>>>>>>>>>>>>>> FEATURE_REGEXP: Yes > >>>>>>>>>>>>>>>>>> GSSAPI Kerberos 5 support: No > >>>>>>>>>>>>>>>>>> FEATURE_DEBUG (debug build, slow code): No > >>>>>>>>>>>>>>>>>> 32bit Atomic operations supported: Yes > >>>>>>>>>>>>>>>>>> 64bit Atomic operations supported: Yes > >>>>>>>>>>>>>>>>>> memory allocator: system default > >>>>>>>>>>>>>>>>>> Runtime Instrumentation (slow code): No > >>>>>>>>>>>>>>>>>> uuid support: Yes > >>>>>>>>>>>>>>>>>> Number of Bits in RainerScript integers: 64 > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> See http://www.rsyslog.com for more information. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> [root valgrind-3.11.0]# dmesg > >>>>>>>>>>>>>>>>>> .. > >>>>>>>>>>>>>>>>>> rs:main Q:Reg[1776]: segfault at 65 ip 00007f0769bcc2b6 > sp > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> 00007f0762de87e8 > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> error 4 in libc-2.12.so[7f0769aa4000+18a000] > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Alec > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 11:32 PM, Rainer Gerhards < > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> 2016-07-20 23:02 GMT+02:00 Alec Swan < > [email protected]>: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> I used test RPMs to upgrade from 8.19.0 to 8.20.0. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> However, > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> now > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> when I > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> start rsyslog it logs the following error message to > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> dmesg > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> and > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> shuts > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> rsyslog down: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> "rs:main Q:Reg[23801]: segfault at 1a ip > >>>>>>>>>>>>>>>>>>>> 00007fa95ac9f778 > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> sp > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 00007fa94f5fd878 error 6 in > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> libfastjson.so.4.0.0[7fa95ac9c000+9000]" > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> These are the packages that were installed/updated: > >>>>>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Installed: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> libfastjson4-0.99.3-2.el6.x86_64 > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> rsyslog-8.20.0-1.el6.x86_64 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> rsyslog-mmutf8fix-8.20.0-1.el6.x86_64 > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> rsyslog-mmnormalize-8.20.0-1.el6.x86_64 > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Jul 20 20:53:11 yum[22693]: Updated: > >>>>>>>>>>>>>>>>>>>> rsyslog-elasticsearch-8.20.0-1.el6.x86_64 > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> would it be possible that you do a run under valgrind > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> control? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Rainer > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Alec > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 12:11 PM, Michael Biebl < > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> [email protected]> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> 2016-07-12 23:12 GMT+02:00 David Lang <[email protected] > >: > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On Tue, 12 Jul 2016, Michael Biebl wrote: > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Fwiw, this is the first release I felt comfortable > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> enough > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> uploading > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Debian unstable. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> It just hit the archive a few hours ago: > >>>>>>>>>>>>>>>>>>>>>>> https://tracker.debian.org/pkg/libfastjson > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> I hope we don't break the API that much anymore > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> before > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 1.0.0. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> I'm now waiting for a liblognorm 1.1.4 release > which > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> compiles > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> against > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> libfastjson 0.99.3. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> My understanding is that we are going to get > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> liblognorm 2 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> this > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> cycle > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> (after > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> being delayed for the last couple of cycles) > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Ok, so what's the plan here? Which liblognorm version > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> should I > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> pick? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> At least they currently released version doesn't compile > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> against > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> latest libfastjson release (which is not a surprise, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> after > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> the > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> API > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> rework, which I'm thankful for). > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>>>>>>>>> Michael > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>> Why is it that all of the instruments seeking > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> intelligent > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> life > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> in > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> the > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> universe are pointed away from Earth? > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>>>>>>>>> rsyslog mailing list > >>>>>>>>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>>>>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ > >>>>>>>>>>>>>>>>>>>>> What's up with rsyslog? Follow > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> https://twitter.com/rgerhards > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> ARCHIVED > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> by a > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> myriad > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> NOT > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> POST if > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> you > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> DON'T LIKE THAT. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> _______________________________________________ > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> rsyslog mailing list > >>>>>>>>>>>>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>>>>>>>>>>>>>>>> http://www.rsyslog.com/professional-services/ > >>>>>>>>>>>>>>>>>>>> What's up with rsyslog? Follow > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> https://twitter.com/rgerhards > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> ARCHIVED > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> by > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> a > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> myriad > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO > NOT > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> POST if > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> you > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> DON'T LIKE THAT. > >> > >> ... > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
