I haven't run it with valgrind yet... honestly I don't think I've used valgrind more than once or twice, I'd need to do a bit of reading up to remember how to use it.
This isn't a matter of *if* the issue reappears. Every time rsyslogd tries to write out a message matching a dynafile output, it dies (within a few ms of starting, or whenever it actually tries to write the first line out to disk). At the moment, I have the issue localized to the call to getClockFileAccess() on line 517 of tools/omfile.c, and am continuing to trace it through the code. I'll see if one of the more c-skilled guys can give me a hand with valgrind. -Jason Rainer Gerhards wrote: > An you run it under valgrind control? If the issue reappears, it should give > us *very* good hints of where to look at... > > RAiner > > >> -----Original Message----- >> From: [email protected] [mailto:rsyslog- >> [email protected]] On Behalf Of Jason Antman >> Sent: Friday, November 05, 2010 5:26 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] segfault with dynamic filenames >> >> Thanks for the info! >> >> Just sent this to Tom, forgot to CC the list.... >> >> I just rebuilt the RPM with all debugging features enabled and am >> experiencing the same issue. >> >> I have: >> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env variable >> set to "LogFuncFlow FileTrace=omfile.c Debug") >> - strace output >> - RPM spec file including configure options, etc. >> - the Fedora patch that is being applied >> - the actual RPMs, if they're of interest, and a source rpm >> >> Which of this is of use? Should I email it? Open a ticket and attach? >> >> Here are the last few lines before the crash: >> 2954.318300688:b7f06b90: Processing next action >> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to builtin- >> file >> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin-file >> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter >> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 >> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx >> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx >> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter >> 2954.318418085:b7f06b90: file to log to: LocalHost >> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter >> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter >> (and then it dies, in the middle of a select() call) >> >> I'm going to start looking through omfile.c and perhaps adding some of >> my own instrumentation just so I can localize it a bit more. >> >> Thanks for any advice you can provide, >> Jason Antman >> >> Tom Bergfeld wrote: >> >>> Hi Jason, >>> >>> unfortunately we were not able to reproduce your issue in Fedora 13. >>> We will create an environment with centos and try it again. >>> >>> Tom >>> >>> >>> -----Ursprüngliche Nachricht----- >>> Von: [email protected] >>> [mailto:[email protected]] Im Auftrag von Jason >>> >> Antman >> >>> Gesendet: Donnerstag, 4. November 2010 22:47 >>> An: rsyslog-users >>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog >>> evaluationquestions) >>> >>> Thanks to both of you guys for your answers. FYI, our days have 24 >>> >> hours >> >>> in them :) >>> >>> At this point I just got a config file worked up. I got the regexes >>> working to do all of the matching I need into separate fields (the >>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS >>> >> 5.5, >> >>> in-house RPM build) in order to of bind imudp to a ruleset (so we can >>> have one ruleset for localhost/unix socket, and one ruleset for all >>> remote hosts). >>> >>> Unfortunately, I'm getting a segfault almost immediately on startup >>> >> due >> >>> to the following lines: >>> $template >>> >>> >> RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa >> cility >> >>> -text%.log" >>> *.* ?RemoteHost >>> >>> I believe I've narrowed the problem down to any templated (dynamic) >>> filename... even if I dump everything external directly to >>> /var/log/remote and add the following in my local ruleset: >>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" >>> *.* ?LocalHost >>> >>> I still get a segfault. The last line in the output running with -dn >>> >> is >> >>> always: >>> "file to log to: RemoteHost" (or LocalHost, when I used that config). >>> >>> Has anyone seen anything like this? I have rsyslog 2.0.6 running at >>> another site and dynamic filenames work fine... sort of troubling >>> >> that >> >>> 5.6.0 is having issues with them... >>> >>> Thanks, >>> Jason Antman >>> >>> Aaron Wiebe wrote: >>> >>> >>>>> 3) Assuming an even distribution over time (not quite accurate), >>>>> >> any >> >>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM >>>>> >> (Xen) >> >>>>> with a single 2.8GHz CPU and 512MB RAM would go? >>>>> >>>>> >>>>> >>>> 2 million lines a day is 70 lines per second assuming an 8 hour day. >>>> MySQL, properly configured, will eat that. Rsyslog won't even >>>> >> notice >> >>>> that log level. >>>> >>>> Your table sizes may be problematic over time, and your ability to >>>> query may impact things. Regardless, your concern should be mysql, >>>> not rsyslog. >>>> >>>> -Aaron >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com >>> >>> >>> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
