Hi Rainer, I talked to my manager. we are OK to contribute the output plugin to the rsyslog src tree. The code needs some clean-up before I can do that though.
Thanks, Liwei On Thu, Oct 9, 2014 at 10:59 AM, Liwei Peng <[email protected]> wrote: > Hi Rainer, I got a stripped down version of my output plugin, removed most > of the code except creating a new thread in BEGINcreateInstance(). See > attached file. > > On Wed, Oct 8, 2014 at 4:07 PM, Liwei Peng <[email protected]> wrote: > >> Thanks Rainer. I'll see whether I can strip down the code to have a >> repro. For the plugin to be open source, we didn't make a decision yet. I >> need to get permission from my manager. >> >> On Wed, Oct 8, 2014 at 12:25 AM, Rainer Gerhards < >> [email protected]> wrote: >> >>> 2014-10-08 2:33 GMT+02:00 Liwei Peng <[email protected]>: >>> >>> > I did more debugging, not root cause yet. but it could be related to >>> the >>> > fact that my new thread is created in the BEGINcreateInstance. >>> > >>> > >>> I don't think so. >>> >>> >>> > Is this the right place to create a helper thread? If not, what other >>> place >>> > is best? >>> > >>> > I only need to create one helper thread. it will run forever. >>> > >>> > >>> Here is one place which is used by omfile when additional threads are >>> created: >>> >>> https://github.com/rsyslog/rsyslog/blob/master/runtime/stream.c#L874 >>> >>> maybe this is useful to look at. >>> >>> Would it be possible that you strip down your plugin so that it only does >>> dbgprintf's and nothing else. And add the autotools plumbing so that I >>> can >>> build it together with rsyslog. If so, I'd build it and see what happens >>> on >>> my machine. >>> >>> As a related note, would you be willing to contribute the plugin to the >>> rsyslog community? If so, I could integrate it into the source tree (with >>> full credits, of course). >>> >>> Rainer >>> >>> >>> > On Tue, Oct 7, 2014 at 1:00 PM, David Lang <[email protected]> wrote: >>> > >>> > > On Tue, 7 Oct 2014, Liwei Peng wrote: >>> > > >>> > > Once I ran rsyslog with "-dn" mode, everything works as expected. >>> > >> >>> > > >>> > > ouch, that makes things hard :-( >>> > > >>> > > just to be sure we're on the same page, you were running it with -dn >>> as a >>> > > service, right? >>> > > >>> > > the new >>> > >> thread is created and running properly. With only "-d", the thread >>> will >>> > >> exit within 2 seconds, just like in service mode. in the '-dn' >>> mode, >>> > the >>> > >> rsyslog is running as the proper user (syslog/syslog), so that >>> shouldn't >>> > >> be >>> > >> the problem. >>> > >> >>> > >> I also made my new thread very simple by making it a do a dummy() >>> > while(1) >>> > >> loop. In the loop, just print out some msg, sleep for X seconds. >>> then do >>> > >> the doop again. >>> > >> >>> > >> What could cause this behaviour difference? >>> > >> >>> > > >>> > > if it works when you run it as root, but not when the system runs it, >>> > that >>> > > means that there is something different between the two. On RHEL >>> systems >>> > > this is typically SELinux, which is why we looked there first. >>> > > >>> > > Please take a very close look at the system startup scripts and >>> > > parameters, there has got to be something different there compared to >>> > when >>> > > you start it manually. >>> > > >>> > > -d says to spit out debug output, -n says to not go into the >>> background >>> > as >>> > > a daemon, but also ends up changing timing of some things >>> > > >>> > > if -n makes it work, then look for oddities or race conditions >>> related to >>> > > your thread. >>> > > >>> > > if -d spits out data all the way until it dies, look through that >>> output >>> > > to see if it says why it's dieing. >>> > > >>> > > David Lang >>> > > >>> > > >>> > > Thanks, >>> > >> >>> > >> Liwei >>> > >> >>> > >> >>> > >> On Tue, Oct 7, 2014 at 10:45 AM, David Lang <[email protected]> wrote: >>> > >> >>> > >> can you configure the startup to start it in debug mode (-dn) and >>> dump >>> > >>> the >>> > >>> output to a file? that could give us a clue about what's failing. >>> > >>> >>> > >>> David Lang >>> > >>> >>> > >>> On Tue, 7 Oct 2014, Liwei Peng wrote: >>> > >>> >>> > >>> Date: Tue, 7 Oct 2014 10:42:55 -0700 >>> > >>> >>> > >>>> From: Liwei Peng <[email protected]> >>> > >>>> Reply-To: rsyslog-users <[email protected]> >>> > >>>> To: rsyslog-users <[email protected]> >>> > >>>> Subject: Re: [rsyslog] output plugin: new thread is not running >>> > properly >>> > >>>> in >>> > >>>> service mode >>> > >>>> >>> > >>>> >>> > >>>> Thanks David. >>> > >>>> >>> > >>>> I found that rsyslogd is in the disabled profile directory. Also, >>> even >>> > >>>> after I totally stop and teardown the apparmor service, my new >>> thread >>> > >>>> still >>> > >>>> has trouble to start up. >>> > >>>> >>> > >>>> /etc/apparmor.d/disable$ ls >>> > >>>> usr.bin.firefox usr.sbin.rsyslogd >>> > >>>> >>> > >>>> >>> > >>>> On Tue, Oct 7, 2014 at 8:43 AM, David Lang <[email protected]> wrote: >>> > >>>> >>> > >>>> Ubuntu uses AppArmor instead of SELinux >>> > >>>> >>> > >>>>> >>> > >>>>> /etc/apparmor.d/usr.sbin.rsyslogd controls the AA config for >>> rsyslog. >>> > >>>>> >>> > >>>>> I don't know exactly what your additional thread is trying to >>> access, >>> > >>>>> but >>> > >>>>> you probably just need to add the permissions here to grant it >>> the >>> > >>>>> access. >>> > >>>>> >>> > >>>>> David Lang >>> > >>>>> >>> > >>>>> >>> > >>>>> On Tue, 7 Oct 2014, Liwei Peng wrote: >>> > >>>>> >>> > >>>>> Thanks David and Rainer. >>> > >>>>> >>> > >>>>> >>> > >>>>>> I am using Ubuntu 14.04 LTS 64-bit. However, when I ran >>> 'getenforce' >>> > >>>>>> on >>> > >>>>>> my >>> > >>>>>> computer, it showed "Disabled". SeLinux package was not >>> installed on >>> > >>>>>> my >>> > >>>>>> computer either. >>> > >>>>>> >>> > >>>>>> >>> > >>>>>> On Tue, Oct 7, 2014 at 12:31 AM, Rainer Gerhards < >>> > >>>>>> [email protected]> >>> > >>>>>> wrote: >>> > >>>>>> >>> > >>>>>> 2014-10-07 5:04 GMT+02:00 David Lang <[email protected]>: >>> > >>>>>> >>> > >>>>>> >>> > >>>>>>> Are you using a system that has SELinux installed? on such >>> systems >>> > >>>>>>> the >>> > >>>>>>> >>> > >>>>>>> permissions that rsyslog has when started as a service are >>> > different >>> > >>>>>>>> than >>> > >>>>>>>> when you run it manually as root. >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> Yeah, as David says, it's most probably SELinux or something >>> > along >>> > >>>>>>>> >>> > >>>>>>>> these >>> > >>>>>>> lines. Rsyslog does not impose any limits on additional >>> threads a >>> > >>>>>>> plugin >>> > >>>>>>> uses internally. For example, omfile uses multiple helper >>> threads >>> > >>>>>>> when >>> > >>>>>>> async writes are configured. Also, there is no real difference >>> from >>> > >>>>>>> rsyslog's PoV between running in a console session vs. as >>> > background >>> > >>>>>>> service. With systemd, the code path is actually the same, as >>> > systemd >>> > >>>>>>> uses >>> > >>>>>>> -n itself. So if it runs in a console session but not as >>> service, >>> > the >>> > >>>>>>> reason most probably is somewhere in the system config. >>> > >>>>>>> >>> > >>>>>>> HTH >>> > >>>>>>> Rainer >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>> David Lang >>> > >>>>>>> >>> > >>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> On Mon, 6 Oct 2014, Liwei Peng wrote: >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> In my output plugin (see attached code), because I need to >>> > process >>> > >>>>>>>> and >>> > >>>>>>>> >>> > >>>>>>>> send >>> > >>>>>>>>> the messages to some other daemon, I created a new pthread to >>> > >>>>>>>>> >>> > >>>>>>>>> listen/read >>> > >>>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>> response from that daemon. All the code works well when I >>> ran by >>> > >>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>>> starting >>> > >>>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>> rsyslog in manual mode. >>> > >>>>>>> >>> > >>>>>>>> >>> > >>>>>>>> >>> > >>>>>>>>> /usr/sbin/rsyslogd -f/etc/rsyslog.conf -u2 -n -irsyslog.pid >>> > >>>>>>>>> -M/usr/lib/rsyslog >>> > >>>>>>>>> >>> > >>>>>>>>> However, when I ran my plugin with rsyslogd as service, it >>> seems >>> > >>>>>>>>> that >>> > >>>>>>>>> my >>> > >>>>>>>>> new thread is started but it is not running properly. I saw >>> the >>> > >>>>>>>>> thread >>> > >>>>>>>>> print out a startup msg but it didn't continue printing new >>> msg >>> > as >>> > >>>>>>>>> expected. >>> > >>>>>>>>> >>> > >>>>>>>>> What's wrong with my code? is there limitation on using new >>> > >>>>>>>>> pthread? >>> > >>>>>>>>> >>> > >>>>>>>>> Liwei >>> > >>>>>>>>> >>> > >>>>>>>>> _______________________________________________ >>> > >>>>>>>>> >>> > >>>>>>>>> rsyslog mailing list >>> > >>>>>>>>> >>> > >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>>>>>>> http://www.rsyslog.com/professional-services/ >>> > >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>> by a >>> > >>>>>>>> myriad >>> > >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>> POST if >>> > >>>>>>>> you >>> > >>>>>>>> DON'T LIKE THAT. >>> > >>>>>>>> >>> > >>>>>>>> _______________________________________________ >>> > >>>>>>>> >>> > >>>>>>>> rsyslog mailing list >>> > >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>>>>>> http://www.rsyslog.com/professional-services/ >>> > >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED >>> by a >>> > >>>>>>> myriad >>> > >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT >>> POST if >>> > >>>>>>> you >>> > >>>>>>> DON'T LIKE THAT. >>> > >>>>>>> >>> > >>>>>>> _______________________________________________ >>> > >>>>>>> >>> > >>>>>>> rsyslog mailing list >>> > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>>>>> http://www.rsyslog.com/professional-services/ >>> > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by >>> a >>> > >>>>>> myriad >>> > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>> if >>> > you >>> > >>>>>> DON'T LIKE THAT. >>> > >>>>>> >>> > >>>>>> _______________________________________________ >>> > >>>>>> >>> > >>>>>> rsyslog mailing list >>> > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>>>> http://www.rsyslog.com/professional-services/ >>> > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> > >>>>> myriad >>> > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>> if >>> > you >>> > >>>>> DON'T LIKE THAT. >>> > >>>>> >>> > >>>>> _______________________________________________ >>> > >>>>> >>> > >>>> rsyslog mailing list >>> > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>>> http://www.rsyslog.com/professional-services/ >>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> > myriad >>> > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >>> if you >>> > >>>> DON'T LIKE THAT. >>> > >>>> >>> > >>>> _______________________________________________ >>> > >>>> >>> > >>> rsyslog mailing list >>> > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >>> http://www.rsyslog.com/professional-services/ >>> > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> > myriad >>> > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>> you >>> > >>> DON'T LIKE THAT. >>> > >>> >>> > >>> _______________________________________________ >>> > >> rsyslog mailing list >>> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > >> http://www.rsyslog.com/professional-services/ >>> > >> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad >>> > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>> you >>> > >> DON'T LIKE THAT. >>> > >> >>> > >> _______________________________________________ >>> > > rsyslog mailing list >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > http://www.rsyslog.com/professional-services/ >>> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad >>> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>> you >>> > > DON'T LIKE THAT. >>> > > >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> > What's up with rsyslog? Follow https://twitter.com/rgerhards >>> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>> myriad >>> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> > DON'T LIKE THAT. >>> > >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >> >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
