Thanks Rainer. I'll see whether I can strip down the code to have a repro. For the plugin to be open source, we didn't make a decision yet. I need to get permission from my manager.
On Wed, Oct 8, 2014 at 12:25 AM, Rainer Gerhards <[email protected]> wrote: > 2014-10-08 2:33 GMT+02:00 Liwei Peng <[email protected]>: > > > I did more debugging, not root cause yet. but it could be related to the > > fact that my new thread is created in the BEGINcreateInstance. > > > > > I don't think so. > > > > Is this the right place to create a helper thread? If not, what other > place > > is best? > > > > I only need to create one helper thread. it will run forever. > > > > > Here is one place which is used by omfile when additional threads are > created: > > https://github.com/rsyslog/rsyslog/blob/master/runtime/stream.c#L874 > > maybe this is useful to look at. > > Would it be possible that you strip down your plugin so that it only does > dbgprintf's and nothing else. And add the autotools plumbing so that I can > build it together with rsyslog. If so, I'd build it and see what happens on > my machine. > > As a related note, would you be willing to contribute the plugin to the > rsyslog community? If so, I could integrate it into the source tree (with > full credits, of course). > > Rainer > > > > On Tue, Oct 7, 2014 at 1:00 PM, David Lang <[email protected]> wrote: > > > > > On Tue, 7 Oct 2014, Liwei Peng wrote: > > > > > > Once I ran rsyslog with "-dn" mode, everything works as expected. > > >> > > > > > > ouch, that makes things hard :-( > > > > > > just to be sure we're on the same page, you were running it with -dn > as a > > > service, right? > > > > > > the new > > >> thread is created and running properly. With only "-d", the thread > will > > >> exit within 2 seconds, just like in service mode. in the '-dn' mode, > > the > > >> rsyslog is running as the proper user (syslog/syslog), so that > shouldn't > > >> be > > >> the problem. > > >> > > >> I also made my new thread very simple by making it a do a dummy() > > while(1) > > >> loop. In the loop, just print out some msg, sleep for X seconds. then > do > > >> the doop again. > > >> > > >> What could cause this behaviour difference? > > >> > > > > > > if it works when you run it as root, but not when the system runs it, > > that > > > means that there is something different between the two. On RHEL > systems > > > this is typically SELinux, which is why we looked there first. > > > > > > Please take a very close look at the system startup scripts and > > > parameters, there has got to be something different there compared to > > when > > > you start it manually. > > > > > > -d says to spit out debug output, -n says to not go into the background > > as > > > a daemon, but also ends up changing timing of some things > > > > > > if -n makes it work, then look for oddities or race conditions related > to > > > your thread. > > > > > > if -d spits out data all the way until it dies, look through that > output > > > to see if it says why it's dieing. > > > > > > David Lang > > > > > > > > > Thanks, > > >> > > >> Liwei > > >> > > >> > > >> On Tue, Oct 7, 2014 at 10:45 AM, David Lang <[email protected]> wrote: > > >> > > >> can you configure the startup to start it in debug mode (-dn) and > dump > > >>> the > > >>> output to a file? that could give us a clue about what's failing. > > >>> > > >>> David Lang > > >>> > > >>> On Tue, 7 Oct 2014, Liwei Peng wrote: > > >>> > > >>> Date: Tue, 7 Oct 2014 10:42:55 -0700 > > >>> > > >>>> From: Liwei Peng <[email protected]> > > >>>> Reply-To: rsyslog-users <[email protected]> > > >>>> To: rsyslog-users <[email protected]> > > >>>> Subject: Re: [rsyslog] output plugin: new thread is not running > > properly > > >>>> in > > >>>> service mode > > >>>> > > >>>> > > >>>> Thanks David. > > >>>> > > >>>> I found that rsyslogd is in the disabled profile directory. Also, > even > > >>>> after I totally stop and teardown the apparmor service, my new > thread > > >>>> still > > >>>> has trouble to start up. > > >>>> > > >>>> /etc/apparmor.d/disable$ ls > > >>>> usr.bin.firefox usr.sbin.rsyslogd > > >>>> > > >>>> > > >>>> On Tue, Oct 7, 2014 at 8:43 AM, David Lang <[email protected]> wrote: > > >>>> > > >>>> Ubuntu uses AppArmor instead of SELinux > > >>>> > > >>>>> > > >>>>> /etc/apparmor.d/usr.sbin.rsyslogd controls the AA config for > rsyslog. > > >>>>> > > >>>>> I don't know exactly what your additional thread is trying to > access, > > >>>>> but > > >>>>> you probably just need to add the permissions here to grant it the > > >>>>> access. > > >>>>> > > >>>>> David Lang > > >>>>> > > >>>>> > > >>>>> On Tue, 7 Oct 2014, Liwei Peng wrote: > > >>>>> > > >>>>> Thanks David and Rainer. > > >>>>> > > >>>>> > > >>>>>> I am using Ubuntu 14.04 LTS 64-bit. However, when I ran > 'getenforce' > > >>>>>> on > > >>>>>> my > > >>>>>> computer, it showed "Disabled". SeLinux package was not installed > on > > >>>>>> my > > >>>>>> computer either. > > >>>>>> > > >>>>>> > > >>>>>> On Tue, Oct 7, 2014 at 12:31 AM, Rainer Gerhards < > > >>>>>> [email protected]> > > >>>>>> wrote: > > >>>>>> > > >>>>>> 2014-10-07 5:04 GMT+02:00 David Lang <[email protected]>: > > >>>>>> > > >>>>>> > > >>>>>>> Are you using a system that has SELinux installed? on such > systems > > >>>>>>> the > > >>>>>>> > > >>>>>>> permissions that rsyslog has when started as a service are > > different > > >>>>>>>> than > > >>>>>>>> when you run it manually as root. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> Yeah, as David says, it's most probably SELinux or something > > along > > >>>>>>>> > > >>>>>>>> these > > >>>>>>> lines. Rsyslog does not impose any limits on additional threads a > > >>>>>>> plugin > > >>>>>>> uses internally. For example, omfile uses multiple helper threads > > >>>>>>> when > > >>>>>>> async writes are configured. Also, there is no real difference > from > > >>>>>>> rsyslog's PoV between running in a console session vs. as > > background > > >>>>>>> service. With systemd, the code path is actually the same, as > > systemd > > >>>>>>> uses > > >>>>>>> -n itself. So if it runs in a console session but not as service, > > the > > >>>>>>> reason most probably is somewhere in the system config. > > >>>>>>> > > >>>>>>> HTH > > >>>>>>> Rainer > > >>>>>>> > > >>>>>>> > > >>>>>>> David Lang > > >>>>>>> > > >>>>>>> > > >>>>>>>> > > >>>>>>>> On Mon, 6 Oct 2014, Liwei Peng wrote: > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> In my output plugin (see attached code), because I need to > > process > > >>>>>>>> and > > >>>>>>>> > > >>>>>>>> send > > >>>>>>>>> the messages to some other daemon, I created a new pthread to > > >>>>>>>>> > > >>>>>>>>> listen/read > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>> response from that daemon. All the code works well when I ran > by > > >>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>> starting > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>> rsyslog in manual mode. > > >>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>>> /usr/sbin/rsyslogd -f/etc/rsyslog.conf -u2 -n -irsyslog.pid > > >>>>>>>>> -M/usr/lib/rsyslog > > >>>>>>>>> > > >>>>>>>>> However, when I ran my plugin with rsyslogd as service, it > seems > > >>>>>>>>> that > > >>>>>>>>> my > > >>>>>>>>> new thread is started but it is not running properly. I saw the > > >>>>>>>>> thread > > >>>>>>>>> print out a startup msg but it didn't continue printing new msg > > as > > >>>>>>>>> expected. > > >>>>>>>>> > > >>>>>>>>> What's wrong with my code? is there limitation on using new > > >>>>>>>>> pthread? > > >>>>>>>>> > > >>>>>>>>> Liwei > > >>>>>>>>> > > >>>>>>>>> _______________________________________________ > > >>>>>>>>> > > >>>>>>>>> rsyslog mailing list > > >>>>>>>>> > > >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>>>>>>> http://www.rsyslog.com/professional-services/ > > >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by > a > > >>>>>>>> myriad > > >>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if > > >>>>>>>> you > > >>>>>>>> DON'T LIKE THAT. > > >>>>>>>> > > >>>>>>>> _______________________________________________ > > >>>>>>>> > > >>>>>>>> rsyslog mailing list > > >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>>>>>> http://www.rsyslog.com/professional-services/ > > >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > >>>>>>> myriad > > >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if > > >>>>>>> you > > >>>>>>> DON'T LIKE THAT. > > >>>>>>> > > >>>>>>> _______________________________________________ > > >>>>>>> > > >>>>>>> rsyslog mailing list > > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>>>>> http://www.rsyslog.com/professional-services/ > > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > >>>>>> myriad > > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > >>>>>> DON'T LIKE THAT. > > >>>>>> > > >>>>>> _______________________________________________ > > >>>>>> > > >>>>>> rsyslog mailing list > > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>>>> http://www.rsyslog.com/professional-services/ > > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > >>>>> myriad > > >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > >>>>> DON'T LIKE THAT. > > >>>>> > > >>>>> _______________________________________________ > > >>>>> > > >>>> rsyslog mailing list > > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>>> http://www.rsyslog.com/professional-services/ > > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > >>>> DON'T LIKE THAT. > > >>>> > > >>>> _______________________________________________ > > >>>> > > >>> rsyslog mailing list > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> http://www.rsyslog.com/professional-services/ > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > >>> DON'T LIKE THAT. > > >>> > > >>> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com/professional-services/ > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > >> DON'T LIKE THAT. > > >> > > >> _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > > DON'T LIKE THAT. > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
