bug 530 was filed for this issue. *Bug 530* <http://bugzilla.adiscon.com/show_bug.cgi?id=530> - 2 messages go to the same worker thread in one doAction() http://bugzilla.adiscon.com/show_bug.cgi?id=530
On Mon, Oct 6, 2014 at 9:44 AM, Liwei Peng <[email protected]> wrote: > Thanks Rainer. Will do. > > On Sun, Oct 5, 2014 at 8:46 AM, Rainer Gerhards <[email protected]> > wrote: > >> 2014-10-02 18:09 GMT+02:00 Liwei Peng <[email protected]>: >> >> > Thanks. I attached my output plugin code. also, I attached the wrong >> > console output in my previous email. See the new one in c2.txt.zip. >> > >> > More details of my problem: >> > 1) I tracked for each msg going to doAction(). I defined a counter in >> > wrkrInstanceData. And increase for the counter for each new msg. >> > 2) However, I found 2 msgs that have the same PID, thread id, and end up >> > with the same counter. Is this by design for the batching? >> > >> > >> I checked the plugin code, and it looks ok. So it smells a little bit like >> a bug inside the core engine. Could you open a bug on github? That would >> make sure I do not forget to look at it as soon as I find time. >> >> Thx, >> Rainer >> >> >> > >> > Summary of the two messages: >> > >> > 5758.832554580:action 1 queue[DA]:Reg/w0: omazuremds: pid *23132* thread >> > *139923907188480* tag=[1412265758:0x7f428c004e60:139923907188480 1] >> > logstring=["syslog","6","2014-10-02T09:02:35.516382-07:00","127.0.0.1"," >> > [origin software=\"rsyslogd\" swVersion=\"8.4.0\" x-pid=\"23109\" >> > x-info=\"http:\/\/www.rsyslog.com\"] exiting on signal 15.",""] >> > >> > 5758.833384371:action 1 queue:Reg/w0: omazuremds: pid *23132* thread >> > *139923907188480* tag=[1412265758:0x7f428c004e60:139923907188480 1] >> > logstring=["syslog","6","2014-10-02T09:02:38.832096-07:00","127.0.0.1"," >> > [origin software=\"rsyslogd\" swVersion=\"8.4.0\" x-pid=\"23132\" >> > x-info=\"http:\/\/www.rsyslog.com\"] start",""] >> > >> > >> > On Wed, Oct 1, 2014 at 11:20 PM, Rainer Gerhards < >> [email protected] >> > > >> > wrote: >> > >> > > Please also send the plugin code, as this helps to understand... >> > > >> > > Thx, >> > > Rainer >> > > >> > > 2014-10-02 2:48 GMT+02:00 Liwei Peng <[email protected]>: >> > > >> > > > Hi, >> > > > >> > > > I am writing a new output module with batching with multiple worker >> > > > threads. I am using disk-assisted fixed array queue. >> > > > >> > > > One thing I'd like to do is to uniquely identify each message sent >> > > through >> > > > the module. I created a counter in worker instance. so that each new >> > > > message in doAction() will make counter + 1. >> > > > >> > > > However, I found that there are cases (not 100% case) that 2 >> messages >> > go >> > > to >> > > > the same worker thread and the counter are only increased once. >> > > > >> > > > Can you help on what's going on? >> > > > >> > > > The only diff I found is that the one msg is from queue[DA]; >> another is >> > > > from queue. but I don't know why this won't cause counter increase. >> > > > >> > > > See attached debug log. >> > > > >> > > > Example: the following 2 messages to the same thread >> > > > >> > > > 0704.006084403:action 1 *queue[DA]:*Reg/w0: omazuremds: pid 20121 >> > thread >> > > > 139993705387776 tag=[1412210704:0x7f52cc004e60:1] >> > > > >> > logstring=["syslog","6","2014-10-01T17:45:00.675677-07:00","127.0.0.1"," >> > > > [origin software=\"rsyslogd\" swVersion=\"8.4.0\" x-pid=\"19917\" >> > > > x-info=\"http:\/\/www.rsyslog.com\"] exiting on signal 15.",""] >> > > > >> > > > 0704.006770935:action 1 q*ueue:Reg/w0*: omazuremds: pid 20121 thread >> > > > 139993680209664 tag=[1412210704:0x7f52cc004e60:1] >> > > > >> > logstring=["syslog","6","2014-10-01T17:45:04.005700-07:00","127.0.0.1"," >> > > > [origin software=\"rsyslogd\" swVersion=\"8.4.0\" x-pid=\"20121\" >> > > > x-info=\"http:\/\/www.rsyslog.com\"] start",""] >> > > > >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > myriad >> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >> you >> > > > DON'T LIKE THAT. >> > > > >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
