A PR would be very welcome by me. Rainer
Sent from phone, thus brief. Am 24.06.2017 06:02 schrieb "Derek DiFilippo" <de...@jsonar.com>: > Hi, > > This is my first email to the rsyslog list. I've been getting to know > rsyslog over the last few months and it's been a pleasure to work with, > thank you. > > ****** > > I would like file offset metadata to reliably order messages on a receiving > machine. > > I'm hoping for something like the following in imfile.c. This diff wouldn't > compile, I know -- it's just here to make the idea concrete: > diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c > index 758d8273..b5b2ce64 100644 > --- a/plugins/imfile/imfile.c > +++ b/plugins/imfile/imfile.c > @@ -547,6 +547,7 @@ static rsRetVal enqLine(lstn_t *const __restrict__ > pLstn, > MsgSetRuleset(pMsg, pLstn->pRuleset); > if(pLstn->addMetadata) > msgAddMetadata(pMsg, (uchar*)"filename", > pLstn->pszFileName); > + msgAddMetadata(pMsg, (uchar*)"fileoffset", > pLstn->pStrm->iCurrOffs); > ratelimitAddMsg(pLstn->ratelimiter, &pLstn->multiSub, pMsg); > > to enable a template like: > $template > fwdtemplate,"%hostname%^%$!metadata!filename%^$!metadata! > fileoffset%^%msg%" > > Is this change something I could submit as a pull request or one that a > regularly contributing developer could easily implement? > > ****** > > Here's the use case. As I mentioned, I want the file offset metadata to > reliably order messages on the receiving machine. > > I have three machines running rsyslog. > > Messages are written to files on machine A and we process those messages on > machine C after passing through a queuing rsyslog on machine B. > > A: imfile (matching by glob) --> omfwd (adds filename metadata via > template) --> to B > B: imtcp --> omfwd (with disk-backed linkedlist action queue) --> to C > C: imtcp --> omprog (running a custom message parser) --> to our > application > > Occasionally the files on machine A are raw XML and I'm seeing the file > reconstructed 'perfectly' on machine C, same number of lines, same number > of bytes, but out of order and therefore not valid XML. > > I am happily using rsyslog for other traditional syslog message processing > and could keep that system in place if I had access to the fileoffset in > addition to the file name. > > Thank you for your time, > -Derek. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
