Hi, I would use the new format, but I don't know how to do it :).
I will experiment more and let you know if I manage this. On Wed, Jul 23, 2014 at 3:19 PM, Rainer Gerhards <[email protected]> wrote: > Don't use legacy format! > > Sent from phone, thus brief. > Am 23.07.2014 14:15 schrieb "Edmonds, Alan" <[email protected]>: > > > Sounds like a good plan. I'm revamping my setup to deal with some ES > > problems. The config format is a nightmare to me. > > > > > -----Original Message----- > > > From: [email protected] [mailto:rsyslog- > > > [email protected]] On Behalf Of Cristian Falcas > > > Sent: 23 July 2014 13:12 > > > To: rsyslog-users > > > Subject: Re: [rsyslog] separate remote logs and elasticsearch logs > > > > > > I think I managed to solve this by calling the elasticsearch remote > > > rule > > > from inside the remote rule: > > > > > > # the remote rule is defined like: > > > $RuleSet remote > > > $RulesetCreateMainQueue on > > > call remote_elasticsearch > > > > > > # and my elastic search config: > > > $ModLoad omelasticsearch > > > > > > $RuleSet remote_elasticsearch > > > $RulesetCreateMainQueue on > > > > > > > > > From what I've read, this should send the messages asynchronously > > > between > > > the rules. > > > > > > Best regards, > > > Cristian Falcas > > > > > > > > > > > > On Wed, Jul 23, 2014 at 2:16 PM, Cristian Falcas > > > <[email protected]> > > > wrote: > > > > > > > Sorry for being so vague. > > > > > > > > What I'm trying to do is to send messages received via tcp module to > > > 2 > > > > different output modules: > > > > - one to write to local files > > > > - second one to send messages to elasticsearch > > > > > > > > Because in my elasticsearch configuration I have some drop rules, > > > nothing > > > > reaches the omfile rules. How can I decouple those 2? I want > > > everything > > > > that comes from tcp to go to both output modules and not be tied to > > > each > > > > other. > > > > > > > > Is there a way to achive this? > > > > > > > > Best regards, > > > > Cristian Falcas > > > > > > > > > > > > > > > > > > > > On Tue, Jul 22, 2014 at 9:10 PM, David Lang <[email protected]> wrote: > > > > > > > >> you have the right idea, I don't understand what you are asking for > > > help > > > >> on. > > > >> > > > >> David Lang > > > >> > > > >> > > > >> On Tue, 22 Jul 2014, Cristian Falcas wrote: > > > >> > > > >> Hi, > > > >>> > > > >>> I have configured a rsyslog server where I want to send the logs > > > from the > > > >>> other machines. > > > >>> > > > >>> Here I want to keep local logs and also to send them to elastic > > > search. > > > >>> For > > > >>> elasticsearch I have multiple actions, because I want httpd (for > > > ex.) to > > > >>> go > > > >>> to searchType="httpd" and so on. After each action I would like to > > > drop > > > >>> the previously catched lines, because I want a last action with the > > > >>> default > > > >>> search type. Because of my dop rule, nothing reaches past the first > > > >>> action: > > > >>> not the omfile writer and not the elasticsearch actions. > > > >>> > > > >>> I'm using the same $RuleSet remote for both output modules. > > > >>> > > > >>> Can anyone help me in setting this correctly? > > > >>> > > > >>> Something like this (with multiple templates and actions): > > > >>> > > > >>> $RuleSet remote > > > >>> > > > >>> $ModLoad omelasticsearch > > > >>> > > > >>> template(name="10-audit" > > > >>> type="list" > > > >>> option.json="on"). > > > >>> { > > > >>> constant(value="{") > > > >>> constant(value="\"@timestamp\":\"") > > > >>> property(name="timereported" dateFormat="rfc3339") > > > >>> constant(value="\",\"timereported\":\"") > > > >>> property(name="timereported" dateFormat="rfc3339") > > > >>> constant(value="\",\"timegenerated\":\"") > > > >>> property(name="timegenerated" dateFormat="rfc3339") > > > >>> constant(value="\",\"message\":\"") > > > >>> property(name="msg") > > > >>> constant(value="\",\"host\":\"") > > > >>> property(name="hostname") > > > >>> constant(value="\",\"severity\":\"") > > > >>> property(name="syslogseverity-text") > > > >>> constant(value="\",\"priority\":\"") > > > >>> property(name="syslogpriority-text") > > > >>> constant(value="\",\"facility\":\"") > > > >>> property(name="syslogfacility-text") > > > >>> constant(value="\",\"tag\":\"") > > > >>> property(name="syslogtag") > > > >>> constant(value="\",\"program_name\":\"") > > > >>> property(name="programname") > > > >>> constant(value="\"}") > > > >>> } > > > >>> *.* action(type="omelasticsearch" > > > >>> name="action_10-audit" > > > >>> server="v-so-repo-02" > > > >>> serverport="9200" > > > >>> template="10-audit" > > > >>> searchIndex="default-index" > > > >>> searchType="audit" > > > >>> bulkmode="on" # use the Bulk API > > > >>> queue.dequeuebatchsize="5000" # ES bulk size > > > >>> queue.size="100000" # capacity of the action > > > queue > > > >>> queue.workerthreads="15" # 5 workers for the action > > > >>> queue.type="linkedlist" > > > >>> queue.FileName="es_queue" > > > >>> queue.MaxDiskSpace="1g" > > > >>> queue.SaveOnShutdown="on" > > > >>> action.resumeretrycount="-1" > > > >>> errorFile="/srv/log/rsyslog_ES-error_default-index_audit.log" > > > >>> ) > > > >>> & stop > > > >>> > > > >>> Best regards, > > > >>> Cristian Falcas > > > >>> _______________________________________________ > > > >>> rsyslog mailing list > > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > >>> http://www.rsyslog.com/professional-services/ > > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > myriad > > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > > you > > > >>> DON'T LIKE THAT. > > > >>> > > > >>> _______________________________________________ > > > >> rsyslog mailing list > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > > >> http://www.rsyslog.com/professional-services/ > > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > myriad > > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > > you > > > >> DON'T LIKE THAT. > > > >> > > > > > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > > > if you DON'T LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
