Don't use legacy format! Sent from phone, thus brief. Am 23.07.2014 14:15 schrieb "Edmonds, Alan" <[email protected]>:
> Sounds like a good plan. I'm revamping my setup to deal with some ES > problems. The config format is a nightmare to me. > > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Cristian Falcas > > Sent: 23 July 2014 13:12 > > To: rsyslog-users > > Subject: Re: [rsyslog] separate remote logs and elasticsearch logs > > > > I think I managed to solve this by calling the elasticsearch remote > > rule > > from inside the remote rule: > > > > # the remote rule is defined like: > > $RuleSet remote > > $RulesetCreateMainQueue on > > call remote_elasticsearch > > > > # and my elastic search config: > > $ModLoad omelasticsearch > > > > $RuleSet remote_elasticsearch > > $RulesetCreateMainQueue on > > > > > > From what I've read, this should send the messages asynchronously > > between > > the rules. > > > > Best regards, > > Cristian Falcas > > > > > > > > On Wed, Jul 23, 2014 at 2:16 PM, Cristian Falcas > > <[email protected]> > > wrote: > > > > > Sorry for being so vague. > > > > > > What I'm trying to do is to send messages received via tcp module to > > 2 > > > different output modules: > > > - one to write to local files > > > - second one to send messages to elasticsearch > > > > > > Because in my elasticsearch configuration I have some drop rules, > > nothing > > > reaches the omfile rules. How can I decouple those 2? I want > > everything > > > that comes from tcp to go to both output modules and not be tied to > > each > > > other. > > > > > > Is there a way to achive this? > > > > > > Best regards, > > > Cristian Falcas > > > > > > > > > > > > > > > On Tue, Jul 22, 2014 at 9:10 PM, David Lang <[email protected]> wrote: > > > > > >> you have the right idea, I don't understand what you are asking for > > help > > >> on. > > >> > > >> David Lang > > >> > > >> > > >> On Tue, 22 Jul 2014, Cristian Falcas wrote: > > >> > > >> Hi, > > >>> > > >>> I have configured a rsyslog server where I want to send the logs > > from the > > >>> other machines. > > >>> > > >>> Here I want to keep local logs and also to send them to elastic > > search. > > >>> For > > >>> elasticsearch I have multiple actions, because I want httpd (for > > ex.) to > > >>> go > > >>> to searchType="httpd" and so on. After each action I would like to > > drop > > >>> the previously catched lines, because I want a last action with the > > >>> default > > >>> search type. Because of my dop rule, nothing reaches past the first > > >>> action: > > >>> not the omfile writer and not the elasticsearch actions. > > >>> > > >>> I'm using the same $RuleSet remote for both output modules. > > >>> > > >>> Can anyone help me in setting this correctly? > > >>> > > >>> Something like this (with multiple templates and actions): > > >>> > > >>> $RuleSet remote > > >>> > > >>> $ModLoad omelasticsearch > > >>> > > >>> template(name="10-audit" > > >>> type="list" > > >>> option.json="on"). > > >>> { > > >>> constant(value="{") > > >>> constant(value="\"@timestamp\":\"") > > >>> property(name="timereported" dateFormat="rfc3339") > > >>> constant(value="\",\"timereported\":\"") > > >>> property(name="timereported" dateFormat="rfc3339") > > >>> constant(value="\",\"timegenerated\":\"") > > >>> property(name="timegenerated" dateFormat="rfc3339") > > >>> constant(value="\",\"message\":\"") > > >>> property(name="msg") > > >>> constant(value="\",\"host\":\"") > > >>> property(name="hostname") > > >>> constant(value="\",\"severity\":\"") > > >>> property(name="syslogseverity-text") > > >>> constant(value="\",\"priority\":\"") > > >>> property(name="syslogpriority-text") > > >>> constant(value="\",\"facility\":\"") > > >>> property(name="syslogfacility-text") > > >>> constant(value="\",\"tag\":\"") > > >>> property(name="syslogtag") > > >>> constant(value="\",\"program_name\":\"") > > >>> property(name="programname") > > >>> constant(value="\"}") > > >>> } > > >>> *.* action(type="omelasticsearch" > > >>> name="action_10-audit" > > >>> server="v-so-repo-02" > > >>> serverport="9200" > > >>> template="10-audit" > > >>> searchIndex="default-index" > > >>> searchType="audit" > > >>> bulkmode="on" # use the Bulk API > > >>> queue.dequeuebatchsize="5000" # ES bulk size > > >>> queue.size="100000" # capacity of the action > > queue > > >>> queue.workerthreads="15" # 5 workers for the action > > >>> queue.type="linkedlist" > > >>> queue.FileName="es_queue" > > >>> queue.MaxDiskSpace="1g" > > >>> queue.SaveOnShutdown="on" > > >>> action.resumeretrycount="-1" > > >>> errorFile="/srv/log/rsyslog_ES-error_default-index_audit.log" > > >>> ) > > >>> & stop > > >>> > > >>> Best regards, > > >>> Cristian Falcas > > >>> _______________________________________________ > > >>> rsyslog mailing list > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> http://www.rsyslog.com/professional-services/ > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > >>> DON'T LIKE THAT. > > >>> > > >>> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com/professional-services/ > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > > you > > >> DON'T LIKE THAT. > > >> > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > > if you DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
