Unfortunately I have drop rules in omfile also, because I want to separate the logs to different files.
On Wed, Jul 23, 2014 at 3:16 PM, Rainer Gerhards <[email protected]> wrote: > On Wed, Jul 23, 2014 at 2:12 PM, Cristian Falcas <[email protected]> > wrote: > > > I think I managed to solve this by calling the elasticsearch remote rule > > from inside the remote rule: > > > > # the remote rule is defined like: > > $RuleSet remote > > $RulesetCreateMainQueue on > > call remote_elasticsearch > > > > # and my elastic search config: > > $ModLoad omelasticsearch > > > > $RuleSet remote_elasticsearch > > $RulesetCreateMainQueue on > > > > > > From what I've read, this should send the messages asynchronously between > > the rules. > > > > > I think this work, but don't remember the legacy syntax well enough to say > that 100% sure. > > However, a simpler solution (and faster) is to simply move the omfile > actions in front of the omelasticsearch ones. > > Rainer > > > Best regards, > > Cristian Falcas > > > > > > > > On Wed, Jul 23, 2014 at 2:16 PM, Cristian Falcas < > [email protected]> > > wrote: > > > > > Sorry for being so vague. > > > > > > What I'm trying to do is to send messages received via tcp module to 2 > > > different output modules: > > > - one to write to local files > > > - second one to send messages to elasticsearch > > > > > > Because in my elasticsearch configuration I have some drop rules, > nothing > > > reaches the omfile rules. How can I decouple those 2? I want everything > > > that comes from tcp to go to both output modules and not be tied to > each > > > other. > > > > > > Is there a way to achive this? > > > > > > Best regards, > > > Cristian Falcas > > > > > > > > > > > > > > > On Tue, Jul 22, 2014 at 9:10 PM, David Lang <[email protected]> wrote: > > > > > >> you have the right idea, I don't understand what you are asking for > help > > >> on. > > >> > > >> David Lang > > >> > > >> > > >> On Tue, 22 Jul 2014, Cristian Falcas wrote: > > >> > > >> Hi, > > >>> > > >>> I have configured a rsyslog server where I want to send the logs from > > the > > >>> other machines. > > >>> > > >>> Here I want to keep local logs and also to send them to elastic > search. > > >>> For > > >>> elasticsearch I have multiple actions, because I want httpd (for ex.) > > to > > >>> go > > >>> to searchType="httpd" and so on. After each action I would like to > > drop > > >>> the previously catched lines, because I want a last action with the > > >>> default > > >>> search type. Because of my dop rule, nothing reaches past the first > > >>> action: > > >>> not the omfile writer and not the elasticsearch actions. > > >>> > > >>> I'm using the same $RuleSet remote for both output modules. > > >>> > > >>> Can anyone help me in setting this correctly? > > >>> > > >>> Something like this (with multiple templates and actions): > > >>> > > >>> $RuleSet remote > > >>> > > >>> $ModLoad omelasticsearch > > >>> > > >>> template(name="10-audit" > > >>> type="list" > > >>> option.json="on"). > > >>> { > > >>> constant(value="{") > > >>> constant(value="\"@timestamp\":\"") > > >>> property(name="timereported" dateFormat="rfc3339") > > >>> constant(value="\",\"timereported\":\"") > > >>> property(name="timereported" dateFormat="rfc3339") > > >>> constant(value="\",\"timegenerated\":\"") > > >>> property(name="timegenerated" dateFormat="rfc3339") > > >>> constant(value="\",\"message\":\"") > > >>> property(name="msg") > > >>> constant(value="\",\"host\":\"") > > >>> property(name="hostname") > > >>> constant(value="\",\"severity\":\"") > > >>> property(name="syslogseverity-text") > > >>> constant(value="\",\"priority\":\"") > > >>> property(name="syslogpriority-text") > > >>> constant(value="\",\"facility\":\"") > > >>> property(name="syslogfacility-text") > > >>> constant(value="\",\"tag\":\"") > > >>> property(name="syslogtag") > > >>> constant(value="\",\"program_name\":\"") > > >>> property(name="programname") > > >>> constant(value="\"}") > > >>> } > > >>> *.* action(type="omelasticsearch" > > >>> name="action_10-audit" > > >>> server="v-so-repo-02" > > >>> serverport="9200" > > >>> template="10-audit" > > >>> searchIndex="default-index" > > >>> searchType="audit" > > >>> bulkmode="on" # use the Bulk API > > >>> queue.dequeuebatchsize="5000" # ES bulk size > > >>> queue.size="100000" # capacity of the action queue > > >>> queue.workerthreads="15" # 5 workers for the action > > >>> queue.type="linkedlist" > > >>> queue.FileName="es_queue" > > >>> queue.MaxDiskSpace="1g" > > >>> queue.SaveOnShutdown="on" > > >>> action.resumeretrycount="-1" > > >>> errorFile="/srv/log/rsyslog_ES-error_default-index_audit.log" > > >>> ) > > >>> & stop > > >>> > > >>> Best regards, > > >>> Cristian Falcas > > >>> _______________________________________________ > > >>> rsyslog mailing list > > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >>> http://www.rsyslog.com/professional-services/ > > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > > myriad > > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > > >>> DON'T LIKE THAT. > > >>> > > >>> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com/professional-services/ > > >> What's up with rsyslog? Follow https://twitter.com/rgerhards > > >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > >> DON'T LIKE THAT. > > >> > > > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
