I think I managed to solve this by calling the elasticsearch remote rule from inside the remote rule:
# the remote rule is defined like: $RuleSet remote $RulesetCreateMainQueue on call remote_elasticsearch # and my elastic search config: $ModLoad omelasticsearch $RuleSet remote_elasticsearch $RulesetCreateMainQueue on >From what I've read, this should send the messages asynchronously between the rules. Best regards, Cristian Falcas On Wed, Jul 23, 2014 at 2:16 PM, Cristian Falcas <[email protected]> wrote: > Sorry for being so vague. > > What I'm trying to do is to send messages received via tcp module to 2 > different output modules: > - one to write to local files > - second one to send messages to elasticsearch > > Because in my elasticsearch configuration I have some drop rules, nothing > reaches the omfile rules. How can I decouple those 2? I want everything > that comes from tcp to go to both output modules and not be tied to each > other. > > Is there a way to achive this? > > Best regards, > Cristian Falcas > > > > > On Tue, Jul 22, 2014 at 9:10 PM, David Lang <[email protected]> wrote: > >> you have the right idea, I don't understand what you are asking for help >> on. >> >> David Lang >> >> >> On Tue, 22 Jul 2014, Cristian Falcas wrote: >> >> Hi, >>> >>> I have configured a rsyslog server where I want to send the logs from the >>> other machines. >>> >>> Here I want to keep local logs and also to send them to elastic search. >>> For >>> elasticsearch I have multiple actions, because I want httpd (for ex.) to >>> go >>> to searchType="httpd" and so on. After each action I would like to drop >>> the previously catched lines, because I want a last action with the >>> default >>> search type. Because of my dop rule, nothing reaches past the first >>> action: >>> not the omfile writer and not the elasticsearch actions. >>> >>> I'm using the same $RuleSet remote for both output modules. >>> >>> Can anyone help me in setting this correctly? >>> >>> Something like this (with multiple templates and actions): >>> >>> $RuleSet remote >>> >>> $ModLoad omelasticsearch >>> >>> template(name="10-audit" >>> type="list" >>> option.json="on"). >>> { >>> constant(value="{") >>> constant(value="\"@timestamp\":\"") >>> property(name="timereported" dateFormat="rfc3339") >>> constant(value="\",\"timereported\":\"") >>> property(name="timereported" dateFormat="rfc3339") >>> constant(value="\",\"timegenerated\":\"") >>> property(name="timegenerated" dateFormat="rfc3339") >>> constant(value="\",\"message\":\"") >>> property(name="msg") >>> constant(value="\",\"host\":\"") >>> property(name="hostname") >>> constant(value="\",\"severity\":\"") >>> property(name="syslogseverity-text") >>> constant(value="\",\"priority\":\"") >>> property(name="syslogpriority-text") >>> constant(value="\",\"facility\":\"") >>> property(name="syslogfacility-text") >>> constant(value="\",\"tag\":\"") >>> property(name="syslogtag") >>> constant(value="\",\"program_name\":\"") >>> property(name="programname") >>> constant(value="\"}") >>> } >>> *.* action(type="omelasticsearch" >>> name="action_10-audit" >>> server="v-so-repo-02" >>> serverport="9200" >>> template="10-audit" >>> searchIndex="default-index" >>> searchType="audit" >>> bulkmode="on" # use the Bulk API >>> queue.dequeuebatchsize="5000" # ES bulk size >>> queue.size="100000" # capacity of the action queue >>> queue.workerthreads="15" # 5 workers for the action >>> queue.type="linkedlist" >>> queue.FileName="es_queue" >>> queue.MaxDiskSpace="1g" >>> queue.SaveOnShutdown="on" >>> action.resumeretrycount="-1" >>> errorFile="/srv/log/rsyslog_ES-error_default-index_audit.log" >>> ) >>> & stop >>> >>> Best regards, >>> Cristian Falcas >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
