I think I may have solved this issue. I disabled Selinux and didn't change anything in my config and now its picking up in kibana. So I think selinux was the issue.
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of David Lang Sent: Friday, May 02, 2014 4:20 PM To: rsyslog-users Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server Ok, thanks things were wrapped oddly. I don't see anything obvious here, I would suggest writing to a file with the format plain-syslog and to another file with the format logstash-index It's very possible that something is going in there that's odd David Lang On Fri, 2 May 2014, Josh Bitto wrote: > Date: Fri, 2 May 2014 15:03:20 -0700 > From: Josh Bitto <[email protected]> > Reply-To: rsyslog-users <[email protected]> > To: rsyslog-users <[email protected]> > Subject: Re: [rsyslog] Rsyslog w/ logstash-elasticsearch-kibana server > > I will repost the entire config. > > ###################################################################### > #### > > > # rsyslog configuration file > # note that most of this config file uses old-style format, # because > it is well-known AND quite suitable for simple cases # like we have > with the default config. For more advanced # things, RainerScript > configuration is suggested. > > # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html > # If you experience problems, see > http://www.rsyslog.com/doc/troubleshoot.html > > #### MODULES #### > > module(load="imuxsock") # provides support for local system logging (e.g. via > logger command) > module(load="imklog") # provides kernel logging support (previously done by > rklogd) > #module(load"immark") # provides --MARK-- message capability > > # Provides UDP syslog reception > # for parameters see http://www.rsyslog.com/doc/imudp.html > module(load="imudp") # needs to be done just once input(type="imudp" > port="514") > > # Provides TCP syslog reception > # for parameters see http://www.rsyslog.com/doc/imtcp.html > #module(load="imtcp") # needs to be done just once #input(type="imtcp" > port="514") > > syslog.* /var/log/rsyslogd.log # per Rainers suggestion..note email > > > #### GLOBAL DIRECTIVES #### > > # Use default timestamp format > $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat > > # File syncing capability is disabled by default. This feature is > usually not required, # not useful and an extreme performance hit > #$ActionFileEnableSync on > > # Include all config files in /etc/rsyslog.d/ $IncludeConfig > /etc/rsyslog.d/*.conf > > module(load="impstats" interval="30" severity="7") > > > #### RULES #### > > # Log all kernel messages to the console. > # Logging much else clutters up the screen. > #kern.* /dev/console > > # Log anything (except mail) of level info or higher. > # Don't log private authentication messages! > *.info;mail.none;authpriv.none;cron.none /var/log/messages > > # The authpriv file has restricted access. > authpriv.* /var/log/secure > > # Log all the mail messages in one place. > mail.* /var/log/maillog > > > # Log cron stuff > cron.* /var/log/cron > > # Everybody gets emergency messages > *.emerg :omusrmsg:* > > # Save news errors of level crit and higher in a special file. > uucp,news.crit /var/log/spooler > > # Save boot messages also to boot.log > local7.* /var/log/boot.log > > template(name="logstash-index" > type="list") { > constant(value="logstash-") > property(name="timereported" dateFormat="rfc3339" position.from="1" > position.to="4") > constant(value=".") > property(name="timereported" dateFormat="rfc3339" position.from="6" > position.to="7") > constant(value=".") > property(name="timereported" dateFormat="rfc3339" position.from="9" > position.to="10") } > > # this is for formatting our syslog in JSON with @timestamp > template(name="plain-syslog" > type="list") { > constant(value="{") > constant(value="\"@timestamp\":\"") property(name="timereported" > dateFormat="rfc3339") > constant(value="\",\"host\":\"") property(name="hostname") > constant(value="\",\"severity\":\"") > property(name="syslogseverity-text") > constant(value="\",\"facility\":\"") > property(name="syslogfacility-text") > constant(value="\",\"tag\":\"") property(name="syslogtag" > format="json") > constant(value="\",\"message\":\"") property(name="msg" format="json") > constant(value="\"}") > } > > # this is where we actually send the logs to Elasticsearch > (localhost:9200 by default) action(type="omelasticsearch" > template="plain-syslog" > searchIndex="logstash-index" > dynSearchIndex="on") > > $ModLoad imfile # Load the imfile input module > > # Watch /var/log/httpd/access_log > #$InputFileName /var/log/httpd/access_log #$InputFileTag > apache-access: > #$InputFileStateFile state-apache-access #$InputRunFileMonitor > > # Watch /var/log/httpd/error_log > #$InputFileName /var/log/httpd/error_log #$InputFileTag apache-error: > #$InputFileStateFile state-apache-error #$InputRunFileMonitor > > > # ### begin forwarding rule ### > # The statement between the begin ... end define a SINGLE forwarding # > rule. They belong together, do NOT split them. If you create multiple > # forwarding rules, duplicate the whole block! > # Remote Logging (we use TCP for reliable delivery) # # An on-disk > queue is created for this action. If the remote host is # down, > messages are spooled to disk and sent when it is up again. > #$WorkDirectory /var/lib/rsyslog # where to place spool files > #$ActionQueueFileName fwdRule1 # unique name prefix for spool files > #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) > #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown > #$ActionQueueType LinkedList # run asynchronously > #$ActionResumeRetryCount -1 # infinite retries if host is down > # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional > #*.* @@192.168.1.88:514 > # ### end of the forwarding rule ### > > ###################################################################### > ################### _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE > WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites > beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
