Ciprian, your issue looks very similar. Have you found a work-around? David, permissions seem to be fine and SELinux is disabled (see below). Any other thoughts?
[root@m0058601 rsyslog.d]# ls -la /var/lib/ | grep rsyslog drwx------ 2 root root 4096 Nov 3 13:00 rsyslog [root@m0058601 rsyslog.d]# ls -la /var/spool/ | grep rsyslog drwxr-xr-x 2 root root 4096 Mar 27 22:34 rsyslog [root@m0058601 rsyslog.d]# sestatus SELinux status: disabled Thanks, Alec On Mon, Mar 28, 2016 at 12:59 PM, Ciprian Hacman < [email protected]> wrote: > Seems very similar to this discussion. Unfortunately, never got the chance > to understand what happened. > http://lists.adiscon.net/pipermail/rsyslog/2015-August/041020.html > > Ciprian > > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > On Mon, Mar 28, 2016 at 7:25 PM, David Lang <[email protected]> wrote: > > > On Sat, 26 Mar 2016, Alec Swan wrote: > > > > Hi there, > >> > >> I am using omelasticsearch module to send logs to elasticsearch server > and > >> started noticing the "fatal error on disk queue" error shown below. I > also > >> noticed a 560 byte .qi file created for the queue configured for > >> omelasticsearch action as shown below. Once I removed the .qi file the > >> error went away. > >> > >> Is there anything wrong with the configuration? If not, how do I go > about > >> troubleshooting this issue? > >> > >> CONFIGURATION > >> > >> action( > >> type = "omelasticsearch" > >> template = "es-payload" > >> dynSearchIndex = "on" > >> searchIndex = "logstash-index" > >> searchType = "syslog" > >> server = "127.0.0.1" > >> serverport = "9200" > >> uid = "xxx" > >> pwd = "yyy" > >> errorFile = "/var/log/rsyslog/ES-error.log" > >> bulkmode = "on" > >> action.resumeretrycount="-1" # retry if ES is unreachable (-1 > for > >> infinite retries) > >> action.resumeInterval="60" > >> queue.dequeuebatchsize="1000" # ES bulk size > >> queue.type="linkedlist" > >> queue.size="100000" > >> queue.workerthreads="5" > >> queue.timeoutworkerthreadshutdown="2000" > >> queue.spoolDirectory="/var/spool/rsyslog" > >> queue.filename="omelasticsearch-queue" > >> queue.maxfilesize="100m" > >> queue.maxdiskspace="1g" > >> queue.highwatermark="80000" # when to start spilling to disk > >> queue.lowwatermark="20000" # when to stop spilling to disk > >> queue.saveonshutdown="on" > >> ) > >> > >> > >> ERROR > >> > >> Mar 27 04:02:04 m0058180 rsyslogd-2040: fatal error on disk queue > 'action > >> 4 > >> queue[DA]', emergency switch to direct mode [v8.14.0 try > >> http://www.rsyslog.com/e/2040 ] > >> Mar 27 04:02:04 m0058180 rsyslogd: [origin software="rsyslogd" > >> swVersion="8.14.0" x-pid="2648" x-info="http://www.rsyslog.com";] start > >> Mar 27 04:02:04 m0058180 rsyslogd-2040: fatal error on disk queue > 'action > >> 2 > >> queue[DA]', emergency switch to direct mode [v8.14.0 try > >> http://www.rsyslog.com/e/2040 ] > >> > > > > usually this means that there is a permission problem (including SELinux > > permissions) when trying to access the files in the work directory. > > > > David Lang > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
