Re: [rsyslog] Help select a new logo

Nice. Logo 1 from me also (voted). Seems the cleanest one. Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Fri, Feb 2, 2018 at 9:19 AM, Rainer Gerhards wrote: > 2018-02-01 19:20 GMT+01:00

Re: [rsyslog] ppa:adiscon/v8-stable ==> 404

Hi Florian, We figured out that this was an error on our part. Sorry for that. One of our customers is trying to install Rsyslog on Debian Jessie, but seems that is not possible at the moment. The only Rsyslog repo is for Wheezy. Any plans to build Rsyslog for Jessie? Thanks, Ciprian --

Re: [rsyslog] Fatal error on disk queue

tus > SELinux status: disabled > > Thanks, > > Alec > > On Mon, Mar 28, 2016 at 12:59 PM, Ciprian Hacman < > ciprian.hac...@sematext.com> wrote: > > > Seems very similar to this discussion. Unfortunately, never got the > chance > > to un

Re: [rsyslog] Fatal error on disk queue

Seems very similar to this discussion. Unfortunately, never got the chance to understand what happened. http://lists.adiscon.net/pipermail/rsyslog/2015-August/041020.html Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On

Re: [rsyslog] Rsyslog using huge amounts of memory

just traced down to json-c) and > have had no memory issues. > > But I'm not using imfile or omelasticsearch. > > David Lang > > On Wed, 16 Dec 2015, Ciprian Hacman wrote: > > Date: Wed, 16 Dec 2015 18:00:48 +0200 >> From: Ciprian Hacman <ciprian.hac...@sematext.com>

Re: [rsyslog] Rsyslog using huge amounts of memory

3 PM, Rainer Gerhards <rgerha...@hq.adiscon.com> wrote: > 2015-12-16 13:48 GMT+01:00 Ciprian Hacman <ciprian.hac...@sematext.com>: > > Hi, > > > > I upgraded a server to Rsyslog 8.15 last night and today the process was > > using almost 200MB of RAM (raisi

Re: [rsyslog] Rsyslog using huge amounts of memory

Done. Can you check the gist again? Thanks, Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Wed, Dec 16, 2015 at 3:40 PM, Rainer Gerhards <rgerha...@hq.adiscon.com> wrote: > 2015-12-16 14:16 GMT+01:00 C

Re: [rsyslog] Rsyslog using huge amounts of memory

iner Gerhards <rgerha...@hq.adiscon.com> wrote: > 2015-12-16 15:10 GMT+01:00 Ciprian Hacman <ciprian.hac...@sematext.com>: > > Done. Can you check the gist again? > > yup. It now contains better info, but I wonder where these leaks stem > from. One suspect is inotify po

Re: [rsyslog] Rsyslog using huge amounts of memory

Lang <da...@lang.hm> wrote: > On Tue, 15 Dec 2015, Ciprian Hacman wrote: > > Hi David, >> >> maxMessageSize="1" >> queue.size="1" - main queue >> queue.size="1" - elasticsearch queue >> >> Based on my calc

Re: [rsyslog] Rsyslog using huge amounts of memory

2015 at 5:23 PM, Rainer Gerhards <rgerha...@hq.adiscon.com> wrote: > 2015-12-16 16:21 GMT+01:00 Ciprian Hacman <ciprian.hac...@sematext.com>: > > Not sure how easy is to reproduce on your side. > > If you need help narrowing the leak to a certain commit or release, I > cou

Re: [rsyslog] rsyslog 8.15.0 (v8-stable) released

Thanks! -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Tue, Dec 15, 2015 at 7:33 PM, Peter Portante wrote: > This is great, nice work! > > On Tue, Dec 15, 2015 at 12:08 PM, Florian Riedl

Re: [rsyslog] Rsyslog using huge amounts of memory

me fixes for memleaks that very seldom show up. If the problem > persists, it makes sense to debug, but I'd prefer not to hunt > potentially fixed bugs... > > Rainer > > > > David Lang > > > > On Tue, 15 Dec 2015, Ciprian Hacman wrote: > > > >> Da

Re: [rsyslog] Delayed batch processing

trycount="5" # retry if ES is unreachable (-1 > for > >> infinite retries) > >> action.resumeInterval="60" > >> queue.dequeuebatchsize="1000" # ES bulk size > >> queue.type="FixedArray" > >

Re: [rsyslog] Rsyslog using huge amounts of memory

-- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Tue, Dec 15, 2015 at 7:57 AM, Peter Portante <peter.a.porta...@gmail.com> wrote: > On Tue, Dec 15, 2015 at 12:30 AM, Ciprian Hacman < > ciprian.hac...@sematext.c

Re: [rsyslog] Rsyslog using huge amounts of memory

of (sometime esoteric) memory leaks in 8.15 > upcoming tomorrow. Maybe there is already a fix included. > > Rainer > > 2015-12-14 16:04 GMT+01:00 Ciprian Hacman <ciprian.hac...@sematext.com>: > > Hi, > > > > We are noticing some Rsyslog instances that push

[rsyslog] Rsyslog using huge amounts of memory

Hi, We are noticing some Rsyslog instances that push about 500MB of logs daily to an Elasticsearch instance, so not that much. We noticed one of the Rsyslog processes using about 6GB of RAM. Usually this is less than 1MB. I plan on debugging this in the next few days, but wanted to see also if

Re: [rsyslog] Rsyslog using huge amounts of memory

log redundant log relay servers in each datacenter and > then have all the systems log to these relays via UDP. UDP is reliable over > a local network, but if there is a problem with the receiving system, it > will go ahead and loose logs rather than affecting the sending system. > > David Lan

Re: [rsyslog] Rsyslog using huge amounts of memory

Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Mon, Dec 14, 2015 at 8:17 PM, David Lang <da...@lang.hm> wrote: > On Mon, 14 Dec 2015, Ciprian Hacman wrote: > > Hi, >> >> We are noticing some Rsyslog instances that

Re: [rsyslog] Delayed batch processing

I see Alec is using imfile and omelasticsearch so I suppose he is tailing a log file. By any chance, are you using Cassandra to capture multiline logs (using the startmsg.regex setting)? That would explain why the last log line is not sent until restart. Ciprian -- Performance Monitoring * Log

Re: [rsyslog] Delayed batch processing

Hi Alec, Actually, Rsyslog doesn't wait for the batch size to fill. Once it gets a log, it starts preparing a batch and sends it as soon as it's ready to send it. We use queue.dequeueslowdown to slow it down to get more logs in a batch. Do you log suspensions? global (

Re: [rsyslog] Problems with queue settings

Hi Alec, I think the names you were looking for are: - queue.timeoutshutdown - https://github.com/rsyslog/rsyslog/blob/master/runtime/queue.c#L118 - queue.timeoutworkerthreadshutdown - https://github.com/rsyslog/rsyslog/blob/master/runtime/queue.c#L121 Regards, Ciprian -- Performance Monitoring

Re: [rsyslog] Time Format

iscon.com> wrote: > 2015-11-24 23:55 GMT+01:00 David Lang <da...@lang.hm>: > > On Tue, 24 Nov 2015, Rainer Gerhards wrote: > > > >> 2015-11-24 11:36 GMT+01:00 Ciprian Hacman <ciprian.hac...@sematext.com > >: > >>> > >>> Hi David, &g

Re: [rsyslog] Time Format

) that don't comply to either of the syslog >>> RFCs. And then we could use mmnormalize to parse them. Goes into the >>> direction of "rsyslog is not only for syslog". >>> -- >>> Performance Monitoring * Log Analytics * Search Analytics >>> Sol

Re: [rsyslog] Time Format

Hi, I was actually thinking of creating a PR for accepting " " instead of "T" between date and time. @Rainer: Would it be ok? Thanks, Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Mon, Nov 16, 2015 at 2:13 PM, Radu

Re: [rsyslog] Parse multiple files with different mmnormalize rules

Hi Alec, For each file input you can assign a ruleset. Each ruleset can contain various actions like normalizing. http://www.rsyslog.com/doc/v8-stable/concepts/multi_ruleset.html Regards, Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support *

Re: [rsyslog] getting core dumps in ubuntu 14.04

Thanks for the tips. I was really looking for something like that deb option a month ago. Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Thu, Oct 22, 2015 at 3:21 AM, David Lang wrote: > also, to make

Re: [rsyslog] segfault problems

rs:main Q:Reg[14469]: segfault at 7fc31c023000 ip 7fc34d85cf2c sp > > 7fc3418412d0 error 6 in rsyslogd[7fc34d838000+8d000] > > > > Error 6 seems to be no such device or address. > > > > I think we have a problem here . > > > > On 10/7/15 1:29 P

Re: [rsyslog] segfault problems

I strongly recommend using 8.13 if you push logs to ES. There are many useful patches since 8.10 (some of them might not be in the changelog though). Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Wed, Oct 7, 2015 at

Re: [rsyslog] Accepting and parsing GELF?

Hi, I tried to add GELF parsing to our servers but I have no idea how to process the timestamp. GELF requires timestamp to be in "Seconds since UNIX epoch with optional decimal places for milliseconds". https://www.graylog.org/resources/gelf/ Extracting it is not an issue, but is there a way to

[rsyslog] Daily builds for Trusty

Hi, Is there any plan to make daily builds for Trusty (latest Ubuntu LTS version)? I think daily builds are now done only for Vivid and Wily. http://ppa.launchpad.net/adiscon/v8-devel/ubuntu/pool/main/r/rsyslog/ Thanks, Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr &

Re: [rsyslog] recommendations for omelasticsearch queue sizes

Hi Risto, I think your queue sizes are quite small for the amount of logs you receive. It should be able to hold data for at least N seconds, so N x 5000. dequeuebatchsize should also be higher, maybe 5000. resumeretrycount="-1" is nice in theory, but a reasonable value would work better in

[rsyslog] Infinite amplified loop in action.c

Hi, Lately we stared noticing issues with out Rsyslog receiver which, about once a week, starts using a lot of CPU and memory. At the same time, we also see some logs duplicated 1000+ times. We are also using Rsyslog in our infrastructure to forward logs from all servers to Elasticsearch. When

[rsyslog] Rsyslog 8.12 Ubuntu builds missing

Hi, I tried installing rsyslog 8.12 on some of our systems to fix the issues we encountered with imfile. Unfortunately, I discovered that most packages for Ubuntu were not built: http://ppa.launchpad.net/adiscon/v8-devel/ubuntu/pool/main/r/rsyslog/ Can something be done to build remaining