Re: [rsyslog] Reject incoming instead of discarding messages when queue is full

El 14/09/16 a las 17:13, Radu Gheorghe escribió: Hi, Oh, I got it. Actually, that's the default behavior. So if Redis output wouldn't drop messages, the queue would get full and then the input wouldn't accept messages. Actually, rsyslog is a bit smarter than that: there are delay marks (search

[rsyslog] Reject incoming instead of discarding messages when queue is full

Hi Newbie here with a bunch of questions about rsyslog. Hope someone could help me. In our scenario, we are receiving messages from a central rsyslog server using RELP and forwarding them to Redis. CENTRAL_SYSLOG --(relp)--> MY_SYSLOG > REDIS According to [1], rsyslog supports

[rsyslog] Newbie: I'm not able to forward RELP to Redis. Any clue?

Hi I'm trying to forward RELP messages to REDIS, but I haven't succeeded so far. *That's my /etc/rsyslog.conf (default's ugly-legacy format):* $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd)

Re: [rsyslog] Newbie: I'm not able to forward RELP to Redis. Any clue?

Related to this: http://www.rsyslog.com/doc/master/configuration/modules/omhiredis.html has a few erratas on examples, where it states */port/* instead of /*serverport*/. Does it worth to open an issue? Where? Regards El 16/09/16 a las 12:08, mosto...@gmail.com escribió: Hi I'm trying

Re: [rsyslog] liblognorm vs grok

El 04/10/16 a las 20:31, Joe Blow escribió: Regex should be avoided like the plague, at all costs. If you know your logs well enough to write a regex for them, why wouldn't you write a liblognorm rule instead? Totally agree...(actually, liblognorm is giving me segfaults :P) I use

Re: [rsyslog] liblognorm vs grok

Totally agree...(actually, liblognorm is giving me segfaults :P) I'll try to check next week when my current task is done. I know you're busy...trying to contribute as much as I can with everything I deal with on my daily work. Liblognorm is based on work from my MSc Thesis. The thesis

Re: [rsyslog] Does indented readmode supports TAB?

ping? El 03/10/16 a las 10:25, mosto...@gmail.com escribió: Hi According to http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html /readMode [mode]// //...// //2 - indented (new log messages start at the beginning of a line. If a line starts with a *space* it is part

[rsyslog] liblognorm vs grok

Hi Radu After reading http://lists.adiscon.net/pipermail/rsyslog/2013-December/035122.html and considering several years have passed, I would like to get some feedback of your experience, to help me choose between raw forwarding messages+logstash or split before forwarding with mmnormalize.

[rsyslog] Dynamic value for action parameter (workaround?)

Hi As exposed a few days ago, I'm trying to do something like: action(type="omfile" file="$!dynamicvalue") According to some comments that's not possible. Otherwise, please, let me know!!!. Can you figure out a workaround to achieve this? ...like using queue="$variable" and then reading

Re: [rsyslog] Dynamic value for action parameter (workaround?)

file="c" } ) ...one action for each field-value seems enough to kill many cats at once. El 21/09/16 a las 13:46, mosto...@gmail.com escribió: Hi As exposed a few days ago, I'm trying to do something like: action(type="omfile" file="$!dynamicvalue&qu

Re: [rsyslog] Dynamic value for action parameter (workaround?)

ards escribió: Look at the dynafile Parameter. Sent from phone, thus brief. Am 21.09.2016 13:46 schrieb "mosto...@gmail.com" <mosto...@gmail.com>: Hi As exposed a few days ago, I'm trying to do something like: action(type="omfile" file="$!dynamicvalue") A

[rsyslog] Compile warning advice

Hi While building rsyslog I'm getting a probably-false warning: rsyslogd.c: In function 'writePidFile': rsyslogd.c:185:2: warning: ignoring return value of 'asprintf', declared with attribute warn_unused_result [-Wunused-result] asprintf((char **), "%s.tmp", PidFile); ^ Checking

Re: [rsyslog] Newbie: I'm not able to forward RELP to Redis. Any clue?

After HOURS and at the edge of craziness I realized I missed /serverpassword/ property. Damn code blindness. El 16/09/16 a las 12:08, mosto...@gmail.com escribió: Hi I'm trying to forward RELP messages to REDIS, but I haven't succeeded so far. *That's my /etc/rsyslog.conf (default's

[rsyslog] Does indented readmode supports TAB?

Hi According to http://www.rsyslog.com/doc/v8-stable/configuration/modules/imfile.html /readMode [mode]// //...// //2 - indented (new log messages start at the beginning of a line. If a line starts with a *space* it is part of the log message before it)/ I know a I could try it by

Re: [rsyslog] Split property into multiple fields

ping? El 21/10/16 a las 15:11, mosto...@gmail.com escribió: Hi Having logs like: /TIMESTAMP host app/pkg/class[1234]: this is my msg/ /TIMESTAMP host2 anotherApp/aNoTherPkg/aClass[1234]: this is another msg TIMESTAMP super-host thisisanotherapp/aClassWithoutPackage[1234

[rsyslog] Split property into multiple fields

Hi Having logs like: /TIMESTAMP host app/pkg/class[1234]: this is my msg/ /TIMESTAMP host2 anotherApp/aNoTherPkg/aClass[1234]: this is another msg TIMESTAMP super-host thisisanotherapp/aClassWithoutPackage[1234]: guess what? this is another msg!/ *How could I split syslogtag* (I

Re: [rsyslog] Would imhiredis make sense?

Logstash needs something like redis because it can't do any queueing itself. Rsyslog is built around queues, and has the ability to create multiple queues and piplines internally, you don't need to run multiple instances. I want multiples instances in order to: * Being able to process

[rsyslog] A couple of questions regarding rsyslog debug.

Hi In the past I have to build a few applications with C/C++ and I found #ifdef directives everywhere to be ugly and confusing. For example, using ifdef DEBUG to enable/disable debug messages. That's why I opt to use functions which behavior was defined depending on compile headers/source

Re: [rsyslog] Feedback request: minimal log shipper project

Could any of you, aware of rsyslog internals, provide a design diagram/architecture view of how the components are working? Perhaps you could use something like: https://docs.google.com/presentation/d/1UJSO0t2J6mrIyBNSkCUvrR5Q4RWbLcOof2WgB2_qA3g/edit#slide=id.g13bdc00843_0_3 I think, apart

[rsyslog] mmnormalize with mutiple input: conditionals?

Hi Considering our imrelp module receives a lot of messages from different applications/hosts (with different TAGs), what would be the best way to split/forward each message to proper ruleset/parser? Should I use /_if/_? Does

Re: [rsyslog] Are we building an ERK stack?

+1 Our current scenario (dockerized!): imfile_forwarder-->imrelp-->rsyslog-->redis-->logstash(grok+geoip)-->elastic We are using redis as memory buffer and to split into multiple channels/lists (using dynakey ATM). We see kafka on the horizon. We are also using several logstash

Re: [rsyslog] Feedback request: minimal log shipper project

./configure --help shows these options. I'm wondering if having a minimal build that created a different binary, used a different config and stripped everything down would be useful Didn't notice I could use --disable-whatever on those features having [default=yes]. Anyway, imjournal is

Re: [rsyslog] Are we building an ERK stack?

there are probably ways to simplify the configs, 5K lines of configs seems excessive :-) how much of this is rulebase config vs rsyslog config? Each app generates app-access.log, app-tomcat.log, app-application.log files. imfile allow me to add filename as metadata, but nothing more. As each

Re: [rsyslog] Are we building an ERK stack?

. If not, what would give you the Mets data? Rainer Sent from phone, thus brief. Am 23.11.2016 15:51 schrieb "mosto...@gmail.com" <mosto...@gmail.com>: there are probably ways to simplify the configs, 5K lines of configs seems excessive :-) how much of this is rulebase config v

Re: [rsyslog] Feedback request: minimal log shipper project

brief. Am 23.11.2016 16:03 schrieb "David Lang" <da...@lang.hm>: On Wed, 23 Nov 2016, mosto...@gmail.com wrote: Could any of you, aware of rsyslog internals, provide a design diagram/architecture view of how the components are working? Perhaps you could use somet

Re: [rsyslog] Are we building an ERK stack?

http://www.slideshare.net/chenryn/elk-stack-at-weibocom I NEED the english version :P ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow

Re: [rsyslog] Feedback request: minimal log shipper project

Open question to the community: does it really make sense to create a new repo at this stage? I'll try to start as clean as possible, but I agree I can start writing anywhere Name + Scope I mean this project will NOT host rsyslog code, so what will it host? Shouldn't we benefit from

Re: [rsyslog] Feedback request: minimal log shipper project

I don't think a fork make sense, why keep two versions? As a branch in rsyslog-repo, that makes a lot of sense, but I'd move in smaller refactoring steps and merge each one as early as possible. A total rewrite from scratch is out of question, except if somone has a couple of month to years

Re: [rsyslog] Feedback request: minimal log shipper project

That's a permission issue: We need to be much more restrictive (security) with who has permissions to the code than to the doc. Thus we have two repos. I'd prefer a single one, too, but that's not possible. Understood...does this happened in real life or just in paper? :P I mean: if there are

Re: [rsyslog] Feedback request: minimal log shipper project

Let me start writing, cause it may be helpful as documentation too! Please do. I still need a codename! Sorry for that...but I NEED IT! ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog

Re: [rsyslog] Feedback request: minimal log shipper project

1:00 mosto...@gmail.com <mosto...@gmail.com>: https://en.wikipedia.org/wiki/Erk ...I know you can do it better... :P El 23/11/16 a las 17:34, Rainer Gerhards escribió: ERK 2016-11-23 17:34 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: Let me start writing, cause it may be helpfu

Re: [rsyslog] Feedback request: minimal log shipper project

Does this help? Although I had all these clear, reading it helped me to better understand. Thank you. A few questions appear: * You talk about threads...may we considered a distributed application architecture/other process reading that queue? That could be done with a redis/kafka

Re: [rsyslog] Feedback request: minimal log shipper project

That's a permission issue: We need to be much more restrictive (security) with who has permissions to the code than to the doc. Thus we have two repos. I'd prefer a single one, too, but that's not possible. Understood...does this happened in real life or just in paper? :P I mean: if there are

Re: [rsyslog] Feedback request: minimal log shipper project

Totally agreed with your comments. That's why I want a collaborative doc, where I can randomly write and you'll wisely answer. Let me start writing, cause it may be helpful as documentation too! ___ rsyslog mailing list

Re: [rsyslog] Feedback request: minimal log shipper project

https://en.wikipedia.org/wiki/Erk ...I know you can do it better... :P El 23/11/16 a las 17:34, Rainer Gerhards escribió: ERK 2016-11-23 17:34 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: Let me start writing, cause it may be helpful as documentation too! Please do. I stil

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

My recommended architecture [1] is to have a local relay picking up logs from each network/datacenter, add useful metadata (fromhost-ip, what environment this is, which relay processed it, timestamp of when the log was processed on the relay, etc) and then forward the message to a central

Re: [rsyslog] Feedback request: minimal log shipper project

why add the overhead of transporting the data between machines if you don't need to? Rsyslog is FAST, it routinely does things on one machine that other systems need many farms of machines to do. adding more machines and network connectivity will add complexity and delays, reducing

Re: [rsyslog] Are we building an ERK stack?

Hi all In order to improve the first draft of ERK project, I would like to get some feedback from you. What features are you missing/you think it may be improved in rsyslog? Please, try to be as more clear/self-explanatory/simple as you can for better understanding. * logstash memory

Re: [rsyslog] Are we building an ERK stack?

The ugly way to do this would be a series of if $programname = "group/appX" then set $.owner = "\"group\":\"group\","\"unit\":\"unit\",\"app\":\"appX\","; do having multiple templates affect performance? (what I really noticed it's they affect loading time!) a far more elegant way to do

Re: [rsyslog] Feedback request: minimal log shipper project

We both reached an agreement about exceptions and I'm happy with that :) Actually my main concern is, each time I read a line like: CHKiRet(statsobj.Construct(>statsobj)); I have to start digging what CHKiRet is, or what the hell is: CODE_STD_STRING_REQUESTparseSelectorAct(1) Anyway,

Re: [rsyslog] Are we building an ERK stack?

The problem is the fact that there are so many ways timestamp data can be scattered in a log message. take a look at the output of date --help and look at all the formatting options. I guarantee that some log somewhere will use every one of them. IIRC, you had found a solution to this...

Re: [rsyslog] A couple of questions regarding rsyslog debug.

yup, the function is named DBGPRINTF() and some helpers. Being used since ~1970 (the original dbgprintf() call stems back to the original code ;)). Sometimes we use #ifdef DEBUG, but only if we actually need to compute something beyond simple output generation (e.g. iterate over a structure).

Re: [rsyslog] omriemann Re: Are we building an ERK stack?

As main promoter (ring the bell and run like hell), could you make some tests comparing filebeat vs imfile performance and footprint? El 23/11/16 a las 19:32, Bob Gregory escribió: I can easily enough knock together an omriemann - it's protobuf over TCP or UDP. TCP allows for message ack.

Re: [rsyslog] Are we building an ERK stack?

you can combine variables to form a string that looks like a date in the output, but you can't take arbitrary date parts in a log message and parse them into a real timestamp field that would let you output it in different formats. back on my pipeline proposal, wouldn't this solve the issue?

Re: [rsyslog] Feedback request: minimal log shipper project

do you have an example of what was sent over the wire and how it was mishandled? It may be that we need to tweak the default parsers. https://github.com/rsyslog/rsyslog/issues/1191 no need to convert, just add the ability to use the slower, but more flexible mmnormalize. You said it has

[rsyslog] Would imhiredis make sense?

Hi We've been playing with logstash, rsyslog and redis for a while in order to *index into elasticsearch a bunch of application logs*. Briefly: app1-file1.log, app1-file2.log...appN-fileX.log -> pipeline -> elasticsearch. So far, we are using *redis queues and _each application_ processing

Re: [rsyslog] Feedback request: minimal log shipper project

Quite interested on this thread http://blog.gerhards.net/2016/11/would-creating-simple-linux-log-file.html IMHO: the "issue" we're having with rsyslog is understanding the code rather than performance. Why you'll invest time doing a "faster reader" if you already have a "fast-enough

Re: [rsyslog] Would imhiredis make sense?

What sort of log volume are you talking about here? (logs/sec type of thing) From 0 to thousand-thousands/sec Logstash needs something like redis because it can't do any queueing itself. Rsyslog is built around queues, and has the ability to create multiple queues and piplines internally,

Re: [rsyslog] making config changes to a running rsyslog

what are people's thoughts on these ideas? Notice there can be multiple reload scenarios: - reload rsyslog config (new modules, inputs, rulesets, actions...) - add new inputs - modify a template - resize a queue The simplest approach I can imagine is to signal HUP to reload: when signal

Re: [rsyslog] Are we building an ERK stack?

. On Wed, 23 Nov 2016 at 19:38 mosto...@gmail.com <mosto...@gmail.com> wrote: Working, spamming mail list and writing on wiki at the same time. A lovely afternoon... Please, add your lines: https://github.com/rsyslog/rsyslog/wiki ___ rsyslo

Re: [rsyslog] rsyslog fails to start due to high queue

TBH, it depends if you prefer building from source or instability Kidding away, I know there are problems in older versions, they are Isn't adiscon repo valid? ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog

Re: [rsyslog] Are we building an ERK stack?

Thanks! It's your mmdblookup opensourced? El 25/11/16 a las 03:46, chenlin rao escribió: re-upload an english version. The content was a little old though. 2016-11-23 22:39 GMT+08:00 mosto...@gmail.com <mosto...@gmail.com>: http://www.slideshare.net/chenryn/elk-stack-at-weibocom

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

After sleeping on it, I'm still thinking about "defining a separate pipeline for each application". To sum up, each application could do his own thing and return the message to the queue, in order to be processed by other modules...until it's done, and indexed into ES Is there a way to put the

Re: [rsyslog] rsyslog fails to start due to high queue

ri, 25 Nov 2016, mosto...@gmail.com wrote: TBH, it depends if you prefer building from source or instability Kidding away, I know there are problems in older versions, they are Isn't adiscon repo valid? It looks like we don't have a repo for Jessie, at least it's not included in the scripts/con

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

What about...? *remote.conf*(Please, notice there are commented questions) global( MaxMessageSize="32k" ) template(name="json" type="string" string="%timestamp:::date-rfc3339% %hostname% logs/$!data!group/$!data!app %$!data%") module(load="omrelp")

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

I may be confused about which part is on the sender and which part is on the receiver. sender: a bunch of imfiles forwarded using RELP receiver: receives a JSON with msg=plain/original message, normalize and extract fields (that should be added to JSON) each "application" should define his

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

string="<%pri%>%timestamp:::date-rfc3339% %hostname% logs/$!data!group/$!data!app: %$!data%") to be fully correct Done. also watch out, the programname is limited to 32 characters, don't let your group and app names get too long. Wasn't it possible to change that? IIRC we had some issues

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Last hours have been quite busy! After reading your comments, here's where we are. Please, advice and comment. On some remote hosts, logs are readed from files. Each remote will have a configuration like: # remote.conf global( MaxMessageSize="32k" )

Re: [rsyslog] Are we building an ERK stack?

not really, but we haven't had anyone experiment with thousands of them, so it's possible, but unlikely that there would be a measureable slowdown as rsyslog finds the right one to use. The bigger overhead is in interpreting the template, that's where simplifying it to be $! or $!foo would

Re: [rsyslog] Are we building an ERK stack?

or docker swarm mode :D El 24/11/16 a las 16:22, David Lang escribió: On Thu, 24 Nov 2016, mosto...@gmail.com wrote: As we are concerned about high availability and load balancing, we plan to deploy multiple instances. just a note that while rsyslog doesn't implement load balancing itself

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Once I use it ;) Yet have to change overview to RST... El 24/11/16 a las 17:36, Rainer Gerhards escribió: 2016-11-24 17:21 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: # I would consider adding a section here to look for parsing failures and log them to someplace for

Re: [rsyslog] Coding doc

I mean https://github.com/rsyslog/rsyslog-doc/pull/188 El 23/11/16 a las 21:25, Rainer Gerhards escribió: Sent from phone, thus brief. Am 23.11.2016 21:01 schrieb "mosto...@gmail.com" <mosto...@gmail.com>: What did happen to the "new documentation format" effort? I

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

this way you use the tag that you set on the input line to create the metadata fields you want with only one ruleset for all the app inputs. cool! The other thing you could do is to create a table lookup to map the programname (the value you set as the tag in the input) to the values you

Re: [rsyslog] Are we building an ERK stack?

El 24/11/16 a las 17:42, Rainer Gerhards escribió: I added a project to rsyslog on github, where we can bind Issues to: shouldn't that work for documentation? (as commented previously, I would love to have 1 repo!) https://github.com/rsyslog/rsyslog/projects/1 I guess I must make the

Re: [rsyslog] Are we building an ERK stack?

Working, spamming mail list and writing on wiki at the same time. A lovely afternoon... Please, add your lines: https://github.com/rsyslog/rsyslog/wiki ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog

Re: [rsyslog] Coding doc

What did happen to the "new documentation format" effort? I think wiki could be fair enough, but I don't know how it will behave for old-releases. If I'm not wrong, current system allow to have different documentation pages according to each version, so perhaps it worths keep it. On the other

Re: [rsyslog] lowercasing structured-data SD names

lowercase? Regards El 16/11/16 a las 16:05, Rainer Gerhards escribió: 2016-11-16 15:38 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: A quick look up returned: https://github.com/rsyslog/rsyslog/search?utf8=%E2%9C%93=tolower This is relevant: https://github.com/rsyslog/rsyslog/issues/481 R

Re: [rsyslog] structured data parsing

To be more clear: seems by default jsonroot = NULL, which IMHO should use "root as parent", instead of "rfc5424-sd". Having "rfc5424-sd" as default it's mandatory for backwards compatibility? El 16/11/16 a las 09:31, mosto...@gmail.com escribió: Seems http

Re: [rsyslog] structured data parsing

that if jsonroot is set to '', the fields become top-level objects. David Lang On Wed, 16 Nov 2016, mosto...@gmail.com wrote: Date: Wed, 16 Nov 2016 09:45:08 +0100 From: "mosto...@gmail.com" <mosto...@gmail.com> Reply-To: rsyslog-users <rsyslog@lists.adiscon.com> To: rsyslog@lists.a

Re: [rsyslog] lowercasing structured-data SD names

Rainer, what do you think? A "removed-lowercase" branch could work for this, while helping us to solve #1116 El 16/11/16 a las 11:11, David Lang escribió: On Wed, 16 Nov 2016, mosto...@gmail.com wrote: According to https://tools.ietf.org/html/rfc5424#section-6.3.2

Re: [rsyslog] lowercasing structured-data SD names

A quick look up returned: https://github.com/rsyslog/rsyslog/search?utf8=%E2%9C%93=tolower El 16/11/16 a las 11:48, mosto...@gmail.com escribió: Rainer, what do you think? A "removed-lowercase" branch could work for this, while helping us to solve #1116 El 16/11/16 a las 11

[rsyslog] lowercasing structured-data SD names

Hi According to https://tools.ietf.org/html/rfc5424#section-6.3.2 and https://tools.ietf.org/html/rfc5424#section-6.3.3 both are *case-sensitive*. As stated in http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html : "Note that the case of the RFC5424 parameter names

[rsyslog] structured data parsing

Hi According to http://www.rsyslog.com/doc/v8-stable/configuration/modules/mmpstrucdata.html, structure data ID and parameters are stored under *rfc5424-sd* element in JSON. It's there any way to store them as /firstchild/ elements (not using rfc5424-sd as parent, but !) ? Otherwise,

Re: [rsyslog] I'm back

Did anyone in the list noticed more traffic since David is back!? El 27/10/16 a las 20:51, singh.janmejay escribió: I was wondering about your silence on the mailing list. Welcome back. On Oct 28, 2016 12:03 AM, "David Lang" wrote: I'm back on my feet, but don't have a new

Re: [rsyslog] liblognorm vs grok

a literal space is always more efficent than whitespace, only use whitespace if there can be more than one space, or tabs Ok. just a note, the new syntax is not always better than the old syntax 127.0.0.1 - - [17/Mar/2016:18:15:06 +0100] "GET /redacted HTTP/1.1" 200 59506

Re: [rsyslog] liblognorm vs grok

when troubleshooting things like this, create a rule file that is as minimal as you can get and parse with the -v option, it will show you what it's doing as it walks through the line. Ok :) In this case, I seem to remember that number is defined as being followed by a space, so you can't

Re: [rsyslog] liblognorm vs grok

In this case, I seem to remember that number is defined as being followed by a space, so you can't use it if the number is followed by a newline. I'll have to confirm that...but may I know why? Should I fill an issue if it's indeed that way? I think it's a problem, several of the types

Re: [rsyslog] liblognorm vs grok

Should something like this work? {"type":"@apache"}, {"type":"alternative","parser":[ {}, { {"type":"whitespace"}, ... } ]} El 07/12/16 a las 11:08, Rainer Ge

Re: [rsyslog] liblognorm vs grok

I think it's a problem, several of the types require a space at the end, and I think they should all be modified to allow either a space or a end-of-line. ack. It's on my list for early next year. better check if one exists, I also think David created one. This is for the liblognorm project.

Re: [rsyslog] liblognorm vs grok

almost, %@apache% makes no more sense than %word%, you need to give the match a name so %log:@apache% would work, or if you want to move everything up a later (rather than having $!apache!ip) you could do %.:@apache% That should work How would that be using JSON syntax? {"type":"@apache"

Re: [rsyslog] liblognorm vs grok

when troubleshooting things like this, create a rule file that is as minimal as you can get and parse with the -v option, it will show you what it's doing as it walks through the line. I don't see how it parsed each message. Perhaps a debug option must be enabled? number of tree nodes:

Re: [rsyslog] liblognorm vs grok

I'm still trying to reproduce/understand what is happening and building a test case for the github issue if needed. Consider the following HTTP access lines: 127.0.0.1 - - [17/Mar/2016:18:15:06 +0100] "GET /redacted HTTP/1.1" 200 59506 127.0.0.1 - - [17/Mar/2016:18:15:24 +0100]

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

chnical Product Management On Dec 1, 2016, at 8:28 AM, mosto...@gmail.com<mailto:mosto...@gmail.com> wrote: After meal, as usually happens, those quotes sparkled. Doesn't rsyslog conf grammar allows that neither? /(eg: set $!data!foo="this_doesnt_seem_to_work_$programname";)/

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

After meal, as usually happens, those quotes sparkled. Doesn't rsyslog conf grammar allows that neither? /(eg: set $!data!foo="this_doesnt_seem_to_work_$programname";)/ El 01/12/16 a las 14:57, mosto...@gmail.com escribió: This worked, but I have lost 2 hours and still don't s

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

This worked, but I have lost 2 hours and still don't see where's the evil. Works: module(load="omrelp") ruleset(name="relp") { set $!data!group=field($programname,47,1); set $!data!msg=$msg; action( action.reportSuspension="on"

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Hi I'm still not able to get it, and *perhaps reproduced an issue related to properties not being accessible*... Consider the following relay.conf: global( MaxMessageSize="32k" workDirectory="/var/spool/rsyslog" parser.escapeControlCharactersOnReceive="off" ) template(

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

El 01/12/16 a las 19:04, Rainer Gerhards escribió: 2016-12-01 18:56 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: El 01/12/16 a las 18:37, Rainer Gerhards escribió: 2016-12-01 18:33 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: Hi Is there any way to dynamically invo

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Hi Is there any way to dynamically invoke a ruleset? eg: call $var (I'm trying to avoid having +200 if statements... El 25/11/16 a las 14:13, David Lang escribió: On Fri, 25 Nov 2016, mosto...@gmail.com wrote: I may be confused about which part is on the sender and which part

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

El 01/12/16 a las 18:37, Rainer Gerhards escribió: 2016-12-01 18:33 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: Hi Is there any way to dynamically invoke a ruleset? eg: call $var (I'm trying to avoid having +200 if statements... not yet, but 90% sure evrything is in place t

[rsyslog] about imfile

According to documentation: State files are used to track which parts of the monitored file are already processed. Do state files keep just "last reading position" or as doc suggests a file can be processed in multiple chunks(parts)? Note that when $WorkDirectory is not set or set to a

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

yup, thx. IMHO makes sense. Let's wait what David says, but I think it would make sense to open an issue refering to this thread. ... I know, I also must find time to actually work on some of them... ;-) This is our current /core.conf/ draft: global( MaxMessageSize="32k"

Re: [rsyslog] Are we building an ERK stack?

.com/blog/rek-it/ I wrote this up earlier. On Wed, 23 Nov 2016 at 19:38 mosto...@gmail.com <mosto...@gmail.com> wrote: Working, spamming mail list and writing on wiki at the same time. A lovely afternoon... Please, add your lines: https://githu

Re: [rsyslog] rsyslog-doc github include rst?

availability. Rainer Sent from phone, thus brief. Am 01.12.2016 19:52 schrieb "mosto...@gmail.com" <mosto...@gmail.com>: within github I guessed sphinx would do it for web-fronted, but I was asking for github. El 01/12/16 a las 19:50, Rainer Gerhards escribió: The headers

Re: [rsyslog] rsyslog-doc github include rst?

put of a generation run. Let me know if this helps or more detail is required. Rainer Sent from phone, thus brief. Am 01.12.2016 19:12 schrieb "mosto...@gmail.com" <mosto...@gmail.com>: Hi @radu-gheorghe @rgerhards: Is there any way to include a rst document into another within g

[rsyslog] rsyslog-doc github include rst?

Hi @radu-gheorghe @rgerhards: Is there any way to include a rst document into another within github? I have been trying /raw/ and other directives without success (seems due to security concerns) (Trying to include legal foot/license for every page) Thanks.

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Hi Combined threads reply follow. On Thu, 1 Dec 2016, David Lang wrote sigh, this is getting a wee bit frustrating, you keep saying "it hurts when I do X", we say "that doesn't work well, do Y" and you come back a day or so later saying "but it really huts when I do X"... (it doesn't help

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

Gerhards escribió: 2016-12-02 11:45 GMT+01:00 mosto...@gmail.com <mosto...@gmail.com>: El 02/12/16 a las 10:50, David Lang escribió: remember that values in logs may be under the control of an attacker (just about every field can be hand-crafted by a person sending you a log message). I kn

Re: [rsyslog] Are we building an ERK stack?

El 01/12/16 a las 23:08, David Lang escribió: On Thu, 1 Dec 2016, mosto...@gmail.com wrote: I think that you are going to end up with some grief, if the message could not be insterted into ES for some reason, I think the odds are good that you will find that rawmsg can't be inserted either

Re: [rsyslog] mmnormalize with mutiple input: conditionals?

(as an optimization, make the most common apps the earliest in the directory) And that's where that /if exists call $!tag/ idea makes more sense, cause you don't need to compare, neither to arrange most used applications at top. ___ rsyslog mailing

Re: [rsyslog] REK stack

Rek Project makes us sound like dangerous anarchists, or possibly a dub-techno outfit. LOL ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow

  1   2   >