Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

On 10/25/2017 4:18 PM, Naftuli Kay via rsyslog wrote: So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then reference this variable? I'm still trying to learn more about rsyslog variables and how to use them in templates. I'm still learning myself, so I completely

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

So would I do "set $deploy_env = getenv('DEPLOY_ENV')"? How would I then reference this variable? I'm still trying to learn more about rsyslog variables and how to use them in templates. Thanks, - Naftuli Kay On Wed, Oct 25, 2017 at 2:05 PM, deoren <

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

On 10/25/2017 3:48 PM, Naftuli Kay via rsyslog wrote: I have a few environment variables that I'd like to include in my log messages that I'm formatting in JSON format. I have a service that runs on boot which generates /etc/sysconfig/ec2 which contains variables like EC2_INSTANCE_ID,

[rsyslog] handling oversized messages

given the global setting of "maxmessagesize=N", what is my recourse if i need to process a message > N in imfile? in other i/o modules? it appears the message is truncated at ~N, and not split (which is what i thought i had seen in the past...) ___

Re: [rsyslog] handling oversized messages

It may sound dumb, but: increase n! That's why this setting exists. Rainer Sent from phone, thus brief. Am 25.10.2017 19:48 schrieb "Randall Diffenderfer via rsyslog" < rsyslog@lists.adiscon.com>: > > given the global setting of "maxmessagesize=N", what is my recourse if i > need to process a

Re: [rsyslog] handling oversized messages

Yeah but if they are too large... They are. Especially with json payload, what will you do against this? Rainer Sent from phone, thus brief. Am 25.10.2017 20:51 schrieb "Randall Diffenderfer" < rdiffender...@proofpoint.com>: > i suppose i deserved that… :-) > > however, i have to interoperate

Re: [rsyslog] Fwd: Full Journal Output Not Appearing in omfwd

Ok, thanks. I will downgrade. On Oct 25, 2017 12:05, "Rainer Gerhards" wrote: There is unfortunately a regression in 8.30.0. the current cure is to pin 8.29 if you run journal. Sorry I have no better news. Rainer Sent from phone, thus brief. Am 25.10.2017 21:00

Re: [rsyslog] handling oversized messages

i am working with a backlevel version of rsyslogd, so i don't have any hint of that in there. oh well... the remote endpoint is, for all intents and purposes, a black hole; it can be any number of different SIEM or log transport systems, but the main limiter is the "default" 8k barrier. my json

Re: [rsyslog] handling oversized messages

Well, honestly, this sounds like there is need to talk to each other or at least to management... This is a loose-loose-loose (you, the others, company) situation. Rainer Sent from phone, thus brief. Am 25.10.2017 21:18 schrieb "Randall Diffenderfer" < rdiffender...@proofpoint.com>: > i am

[rsyslog] Fwd: Full Journal Output Not Appearing in omfwd

I'm running CentOS 7.4 with systemd 219 with a compiled rsyslog 8.30.0. My rsyslog configuration can be found here: https://gist.github.com/ naftulikay/605e6fcfc90dc1ca7c924baa11bf87db Essentially, I have a TCP 127.0.0.1:514, UDP 127.0.0.1:514, Unix socket /run/systemd/journa/syslog, and the

Re: [rsyslog] handling oversized messages

thanks for the followup. i had come to a similar conclusion. 10# in a 5# sack is gonna leave some stuff on the floor... :-) On 10/25/17, 12:09, "David Lang" wrote: >you can increase N so that messages you receive/read from file are intact >within >rsyslog, but then you need to

Re: [rsyslog] handling oversized messages

There was a recent config option to imfile to allow you to configure between trucating the message and splitting the message to have more of it appear in another message There is no way for rsyslog to combine messages once they have been split, it processes messages one at a time. David

[rsyslog] Sourcing Environment Variables for Use in Templates?

I have a few environment variables that I'd like to include in my log messages that I'm formatting in JSON format. I have a service that runs on boot which generates /etc/sysconfig/ec2 which contains variables like EC2_INSTANCE_ID, EC2_AMI_ID, etc. If I expose these environment variables to the

Re: [rsyslog] handling oversized messages

i suppose i deserved that… :-) however, i have to interoperate with other folks who can't/won't increase N … so, i am pegged at their "N" in order to see no data corruption, as the messages are structured, in that they are json, and busting them up will be a bit problematic… From: Rainer

Re: [rsyslog] handling oversized messages

you can increase N so that messages you receive/read from file are intact within rsyslog, but then you need to figure out what to do with them before you send them you can write them to a local 'oversized' file and send some subset of the log with a tag to say that this isn't the full

Re: [rsyslog] Fwd: Full Journal Output Not Appearing in omfwd

There is unfortunately a regression in 8.30.0. the current cure is to pin 8.29 if you run journal. Sorry I have no better news. Rainer Sent from phone, thus brief. Am 25.10.2017 21:00 schrieb "Naftuli Kay via rsyslog" < rsyslog@lists.adiscon.com>: > I'm running CentOS 7.4 with systemd 219

Re: [rsyslog] handling oversized messages

that's my point — i am trying to figure out what i can do … at the moment i am trying to reconcile my "cloudy" memory of message splitting. i thought that did occur in certain i/o paths, so i was surprised to see a pure truncation in the imfile path. From: Rainer Gerhards

Re: [rsyslog] handling oversized messages

There were a couple of bugs, I think imfile proprably did not abide the max size. But even if it would split the message, the json content would be damaged. Sent from phone, thus brief. Am 25.10.2017 21:05 schrieb "Randall Diffenderfer" < rdiffender...@proofpoint.com>: > that's my point — i am

Re: [rsyslog] handling oversized messages

On Wed, 25 Oct 2017, Randall Diffenderfer wrote: thanks for the followup. i had come to a similar conclusion. 10# in a 5# sack is gonna leave some stuff on the floor... :-) I do the 'write to a local file and send a truncated message', if it's a known source with json, you probably also

Re: [rsyslog] Sourcing Environment Variables for Use in Templates?

Rsyslog variables are JSON based, so everything can be represented in aJSON structure. To address it, you separate keys with ! There are two 'normal' trees of variables $! and $. Sincd $! existed first, and several things use it by default, it's considered the 'normal' variables. But since